This pipeline converts Hugo content pages to page leaf bundles.
From:
`-- content
|-- about.md
`-- post
|-- content-1.md
# run this like follows: | |
# Run only the common/shared dependency: | |
# $ pypyr deps-example | |
# | |
# Run either my_task or another_task: | |
# $ pypyr deps-example my_task | |
# $ pypyr deps-example another_task | |
# | |
# And to run both tasks: | |
# $ pypyr deps-example my_task another_task |
AWS_ACCESS_KEY_ID="your_object_storage_access_key" | |
AWS_SECRET_ACCESS_KEY="your_object_storage_secret_key" | |
BUCKET_NAME="thebucket" | |
ENDPOINT_URL="https://endpoint" | |
REGION_NAME="theregion" |
AWS_ACCESS_KEY_ID: mykeyID | |
AWS_SECRET_ACCESS_KEY: mysecretkey | |
endpoint_url: myendpoint | |
region: myregion | |
bucket: mybucket | |
destination_dir: models/spam-sms | |
files: | |
- exports/spam-sms/spam-model.h5 | |
- exports/spam-sms/spam-classifer-tokenizer.json | |
- exports/spam-sms/spam-classifer-metadata.json |
context_parser: pypyr.parser.keyvaluepairs | |
steps: | |
- name: pypyr.steps.default | |
in: | |
defaults: | |
docker_image: pytorch/pytorch:nightly-devel-cuda10.0-cudnn7 | |
task_name: MRPC | |
model_name: bert-base-uncased | |
parameters: | |
- name: model_type |
steps: | |
- name: pypyr.steps.echo | |
comment: if you want to see the echo output, remember to set loglevel <=25 | |
foreach: '{input_collection}' | |
in: | |
echoMe: just an echo with {i.arbfield} | |
- name: pypyr.steps.contextsetf | |
in: | |
contextSetf: | |
newkey: arbitrary value set in pipeline |
if you do all your python in virtualenvs, and you probably should be, the one package that sits in global is virtualenv.
when you upgrade your python installation, those virtualenvs do not upgrade themselves. you have to go and re-create your virtualenv when you upgrade python. this is annoying. this is even more annoying when you have pip.conf set with
[install]
require-virtualenv = true
[uninstall]