I have amnesia when it comes to setting up bash files correctly. I will spend hours upon hours debugging the dumbest things simply because I can't remember the nuances of setting up my bash and shell files correct.
-
Open a crontab:
crontab -e
-
List the contents of your crontab:
crontab -l
-
Use bash, not sh, but adding the following line to your crontab:
SHELL=/bin/bash
-
Add any files to your path that you need. This is the default system one:
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
- You might need to append e.g.
:~/bin
to the end. - You could also run
echo PATH=$PATH
to get what you should put there
- You might need to append e.g.
-
Using a special python configuration? Maybe add your PYTHONPATH
PYTHONPATH=/usr/lib/python2.7/dist-packages:$PYTHONPATH
- You should probably be using virtualenv tbh
-
Use the following format:
* * * * * . /full/path/to/file/shell_script.sh > /full/path/to/file/.cron.log 2>&1
- Use a period, i.e.
. /full....
to start it. Don't just put the file. - Use
2>&1
to redirect all output to a log file - Make sure there's a space after
2>&1
- via https://www.thegeekstuff.com/2012/07/crontab-log
- Use a period, i.e.
##Shell Script
-
When naming your shell script, name it the exact same thing as your python file, but with a
.sh
extension. -
Add
#!/bin/bash
to the top -
If your script is shelling out to environment commands, make sure you add that to your path. You can get that by running
echo PATH=$PATH
-
You need to make sure you CD into the directory running the python script. make sure you have:
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" cd "$DIR"
-
Of course, if you want to end up in the directory you started, add the following towards the top:
CURRENTDIR="$PWD"
and the following to the bottom:
cd $CURRENTDIR
-
If you're using virtualenv to load packages, you'll want to make sure you set that up before calling your python script. I don't need that right now, but something to keep in mind.
##Logging in your python file
Don't rely on bash for your only logging. It's pretty simple to set up logging in Python. Here's an example using rotating files. You need to add this to the top of each file, though :-/
import logging
from logging.handlers import RotatingFileHandler
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# add a rotating handler
handler = RotatingFileHandler(".python_script.log", maxBytes=2000000, backupCount=5)
formatter = logging.Formatter('%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
And then when logging, do something like:
logger.info("this is a message about %s", "something that happened")
SHELL=/bin/bash
saved my day!