mjhjmtu

I have a Anaconda Python Virtual Environment set up and if I run my project while that virutal environment is activated everything runs great.

But I have a cronjob configured to run it every hour. I piped the output to a log because it wasn't running correctly.

crontab -e:

10 * * * * bash /work/sql_server_etl/src/python/run_parallel_workflow.sh >> /home/etlservice/cronlog.log 2>&1

I get this error in the cronlog.log:

Traceback (most recent call last):
 File "__parallel_workflow.py", line 10, in <module>
 import yaml
ImportError: No module named yaml

That is indicative of the cronjob somehow not running the file without the virtual environment activated.

To remedy this I added a line to the /home/user/.bash_profile file:

conda activate ~/anaconda3/envs/sql_server_etl/

Now when I login the environment is activated automatically.

However, the problem persists.

I tried one more thing. I changed the cronjob, (and I also tried this in the bash file the cronjob runs) to explicitly manually activate the environment each time it runs, but to no avail:

10 * * * * conda activate ~/anaconda3/envs/sql_server_etl/ && bash /work/sql_server_etl/src/python/run_parallel_workflow.sh >> /home/etlservice/cronlog.log 2>&1

Of course, nothing I've tried has fixed it. I really know nothing about linux so maybe there's something obvious I need to change.

So, is there anyway to specify that the cronjob should run under a virutal environment?

Found answer on stack over flow:

https://stackoverflow.com/questions/3287038/cron-and-virtualenv

The solution is to reference the python executable within the virtual environment itself. In my case, I changed the bash file to run this executable:

/home/etlservice/anaconda3/envs/sql_server_etl/bin/python