Use Supervisor to run your Python Tests

Photo by Christoffer Engström on Unsplash

For most scenarios where you need your test to run for a long time, with a bad connection, keeping your ssh connection up might be almost be impossible at times.

A simple fix for this problem run your python script as a cronjob using crontab -e . However there might be some pitfalls to this simple fix. Instead, we can let supervisor execute your python scripts. You can then log in via SSH or visit a web page to see the progress of the task. Here’s what we want to achieve.

Requirements

1. Be able to see what tasks are running now on supervisor
2. Be able to see the status of the task
3. If there is a failure, we should be able to visit a log file
4. When the task is completed, notify via email

With the above in mind. The next part of the article documents my process of installing and using supervisor.

Setup

1. Google Cloud Platform with GPU
2. Ubuntu 16.04
3. Python 3 managed via Mini-conda

Installation

Installation of supervisor on ubuntu is fairly straight forward. Just run the following commands to install and start supervisord.

$ sudo apt-get…