Python Module not Found from `systemd` service Script

systemd python module not found
python-systemd
systemd pythonpath
run python script as service ubuntu
systemd python github
python systemd journal
systemd networkd python
python script to restart service in linux

With the dozens of previous answers, I'm surprised I couldn't find anything...

I'm using the paho mqtt library for a very simply python program to report out some data (running on a Raspberry Pi). My import from within the python program (my_program.py) is:

import paho.mqtt.client as mqtt

If I run the program from the command line using python my_program.py it runs without error. However, I'm trying to get it setup as a system service to manage its execution. I've done this a dozen times with similar python programs, setting up the various bash scripts and service files. They all work except this one. I mention this because I don't think it is related to the bash or the service per se. For completeness, here is the setup.

From a bash script/systemd that runs -

## Service (my_program.service):
ExecStart=/home/pi/my_program.sh

## and bash (my_program.sh)
python /home/pi/my/directory/my_program.py

When I go to start the service, I get:

pi@ArmstrongSE:/etc/systemd/system $ sudo systemctl status my_program
* my_program.service
   Loaded: loaded (/etc/systemd/system/my_program.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sat 2019-11-23 13:59:58 PST; 22s ago
  Process: 31100 ExecStart=/home/pi/my_program.sh (code=exited, status=1/FAILURE)
 Main PID: 31100 (code=exited, status=1/FAILURE)

Nov 23 13:59:54 ArmstrongSE systemd[1]: Started my_program.service.
Nov 23 13:59:54 ArmstrongSE my_program.sh[31100]: Starting MQTT Transmitter
Nov 23 13:59:55 ArmstrongSE my_program.sh[31100]: /home/pi/data/solar/20191123135605.json
Nov 23 13:59:58 ArmstrongSE my_program.sh[31100]: Traceback (most recent call last):
Nov 23 13:59:58 ArmstrongSE my_program.sh[31100]:   File "/home/pi/my/directory/my_program.py", line 25, in <module>
Nov 23 13:59:58 ArmstrongSE my_program.sh[31100]:     import paho.mqtt.client as mqtt
Nov 23 13:59:58 ArmstrongSE my_program.sh[31100]: ImportError: No module named paho.mqtt.client

To confirm that the path assignments are there - -

From the python interpreter I get:

>>> import paho.mqtt.client as mqtt
>>> print(mqtt.__file__)
/home/pi/.local/lib/python2.7/site-packages/paho/mqtt/client.pyc

The sys.path reports:

>>> import sys
>>> print(sys.path)
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-arm-linux-gnueabihf', 
'/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', 
'/home/pi/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', 
'/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/gtk-2.0']

I'm stuck... Any thoughts on why this won't load?

Update/Clarification:

I can run the bash script from anywhere (any directory) and it works. So, it must be something in the systemd setup (?).

I think I figured out what the issue is... systemd runs its services by default as root. Which would/should, I would think (see below), allow for the equivalent of running things as root using sudo for another user or the actual root user on the cli.

Just focusing on the services file for the systemd, I found in the man pages that there is a [Service] parameter that allows you to designate who runs the service. In this case, I tried setting the parameter as:

[Service]
User=pi
ExecStart=/home/pi/my_program.sh

This allowed the service to run without the error (the error noted above).

I can also use these settings and execute without error:

[Service]
User=root
ExecStart=/home/pi/my_program.sh

Why:

For others who find themselves here, if you omit the User= parameter from the [Service] section the default is root. HOWEVER, that is different than setting the User= parameter to root (e.g., User=root) or to another user (in my case pi).

I don't really understand the nuances, but from the documentation on environmental variables, the User= parameter is used to set the $PATH variable. if the service file doesn't have a User= set I think it causes a problem with the path assignments. I think that's why setting User= to either pi or root solves this problem, while using the default seems to cause the problem.

Explanations and education welcome. Thanks.

Running Python script via systemd fails to load module, However as soon as I attempt to have a systemd unit call the script, running systemctl status myservice.service shows ImportError: No module� The most likely explanation is that you have some environment variables set (e.g. an extension of your PYTHONPATH?) which is not set when the script is being run by systemd. You could try using the Environment parameter (see [0]) so set PYTHONPATH (and whatever else might influence this) to whatever it is in your console session.

Try to install module with root.

Systemd service not recognizing python library, Systemd starts the processes with a minimal environment. In this case you are probably missing PYTHONPATH and maybe more. Try it yourself� It appears that you have been operating under the assumption that whenever you called source /home/user/venv/activate, the python3 command (and the pip3 command) would subsequently call the relevant executable from /home/user/venv/bin.

This may be a linking issue that python has. If you were to install a library via pip, it may or may not link or install properly. I would suggest re-installing the mqtt library using something like...

sudo -H pip3 install --system paho-mqtt

Also, I noticed that your import is wrong. You wrote: import pho.mqtt.client as mqtt.

It should be :

import paho.mqtt.client as mqtt

Running a Python Script with Tensorflow via systemd doesn't work , I don't believe you really need this (man systemd.service):. This may be The python3 interpreter does not find the module utils . It searches� my python script imports modules installed using pip . this module is not getting imported when running through systemd. but when i run the python script it runs without any problems Reply Carlos on February 14, 2020 6:08 pm

torfsen/python-systemd-tutorial: A tutorial for writing a , GitHub is home to over 50 million developers working together to host and review Writing a systemd service in Python turns out to be easy, but the complexity of import time while True: print('Hello from the Python Demo Service') time.sleep( 5) You might have noticed that the output of our script's print calls did not show � Run Python script as systemd service. GitHub Gist: instantly share code, notes, and snippets.

How to create a systemd service for python script with virtualenv , ExecStart=/home/user/venv/bin/python script.py systemd service config: Documentation=https://docs.python.org/3.7/library/http.server.html After=network. target [Service] Also the "command not found" might be from inside the .bashrc. and yes the path is set to Python 3.7, which means we can now access python 3.7 via python3. So let’s install the module. command: python3 -m pip install matplotlib. and as you can see the module is installed successfully. Even though it uses the cached memory because in this case matplotlib is already installed in other versions.

Python script on boot, module import error, service: Main process exited, code=exited, status=1/FAILURE Jul 11 13:49:32 localhost systemd[1]: sample.service: Failed with result 'exit-code'. Writing a systemd Service in Python. Many Linux distributions use systemd to manage the system's services (or daemons), for example to automatically start certain services in the correct order when the system boots.

Comments
  • "from the python interpreter I get". You did that as root...?
  • No, that was done as pi (the only user on the system); to be clear it wasn't done with sudo either. I think it has something to do with the system file - it runs fine as pi, but not when started in systemd. It's not lost on me that I have everything installed locally under user pi - but I'm also running everything as user pi. (?)