Why gUnicorn spaws 2 process when running a Flask

Why gUnicorn spaws 2 process when running a Flask

gunicorn pass arguments to app
gunicorn flask
gunicorn config file
gunicorn workers
gunicorn logging flask
gunicorn gevent flask
gunicorn restart workers
gunicorn slow

I'm running a Flask application that is basically pulling tweets from Twitter. While running the app with the embedded Flask server gives no troubles, when running within gUnicorn I get duplicated tweets, mostly because I have 2 threads receiving the callback from Twitter.

For instance, if I start my app using

python app.py

When receiving tweets I'm getting this expected output, see that I've attached thread info (first param) in the logger output:

140721974449920 2015-03-12 17:59:13,030 INFO: Got message from streaming Twitter API! [in /home/mosquito/git/opencoast_streamer/app.py:83]
140721974449920 2015-03-12 17:59:14,646 INFO: Got message from streaming Twitter API! [in /home/mosquito/git/opencoast_streamer/app.py:83]
140721974449920 2015-03-12 17:59:49,031 INFO: Got message from streaming Twitter API! [in /home/mosquito/git/opencoast_streamer/app.py:83]

As you can see, timestamp looks valid too, checking at the mongo collection where I'm storing this, I see documents are OK. Then, if I start the app using gunicorn:

gunicorn app:app -b localhost:8000 --debug

And then check the logs, I can see that 2 different threads are getting data:

139883969844992 2015-03-12 17:52:05,104 INFO: Got message from streaming Twitter API! [in /home/mosquito/git/opencoast_streamer/app.py:83]
139883961452288 2015-03-12 17:52:05,106 INFO: Got message from streaming Twitter API! [in /home/mosquito/git/opencoast_streamer/app.py:83]
139883969844992 2015-03-12 17:53:36,480 INFO: Got message from streaming Twitter API! [in /home/mosquito/git/opencoast_streamer/app.py:83]
139883961452288 2015-03-12 17:53:36,481 INFO: Got message from streaming Twitter API! [in /home/mosquito/git/opencoast_streamer/app.py:83]

As you can see something weird is going on....then I went to see and check gunicorn:

ps aux | grep gunicorn

mosquito 25035  3.1  0.3  54612 12516 pts/1    S    15:31   0:01 /home/mosquito/www/env/bin/python /home/mosquito/www/env/bin/gunicorn app:app -b localhost:8000
mosquito 25606  0.0  0.4  66904 17016 pts/1    R    15:32   0:00 /home/mosquito/www/env/bin/python /home/mosquito/www/env/bin/gunicorn app:app -b localhost:8000
mosquito 25610  0.0  0.0  13220   956 pts/3    S+   15:32   0:00 grep --color=auto gunicorn

Thus, I'm starting to think that this has to do with gUnicorn...any ideas why gUnicorn is spawining 2 process for my Flask app?

Thanks!


I believe this is not gUnicorn's fault but rather the intended behavior of Werkzeug. Werkzeug has a "reloader" process that monitors for file changes (and hence reloads if it detects a change in your .py files.

For more information on the reloader go here.

To get you through your trouble, I believe adding use_reloader=False to your call to app.run: app.run(use_reloader=False) would do the trick.

You can also see this SO answer for more information.

How to use Flask with gevent (uWSGI and Gunicorn editions), Create asynchronous Flask application and run it with uWSGI or probably increasing this number by 1-2 orders of magnitude from the previous limit. In the multi-threaded mode, Flask spawns a thread for every incoming HTTP request. Gunicorn uses the worker process model to serve HTTP requests. And then run 50 000 requests with different concurrency. Seen Errors Connection reset by peer (104) I got a lot of apr_socket_recv: Connection reset by peer (104) with gunicorn. See also. digitalocean.com: How To Serve Flask Applications with Gunicorn and Nginx on Ubuntu 16.04


Gunicorn would always spawn at least 2 processes, even when you set --workers=1. One of the processes is a master process, which spawns other worker processes to handle requests.

Running Docker Container With Gunicorn and Flask - DEV, The parameters are pretty much self-explanatory: We are telling Gunicorn to spawn 2 worker processes, running 2 threads each. We are also� The “webserver” that ships with flask (e.g. “app.run()”) is really great for local development. And that’s about it. It isn’t designed to take the abuse that a normal webserver receives. Our microservice was using that runner. A WSGI HTTP server like gunicorn (or uWsgi) is designed to take that kind of abuse from the web. It turns


gunicorn --workers=1 app:app -b localhost:8000 --debug

Source

Best practices for starting new threads/processes (that are not part of , EB123 opened this issue on Nov 23, 2018 � 2 comments all of them are a flask app wrapped by gunicorn (currently we're mostly using the gevent worker type). but the worker process is currently also running a pika thread that is doing some heavy entry points rather than try to spawn a background thread for Gunicorn. But the schroot process itself continues to live and is now a child of the init process: schroot(8067)---gunicorn(8068)---gunicorn(8073)---{gunicorn}(8078) This whole behavior seems to be related to both the way schroot and gunicorn work. How can I let supervisor stop my schroot-hosted process correctly?


Gunicorn's settings, Could someone clarify what workers and threads in gunicorn's Threads are the number of threads in each process. Start with sync workers and CPUs * 2 + 1 . --threads 5 without the preload option, each worker thread will spawn a are referring to the Flask server, which can be started via app.run() . description "Gunicorn application server running myproject" start on runlevel [2345] stop on runlevel [!2345] We’ll tell the init system that it should restart the process if it ever fails. Next, we need to define the user and group that Gunicorn should be run as.


Running Gunicorn — Gunicorn 20.0.4 documentation, -w WORKERS, --workers=WORKERS - The number of worker processes. This number should generally be between 2-4 workers per core in the server. With Docker and Gunicorn we can deploy our flask application in an easy and quick way. We can also generate SSL/TLS certificates for free thanks to Let’s Encrypt and automate the renewal process


Selecting gunicorn worker types for different python web , Gunicorn provides serval types of worker: sync, gthread, gevent, evenlet 2. request per thread: recommended for high CPU bounded A web application: an actual logic application written using web framework (such as Django, Flask). worker process spawns a number of threads, gunicorn delegates a� Note: this is a manual procedure to deploy Python Flask app with gunicorn, supervisord and nginx. A full automated CI/CD method is described in another post. Then locate the private & public keys