Truncated or oversized response headers received from daemon process

timeout when reading response headers from daemon process
wsgiapplicationgroup
wsgiprocessgroup
mod_wsgi memory limit
invalid option to wsgi daemon process definition
no wsgi daemon process called has been configured
mod_wsgi graceful-timeout
mod_wsgi python 3

I recently migrated a python django application from a debian system to a redhat enterprise distribution. The application is hosted using httpd, mod_wsgi and running in a venv in an daemon process. On large requests I now get following error message in the log file:

"Truncated or oversized response headers received from daemon process" 

I have never experienced anything like this and Google is not the key here as well. I checked configuration of apache, but no config is related to response headers in there.

My httpd.conf configuration looks like this ( pretty standard):

WSGIPassAuthorization On
WSGIScriptAlias / /var/www/myapp/wsgi.py
WSGIDaemonProcess my.name python-path=/path/to/myapp/:/path/to/venv/lib/python2.7/site-packages display-name=%{GROUP}
WSGIProcessGroup my.name

Does any Guru have a hint in which direction I should look?

We recently ran into this issue, and after days of vigorous ugoogleizing and massive headache, we discovered that we were using psycopg2-binary as our database connector dependency (I know, newbs)! It states right in their documentation not to use the package in a production environment.

We did add all the other proposed answers such as adding 'WSGIApplicationGroup %{GLOBAL}' to our settings (which we kept), but all of them alone and together didn't solve the issue.

We also found that other C libraries like numpy, cause issues.

Hope this helps someone some day.

Django Webfaction 'Timeout when reading response headers from daemon process'

http://initd.org/psycopg/docs/install.html#prerequisites

Truncated or oversized response headers received from daemon , The error message is also more specific as you have seen. The default maximum header size for what is returned from mod_wsgi daemon mode, that is header name and value, is about 8192 bytes. You can override this by using the 'header-buffer-size' option to WSGIDaemonProcess. I am aware that this problem has been mentioned in several threads. I have followed them for last three days, and I do not seem to be making any headway. I shall be really grateful if somebody could help me figure this out.

The code used from Apache by mod_wsgi applies a limit on the size of a single response header returned from mod_wsgi daemon mode processes. This would result in a really cryptic error message from Apache which didn't at all point to the problem. From memory the previous error was:

Premature end of script headers

The size limit was also hard coded in Apache and couldn't be changed. This has caused problems for some Python web applications such as Keystone in OpenStack which generates very large authentication headers.

In mod_wsgi 4.1+, the reliance on the Apache code has been removed and the limit is now configurable. The error message is also more specific as you have seen.

The default maximum header size for what is returned from mod_wsgi daemon mode, that is header name and value, is about 8192 bytes. You can override this by using the 'header-buffer-size' option to WSGIDaemonProcess.

Can you indicate what application and what header was so large that the limit was reached as would like to know what other Python web applications besides Keystone are generating such large headers if is a commonly used application.

A second possibility, deriving from the 'truncated' reference in that message, is that your mod_wsgi daemon process crashed. You don't say though that you saw a 'Segmentation fault' or similar message indicating that a crash occurred. Check for that and if there are other messages in the error log at the time, then indicate what they were and can look at that as the primary problem.

wsgi:error Truncated or oversized response headers received from , problem recently: [wsgi:error] [pid 4305] [client 1.1.1.10:9787] Truncated or oversized response headers received from daemon process Later  The steps are to attach gdb to the mod_wsgi daemon process and then trigger the request and see where it crashes. Copy link Quote reply nottheend commented Mar 23, 2019 •

I had an issue with this in CentOS 7 server when deploying Django using httpd with mod_wsgi 4.5.4. I had to revert back to using mod_wsgi 4.3.2 which solved my problem.

"Truncated or oversized response headers received from daemon , "Truncated or oversized response headers received from daemon process" with postgresql #390. Closed. sidark opened this issue on Jan 10,  Hi, The json.loads function where I pass a JSON-string of length 4187288 causes WSGI error: Truncated or oversized response headers received from daemon process 'my process'

I have installed filebeat which changed my ssl version so that psycopy2 needs to be updated and the error was Truncated or oversized response headers received from daemon process

Do the following:-

Uninstall your psycopy2 package using pip. I am using pip3 because my python version is 3.6.8

sudo pip3 uninstall psycopg2

Reinstall psycopy2 using pip.

sudo pip3 install psycopg2

Before psycopg2-2.7.4 now psycopg2-2.8.3

Apache wsgi Truncated or oversized response headers (Page 1 , [Thu Nov 29 01:31:44.366100 2018] [wsgi:error] [pid 6968] [client] Truncated or oversized response headers received from daemon process  Truncated or oversized response headers received from daemon process Showing 1-4 of 4 messages

I suddenly had this same problem after an update. The next update fixed the problem... I run arch, as of the date of this post, the WSGI version in repo works.

Receiving error: "Truncated or oversized response headers received , from django.http.response import HttpResponse Truncated or oversized response headers received from daemon process 'fb_bot':  There is no indication that the daemon process crashed, so it may well be the case that the response headers returned have values which are too long. This can happen when using fancy authentication

Django, headers received from daemon process". (too old to reply). Gregory Gundersen​. 5 years ago. Permalink. Hi all, I posted my question on Stack Overflow Truncated or oversized response headers received from daemon process client 172.24.112.31:44450] Truncated or oversized response headers received from daemon

Conflict between php and django with apache sites-available, 1564:tid 1861219376] [client 127.0.0.1:60308] Truncated or oversized response headers received from daemon process 'app': /home/user/app/app/​wsgi.py,  This earned me a stackdump, because wsgi couldn't find arches in the settings.py import arches line. I just have to try changing that path in settings.py to see whether that just takes me back to the truncated or oversized response headers messages or opens new doors.

Bug #1569081 “Truncated or oversized response headers received f , This might need to be overridden where excessively large response headers are returned, such as in custom authentication challenge 

Comments
  • Thanks... this worked for me. uninstalled psycopg2-binary and installed psycopg2
  • Lifesavers! Reinstalled psycopg2 and it worked.
  • this should be accepted answer. worked for me as well!
  • Reinstalling psycopg2-binary worked for me. Thanks!
  • Just doing 'pip install psycopg2' requires that prerequisites be installed by 'apt' first, but then you'll have a working psycopg2 WITHOUT psycopg2-binary which causes this bewildering problem. See psycopg.org/docs/install.html#prerequisites
  • I will check this tomorrow morning. The problem is caused by Django, I grab svg-xml from some statistical graphs on the webpage and push them to django (1.6.5), where I use Cairo and xhtml2pdf to convert them to png files and put into a pdf file, which is the response. I have run this app on a WSGIDaemonProcess many times before and never had problems. First time this happens when I test it on a new RHEL 6 server. Python is 2.7.3 and Apache is 2.2.
  • OK. No success! I increased header-buffer-size=32768 first to approx. 16000 and now 32768. No changes. Restarted apache, removed .pyc files of wsgi.py and so on. No success. For more info, here is the wsgi.py I use:
  • import os os.environ.setdefault("DJANGO_SETTINGS_MODULE", "app.settings") from django.core.wsgi import get_wsgi_application application = get_wsgi_application()
  • Sorry, my fault. I guess this is an success. I didn't close the app on restart, so the process was not reloaded. As far as I see it works. Let me test tomorrow, then I give the OK to mark as resolved. Thank you very much Graham. Saved me a hell of a time.Cheers Kartopete
  • Just go set 'WSGIApplicationGroup %{GLOBAL}' and see if the issue goes away. It is more likely the process is crashing and you just aren't noting the error message indicating that from the Apache error logs.
  • Are you setting the application group to %{GLOBAL}? It is more likely your daemon mode process is crashing. See modwsgi.readthedocs.io/en/develop/user-guides/…
  • Do you have SELinux extensions enabled? That can cause strange memory errors in some circumstances.
  • Did you ever figure this out?