I check my servers for upgrades on a regular basis and when I noticed yesterday that PHP was upgraded to version 5.5.12, I backed up everything of course (backup guide) and decided to upgrade which can easily be done with sudo apt-get dist-upgrade. Unfortunately I’ve ran into quite an annoying issue which I had not encountered before, all pages returned “502 Bad Gateway” errors.
I’ve done this a numerous time and the upgrade went very smoothly. I was even asked if I wanted to keep local configuration files or not. I choose yes and the installation proceeded. After installation was completed php got restarted and I received a notice that the upgrade was successful. Unfortunately when I went to check my website I got annoying 502 Bad Gateway” errors on every single page.
What does this error message mean? A gateway, is like an access point, a bridge that communicate one service with another, in this case the gateway can be a service/application (WordPress, running on PHP) that is working and recieving requests from NGINX web server. So there is a communication issue between PHP and NGINX.
Problem Solving 1: is PHP-FPM running?
As you can see I am also running the latest version of NGINX (1.6.0 as of May 10th 2014) which was updated a few days earlier. It was clearly an issue with PHP so I started to think of what could’ve gone wrong. The first thing that you should always do is check if PHP is even running. Perhaps something went wrong while restarting PHP. Use the following command to check whether PHP-FPM is running.
ps aux | grep php |
ps aux will output all processes that are running, so we add | grep phpto only output processes with php in the name. If you see PHP processes, then this is not the the issue. Otherwise try to stop/start/restart PHP.
sudo service php5-fpm stop
|
sudo service php5-fpm start
|
sudo service php5-fpm restart
|
If there are still no PHP-FPM processes running, you might try to remove PHP and reinstall it. If PHP-FPM is running correctly, skip this step and go to the following section.
sudo apt-get remove php5 php5-cgi php5-fpm |
sudo apt-get install php5 php5-cgi php5-fpm |
Problem Solving 2a: is PHP-FPM listening correctly?
A common issue is that the PHP-FPM service is not listening to the host/port which is configured in NGINX. Find the www.conf file on your server in the PHP folder. On Ubuntu this can be found here:
/etc/php5/fpm/pool.d/www.conf |
Search for the listen parameter and make note of it:
; The address on which to accept FastCGI requests. ; Valid syntaxes are: ; 'ip.add.re.ss:port' - to listen on a TCP socket to a specific address on ; a specific port; ; 'port' - to listen on a TCP socket to all addresses on a ; specific port; ; '/path/to/unix/socket' - to listen on a unix socket. ; Note: This value is mandatory. listen = 127.0.0.1:9000 |
As you can see this is set to 127.0.0.1:9000, but it is also commonly set to /var/run/php5-fpm.sock. In my config this was set to the latter and I found out that php5-fpm.sock was simply missing (removed during the upgrade), so I changed it to 127.0.0.1:9000.
Now find your NGINX server configuration file, usually located at /etc/nginx/sites-available, and open it. Look for something like this (location ~ \.php$):
location ~ \.php$ { try_files $uri =404; fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi_params; } |
The parameter we are looking for is fastcgi_pass. This has to be the same value as in the listen parameter in the www.conf file. Change it accordingly. In my case I changed fastcgi_pass unix:/var/run/php5-fpm.sock; to fastcgi_pass 127.0.0.1:9000;.
Of course you may never forget to restart the NGINX and PHP5-FPM services! You would be surprised how many people forget this step. It is necessary to do this so that the updated configuration files are loaded.
sudo service php5-fpm restart
|
sudo service nginx restart
|
Problem Solving 2b: permanently fix the issue
While the above fix works perfectly, it is not ideal since we need to change parameters which were probably changed for a reason. It turns out that issues like this are very common due to file permissions and owner issues. That is way php-fpm.sock was missing. The bug report can be found here. Follow these steps to fix permissions and owner on Ubuntu.
So first of all, make sure that your virtual hosts NGINX files are using fastcgi_pass unix:/var/run/php5-fpm.sock; at the php configuration like this:
location ~ \.php$ { try_files $uri =404; fastcgi_pass unix:/var/run/php5-fpm.sock; fastcgi_index index.php; include fastcgi_params; } |
Now edit the NGINX configuration file, found at /etc/nginx/nginx.confand take note of the user (usually located on line 1). This can be www-data, nginx or something else you have set. I will call this “NGINXUSER”.
Now we will change the php-fpm configuration file, located at /etc/php5/fpm/pool.d/www.conf. Simply locate the listen parameter and change/add the appropriate values to match this (change NGINXUSER to the user you have noted down earlier):
listen = /var/run/php5-fpm.sock listen.owner = NGINXUSER listen.group = NGINXUSER |
Restart PHP5-FPM and NGINX with the following commands and now you should be able to upgrade PHP without any issues.
sudo service php5-fpm restart
|
sudo service nginx restart
|
Problem Solving 3: Change NGINX config
Perhaps the buffer and timeout parameters are configured incorrectly for your server or they don’t suffice anymore. You can find the NGINX configuration file at /etc/nginx/nginx.conf. Try increasing the following parameters.
Increase buffer and timeouts inside http block:
Increase buffer and timeouts inside http block:
http { ... fastcgi_buffers 8 16k; fastcgi_buffer_size 32k; fastcgi_connect_timeout 300; fastcgi_send_timeout 300; fastcgi_read_timeout 300; ... } |
Problem Solving 4: Disable APC
APC caching can cause 502 Bad Gateway issues under particular environments causing segmentation faults. I highly suggest using Memcache(d), but XCACHE is also a good alternative.
Conclusion
Getting unexpected are always very annoying and can really cause a lot of issues (decreased potential income, loss of visitors,…). Especially when you have no clue what could be causing the error. Hopefully this article helped you solve this issue.
22 Comments.
listen.owner = www-data
listen.group = www-data
in /etc/php5/fpm/pool.d/www.conf (and restarting the PHP service) did the trick.
“listen.owner = NGINXUSER
listen.group = NGINXUSER”
section that was wrong and I didn’t see this anywhere else.
Thank you very much!
I have a moodle, wordpress and other cms running on campus server. The weird thing is I only get this 502 only on moodle page (/cource/view.php) the others if fine.
Here is my error log:
2015/01/10 12:14:38 [error] 20176#0: *21604 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: xxx.xxx, server: mysite.ac.id, request: “GET /course/view.php?id=4 HTTP/1.1”, upstream: “fastcgi://unix:/tmp/php5-fpm.sock:”, host: “mysite.ac.id”, referrer: “http://mysite.ac.id/my/”
I’ve tries your step mention above but still nothing.
Thank you very much
Thank you very much!
than you have to look at your virtual host configuration on nginx config file. you have to set or correct the fastcgi_pass.
the fastcgi_pass is the variable to set the socket connection between nginx and php cgi
another point of issue is that the binary starter script could missed the following entries(important) open with: nano /usr/bin/php-fastcgi
PIDFILE=/var/run/php-fastcgi/php-fastcgi.pid
FASTCGI_GROUP=www-data
SOCKET=/var/run/php-fastcgi/php-fastcgi.socket
PIDFILE=/var/run/php-fastcgi/php-fastcgi.pid
CHILDREN=6
PHP5=/usr/bin/php5-cgi