Nginx is a very popular webserver used by various websites and web apps, especially those that deal with high traffic. It consists of many modules, one of which is Upstream. In some cases, it leads to errors that impact the availability of the websites. In this article, learn what Nginx Upstream error is and how to fix it.
What are Nginx Upstream Errors?
The way a reverse proxy works is it receives a request from the client and sends it to Upstream server (it is the server that completes the request). It then awaits for the response from the server and returns it to the client.
The Nginx Upstream errors come into picture when the proxy server either doesn’t receive any response or receives an invalid response.
Users are likely to see different variation of the error like this:
502 bad gateway
502 bad gateway nginx
502 Proxy error
504 Gateway Timeout
HTTP 504 Gateway Timeout
These messages doesn’t provide much detail. Therefore, you need to dig into the error log, which contains the following log:
Why the Problem Occurs (Six Reasons and the Respective Solutions)
Here are the six reasons why the Nginx Upstream error occurs:
Load is excess on the origin server
The most common reason for downtime is high server load, and that might be the case for Nginx Upstream error.
Here’s why the origin server might be facing load issues:
The origin server is facing downtime
If the origin server is experiencing downtime, you’ll face Nginx upstream errors. That’s because it won’t respond to the requests.
To fix this, you must get the origin server back online. Kill all the dead processes and restart.
Firewall is blocking requests
If you’re using a firewall (and you should), it could be preventing the requests from the clients to get through.
By default, firewalls are designed to block requests from uncommon port. If your services are running on custom ports, then the firewall may block their access. So Nginx won’t be able to communicate to the necessary services.
To fix this issue, you need to check the ports used by each service using the netstat command. If you find any service is running on custom port, make the necessary configuration changes to allow firewall to accept those requests.
There’s network-related issue
There are certain problems related to network that can trigger this issue. This includes ISP problems, DNS issues, or routing issues.
If you have made DNS changes (like changing nameservers and hosting servers) lately, then it will take some time to reflect in global scope. So during that time, the domain may stay unroutable. Because of this, the ISPs may block the access to the website.
For fixing this issue, you need to check the DNS connectivity of the domain. Execute this command for this purpose:
Besides that, you need to access the domain using proxy servers that are not related to this server. This helps to check if error is only limited to the website or across several websites.
In case you find conflicting DNS settings, correct them at the earliest. Likewise, if there are ISP-related issues, you must contact your ISP regarding the problem and seek assistance.
Server software timeout
If the webserver takes more time than usual, it will lead to the upstream error. Ideally, the response should be within the timeout values of the caching server (the timeout value between upstream server and proxy server).
To fix this issue, you need to open the Nginx configuration file and update the following Nginx timeout values:
To save and apply these values, update Nginx.
There are bugs in application code base
Last but not least, it’s the bugs present in your website or web application code base that can lead to the Nginx Upstream errors. Maybe there are compatibility issues or the code is written in such a way that takes more time to load the files.
Either way, you need to sit down with your developers, figure out the bug, and fix it. Analyze the error log or use a debugging tool for faster results.
So that’s how you fix the Nginx Upstream error. For more assistance, contact the hosting support team.
What are Nginx Upstream Errors?
The way a reverse proxy works is it receives a request from the client and sends it to Upstream server (it is the server that completes the request). It then awaits for the response from the server and returns it to the client.
The Nginx Upstream errors come into picture when the proxy server either doesn’t receive any response or receives an invalid response.
Users are likely to see different variation of the error like this:
502 bad gateway
502 bad gateway nginx
502 Proxy error
504 Gateway Timeout
HTTP 504 Gateway Timeout
These messages doesn’t provide much detail. Therefore, you need to dig into the error log, which contains the following log:
Code:
2018/11/01 18:15:02 [error] 1713#0: *1 upstream timed out (110: Connection timed out) while connecting to upstream, client: xx.xx.xxx.xxx, server: , request: "GET / HTTP/1.1", upstream: "http://xx.xx.xx.xx:8082/", host: "xx.xx.xx.xx"Here are the six reasons why the Nginx Upstream error occurs:
Load is excess on the origin server
The most common reason for downtime is high server load, and that might be the case for Nginx Upstream error.
Here’s why the origin server might be facing load issues:
- High number of people are visiting the website (because of promotion or latest trend)
- The server is compromised and sends spams
- Hackers are carrying out brute force or DDoS attacks
- There are bugs in the web application that are causing memory leaks
The origin server is facing downtime
If the origin server is experiencing downtime, you’ll face Nginx upstream errors. That’s because it won’t respond to the requests.
To fix this, you must get the origin server back online. Kill all the dead processes and restart.
Firewall is blocking requests
If you’re using a firewall (and you should), it could be preventing the requests from the clients to get through.
By default, firewalls are designed to block requests from uncommon port. If your services are running on custom ports, then the firewall may block their access. So Nginx won’t be able to communicate to the necessary services.
To fix this issue, you need to check the ports used by each service using the netstat command. If you find any service is running on custom port, make the necessary configuration changes to allow firewall to accept those requests.
There’s network-related issue
There are certain problems related to network that can trigger this issue. This includes ISP problems, DNS issues, or routing issues.
If you have made DNS changes (like changing nameservers and hosting servers) lately, then it will take some time to reflect in global scope. So during that time, the domain may stay unroutable. Because of this, the ISPs may block the access to the website.
For fixing this issue, you need to check the DNS connectivity of the domain. Execute this command for this purpose:
Code:
dig domain.comIn case you find conflicting DNS settings, correct them at the earliest. Likewise, if there are ISP-related issues, you must contact your ISP regarding the problem and seek assistance.
Server software timeout
If the webserver takes more time than usual, it will lead to the upstream error. Ideally, the response should be within the timeout values of the caching server (the timeout value between upstream server and proxy server).
To fix this issue, you need to open the Nginx configuration file and update the following Nginx timeout values:
Code:
proxy_connect_timeout 1200s;
proxy_send_timeout 1200s;
proxy_read_timeout 1200s;
fastcgi_send_timeout 1200s;
fastcgi_read_timeout 1200s;There are bugs in application code base
Last but not least, it’s the bugs present in your website or web application code base that can lead to the Nginx Upstream errors. Maybe there are compatibility issues or the code is written in such a way that takes more time to load the files.
Either way, you need to sit down with your developers, figure out the bug, and fix it. Analyze the error log or use a debugging tool for faster results.
So that’s how you fix the Nginx Upstream error. For more assistance, contact the hosting support team.
