Nginx Reverse Proxy Cache for WordPress and Apache

I ran a simple test against Nginx v0.5.22 and Apache v2.2.8 using ab (Apache’s benchmarking tool). During the tests, I monitored the system with vmstat and top. The results indicate that Nginx outperforms Apache when serving static content. Both servers performed best with a concurrency of 100. Apache used four worker processes (threaded mode), 30% CPU and 17MB of memory to serve 6,500 requests per second. Nginx used one worker, 15% CPU and 1MB of memory to serve 11,500 requests per second.
–The Linix Journal

This article describes how to set up Nginx as a reverse proxy static cache for WordPress. This web server stack will enable you to scale single or multiple WordPress installations on a medium to low end VPS or dedicated server. It is also the same stack this website runs on along with about 10 other WordPress sites on an unmanaged Xen VPS with only 768M of Ram.

WordPress Performance Plugin Stack

This setup requires Dan Collis-Puro’s Nginx proxy cache integrator plugin and works great with W3 Total Cache

Our W3 Total Cache setup is page caching with disk enhanced, minify with disk and APC as an object and database cache. By setting page cache and minify to disk they will be served by Nginx.

We are also utilizing W3T’s great self hosted CDN feature and have another domain on this same server with 4 cnames pointing to it.

Browsers can only download a few files at once, only 4 in some cases. Pipelining is a technique whereby aliases (subdomains for example) of your server are used to allow your browser to increase the practical limit of files that can be downloaded in parallel. Doing so maximizes the throughput of the your internet connection and allows the browser to render a page faster. W3TC takes care of managing these files transparently once DNS CNAMEs (aliases) and subdomains are properly configured.
–Frederick Townes, Author W3 Total Cache

How to configure Nginx to serve static files and pass the heavy PHP and WordPress lifting to Apache

The problem with using Apache alone is that it opens up a connection and hits php on every request even for static files. This wastes connections because Apache will keep them open and when you have lots of traffic your connections will be bogged down even if they are not being used.

By default Apache listens for requests on port 80 which is the default web port. First we are going to make changes to our Apache conf and virtual hosts files to listen on port 8080.

Apache Config

httpd.conf

set KeepAlive to off

ports.conf

NameVirtualHost *:8080
Listen 8080

Per Site Virtual Host

<VirtualHost 127.0.0.1:8080>
     ServerAdmin info@yoursite.com
     ServerName yoursite.com
     ServerAlias www.yoursite.com
     DocumentRoot /srv/www/yoursite.com/public_html/
     ErrorLog /srv/www/yoursite.com/logs/error.log
     CustomLog /srv/www/yoursite.com/logs/access.log combined
</VirtualHost>

You should also install mod_rpaf so your logs will contain the real ip addresses of your visitors. If not your logs will have 127.0.0.1 as the originating ip address.

Nginx Config

On Debian you can use the repositories to install but they only contain version 0.6.33. To install a later version you have to add the lenny backports packages

$ nano /etc/apt/sources.list

Add this line to the file deb http://www.backports.org/debian lenny-backports main

$ nano /etc/apt/preferences

Add the following to the file:

Package: nginx
Pin: release a=lenny-backports
Pin-Priority: 999

Issue the following commands to import the key from backports.org to verify packages and update your system’s package database:

wget -O - http://backports.org/debian/archive.key | apt-key add -
$ apt-get update


Now install with apt-get

apt-get install nginx

This is much easier than compiling from source.

Nginx conf and server files config

nginx.conf

user www-data;
worker_processes  4;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    access_log  /var/log/nginx/access.log;
    client_body_temp_path /var/lib/nginx/body 1 2;
    gzip_buffers 32 8k;
    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;
    tcp_nodelay        on;

    gzip  on;

  gzip_comp_level   6;
  gzip_http_version 1.0;
  gzip_min_length   0;
  gzip_types        text/html text/css image/x-icon
        application/x-javascript application/javascript text/javascript application/atom+xml application/xml ;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

Now you will need to set up your Nginx virtual hosting. I like to use the sites-enabled method with each v host sym linked to a file in the sites-available directory.

mkdir /etc/nginx/sites-available
$ mkdir /etc/nginx/sites-enabled
$ touch /etc/nginx/sites-available/yourservername.conf
$ touch /etc/nginx/sites-available/default.conf
$ ln -s /etc/nginx/sites-available /etc/nginx/sites-enabled
$ nano /etc/nginx/sites-enabled/default.conf


default.conf

Note:

The static cache settings in the following files will only work if the Nginx proxy cache integrator plugin is enabled.

proxy_cache_path  /var/lib/nginx/cache  levels=1:2   keys_zone=staticfilecache:180m  max_size=500m;
proxy_temp_path /var/lib/nginx/proxy;
proxy_connect_timeout 30;
proxy_read_timeout 120;
proxy_send_timeout 120;

#IMPORTANT - this sets the basic cache key that's used in the static file cache.
proxy_cache_key "$scheme://$host$request_uri";

upstream wordpressapache {
        #The upstream apache server. You can have many of these and weight them accordingly,
        #allowing nginx to function as a caching load balancer
        server 127.0.0.1:8080 weight=1 fail_timeout=120s;
}

Per WordPress site conf (For multi site you will only need one vhost)

server {
        #Only cache 200 responses, and for a default of 20 minutes.
        proxy_cache_valid 200 20m;

        #Listen to your public IP
        listen 80;

        #Probably not needed, as the proxy will pass back the host in "proxy_set_header"
        server_name www.yoursite.com yoursite.com;
        access_log /var/log/nginx/yoursite.proxied.log;

        # "combined" matches apache's concept of "combined". Neat.
        access_log  /var/log/apache2/nginx-access.log combined;
        # Set the real IP.
        proxy_set_header X-Real-IP  $remote_addr;

        # Set the hostname
        proxy_set_header Host $host;

        #Set the forwarded-for header.
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        location / {
                        # If logged in, don't cache.
                        if ($http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) {
                                set $do_not_cache 1;
                        }
                        proxy_cache_key "$scheme://$host$request_uri $do_not_cache";
                        proxy_cache staticfilecache;
                        proxy_pass http://wordpressapache;
        }

        location ~* wp\-.*\.php|wp\-admin {
                        # Don't static file cache admin-looking things.
                        proxy_pass http://wordpressapache;
        }

        location ~* \.(jpg|png|gif|jpeg|css|js|mp3|wav|swf|mov|doc|pdf|xls|ppt|docx|pptx|xlsx)$ {
                        # Cache static-looking files for 120 minutes, setting a 10 day expiry time in the HTTP header,
                        # whether logged in or not (may be too heavy-handed).
                        proxy_cache_valid 200 120m;
                        expires 864000;
                        proxy_pass http://wordpressapache;
                        proxy_cache staticfilecache;
        }

        location ~* \/[^\/]+\/(feed|\.xml)\/? {
 # Cache RSS looking feeds for 45 minutes unless logged in.
                        if ($http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_" ) {
                                set $do_not_cache 1;
                        }
                        proxy_cache_key "$scheme://$host$request_uri $do_not_cache";
                        proxy_cache_valid 200 45m;
                        proxy_cache staticfilecache;
                        proxy_pass http://wordpressapache;
        }

        location = /50x.html {
                root   /var/www/nginx-default;
        }

        # No access to .htaccess files.
        location ~ /\.ht {
                deny  all;
        }

        }

Self Hosted CDN conf

For your self hosted CDN conf you only need to set it up to serve static files without the proxy pass

server {

        proxy_cache_valid 200 20m;
        listen 80;
        server_name yourcdndomain.com;
        access_log   /srv/www/yourcdndomain.com/logs/access.log;
        root   /srv/www/yourcdndomain.com/public_html/;

 proxy_set_header X-Real-IP  $remote_addr;

      location ~* \.(jpg|png|gif|jpeg|css|js|mp3|wav|swf|mov|doc|pdf|xls|ppt|docx|pptx|xlsx)$ {
                                # Cache static-looking files for 120 minutes, setting a 10 day expiry time in the HTTP header,
                                # whether logged in or not (may be too heavy-handed).

                                proxy_cache_valid 200 120m;
                        expires 7776000;
                        proxy_cache staticfilecache;
                }

location = /50x.html {
                root   /var/www/nginx-default;
        }

 # No access to .htaccess files.
        location ~ /\.ht {
          deny  all;
        }

    }

Now start the servers

/etc/init.d/apache2 restart
$/etc/init.d/nginx start


The Benchmark Results

To test this setup and demonstrate the potential I ran Apache Bench on the home page of this site and set it to make 5000 requests at a concurrency level of 100. The results:

 Requests per second:    8242.83 [#/sec] (mean)

While running the benchmark I had htop open in another window and as you can see by the screen shot below, Nginx only opened 3 worker processes and barely made a blip on memory or cpu.

ab -n 5000 -c 100 http://wp-performance.com/


[root@xxxxxxxxx ~]#ab -n 5000 -c 100 http://wp-performance.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking wp-performance.com (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests

Server Software:        nginx/0.7.65
Server Hostname:        wp-performance.com
Server Port:            80

Document Path:          /
Document Length:        9348 bytes

Concurrency Level:      100
Time taken for tests:   0.607 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      49172298 bytes
HTML transferred:       46870872 bytes
Requests per second:    8242.83 [#/sec] (mean)
Time per request:       12.132 [ms] (mean)
Time per request:       0.121 [ms] (mean, across all concurrent requests)
Transfer rate:          79163.82 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.9      1       9
Processing:     3   11   1.8     11      61
Waiting:        1    5   2.1      5      54
Total:          5   12   1.7     12      61

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     12
  75%     13
  80%     13
  90%     14
  95%     15
  98%     15
  99%     16
 100%     61 (longest request)