How to Configure Varnish 4 for WordPress

Varnish 4 for Wordpress

Learn how to speed up your WordPress site using a reverse proxy cache called Varnish. In this tutorial I will guide you through setting up a caching server to significantly increase the amount of traffic your site can receive, while decreasing your web server’s work.

Why Caching is Needed with PHP

PHP is known as an interrupted language. In simple terms this means that it is not pre-compiled. Having to interrupt the code, compile into something the computer can understand, and then finally pushing the output to a user is considered computationally expensive. All of this work means your web server, such as Apache 2, for example, requires a lot more resources to serve content.

The affect is a signficant decrease in the amount of users you can serve without having to increase your CPU or RAM.

PHP has introduced a number of features that help tackle this issue. One of the earlier features was caching the byte code that the inturreptor generated when a page was requested. Rather than having to re-interpret the PHP file and convert it to byte code every time it is requested, PHP can simply save and reuse the existing byte code.

With the introduction of PHP 7 we saw a number of major changes that significantly increased PHP’s performance.

What is Varnish

Varnish is a reverse proxy cache.  A reverse proxy simply means that all requests to your web server are funnelled through it before going to your web server, and all responses flow through the proxy back to the end user. This allows Varnish to monitor all traffic and store pre-compiled, static versions of your content.

Serving static contents is enormously faster and less resource hogging than having to compile code before serving it.

Installing Varnish and Configuring the Service

Varnish can be installed directly from Ubuntu’s official repositories. As long as you running at least 16.04, the version available will be 4. By default Varnish will run on port 8080. That may be find, as you may have your caching server sitting behind a load balancer listening on port 80. If this doesn’t match your configuration, and traffic flows directly to your server, you will need to adjust the settings to have Varnish listen on port 80.

Install Varnish

To install Varnish, run the following command.

sudo apt install varnish -y

Configuring Varnish to listen on Port 80 (optional)

To configure Varnish to listen on port 80, do the following:

  1. Create a the directory for the Varnish service configuration.
    sudo mkdir /etc/systemd/system/varnish.service.d
  2. Create the new service file.
    sudo touch /etc/systemd/system/varnish.service.d/customexec.conf
  3. Open the file in a text editor, such as VI.
    sudo vi /etc/systemd/system/varnish.service.d/customexec.conf
  4. Add the following lines.
    [Service]
    ExecStart=
    ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m
  5. We are setting the cache to reside in RAM (-s malloc) and for it to occupy 256 MB of space. This is a good setting for a server with 1 GB of memory.

 

Varnish Config for WordPress

  1. Backup the existing vcl file.
    sudo mv /etc/varnish/default.vcl /etc/varnish/default.vcl.bacup
  2. Create a new VCL file.
    sudo touch /etc/varnish/default.vcl
  3. Open the file in a text editor.
    sudo vi /etc/varnish/default.vcl
  4. Set the configuration version to 4.0
    # Marker to tell the VCL compiler that this VCL has been adapted to the
    # new 4.0 format.
    vcl 4.0;
  5. Add a backend server – the Apache 2 web server hosting the site, for example.
    # Default backend definition. Set this to point to your content server.
    backend default {
        .host = "10.0.0.10";
        .port = "80";
        .connect_timeout = 600s;
        .first_byte_timeout = 600s;
        .between_bytes_timeout = 600s;
        .max_connections = 800;
    }
  6. Only allow cache purges from the localhost or our WordPress host.
    # Only allow purging from specific IPs
    acl purge {
        "localhost";
        "127.0.0.1";
        "10.0.0.10";
    }
  7. Setting rules for requests received from an HTTP client (web browser). We need prevent caching things the are either sensitive (admin page access) or may break things (Google Analytics).
    # This function is used when a request is send by a HTTP client (Browser) 
    sub vcl_recv {
            # Normalize the header, remove the port (in case you're testing this on various TCP ports)
            set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
    
            # Allow purging from ACL
            if (req.method == "PURGE") {
                    # If not allowed then a error 405 is returned
                    if (!client.ip ~ purge) {
                            return(synth(405, "This IP is not allowed to send PURGE requests."));
                    }
                    # If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
                    return (purge);
            }
    
            # Post requests will not be cached
            if (req.http.Authorization || req.method == "POST") {
                    return (pass);
            }
    
            # --- WordPress specific configuration
    
            # Did not cache the RSS feed
            if (req.url ~ "/feed") {
                    return (pass);
            }
    
            # Blitz hack
            if (req.url ~ "/mu-.*") {
                    return (pass);
            }
    
    
            # Did not cache the admin and login pages
            if (req.url ~ "/wp-(login|admin)") {
                    return (pass);
            }
    
    
            # Remove the "has_js" cookie
            set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
    
            # Remove any Google Analytics based cookies
            set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
    
            # Remove the Quant Capital cookies (added by some plugin, all __qca)
            set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
    
            # Remove the wp-settings-1 cookie
            set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
    
            # Remove the wp-settings-time-1 cookie
            set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", "");
    
            # Remove the wp test cookie
            set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");
    
            # Are there cookies left with only spaces or that are empty?
            if (req.http.cookie ~ "^ *$") {
                        unset req.http.cookie;
            }
    
            # Cache the following files extensions 
            if (req.url ~ "\.(css|js|png|gif|jp(e)?g|swf|ico)") {
                    unset req.http.cookie;
            }
    
            # Normalize Accept-Encoding header and compression
            # https://www.varnish-cache.org/docs/3.0/tutorial/vary.html
            if (req.http.Accept-Encoding) {
                    # Do no compress compressed files...
                    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
                                    unset req.http.Accept-Encoding;
                    } elsif (req.http.Accept-Encoding ~ "gzip") {
                            set req.http.Accept-Encoding = "gzip";
                    } elsif (req.http.Accept-Encoding ~ "deflate") {
                            set req.http.Accept-Encoding = "deflate";
                    } else {
                            unset req.http.Accept-Encoding;
                    }
            }
    
            # Check the cookies for wordpress-specific items
            if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~ "comment_") {
                    return (pass);
            }
            if (!req.http.cookie) {
                    unset req.http.cookie;
            }
    
            # --- End of WordPress specific configuration
    
            # Do not cache HTTP authentication and HTTP Cookie
            if (req.http.Authorization || req.http.Cookie) {
                    # Not cacheable by default
                    return (pass);
            }
    
            # Cache all others requests
            return (hash);
    }
  8. Add vcl_pipe and vcl_pass directives.
    sub vcl_pipe {
            return (pipe);
    }
    
    sub vcl_pass {
            return (fetch);
    }
  9. Map cacheable items to hash for quick lookup.
    # The data on which the hashing will take place
    sub vcl_hash {
            hash_data(req.url);
            if (req.http.host) {
                 hash_data(req.http.host);
            } else {
                 hash_data(server.ip);
            }
    
            # If the client supports compression, keep that in a different cache
            if (req.http.Accept-Encoding) {
                 hash_data(req.http.Accept-Encoding);
            }
    
            return (lookup);
    }
  10. Strip information from responses sent from the backend servers. We may also choose to add new headers and other information here, too.
    # This function is used when a request is sent by our backend (Nginx server)
    sub vcl_backend_response {
            # Remove some headers we never want to see
            unset beresp.http.Server;
            unset beresp.http.X-Powered-By;
    
            # For static content strip all backend cookies
            if (bereq.url ~ "\.(css|js|png|gif|jp(e?)g)|swf|ico") {
                    unset beresp.http.cookie;
            }
    
            # Only allow cookies to be set if we're in admin area
            if (beresp.http.Set-Cookie && bereq.url !~ "^/wp-(login|admin)") {
                    unset beresp.http.Set-Cookie;
            }
    
            # don't cache response to posted requests or those with basic auth
            if ( bereq.method == "POST" || bereq.http.Authorization ) {
                    set beresp.uncacheable = true;
                    set beresp.ttl = 120s;
                    return (deliver);
            }
    
            # don't cache search results
            if ( bereq.url ~ "\?s=" ){
                    set beresp.uncacheable = true;
                    set beresp.ttl = 120s;
                    return (deliver);
            }
    
            # only cache status ok
            if ( beresp.status != 200 ) {
                    set beresp.uncacheable = true;
                    set beresp.ttl = 120s;
                    return (deliver);
            }
    
            # A TTL of 24h
            set beresp.ttl = 24h;
            # Define the default grace period to serve cached content
            set beresp.grace = 30s;
    
            return (deliver);
    }
  11. Add additional information to the response header. For example, we can add whether the particular URL has been returned from cache or not. This helps with debugging from a client.
    # The routine when we deliver the HTTP request to the user
    # Last chance to modify headers that are sent to the client
    sub vcl_deliver {
            if (obj.hits > 0) {
                    set resp.http.X-Cache = "cached";
            } else {
                    set resp.http.x-Cache = "uncached";
            }
    
            # Remove some headers: PHP version
            unset resp.http.X-Powered-By;
    
            # Remove some headers: Apache version & OS
            unset resp.http.Server;
    
            # Remove some heanders: Varnish
            unset resp.http.Via;
            unset resp.http.X-Varnish;
    
            return (deliver);
    }
  12. Additional options.
    sub vcl_init {
            return (ok);
    }
    
    sub vcl_fini {
            return (ok);
    }
  13. Save your changes and exit the text editor.

Your configuration file should like the following example when done.

#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and https://www.varnish-cache.org/trac/wiki/VCLExamples for more examples.

# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;

# Default backend definition. Set this to point to your content server.
backend default {
    .host = "10.0.0.10";
    .port = "80";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
    .max_connections = 800;
}

# Only allow purging from specific IPs
acl purge {
    "localhost";
    "127.0.0.1";
    "10.0.0.10";
}

# This function is used when a request is send by a HTTP client (Browser) 
sub vcl_recv {
        # Normalize the header, remove the port (in case you're testing this on various TCP ports)
        set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");

        # Allow purging from ACL
        if (req.method == "PURGE") {
                # If not allowed then a error 405 is returned
                if (!client.ip ~ purge) {
                        return(synth(405, "This IP is not allowed to send PURGE requests."));
                }
                # If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
                return (purge);
        }

        # Post requests will not be cached
        if (req.http.Authorization || req.method == "POST") {
                return (pass);
        }

        # --- WordPress specific configuration

        # Did not cache the RSS feed
        if (req.url ~ "/feed") {
                return (pass);
        }

        # Blitz hack
        if (req.url ~ "/mu-.*") {
                return (pass);
        }


        # Did not cache the admin and login pages
        if (req.url ~ "/wp-(login|admin)") {
                return (pass);
        }


        # Remove the "has_js" cookie
        set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");

        # Remove any Google Analytics based cookies
        set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");

        # Remove the Quant Capital cookies (added by some plugin, all __qca)
        set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");

        # Remove the wp-settings-1 cookie
        set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");

        # Remove the wp-settings-time-1 cookie
        set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", "");

        # Remove the wp test cookie
        set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");

        # Are there cookies left with only spaces or that are empty?
        if (req.http.cookie ~ "^ *$") {
                    unset req.http.cookie;
        }

        # Cache the following files extensions 
        if (req.url ~ "\.(css|js|png|gif|jp(e)?g|swf|ico)") {
                unset req.http.cookie;
        }

        # Normalize Accept-Encoding header and compression
        # https://www.varnish-cache.org/docs/3.0/tutorial/vary.html
        if (req.http.Accept-Encoding) {
                # Do no compress compressed files...
                if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
                                unset req.http.Accept-Encoding;
                } elsif (req.http.Accept-Encoding ~ "gzip") {
                        set req.http.Accept-Encoding = "gzip";
                } elsif (req.http.Accept-Encoding ~ "deflate") {
                        set req.http.Accept-Encoding = "deflate";
                } else {
                        unset req.http.Accept-Encoding;
                }
        }

        # Check the cookies for wordpress-specific items
        if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~ "comment_") {
                return (pass);
        }
        if (!req.http.cookie) {
                unset req.http.cookie;
        }

        # --- End of WordPress specific configuration

        # Do not cache HTTP authentication and HTTP Cookie
        if (req.http.Authorization || req.http.Cookie) {
                # Not cacheable by default
                return (pass);
        }

        # Cache all others requests
        return (hash);
}

sub vcl_pipe {
        return (pipe);
}

sub vcl_pass {
        return (fetch);
}

# The data on which the hashing will take place
sub vcl_hash {
        hash_data(req.url);
        if (req.http.host) {
             hash_data(req.http.host);
        } else {
             hash_data(server.ip);
        }

        # If the client supports compression, keep that in a different cache
        if (req.http.Accept-Encoding) {
             hash_data(req.http.Accept-Encoding);
        }

        return (lookup);
}


# This function is used when a request is sent by our backend (Nginx server)
sub vcl_backend_response {
        # Remove some headers we never want to see
        unset beresp.http.Server;
        unset beresp.http.X-Powered-By;

        # For static content strip all backend cookies
        if (bereq.url ~ "\.(css|js|png|gif|jp(e?)g)|swf|ico") {
                unset beresp.http.cookie;
        }

        # Only allow cookies to be set if we're in admin area
        if (beresp.http.Set-Cookie && bereq.url !~ "^/wp-(login|admin)") {
                unset beresp.http.Set-Cookie;
        }

        # don't cache response to posted requests or those with basic auth
        if ( bereq.method == "POST" || bereq.http.Authorization ) {
                set beresp.uncacheable = true;
                set beresp.ttl = 120s;
                return (deliver);
        }

        # don't cache search results
        if ( bereq.url ~ "\?s=" ){
                set beresp.uncacheable = true;
                set beresp.ttl = 120s;
                return (deliver);
        }

        # only cache status ok
        if ( beresp.status != 200 ) {
                set beresp.uncacheable = true;
                set beresp.ttl = 120s;
                return (deliver);
        }

        # A TTL of 24h
        set beresp.ttl = 24h;
        # Define the default grace period to serve cached content
        set beresp.grace = 30s;

        return (deliver);
}

# The routine when we deliver the HTTP request to the user
# Last chance to modify headers that are sent to the client
sub vcl_deliver {
        if (obj.hits > 0) {
                set resp.http.X-Cache = "cached";
        } else {
                set resp.http.x-Cache = "uncached";
        }

        # Remove some headers: PHP version
        unset resp.http.X-Powered-By;

        # Remove some headers: Apache version & OS
        unset resp.http.Server;

        # Remove some heanders: Varnish
        unset resp.http.Via;
        unset resp.http.X-Varnish;

        return (deliver);
}

sub vcl_init {
        return (ok);
}

sub vcl_fini {
        return (ok);
}

 

Validate the Configuration

Before restarting Varnish to apply your new settings, a good practice to have is validating your configuration file’s syntax. If you were to restart the Varnish service with an invalid configuration file, you would bring your caching server offline. To save yourself some grief, test your changes every time.

To test our configuration file, run the following command.

varnishd -C -f /etc/varnish/default.vcl

 

Restarting Varnish to Apply Our Settings

We’ve created our configuration file and have tested it for syntax errors. We can now, finally, restart the service to apply our new settings.

  1. Restart Varnish.
    sudo systemctl restart varnish.service
  2. Verify Varnish restarted without errors by viewing the status of the service.
    sudo systemctl status varnish.service