Learn how to speed up your WordPress site using a reverse proxy cache called Varnish. In this tutorial I will guide you through setting up a caching server to significantly increase the amount of traffic your site can receive, while decreasing your web server’s work.
Why Caching is Needed with PHP
PHP is known as an interrupted language. In simple terms this means that it is not pre-compiled. Having to interrupt the code, compile into something the computer can understand, and then finally pushing the output to a user is considered computationally expensive. All of this work means your web server, such as Apache 2, for example, requires a lot more resources to serve content.
The affect is a signficant decrease in the amount of users you can serve without having to increase your CPU or RAM.
PHP has introduced a number of features that help tackle this issue. One of the earlier features was caching the byte code that the inturreptor generated when a page was requested. Rather than having to re-interpret the PHP file and convert it to byte code every time it is requested, PHP can simply save and reuse the existing byte code.
With the introduction of PHP 7 we saw a number of major changes that significantly increased PHP’s performance.
What is Varnish
Varnish is a reverse proxy cache. A reverse proxy simply means that all requests to your web server are funnelled through it before going to your web server, and all responses flow through the proxy back to the end user. This allows Varnish to monitor all traffic and store pre-compiled, static versions of your content.
Serving static contents is enormously faster and less resource hogging than having to compile code before serving it.
Installing Varnish and Configuring the Service
Varnish can be installed directly from Ubuntu’s official repositories. As long as you running at least 16.04, the version available will be 4. By default Varnish will run on port 8080. That may be find, as you may have your caching server sitting behind a load balancer listening on port 80. If this doesn’t match your configuration, and traffic flows directly to your server, you will need to adjust the settings to have Varnish listen on port 80.
Install Varnish
To install Varnish, run the following command.
sudo apt install varnish -y
Configuring Varnish to listen on Port 80 (optional)
To configure Varnish to listen on port 80, do the following:
Create a the directory for the Varnish service configuration.
# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;
Add a backend server – the Apache 2 web server hosting the site, for example.
# Default backend definition. Set this to point to your content server.
backend default {
.host = "10.0.0.10";
.port = "80";
.connect_timeout = 600s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 800;
}
Only allow cache purges from the localhost or our WordPress host.
# Only allow purging from specific IPs
acl purge {
"localhost";
"127.0.0.1";
"10.0.0.10";
}
Setting rules for requests received from an HTTP client (web browser). We need prevent caching things the are either sensitive (admin page access) or may break things (Google Analytics).
# This function is used when a request is send by a HTTP client (Browser)
sub vcl_recv {
# Normalize the header, remove the port (in case you're testing this on various TCP ports)
set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
# Allow purging from ACL
if (req.method == "PURGE") {
# If not allowed then a error 405 is returned
if (!client.ip ~ purge) {
return(synth(405, "This IP is not allowed to send PURGE requests."));
}
# If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
return (purge);
}
# Post requests will not be cached
if (req.http.Authorization || req.method == "POST") {
return (pass);
}
# --- WordPress specific configuration
# Did not cache the RSS feed
if (req.url ~ "/feed") {
return (pass);
}
# Blitz hack
if (req.url ~ "/mu-.*") {
return (pass);
}
# Did not cache the admin and login pages
if (req.url ~ "/wp-(login|admin)") {
return (pass);
}
# Remove the "has_js" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
# Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
# Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", "");
# Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^ *$") {
unset req.http.cookie;
}
# Cache the following files extensions
if (req.url ~ "\.(css|js|png|gif|jp(e)?g|swf|ico)") {
unset req.http.cookie;
}
# Normalize Accept-Encoding header and compression
# https://www.varnish-cache.org/docs/3.0/tutorial/vary.html
if (req.http.Accept-Encoding) {
# Do no compress compressed files...
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
unset req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
unset req.http.Accept-Encoding;
}
}
# Check the cookies for wordpress-specific items
if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~ "comment_") {
return (pass);
}
if (!req.http.cookie) {
unset req.http.cookie;
}
# --- End of WordPress specific configuration
# Do not cache HTTP authentication and HTTP Cookie
if (req.http.Authorization || req.http.Cookie) {
# Not cacheable by default
return (pass);
}
# Cache all others requests
return (hash);
}
Add vcl_pipe and vcl_pass directives.
sub vcl_pipe {
return (pipe);
}
sub vcl_pass {
return (fetch);
}
Map cacheable items to hash for quick lookup.
# The data on which the hashing will take place
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
# If the client supports compression, keep that in a different cache
if (req.http.Accept-Encoding) {
hash_data(req.http.Accept-Encoding);
}
return (lookup);
}
Strip information from responses sent from the backend servers. We may also choose to add new headers and other information here, too.
# This function is used when a request is sent by our backend (Nginx server)
sub vcl_backend_response {
# Remove some headers we never want to see
unset beresp.http.Server;
unset beresp.http.X-Powered-By;
# For static content strip all backend cookies
if (bereq.url ~ "\.(css|js|png|gif|jp(e?)g)|swf|ico") {
unset beresp.http.cookie;
}
# Only allow cookies to be set if we're in admin area
if (beresp.http.Set-Cookie && bereq.url !~ "^/wp-(login|admin)") {
unset beresp.http.Set-Cookie;
}
# don't cache response to posted requests or those with basic auth
if ( bereq.method == "POST" || bereq.http.Authorization ) {
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}
# don't cache search results
if ( bereq.url ~ "\?s=" ){
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}
# only cache status ok
if ( beresp.status != 200 ) {
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}
# A TTL of 24h
set beresp.ttl = 24h;
# Define the default grace period to serve cached content
set beresp.grace = 30s;
return (deliver);
}
Add additional information to the response header. For example, we can add whether the particular URL has been returned from cache or not. This helps with debugging from a client.
# The routine when we deliver the HTTP request to the user
# Last chance to modify headers that are sent to the client
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "cached";
} else {
set resp.http.x-Cache = "uncached";
}
# Remove some headers: PHP version
unset resp.http.X-Powered-By;
# Remove some headers: Apache version & OS
unset resp.http.Server;
# Remove some heanders: Varnish
unset resp.http.Via;
unset resp.http.X-Varnish;
return (deliver);
}
Additional options.
sub vcl_init {
return (ok);
}
sub vcl_fini {
return (ok);
}
Save your changes and exit the text editor.
Your configuration file should like the following example when done.
#
# This is an example VCL file for Varnish.
#
# It does not do anything by default, delegating control to the
# builtin VCL. The builtin VCL is called when there is no explicit
# return statement.
#
# See the VCL chapters in the Users Guide at https://www.varnish-cache.org/docs/
# and https://www.varnish-cache.org/trac/wiki/VCLExamples for more examples.
# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "10.0.0.10";
.port = "80";
.connect_timeout = 600s;
.first_byte_timeout = 600s;
.between_bytes_timeout = 600s;
.max_connections = 800;
}
# Only allow purging from specific IPs
acl purge {
"localhost";
"127.0.0.1";
"10.0.0.10";
}
# This function is used when a request is send by a HTTP client (Browser)
sub vcl_recv {
# Normalize the header, remove the port (in case you're testing this on various TCP ports)
set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");
# Allow purging from ACL
if (req.method == "PURGE") {
# If not allowed then a error 405 is returned
if (!client.ip ~ purge) {
return(synth(405, "This IP is not allowed to send PURGE requests."));
}
# If allowed, do a cache_lookup -> vlc_hit() or vlc_miss()
return (purge);
}
# Post requests will not be cached
if (req.http.Authorization || req.method == "POST") {
return (pass);
}
# --- WordPress specific configuration
# Did not cache the RSS feed
if (req.url ~ "/feed") {
return (pass);
}
# Blitz hack
if (req.url ~ "/mu-.*") {
return (pass);
}
# Did not cache the admin and login pages
if (req.url ~ "/wp-(login|admin)") {
return (pass);
}
# Remove the "has_js" cookie
set req.http.Cookie = regsuball(req.http.Cookie, "has_js=[^;]+(; )?", "");
# Remove any Google Analytics based cookies
set req.http.Cookie = regsuball(req.http.Cookie, "__utm.=[^;]+(; )?", "");
# Remove the Quant Capital cookies (added by some plugin, all __qca)
set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");
# Remove the wp-settings-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-1=[^;]+(; )?", "");
# Remove the wp-settings-time-1 cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wp-settings-time-1=[^;]+(; )?", "");
# Remove the wp test cookie
set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");
# Are there cookies left with only spaces or that are empty?
if (req.http.cookie ~ "^ *$") {
unset req.http.cookie;
}
# Cache the following files extensions
if (req.url ~ "\.(css|js|png|gif|jp(e)?g|swf|ico)") {
unset req.http.cookie;
}
# Normalize Accept-Encoding header and compression
# https://www.varnish-cache.org/docs/3.0/tutorial/vary.html
if (req.http.Accept-Encoding) {
# Do no compress compressed files...
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
unset req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
unset req.http.Accept-Encoding;
}
}
# Check the cookies for wordpress-specific items
if (req.http.Cookie ~ "wordpress_" || req.http.Cookie ~ "comment_") {
return (pass);
}
if (!req.http.cookie) {
unset req.http.cookie;
}
# --- End of WordPress specific configuration
# Do not cache HTTP authentication and HTTP Cookie
if (req.http.Authorization || req.http.Cookie) {
# Not cacheable by default
return (pass);
}
# Cache all others requests
return (hash);
}
sub vcl_pipe {
return (pipe);
}
sub vcl_pass {
return (fetch);
}
# The data on which the hashing will take place
sub vcl_hash {
hash_data(req.url);
if (req.http.host) {
hash_data(req.http.host);
} else {
hash_data(server.ip);
}
# If the client supports compression, keep that in a different cache
if (req.http.Accept-Encoding) {
hash_data(req.http.Accept-Encoding);
}
return (lookup);
}
# This function is used when a request is sent by our backend (Nginx server)
sub vcl_backend_response {
# Remove some headers we never want to see
unset beresp.http.Server;
unset beresp.http.X-Powered-By;
# For static content strip all backend cookies
if (bereq.url ~ "\.(css|js|png|gif|jp(e?)g)|swf|ico") {
unset beresp.http.cookie;
}
# Only allow cookies to be set if we're in admin area
if (beresp.http.Set-Cookie && bereq.url !~ "^/wp-(login|admin)") {
unset beresp.http.Set-Cookie;
}
# don't cache response to posted requests or those with basic auth
if ( bereq.method == "POST" || bereq.http.Authorization ) {
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}
# don't cache search results
if ( bereq.url ~ "\?s=" ){
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}
# only cache status ok
if ( beresp.status != 200 ) {
set beresp.uncacheable = true;
set beresp.ttl = 120s;
return (deliver);
}
# A TTL of 24h
set beresp.ttl = 24h;
# Define the default grace period to serve cached content
set beresp.grace = 30s;
return (deliver);
}
# The routine when we deliver the HTTP request to the user
# Last chance to modify headers that are sent to the client
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "cached";
} else {
set resp.http.x-Cache = "uncached";
}
# Remove some headers: PHP version
unset resp.http.X-Powered-By;
# Remove some headers: Apache version & OS
unset resp.http.Server;
# Remove some heanders: Varnish
unset resp.http.Via;
unset resp.http.X-Varnish;
return (deliver);
}
sub vcl_init {
return (ok);
}
sub vcl_fini {
return (ok);
}
Validate the Configuration
Before restarting Varnish to apply your new settings, a good practice to have is validating your configuration file’s syntax. If you were to restart the Varnish service with an invalid configuration file, you would bring your caching server offline. To save yourself some grief, test your changes every time.
To test our configuration file, run the following command.
varnishd -C -f /etc/varnish/default.vcl
Restarting Varnish to Apply Our Settings
We’ve created our configuration file and have tested it for syntax errors. We can now, finally, restart the service to apply our new settings.
Restart Varnish.
sudo systemctl restart varnish.service
Verify Varnish restarted without errors by viewing the status of the service.
