Varnish is a very fast caching mechanism for web applications but it is not always easy to install. In the latest version there are a number of changes pertinent to Debian servers.
The following assumes you are working as root. If not, then you need to add sudo to the front of all commands.
Installing Varnish on the server
At the command line:
apt-get install apt-transport-https
curl https://repo.varnish-cache.org/GPG-key.txt | apt-key add -
echo "deb https://repo.varnish-cache.org/debian/ jessie varnish-4.1"\
>> /etc/apt/sources.list.d/varnish-cache.list
apt-get update
apt-get install varnish
In Ansible:
- name: Get Varnish key
apt_key: url=https://repo.varnish-cache.org/ubuntu/GPG-key.txt state=present - name: Apt Update
apt: update_cache=yes - name: Install Varnish
apt: name=varnish state=present
The first step is to make Varnish the first call for all http requests. Http normally uses port 80 and so this is the port we will use. Before Debian 8 and Varnish 4.x custom configuration could be made in the file /etc/default/varnish by changing the value for -a :6081 to -a :80
DAEMON_OPTS="-a :6081 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,256m"
In Debian 8, however, configuration is done by systemd. To enable this create a new file in the directory at /etc/systemd/system/ called varnish.service, whose contents can vary depending on your setup, but the following will work on most installs:
[Unit]
Description=Varnish HTTP accelerator[Service]
Type=forking
LimitNOFILE=131072
LimitMEMLOCK=82000
ExecStartPre=/usr/sbin/varnishd -C -f /etc/varnish/default.vcl
ExecStart=/usr/sbin/varnishd -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,512m
ExecReload=/usr/share/varnish/reload-vcl[Install]
WantedBy=multi-user.target
In ansible we can use:
- name: Override Systemd service
template: src=varnish.service dest=/etc/systemd/system/varnish.service
The value -a :80 determines the port at which Varnish listens. This is the key item for this configuration.
The value -T localhost:6082 determines the port for varnish administration. We will revisit this when we configure Varnish for Drupal.
Varnish requires one other configuration file. This too has changed a lot since Varnish 3.x. There are a number of examples available on the net written expressly for Drupal. I got mine from https://github.com/NITEMAN/varnish-bites. Download a suitable Drupal default.vcl file to use on your server. There is one setting that needs to be changed in the Backend Definitions.
backend default {
.host = "127.0.0.1";
.port = "8000";
.max_connections = 100;
.connect_timeout = 60s;
.first_byte_timeout = 60s;
.between_bytes_timeout = 60s;
.probe = basic;
}
For this setup we should change the port to 8000. This means that Varnish will listen on port 80 and pass requests onto port 8000 which is the port we are going to choose from which to load the web pages.
The default.vcl file belongs in /etc/varnish. In Ansible we can use:
- name: Upload default.vcf
template: src=default.conf dest=/etc/varnish/default.vcl
Until you know a bit more about the configuration settings in default.vcl this should do the trick.
Now that we have Varnish ready to go we should restart it. At the command line:
systemctl daemon-reload
systemctl restart varnish.service
Or with Ansible:
- name: reload systemd
command: systemctl daemon-reload - name: restart varnish
command: systemctl restart varnish.service
On the command line you can check that Varnish is running correctly with:
systemctl status varnish.service -l
or
netstat -lp | grep varnish
You will get an error such as:
varnish.service - Varnish HTTP accelerator
Loaded: loaded (/etc/systemd/system/varnish.service; enabled)
Active: failed (Result: exit-code) since Sun 2016-07-03 17:33:29 NZST; 9min ago
Main PID: 10121 (code=exited, status=0/SUCCESS)
Jul 03 17:33:28 server2016 varnishd[13512]: .backend_error_func = VGC_function_vcl_backend_error,
Jul 03 17:33:28 server2016 varnishd[13512]: .init_func = VGC_function_vcl_init,
Jul 03 17:33:28 server2016 varnishd[13512]: .fini_func = VGC_function_vcl_fini,
Jul 03 17:33:28 server2016 varnishd[13512]: };
Jul 03 17:33:29 server2016 varnishd[13522]: bind(): Address already in use
Jul 03 17:33:29 server2016 varnishd[13522]: bind(): Address already in use
Jul 03 17:33:29 server2016 varnishd[13522]: Error: Failed to open (any) accept sockets.
Jul 03 17:33:29 server2016 systemd[1]: varnish.service: control process exited, code=exited status=2
Jul 03 17:33:29 server2016 systemd[1]: Failed to start Varnish HTTP accelerator.
Jul 03 17:33:29 server2016 systemd[1]: Unit varnish.service entered failed state.
This is because Apache is still bound to port 80. Now to change that.
Configuring Apache
When Apache is bound to port 80 it takes precedence over Varnish. In order to allow Varnish to serve incoming http requests, we will assign Apache to port 8000. in all apache.conf files:
- Change Listen 80 to 8000. This might be in your apache2.conf or in ports.conf or elsewhere.
- Note that in Debian 8 NameVirtualHost is deprecated so you can ignore it.
- Go through all vhost configs and change *:80 to *:8000 in each of them. If you are applying Varnish to a server then all vhosts need to lie behind it. You cannot leave any of them assigned to port 80 or Varnish won't work and will give the error above.
You should now have:
Listen 8000
And directives such as
<VirtualHost *:8000>
ServerName example.com
ServerAlias www.example.com # Index file and Document Root (where the public files are located)
DirectoryIndex index.php
DocumentRoot /var/www/html
</VirtualHost>
Upload those changes to the server and restart Apache. Now run the check:
netstat -lp | grep varnish
And you should get something like:
tcp 0 0 *:80 *:* LISTEN 17239/varnishd
tcp 0 0 server.vps.s:6082 *:* LISTEN 17238/varnishd
tcp6 0 0 [::]:80 [::]:* LISTEN 17239/varnishd
This tells us that Varnish is listening on port 80 and using port 6082 as desired. If Varnish does not work for you please check the Troubleshooting list at the end of this page.
Now that we have Varnish running on the server we should install the Drupal Varnish module. Go to the admin section at /admin/config/development/varnish. The Varnish Control Terminal should be 127.0.0.1:6082 for most servers. Since we have installed Varnish with a secret key, then you'll need to obtain this from /etc/vanish/secret. Make sure that Varnish 4.x is selected and when you submit the form it should show that Varnish is running.
Now check that Varnish is working on your website by using this handy widget.
Troubleshooting
If you get the error
varnish.service has more than one ExecStart setting, which is only allowed for Type=oneshot services. Refusing.
I found it difficult to get Varnish to read my varnish.service file and this problem was a time hog.
- It is likely that your script editor / IDE is adding extra characters or newlines to you custom varnish.service file and you need to use a simple editor like notepad with no extra characters. Try copying from the source file at /lib/systemd/system/varnish.service and making the customizations you require in TextWrangler (Mac) or Notepad (Windows).
- As shown in the Varnish documentation, it is sometimes wise to put an empty ExecStart declaration in place especially when you are repeatedly restarting Varnish. This has the effect of clearing the variable out before we add a new value.
Useful commands
Purge a domain
curl -X PURGE http://domain.org
Useful resources