WordPress on Apache 2.4 Varnished CHROOTED PHP-FPM Virtual Host

Inspired by some of the talks at WordCamp Melbourne I decided to improve the configuration of my WordPress server. This tutorial works through how I setup Apache 2.4, PHP-FPM, the Alternative PHP Cache, chrooted virtual hosts and a Varnish Cache web accelerator. I primarily did this for the challenge however, anecdotal evidence suggests, this loads pages over HTTP approximately in half the time of the old Apache 2.2 mod_php stack. Please note this benchmark is NOT meaningful or calibrated or even remotely suggestive of any performance difference.

Updated 31st July 2013: This tutorial was originally written for Apache 2.4.4 but also works for Apache 2.4.6. If you’re upgrading PHP5-FPM you will also need the ppa:ondrej/systemd PPA (add-apt-repository -y ppa:ondrej/systemd) – otherwise PHP5-FPM will NOT be upgraded.

Updated 27th November 2013: Apache 2.4.7 has been released which requires APR branch 1.5.x. The build_apache.sh script has been updated. The only other change of note is that the libnss3-dev package is now included in the dependency install step.

Updated 31st January 2014: This tutorial also works for PHP 5.5 however the APC opcode cache has been replaced with ZendOptimizer+ – this means certain WordPress plugins (such as W3 Total Cache) will NOT work. When using PHP 5.5 the section on configuring APC should be skipped.

Updated 6th May 2014: Apache 2.4.9 has been released and the build_apache.sh script has been updated to use this. This will be the final update for this script as v2.4.7 is included in Ubuntu 14.04 LTS (Trusty Tahr). Please recompile Apache if you are using this guide! Updating the OpenSSL library won’t fix Heartbleed – you need to recompile.

This tutorial covers Ubuntu Server 12.04 LTS. Future versions of Ubuntu and some other Debian distributions will probably work with only minor modifications however other non-Debian based distributions will need to make some significant changes. NOTE: From Ubuntu Saucy Salamander 13.10 onwards Apache 2.4 is included in the release packages.

DISCLAIMER: This works for me for my server setup. Don’t just copy and paste these commands / configuration without understanding what they do or how they may affect your server and security arrangements. I am not responsible for anything you do as a result of following this tutorial.

I made a range of helper bash scripts during this task, these are available from my GitHub repository for this project https://github.com/andrewbevitt/apachevarnishchroot. I’ve tried to document the scripts and their usage in this tutorial but as I said in the disclaimer above – you shouldn’t assume they will work for you.

This is a pretty long tutorial give yourself plenty of time to work through it.

Part 1: PHP 5.4 / 5.5

This is safe to install even if you already have PHP 5.3 installed.

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:ondrej/php5
sudo add-apt-repository ppa:ondrej/systemd
sudo apt-get update
sudo apt-get upgrade

Depending on what you already have installed you may also need to:

sudo apt-get dist-upgrade
sudo apt-get install php5 php-apc

If you have made any changes to you php.ini files you will be prompted about overwriting or keeping your changes. If you choose to overwrite the old file is saved to /path/to/php.ini.ucf-old so you can always revert back later if needed. You should take this opportunity to review your PHP settings and make sure they match the production recommendations. I change the following:

short_tag_open = Off
upload_max_filesize = 16M
session.name = PICK_A_PHP_SESSION_NAME

You may now have a package libt1-5 which can be removed:

sudo apt-get autoremove

NOTE: APC has been replaced with ZendOptimizer+ in PHP 5.5 – skip this if using 5.5: You may have noticed above that the php-apc package was installed (if it wasn’t already). The Alternative PHP Cache (APC) is a PHP opcode cache, which I won’t explain here, but you almost certainly want to use it for PHP powered sites (e.g. WordPress). You will need to configure APC in the file /etc/php5/mods-available/apc.ini – here is my configuration but you should read the documentation and decide what works for you:

extension=apc.so
apc.enabled=1
apc.shm_size=64M
apc.ttl=3600
apc.user_ttl=7200
apc.gc_ttl=3600

If you have Apache 2.2 installed on this server you should now check that PHP scripts work with PHP 5.4 and APC. The most sensible way to do this is by creating a file phptest.php in your Document Root folder containing the code:

<?php phpinfo(); ?>

Part 2: Setting up your build environment

This tutorial compiles Apache 2.4 from source because there are no packages available yet. There are a few PPA’s but none had what I wanted. If one of the PPA’s meets your requirements then by all means use that instead. If you do use a PPA keep in mind the shell scripts in my git repository are hard coded to assume you followed this tutorial and it’s directory layout.

sudo apt-get install git libnss3-dev
sudo apt-get build-dep apache2

From https://httpd.apache.org/docs/current/install.html we need to do the following:

  1. Build APR and APR-Util with Apache
  2. Check PCRE is installed: which pcre-config
  3. Check you have sufficient disk space: df -h
  4. Have Perl 5 installed – should be already

If you’re using my git repository scripts then you can run: ./build_apache.sh.

Here are the steps in case you want to do it by hand.

First you need to get the Apache and APR code from the Apache git repository:

git clone --branch 2.4.x https://github.com/apache/httpd httpd-2.4.x
cd httpd-2.4.x
git clone --branch 1.4.x https://github.com/apache/apr srclib/apr
git clone --branch 1.5.x https://github.com/apache/apr-util srclib/apr-util
./buildconf

Now we need to configure the Apache 2.4 build, documentation is here: https://httpd.apache.org/docs/current/programs/configure.html. You’ll notice that I have statically linked a few modules into the Apache 2.4 executable – I did this because I will always be using those modules but it doesn’t have to be that way.

CFLAGS="-O2 -pipe -fomit-frame-pointer" \
./configure --prefix=/opt/apache24 --enable-nonportable-atomics=yes --enable-pie --enable-mods-shared=all --enable-mods-static='alias authz_core authz_host log_config proxy proxy_fcgi proxy_http rewrite ssl unixd' --enable-auth-digest --enable-so --disable-include --enable-deflate --enable-http --enable-expires --enable-headers --disable-lua --disable-luajit --enable-mime-magic --enable-proxy --disable-proxy-connect --disable-proxy-ftp --enable-proxy-http --enable-proxy-fcgi --disable-proxy-scgi --disable-proxy-fdpass --disable-proxy-ajp --enable-proxy-balancer --disable-proxy-express --enable-slotmem-shm --enable-ssl --disable-autoindex --enable-negotiation --enable-dir --enable-alias --enable-rewrite --enable-v4-mapped --with-mpm=event --with-included-apr --with-ldap --with-crypto

If the configure process succeeds then you can move onto compiling. If you get an WARNING or ERROR messages then you really should investigate those before moving on – they will probably stop something working if you don’t. Once you’re ready to compile and install:

make
sudo make install
sudo chown -T root:root /opt/apache24

The compile process will take a few minutes (about 5 minutes on a Linode 1024MB).

Part 3: Configuring Apache 2.4

If you followed the above correctly you should now have Apache 2.4 installed under /opt/apache24 lets do a few little tests to make sure it’s working properly:

  1. Edit /opt/apache24/conf/http.conf:
    -Listen 80
    +Listen 10080
  2. Start server: /opt/apache24/bin/apachectl start
  3. Test it’s working: curl -v -X GET http://localhost:10080/ – should get:
    * About to connect() to localhost port 10080 (#0)
    * Trying 127.0.0.1... connected
    > GET / HTTP/1.1
    > User-Agent: curl/7.22.0
    > Host: localhost:10080
    > Accept: */*
    >
    < HTTP/1.1 200 OK
    < Date: ...
    < Server: Apache/2.4.5-dev (Unix) OpenSSL/1.0.1
    < Last-Modified: ...
    < ETag: ...
    < Accept-Ranges: bytes
    < Content-Length: 45
    < Content-Type: text/html
    <
    <html><body><h1>It works!</h1></body>
    * Connection #0 to host localhost left intact
    * Closing connection #0
  4. /opt/apache24/bin/apachectl stop
  5. Check the log files in /opt/apache24/logs

Now for the fun part… configuring Apache. You’re probably used to the Ubuntu config layout – it doesn’t work like that with this custom build. But I’ve attempted to make it quasi-modular. My configuration uses the following files:

  • conf/httpd.conf – system global config
  • conf/modules.d/modules.load – which modules to load
  • conf/modules.d/*.conf – specific config for each module (should wrap IfModule on each)
  • conf/ports.conf – which TCP ports to listen for HTTP requests on
  • conf/vhosts.d/*.conf – virtual host configuration files (symlinks)
  • bin/envvars – environment variables for startup

You can find copies of my default configuration files in the git repository under install/opt/apache24/ to install these default files (this will overwrite any files you already have):

cp -f install/opt/apache24/bin/envvars /opt/apache24/bin
cp -f install/opt/apache24/conf/*.conf /opt/apache24/conf
cp -fr install/opt/apache24/conf/modules.d /opt/apache24/conf
cp -fr install/opt/apache24/conf/vhosts.d /opt/apache24/conf
cp -fr install/srv/www/default /srv/www/

A few notes:

  1. If you use the above you should change the ServerAdmin in /opt/apache24/conf/vhosts.d/000_default.conf.
  2. The NameVirtualHost directive is deprecated in Apache 2.4 – it is assumed by default.
  3. I have provided some mod_expires and mod_headers configuration directives which are designed for use with the Varnish Cache configuration down the page however these should work even if you’re not using Varnish. The configuration files are in:

    /opt/apache24/conf/modules.d/expires.conf
    /opt/apache24/conf/modules.d/headers.conf

Presumably you want to have an initscript for Apache 2.4:

mkdir -p /var/log/apache24 /var/lock/apache24 /var/run/apache24 /var/cache/apache24
chown www-data:www-data /var/cache/apache24
chown www-data:root /var/lock/apache24 # yes root
cp install/etc/init.d/apache24 /etc/init.d/apache24
chown root:root /etc/init.d/apache24
chmod 755 /etc/init.d/apache24
cp install/etc/default/apache24 /etc/default/apache24
chown root:root /etc/default/apache24
chmod 644 /etc/default/apache24

You may also want to use logrotate and logwatch:

cp install/etc/logrotate.d/apache24 /etc/logrotate.d/apache24
chown root:root /etc/logrotate.d/apache24
chmod 644 /etc/logrotate.d/apache24
cp install/etc/logwatch/conf/logfiles/* /etc/logwatch/conf/logfiles/*
chown root:root /etc/logwatch/conf/logfiles/*

That’s about it – now test Apache 2.4 works!

sudo service apache24 start
curl -v -k -X http://localhost/helloworld.html
sudo service apache24 stop

Screenshot of browser

Browser output of the helloworld.html file.

NOTE: The helloworld.html file was installed to above with the Apache configuration files; if you didn’t copy it into /srv/www/default/ then you’ll need to create it first. The log files are now in /var/log/apache24/.

Finally you’ll want Apache 2.4 to start on boot:

sudo update-rc.d apache2 disable
sudo update-rc.d apache24 defaults 91 09

Part 4: Configure PHP-FPM

sudo apt-get install php5-fpm

Once again I’ve provided some default configuration files:

mv /etc/php5/fpm/pool.d/www.conf /etc/php5/fpm/pool.d/www.conf.orig
mv /etc/php5/fpm/php-fpm.conf /etc/php5/fpm/php-fpm.conf.orig
cp install/etc/php5/fpm/php-fpm.conf /etc/php5/fpm
cp install/etc/php5/fpm/pool.d/default.conf /etc/php5/fpm/pool.d

My default configuration files use the ondemand process manager for the default virtual host because it shouldn’t be accessed very often. If you disable PHP for the default virtual host you will need to change the catch all RewriteRule that redirects all requests to /helloworld.php.

Restart the PHP-FPM process and check /var/log/php5-fpm.log:

sudo service php5-fpm restart

Now you will want to make sure the PHP-FPM service is working. As part of the default virtual host we installed /srv/www/default/public/helloworld.php which can be accessed using the default virtual host care of the configuration directive:

ProxyPassMatch ^/helloworld\.php$ fcgi://127.0.0.1:9000/public/helloworld.php

Screenshot of browser

Browser output for the helloworld.php script.

Make sure you check the log files even if the script works.

Part 5: Benchmarking

I used the following command to do the benchmarking. Please note using localhost to do benchmarking does NOT give any sense of the real world performance. My purpose with this test was to see what the new setup would work like when compared under identical conditions. As it turns out because I’m using different Apache MPM’s the tests are sort of like comparing apples and oranges (i.e. they don’t mean much).

/opt/apache24/bin/ab -n 10000 -c 20 -k http://localhost/helloworld.php

Apache 2.2 with mod_php

Document Path:          /helloworld.php
Document Length:        60768 bytes

Concurrency Level:      20
Time taken for tests:   3.350 seconds
Complete requests:      10000
Failed requests:        1029
   (Connect: 0, Receive: 0, Length: 1029, Exceptions: 0)
Write errors:           0
Keep-Alive requests:    0
Total transferred:      609608866 bytes
HTML transferred:       607678866 bytes
Requests per second:    2985.45 [#/sec] (mean)
Time per request:       6.699 [ms] (mean)
Time per request:       0.335 [ms] (mean, across all concurrent requests)
Transfer rate:          177730.14 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       3
Processing:     2    6   1.3      6      26
Waiting:        1    6   1.3      6      26
Total:          2    7   1.2      7      26

Percentage of the requests served within a certain time (ms)
  50%      7
  66%      7
  75%      7
  80%      8
  90%      8
  95%      9
  98%      9
  99%     10
 100%     26 (longest request)

Apache 2.4 with PHP-FPM

Document Path:          /helloworld.php
Document Length:        58154 bytes

Concurrency Level:      20
Time taken for tests:   5.034 seconds
Complete requests:      10000
Failed requests:        9902
   (Connect: 0, Receive: 0, Length: 9902, Exceptions: 0)
Write errors:           0
Keep-Alive requests:    0
Total transferred:      583198809 bytes
HTML transferred:       581558809 bytes
Requests per second:    1987.48 [#/sec] (mean)
Time per request:       10.068 [ms] (mean)
Time per request:       0.503 [ms] (mean, across all concurrent requests)
Transfer rate:          113135.86 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:     4   10   1.6     10      20
Waiting:        3    9   1.6      9      20
Total:          6   10   1.6     10      21

Percentage of the requests served within a certain time (ms)
  50%     10
  66%     10
  75%     11
  80%     11
  90%     12
  95%     13
  98%     14
  99%     16
 100%     21 (longest request)

You will probably notice this looks like Apache 2.4 was SLOWER than Apache 2.2. This is actually due to the configuration that I had in place during testing. Apache 2.2 was using the prefork MPM and Apache 2.4 was using the Event MPM with the ondemand PHP-FPM process manager. These work in two different ways. When I changed the PHP-FPM process manager to use the dynamic process manager with the same number of minimum spare servers as the Apache 2.2 prefork processes the results change to:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1   0.3      1       2
Processing:     1    2   0.4      2       9
Waiting:        1    2   0.3      2       8
Total:          1    3   0.4      3       9

Which is much better the mean is approximately half of Apache 2.2 with mod_php even allowing for the standard deviation there is around a 25% speed increase. I should point out this is for helloworld.php then HTML file helloworld.html was always faster no matter what configuration.

If you want to write your own virtual host config files you need:

ProxyPassMatch ^/(.*\.php)$ fcgi://127.0.0.1:PORT_NUMBER/path/to/your/documentroot/$1

WARNING: Sending all PHP requests to the PHP-FPM process like this could allow arbitrary code execution if a user uploads a PHP script into a path under /path/to/your/documentroot. There are ways to mitigate this problem see the Appendix for my WordPress solution. Do NOT just leave this as is!

Part 6: CHROOT

Assuming you have multiple sites hosted on your server you might want to separate the user accounts which control those sites. You don’t have to do this but I wanted to make sure there was some separation of user accounts for a range of reasons. One example is a user who had a PHP script which scanned the file system. A CHROOT can be used to provide this separation: it “jails” the process to a new root directory.

When you setup a PHP-FPM CHROOT it changes where the PHP process thinks the / (i.e. root) directory is located. This means the ProxyPassMatch directive needs to be changed so that the path is relative to the CHROOT not the true root. For example:

/etc/php5/fpm/pool.d/default.conf:
prefix = /srv/www/default
chroot = $prefix
/opt/apache24/conf/vhost.d/000_default.conf:
DocumentRoot /srv/www/default/public
# NOTE: This assumes CHROOT is set to /srv/www/default ($prefix in FPM pool)
ProxyPassMatch ^/helloworld\.php$ fcgi://127.0.0.1:9000/public/helloworld.php

The alternate approach which can be found online for this is to create a symlink so that the perceived full path does in fact exist. I decided changing the Apache configuration files was more suitable because I had access to them. This is more efficient as the symlink stat call is not required. However the PHP global $_SERVER["DOCUMENT_ROOT"] does NOT get updated so if scripts rely on this they will break unless you create the symlink. The shell script in my git repository creates the symlink automatically to resolve this issue – just in case you’re wondering.

If you are happy to run your PHP scripts as the www-data user (or whatever you have Apache configured to run as) then you can stop here. The PHP scripts are CHROOT’d by PHP-FPM and that is all you need to do. However if you have multiple users and you want to separate those users (e.g. they need FTP or SHELL access) then you need to create a real CHROOT with system libraries etc. Doing this manually is VERY complicated so I have provided the script usersite.sh to do it for you. The script is derived from http://www.fuschlberger.net/programs/ssh-scp-sftp-chroot-jail/ however I have made some significant changes:

  • Stripped out the other Linux distros: is Ubuntu specific!
  • Translated into BASH syntax
  • Added the following programs: php, curl, unzip, tar, false
  • Downloads and installs mini_sendmail so php mail() works
  • Includes ImageMagick if required for a particular user
  • Copies /etc/localtime, /etc/timezone and /usr/share/zoneinfo for mktime()
  • Creates a skeleton /etc/hosts, /etc/resolv.conf and /etc/nsswitch.conf
  • Uses CHROOT root as system home but /home when CHROOT’d
  • Uses /etc/sudoers.d/ files for sudo access to chroot command
  • Does NOT create a group for the new user

WARNING: Please understand a CHROOT can be very insecure. It is possible for users to get out of their CHROOT with very simple configuration mistakes. You should NOT rely on the CHROOT to do anything other than change the root directory of the PHP processor. If you really need to allow FTP, SFTP, SCP or even SHELL access to your CHROOT make sure you know what you’re doing!

The usersite.sh script copies libraries and files into the CHROOT which means they remain stable after creating the CHROOT. This has a small overhead in storage space per user: approximately 45M depending on ImageMagick and your architecture. It also means that the CHROOT does NOT get updated when you install security updates for the system. This could compromise your server security. I have provided a separate script update_chroot.sh /path/to/chroot which will update the CHROOT; this should be run each time you install security updates.

The CHROOT creation and updates work by creating a small file /path/to/chroot/.isvhc which contains a list of the installed applications and extra system libraries. This file is read-only and accessible by the root user only. If you delete this file the CHROOT can not be updated using the script.

Here is the directory structure that my scripts assume:

/srv/www - root path for all hosting
  /default/public - document root for "default" unknown virtual host
  /$USERNAME - user home directory (where php is chrooted to)
    home - the users home dir when inside the chroot
    etc/apache - user apache configuration files 
    var/log - log files 
    var/www/site1.example.com - document root
    var/www/site2.example.com - document root
    tmp - temp folder for sessions etc..
    var/www - symlink to maintain DOCUMENT_ROOT validity

You may recall I said previously that /opt/apache24/conf/vhosts.d/*.conf are symlinks. The symlinks point to files in the users Apache configuration directory. Similarly the PHP-FPM pool configuration file points to the php-fpm.conf file in the users Apache configuration directory.

User accounts are assigned the following script as their SHELL:

#!/bin/bash
CHROOT_DIR=$(grep -e "^$USER:" /etc/passwd | awk '{ split($0,x,":"); print x[6] }')
/usr/bin/sudo /usr/sbin/chroot ${CHROOT_DIR} /bin/su -c /bin/bash -l $USER "[email protected]"

This script is installed as /usr/local/bin/vhost-shell when the first user is created using the script. Do NOT put this script into /etc/shells; if you do the user will be able to break out of the CHROOT. This may mean FTP servers will fail because the user SHELL is invalid – see the Appendix for how to fix this.

How to use the usersite.sh script

Create a new user:
  ./usersite create_user USERNAME FPM_PORT ADMIN_EMAIL
  ./usersite create_user mywebstore 10000 [email protected]
  The email address is the server admin address and php mail() from address

Create a new site (and user if user account does not exist)
  ./usersite create_site USERNAME FPM_PORT DOMAIN ADMIN_EMAIL SSL=Y|N ALIAS ALIAS ALIAS...
  ./usersite create_site mywebstore 10000 my.web.store.com [email protected] Y www.my.web.store.com another.alias.store.com
  If the user already exists use . for the FPM_PORT

SSL and aliases are not required (SSL defaults to N):
  ./usersite create_site mywebstore 10000 my.web.store.com [email protected]
  You can use . for the port if the user already exists

Enable a users site:
  ./usersite enable USERNAME DOMAIN
  ./usersite enable mywebstore my.web.store.com

Disable a site:
  ./usersite disable USERNAME DOMAIN
  ./usersite disable mywebstore my.web.store.com

Disable all users sites:
  ./usersite disable USERNAME
  ./usersite disable mywebstore

Remove a users site:
  ./usersite remove_site USERNAME DOMAIN
  ./usersite remove_site mywebstore my.web.store.com
  This deletes the files for the site!

Remove all sites for a user:
  ./usersite remove_site USERNAME
  This deletes all site files for all sites!

Remove a user from the system:
  ./usersite remove_user USERNAME
  This deletes all files and the user account!

Note that the default virtual host config files that I have provided use cronolog to split the Apache 2.4 log file up into individual files per user. I use cronolog so that Apache doesn’t reach the system file handle limit. You will need to install cronolog for this to work:

sudo apt-get install cronolog

Part 7: Varnish Cache

You can skip this if you want just change /opt/apache24/conf/ports.conf to listen on all interfaces.

sudo apt-get install varnish

In my configuration Apache 2.4 listens to the localhost loopback interface on port 80. This gets around Apache redirect issues when listening on another port. However it also means that Varnish Cache must only listen for HTTP requests on public interface IP addresses.

You can get your server IP addresses with the command:

ip -o addr | grep global | awk '!/^[0-9]*: ?lo|link\/ether/ {gsub("/", " "); print $2" "$4}'

You should edit /etc/default/varnish to change the IP’s:

    START=yes
    NFILES=131072
    MEMLOCK=82000
    VARNISH_IPV4=YOUR_IPV4_ADDRESS
    VARNISH_IPV6=YOUR_IPV6_ADDRESS
    VARNISH_PORT=80
    CACHE_SIZE_MB=128
    DAEMON_OPTS="-a ${VARNISH_IPV4}:${VARNISH_PORT},[${VARNISH_IPV6}]:$VARNISH_PORT \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,${CACHE_SIZE_MB}m"

Remove the IPV6 lines if not required. Note that IPV6 addresses should be surrounded by [].

And then simply change the default backend in /etc/varnish/default.vcl:

    backend default {
        .host = "127.0.0.1";
        .port = "80";
    }

For a really basic Varnish Cache setup that’s all that you need to do (not really but let’s pretend for a moment).

sudo service varnish restart
sudo varnishlog

What normally happens now is that remote client IP (i.e. the browsers IP) in your Apache logs will be 127.0.0.1 because the request is actually coming from Varnish. This also means that $_SERVER["REMOTE_ADDR"]=="127.0.0.1" which is not ideal. Fortunately Apache 2.4 includes mod_remoteip http://httpd.apache.org/docs/current/mod/mod_remoteip.html which enables the %h parameter in Apache’s LogFormat directives. The %h parameter is replaced by the true client IP address. I’ve enabled and setup mod_remoteip in my default configuration files so you don’t need to do anything. Check out /opt/conf/apache24/modules.d/remoteip.conf for more details.

Part 8: WordPress

WordPress is sort of written for Apache. Yes it can run on nginx and lighttpd and some others but it works on Apache out of the box so that’s what I’m writing this for Apache 2.4. But presumably if you’ve read this far you have already decided to use Apache 2.4 anyway.

You can simply extract the WordPress archive into a virtual host document root and then use chown / chmod to setup appropriate user permissions. However the usersite.sh script provided in my git repository provides a WordPress automated installation if you’d prefer to use that. Note that the script will install to the site document root and will automatically create a secure database password for the user/site. The password will be printed to the console and also added to wp-config.php automatically.

./usersite.sh wordpress USERNAME DOMAIN <dbname> <dbprefix> [<dbhost>]

NOTE: If you are using a CHROOT’d PHP either from PHP-FPM or a complete OS CHROOT the MySQL UNIX socket file will NOT be available to the PHP interpreter. This means you MUST use the localhost IP address as the database hostname (i.e. define( 'DB_HOST', '127.0.0.1' );). If you use the ‘localhost’ string the PHP interpreter tries to find the UNIX socket file and therefore won’t be able to connect to the database.

There are a few things you should do to lock down WordPress:

  • Do NOT use ‘admin’ as your username.
  • Use a strong password – something that is really long.
  • Use Fail2Ban to block multiple failed login attempts.
  • Install a login failure limit plugin (personally I don’t but you can).

Recall the ProxyPassMatch directive in the Apache configuration:

ProxyPassMatch ^/(.*\.php)$ fcgi://127.0.0.1:PORT_NUMBER/path/to/your/documentroot/$1

This will work out the box for WordPress but as I mentioned above it will allow arbitrary code execution if the user can upload PHP files to your document root. In the Appendix I have provided a solution to prevent PHP execution in the WordPress uploads folder. You really should consider using this or implementing your own solution.

Part 9: Varnish + WordPress

Unfortunately it is not as simple as the above makes it seem. You probably guessed that from the above. Basically Varnish Cache will NOT cache any request where a cookie is set and it will not cache any request that uses HTTP GET parameters (e.g. ?XYZ=SOME_VALUE). WordPress uses both cookies and GET parameters prolifically. In case you’re wondering: I tried out the default configuration to see what the cache hit/miss ratio would be. Here are the varnishstat numbers from 5 hours:

           9         0.00         0.00 cache_hit - Cache hits
          13         0.00         0.00 cache_hitpass - Cache hits for pass
        1429         0.00         0.08 cache_miss - Cache misses

… hopefully you can tell that this is not good.

I’m not going to try and explain how to configure Varnish. That’s been done. Instead there is a varnish config file provided in my git repository that you can use. This config file is designed to work with all the other settings in this tutorial so if you’ve changed some details make sure you change them here too. Just remember that Varnish caches content in RAM and RAM is not really cheap so cache only what you really need to.

Out of the box this configuration will give you a hit/miss ratio of at least 50% or more depending on site size and how many static files you use. This starts at 50% because I have configured Varnish Cache to NOT cache static files. My goal here was to reduce the cache size and my logic is that a) if you’re running a small site then pulling the static files from disk is not a big issue – browser caching will prevent repeat requests and disk access is generally faster than most internet connections; alternatively b) if you’re running a large site you will be using a CDN so the static files should only be hit for CDN PULL requests.

Install the configuration file:
cp install/etc/varnish/apache24.vcl /etc/varnish/apache24.vcl

Change /etc/default/varnish just the -f parameter:

    DAEMON_OPTS="-a ${VARNISH_IPV4}:${VARNISH_PORT},[${VARNISH_IPV6}]:$VARNISH_PORT \
             -T localhost:6082 \
             -f /etc/varnish/apache24.vcl \
             -S /etc/varnish/secret \
             -s malloc,${CACHE_SIZE_MB}m"

WARNING: I have configured Varnish Cache to perform health checks on the backend server (Apache 2.4). These health checks assume this tutorials default virtual host exists (i.e. it checks for /helloworld.html). If you’re using something else then Varnish Cache will probably decide Apache 2.4 is down and stop serving requests:

backend default {
    .host = "127.0.0.1";
    .port = "80";
    .probe = {
        # THIS FILE MUST EXIST ON THE DEFAULT VIRTUAL HOST
        # OR THE BACKEND WILL GET LABELLED "UNHEALTHY" AND
        # VARNISH WILL STOP FORWARDING REQUESTS.
        .url = "/helloworld.html";
        .interval = 60s;
        .timeout = 3s;
        .window = 10;
        .threshold = 7;
        .initial = 10;
    }
}

Finally you should probably install a WordPress plugin for Varnish Cache management. This will allow WordPress changes to trigger a Varnish Cache purge. There are a couple of plugins but https://wordpress.org/extend/plugins/wordpress-varnish/ seems to be popular. The configuration allows purges from localhost so any plugin should work.

Naturally you should also be using a WordPress caching plugin such as WP Super Cache.

Part 10: Nope that’s it.

If you read all the way to here then well done! That’s all there is. In the Appendix below there are a few specific notes that I’ve made for various situations. I will try to keep this Appendix up-to-date so if you find an issue with any of this then let me know or leave a comment below.

Some general advice though:

  1. Subscribe to the Apache Announcements mailing list
  2. Do some speed testing using Pingdom and Blitz.io

I would also like to reference the following sites:

Appendix

PHP-FPM using TCP Ports

You’ve probably noticed that the Apache configuration directive ProxyPassMatch uses port 9000 for the default virtual host. This is because the UNIX TCP sockets patch had not been merged into the source at the time of writing.

VSFTPD with the CHROOT

check_shell=NO
pam_service_name=ftp

PROFTPD with the CHROOT

Set RequireValidShell to off.

PHP HTTPS API Calls

I use Authy to provide two-factor authentication on several WordPress installs. The Authy for WordPress plugin uses PHP’s CURL module to make a HTTPS request to the Authy servers. This will fail inside the default CHROOT because the Authy API uses HTTPS and the certificate can not be validated.

You can copy the system SSL certificates into the CHROOT to resolve this issue:

cd /etc/ssl
for c in `ls certs`; do B=$(dirname `readlink certs/$c`); if [[ "$B" == "." ]]; then sudo cp -P certs/$c /srv/www/andrewbevitt/etc/ssl/certs; else sudo cp certs/$c /srv/www/andrewbevitt/etc/ssl/certs; fi; done

There will be a few readlink errors where the link is broken you can ignore these.

Once again this is a COPY of the certificates and you must keep them up-to-date.

MOD_WSGI – Python Support

  • I have some Django applications running with this setup
  • Yes mod_wsgi works with the Apache 2.4 Event MPM
  • Configuration is per normal but you’ll need to compile mod_wsgi

cd build
curl -X GET http://modwsgi.googlecode.com/files/mod_wsgi-3.4.tar.gz | tar xzf -
cd mod_wsgi-3.4
./configure --with-apxs=/opt/apache24/bin/apxs --with-python=`which python`
make
sudo make install
sudo echo "LoadModule wsgi_module modules/mod_wsgi.so" >> /opt/apache24/conf/modules.d/modules.load

Fail2Ban with this setup

sudo apt-get install fail2ban
sudo cat > /etc/fail2ban/filter.d/apache-403.conf << EOF
# Fail2Ban Configuration for Apache 403's
[Definition]
failregex = (?P[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}) .+ 403 [0-9]+ "
ignoreregex =
EOF

cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

In /etc/fail2ban/jail.local:

[apache-default]

enabled  = true
port     = http,https
filter   = apache-auth
logpath  = /var/log/apache24/error.log
maxretry = 3

[apache-vhost]

enabled  = true
port     = http,https
filter   = apache-auth
logpath  = /var/log/apache24/vh_*/error_log*
maxretry = 3

[apache-403]

enabled  = true
port     = http,https
filter   = apache-403
logpath  = /var/log/apache24/vh_*/access_log*
maxretry = 3

Prevent Uploaded PHP from Executing

Add something like this to your virtual host configuration files:

RewriteRule ^/content/uploads/sites/3/.*\.php$ - [F,NC]
ProxyPassMatch ^/(.*\.php)$ fcgi://127.0.0.1:FPM_PORT/var/www/site1.example.com/$1

The RewriteRule will [F] forbid any [NC] non-case sensitive file name ending in .php from being accessed.