memcached + WordPress + DreamHost Shared Hosting

The idea of using memcached on my shared hosting account had been bouncing around my head for a while; especially on my DreamHost shared hosting account. There are periods when the performance would suffer because of other users on the server, and I was also looking for a general increase in performance without having to pay for an expensive dedicated server. I should start off by saying that what I’m about to explain only works if you have a VPS or Dedicated server that you can access already. It may seem redundant for a tutorial or howto guide on how to get memcached working in a shared environment if you already have a VPS available, but the practice and procedures will be useful to allow you to run other customer PHP modules on DreamHost shared hosting. In my case, I already had a VPS server I use for Irssi but was not configured to be a web server. It’s a stripped bare DreamHost VPS with sudo / root access, and a handful of IRC applications running in screen. I had stumbled across this deal, for $5/month (on top of your regular DreamHost bill) when they were beta testing VPS servers and I had volunteered for an account; somehow my account got tagged with the discount, so I decided why not and added one to the account. Not only that, I didn’t want to deal with the migration of switching my accounts over from the shared server to the VPS; the “seamless” transition they have setup is fairly deceiving; I find that it usually borks up the websites in the transfer and requires a fair bit of mucking to get everything working again.

The first obvious problem, even before you go about installing memcached support modules for your web application, is that DreamHost’s default PHP modules for their shared hosting environments do not include memcached. Later I found out that they also don’t have a build of Imagemagick, but the procedures in this guide will apply to also allow you to install Imagemagick on your own.

Before we dive in and get our hands dirty, here’s a little background about memcached and why we want it. Memcached’s website says this about what it is:

memcached is a high-performance, distributed memory object caching system, generic in nature, but originally intended for use in speeding up dynamic web applications by alleviating database load.

The gist of the issue is that PHP is a dynamically interpreted language. Unlike traditional languages like C or Java, you don’t “compile” it, you leave it as source. When someone stumbles on your website, the request for the page pulls the source code into the PHP interpreter, which then compiles it and executes the code before sending the result to your happy web visitor. Because the code is pulled, compiled, and ran each time someone visits your website, there is usually a pretty significant overhead as your PHP slowly stumbles through the compiler. This effect is especially bad for intensive CMS systems like WordPress, which is being used on this very website.

Memcached aims to speed the process up by caching frequently used objects that your PHP code uses – in the case of WordPress, this is achieved through the use of a plugin that queries the memcached server for a cached version of the object before recreating and recompiling the object from scratch. Even with the overhead of checking before creating the object, this results in a significant savings in compilation and execution time. The tradeoff is higher memory usage (space-time tradeoff). The memory usage isn’t really significant for a single user, but can easily balloon if there’s high-traffic website that is hitting the cache often. For this application, I’m just using memcached with the default cache size of 64mb.

The first thing you’ll want to do is head on over to the PECL page and download the module of choice – since I’m working with memcached, this is the link for the PECL module. I grabbed the latest stable version, which is 2.2.6 at the time of this writing. You’ll need to download and extract the archive on the DreamHost shared server into a temporary working directory.

exodus:~/.php/5.3/src$ wget -nv http://pecl.php.net/get/memcache-2.2.6.tgz; tar -zxvf memcache-2.2.6.tgz
2011-06-27 06:21:39 URL:http://pecl.php.net/get/memcache-2.2.6.tgz [35957/35957] -> “memcache-2.2.6.tgz” [1]
package.xml
memcache-2.2.6/config.m4
memcache-2.2.6/config9.m4
memcache-2.2.6/config.w32
memcache-2.2.6/CREDITS
memcache-2.2.6/example.php
memcache-2.2.6/memcache.c
memcache-2.2.6/memcache_queue.c
memcache-2.2.6/memcache_session.c
memcache-2.2.6/memcache_standard_hash.c
memcache-2.2.6/memcache_consistent_hash.c
memcache-2.2.6/memcache.dsp
memcache-2.2.6/php_memcache.h
memcache-2.2.6/memcache_queue.h
memcache-2.2.6/README
memcache-2.2.6/memcache.php
atomicfire@exodus:~/.php/5.3/src$

Now that you have extracted the source, we can follow DreamHosts’s procedure on building and installing custom modules:

exodus:~/.php/5.3/src$ cd memcache-2.2.6
exodus:~/.php/5.3/src/memcache-2.2.6$ /usr/local/php53/bin/phpize
Configuring for:
PHP Api Version: 20090626
Zend Module Api No: 20090626
Zend Extension Api No: 220090626
exodus:~/.php/5.3/src/memcache-2.2.6$ ./configure –with-php-config=/usr/local/php53/bin/php-config && make
<lots of configure and build information here>
exodus:~/.php/5.3/src/memcache-2.2.6$ cd modules
exodus:~/.php/5.3/src/memcache-2.2.6/modules$ ls
memcache.la  memcache.so
exodus:~/.php/5.3/src/memcache-2.2.6/modules$

With the build (hopefully) successfully finished, all we need to do now is move them to a folder where we can point our new PHP configuration file, and add the line into our phprc file:

exodus:~/.php/5.3/src/memcache-2.2.6/modules$ mv * ../../../lib/
exodus:~/.php/5.3/src/memcache-2.2.6/modules$ cd ../../../
exodus:~/.php/5.3$ cat phprc
extension = /home/username/.php/5.3/lib/imagick.so
extension = /home/username/.php/5.3/lib/memcache.so

Now all you have to do is log into your DreamHost control panel and make sure your hostname is running PHP 5.3 (since you compiled the PECL module against 5.3), and your Memcache PECL extension should be good to go!

Lets install memcached on our VPS:

~$ sudo apt-get install memcached
Reading package lists… Done
<lots of install-ish stuff here>
~$

I do love Debian-based distributions so, so much. Next, lets get it setup and configured.

Take note; because DreamHost’s VPS servers do NOT have the capability for iptables, you can’t firewall off controlled access to the memcached daemon. By itself, memcached does not have much in the way for access control, and you certainly don’t want it exposed to the Internet. Memcached stores, writes, returns, and can communicate data in clear-text, and is indiscriminate about who accesses the daemon or what the client does to the data. This means any average Joe Bob SquirrelPants can just netcat / telnet / whatever their way into your object cache and read and modify information at will. Not good. If HTML had a tag for “reach out and slap the reader” I would use it here to make sure you understand the risk.

What I wound up doing here is something that I lucked out on. Dreamhost VPS have a private and a public interface:

~$ sudo ifconfig
eth0 Link encap:Ethernet HWaddr 00:00:00:00:00:00
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:151727415 errors:0 dropped:1871 overruns:0 frame:0
TX packets:236389135 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:31861835451 (29.6 GiB) TX bytes:341275420981 (317.8 GiB)
Memory:fb5e0000-fb600000

eth1 Link encap:Ethernet HWaddr 00:00:00:00:00:00
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:3282435188 errors:0 dropped:5610 overruns:0 frame:0
TX packets:3138743657 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:2913519728599 (2.6 TiB) TX bytes:2137298062221 (1.9 TiB)
Memory:fb6e0000-fb700000

eth0:ps358441 Link encap:Ethernet HWaddr 00:25:90:34:c7:4a
inet addr:10.xxx.xxx.xxx Bcast:xxx.xxx.xxx.255 Mask:255.255.128.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Memory:fb5e0000-fb600000

eth1:ps358440 Link encap:Ethernet HWaddr 00:25:90:34:c7:4b
inet addr:173.xxx.xxx.xxx Bcast:xxx.xxx.xxx.255 Mask:255.255.252.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Memory:fb6e0000-fb700000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:234405240 errors:0 dropped:0 overruns:0 frame:0
TX packets:234405240 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:874649533641 (814.5 GiB) TX bytes:874649533641 (814.5 GiB)

~$

See that 10.x.x.x interface? It’s private and only accessible from inside the datacenter; as it turns out I can also access it from my shared server Exodus – perfect. Edit the /etc/memcached.conf configuration file to make the daemon listen only on the private interface:

# Specify which IP address to listen on. The default is to listen on all IP addresses
# This parameter is one of the only security measures that memcached has, so make sure
# it’s listening on a firewalled interface.
-l 10.xxx.xxx.xxx

Obviously you want to use your private interface here instead of the x’s. :wq and go start the daemon:

~$ sudo /etc/init.d/memcached start
Starting memcached: memcached.
~$

You now have a functional and running memcached setup on your VPS server, accessible only from it’s private interface. Now lets concentrate on our WordPress installation. I was using the Memcached Object Cache plugin to enable my websites to utilize memcached. A side-note when you use this plugin; there’s no need to have the directory in your /wp-content/plugins/ directory. When you download the zip file, the only file you need is the object-cache.php file, which you place in the wp-content directory. It activates automatically, and there is no configuration required beyond setting the location of your memcached server your wp-config.php file:

//memcached plugin settings
global $memcached_servers;
$memcached_servers = array(‘default’ => array(’10.xxx.xxx.xxx:11211′));

I have several WordPress sites that I want to use with memcached but one of the things you’ll find out quick is that you will run into key collisions where WordPress will use the same object names in queries to memcached, even across several different installations. This eventually lead to some hilarity as I started working on my sites; I would log into one, while reviewing content another. Pretty soon I was seeing myself logged into the wrong instances of WordPress, and seeing content grafted from one site to another. No bueno!

The problem lies in the fact that the plugin was designed to be used with either one instance of WordPress -> once instance of memcached or with a WordPress MU installation. This wasn’t really what I wanted. I wanted multiple instances of WordPress to be able to hit the same memcached server, so I didn’t have to spawn a separate memcached process for each website I intend to use with WordPress. More research and digging revealed that this particular plugin was more exclusively designed to work with WordPress MU installations, not standard single-seat WordPress installations. Some Google searching and I came upon this blog where Mr. Mohanjith had a modified and patched version of object-cache.php that can be used to allow multiple instances of standard WordPress installations to hit the same memcached server without key collisions. His innovation relies in the use of the $blog_id variable that you can define in the wp-config.php file – thereby prepending each key query to the server with a blog ID unique to that particular instance, eliminating the data key collision problem. In each of my wp-config.php files for my sites, I have this defined now:

//blog_id settings – REQUIRED for memcached
global $blog_id;
$blog_id = ‘andrewpeng_net’;

With this modification in effect along with the patched and updated object-cache.php – all my WordPress sites are now hitting the same memcached server, with a healthy performance boost as well. Great results so far!