A couple of weeks ago I was setting up a few servers for a new client. Part of that setup was an instance of memcache, and I took a few minutes to show a colleague the 'how and why' of a secure memcache install. She said "You should blog that". It made it to my 'to do' list, but I hadn't gotten around to it until now.
This slashdot story was an inspiration to me. I can't believe the high-profile websites that had security problems with their memcache instances; clearly this blog entry is a needed public service.
Understanding the Problem
Before we can secure memcache appropriately, lets understand the problem, and also look at how we can test our secure installation. For the sake of this tutorial I'm going to talk about memcache running on linux systems. My examples will be against a CentOS 5.3 install, but this should apply to just about any flavor of linux. Your mileage on Windows may vary. I'm also not going to go into detail on what memcached is and why you'd use it as part of a web aplication architecture - suffice to say that for high performance web apps, it can be an important part of your application's performance.
The problem is actually pretty simple - memcached is built for speed, not for security, and it does nothing to secure itself. As far as memcache's authors are concerned, this isn't a problem... it just means you have to secure it yourself. The problem is the people don't do that.
Since its not secure, this means that random people on the internet can discover your memcache, read things from it, and even write into it. The kinds of security problems that could create are limited only by your imagination and the kinds of data you put into your cache.
Lets assume I have an insecure memcache instance running at my.example.com. I can log into that server, and connect to my memcache instance with telnet:
[dbock@my.example ~]$ telnet localhost 11211 Trying 127.0.0.1... Connected to my.example.com (127.0.0.1). Escape character is '^]'.
Notice I'm connecting to port 11211 - this is the standard memcache port. In this example, I successfully connected, so lets store a value in the cache:
add catchphrase 0 0 29 intelligence for sale or rent
and memcache returns
STORED
so far, so good. We just stuck a piece of data into our memcache.
but guess what? ANYONE, ANYWHERE on the internet can now get that piece of data. Log out of your machine and try this:
[dbock@my.desktopmachine] ~]$ telnet my.example.com 11211 Trying 192.0.32.10... Connected to my.example.com (192.0.32.10). Escape character is '^]'.
Wow... we connected. That seems risky... Now type:
get catchphrase
and memcache responds
VALUE catchphrase 0 29 intelligence for sale or rent END
Ouch. We just connected to our server and pulled out our super-secret catch phrase. We don't want or need memcache to be listening to the outside world.
Securing the Server
If you are dealing with a multi-server environment, the solution will be more complicated than this - you'll want a firewall, a private network, and other complexity to separate the 'outside world' from your 'inside world' (contact me for advice if thats the case). If you are dealing with one server though, the solution is simple - just don't listen to your ethernet connection.
Mysql has an option to specify what address it listens to... so rather than listen to everything, we ony want to listen to the localhost - 127.0.0.1
On my CentOS box, I can do this by simply editing the
/etc/sysconfig/memcachedfile, changing the line
OPTIONS=""
to
OPTIONS="-l 127.0.0.1"
and restart memcache with
sudo /sbin/service memcached restart
retry the steps above and you'll find that outside the server, telnet can't connect anymore.
The memcached settings file might be someplace else on other linuxes, depending on their file conventions... The memcacheinstall for your platform should give you a clue where this is, as this file is also needed to specify the amount of memory in the cache, etc.
I hope this prevents an attack against your server, and good luck!
