Varnish to the rescue (big math of small numbers)

Simon Nielsen from the FreeBSD project writes about his success with using Varnish to speed up content delivery with an interesting statement close to the end:

The wiki (MoinMoin) doesn’t support explicitly purging changed pages from Varnish, so the wiki pages can’t be cached for very long since we would risk returning out of date pages for a long time. This means I don’t expect Varnish to change responsiveness of the wiki much …

I keep hearing roughly the same thing over and over again from clients – mostly when it comes to content that’s more or less created on the fly. Think forums, blogs and, yes, wikis. There are two things at work here that more or less block the use of a proxy cache like Varnish and Squid in many installations.

1. The “deliver immediately” problem
   This is my personal favorite and is mostly to be found with traditional (read: magazine) publishers. They want to see new content showing up on their front page the very moment they publish it. That’s because they want to present the hottest news to the user, well, immediately. Now, from the users perspective, a news is new news, if he or she hasn’t seen it before. So in reality, it doesn’t matter whether the news is delivered immediately or with a delay of 1, 5, 10 or 15 minutes. In my opinion, I’d say five minutes should do for most situations quite well. And that’s because of the second thing:

2. The magic of the small numbers
   Now comes the really interesting part. Imagine you run a very popular site that gets hit by, say, 100 requests a second. Without a cache, this means your web server will have to “do it’s stuff” 100 times a second. Doing it’s stuff involves loading itself, loading all the modules you may require to run your site, check all the various access, forwarding and redirecting rules that may apply and possibly even interpret some or other web programming language to finally deliver something. Now let’s imagine you run a very
   ineffective proxy cache in front of your web server that only handles 50 percent of the requests itself and forwards the rest to the web servers in the backend with an average caching time of, say, two minutes. Even when caching for such a small time frame, and with the inefficient caching you’ll spare your backends from 2 * 60 * 100 = 12.000 requests. The benefit is twofold. First, static content will get delivered to your users faster by the proxy cache. Second, even the generation of dynamic content will be faster because your backend servers will have more time slices available to actually compute something since they’re freed of the burden of delivering stupid simple static things.

Now, numbers are nice but will that really work? I’ll give you a real world example: On a client’s installation, they received roughly 1500 hits per second. This load was distributed to 15 web servers running Apache, PHP, mod_perl and god knows what else. After deploying a two machine cluster with big Varnish caches and after tweaking Varnish’s config a bit, the cache hit rate is now in the upper 90ies. From the 15 web servers formerly aching under load, nine got removed completely. The remaining six machines handle the load easily. Tests showed that even four machines would be enough but the additional two are kept as fallback systems. Savings from reduced leasing cost made up for the initial setup cost in just a few months. So in the end, for commercial sites, there’s even more to gain from using a proxy cache than faster web sites. There may be even real profit in using them.