More than six months after the release of version 2.0.4 today Varnish 2.0.5 has been released to the public. From the announcement:

This release contains multiple changes, amongst them:
• Performance improvements, particularly on Linux.
• Implement support for HTTP continuation lines
• Handle illegal responses from the backend better by serving a 503 page rather than panic-ing
• Add backtrace to assertion errors. This requires Varnish to be installed unstripped
• Consume less memory when processing ESI
• Better standards compliance with If-None-Match support and emitting more headers on 304 responses
• Add a FetchError log tag which makes it easier to understand why a backend fetch failed.

From a first look it seems to me that not all things from trunk have made it into this release. But then, I’ll have to do some testing before I can be sure about that.

[ More » ]

As you may or may not know, on Linux the number of threads a process can spawn is theoretically unlimited. In reality, however, you will encounter a small problem there: Since the initial stack size for every user is set to a whooping 8 MB you will run out of memory for your thread’s stacks pretty fast. In general, you won’t get more than 280 to 300 threads up and running. And that’s way too few for heavy duty caches. There are two solutions to the problem:

1. You edit /etc/init.d/varnish and put a ulimit -s 256 somewhere at the beginning. This will reduce the stack size for processes started in this shell to 256KB. You can possibly go lower than that since 64KB is the minimum set by the POSIX thread library but my testing showed that going below 256KB won’t allow for any more threads. With a stack size of 256KB I managed to get to 32,757 threads running which is quite enough even for installations that get an awful lot of hits.
2. You use Ken’s patch which allows you to set Varnish’s stack size using the runtime parameter -p thread__pool_stacksize=256. The patch applies cleanly to Varnish 2.0.4 as well as the current trunk. It comes in handy on installations where you can not or do not want to fiddle with environment parameters.

Sidenote: On Linux systems you may also want to do a sysctl -w kernel.pid_max=4194303 to increase the number of available pids. Otherwise you may still be stuck with a much lower number of threads than technically possible.

On deserialized.com there’s a nice pair (Part I & Part II) of articles comparing Squid and Varnish.
While the findings there closely resemble the results of my own testing and live installations, they have to be taken with a grain of salt (as you should do with all test results anyway). The major thing here is that they just measure the raw performance of a single object type of fixed size. So the cache isn’t really populated, eliminating lookups for the requested object. And there are some other things that could be tested to more closely match a real world szenario.
However, what I found even more interesting is the comment section. Looks like we’ve got another holy war here which really makes me wonder. For me, any proxy cache is just that: a tool that helps me to deliver content faster. There are situations where I would use Varnish and other where I would use Squid – depending on which of those fits the job better. So my advice here is: If you ever want to do serious caching, take a look at all the tools available and choose the one you’re most comfortable with.

Simon Nielsen from the FreeBSD project writes about his success with using Varnish to speed up content delivery with an interesting statement close to the end:

The wiki (MoinMoin) doesn’t support explicitly purging changed pages from Varnish, so the wiki pages can’t be cached for very long since we would risk returning out of date pages for a long time. This means I don’t expect Varnish to change responsiveness of the wiki much …

I keep hearing roughly the same thing over and over again from clients – mostly when it comes to content that’s more or less created on the fly. Think forums, blogs and, yes, wikis. There are two things at work here that more or less block the use of a proxy cache like Varnish and Squid in many installations.

1. The “deliver immediately” problem
   This is my personal favorite and is mostly to be found with traditional (read: magazine) publishers. They want to see new content showing up on their front page the very moment they publish it. That’s because they want to present the hottest news to the user, well, immediately. Now, from the users perspective, a news is new news, if he or she hasn’t seen it before. So in reality, it doesn’t matter whether the news is delivered immediately or with a delay of 1, 5, 10 or 15 minutes. In my opinion, I’d say five minutes should do for most situations quite well. And that’s because of the second thing:

2. The magic of the small numbers
   Now comes the really interesting part. Imagine you run a very popular site that gets hit by, say, 100 requests a second. Without a cache, this means your web server will have to “do it’s stuff” 100 times a second. Doing it’s stuff involves loading itself, loading all the modules you may require to run your site, check all the various access, forwarding and redirecting rules that may apply and possibly even interpret some or other web programming language to finally deliver something. Now let’s imagine you run a very
   ineffective proxy cache in front of your web server that only handles 50 percent of the requests itself and forwards the rest to the web servers in the backend with an average caching time of, say, two minutes. Even when caching for such a small time frame, and with the inefficient caching you’ll spare your backends from 2 * 60 * 100 = 12.000 requests. The benefit is twofold. First, static content will get delivered to your users faster by the proxy cache. Second, even the generation of dynamic content will be faster because your backend servers will have more time slices available to actually compute something since they’re freed of the burden of delivering stupid simple static things.

Now, numbers are nice but will that really work? I’ll give you a real world example: On a client’s installation, they received roughly 1500 hits per second. This load was distributed to 15 web servers running Apache, PHP, mod_perl and god knows what else. After deploying a two machine cluster with big Varnish caches and after tweaking Varnish’s config a bit, the cache hit rate is now in the upper 90ies. From the 15 web servers formerly aching under load, nine got removed completely. The remaining six machines handle the load easily. Tests showed that even four machines would be enough but the additional two are kept as fallback systems. Savings from reduced leasing cost made up for the initial setup cost in just a few months. So in the end, for commercial sites, there’s even more to gain from using a proxy cache than faster web sites. There may be even real profit in using them.

Just found this in Varnish’s Trac:

On linux I saw a 10-20x decrease in IO from reading the disk

If that’s true, it clearly show’s the power of a single line of added code:

(void) madvise(p, sz, MADV_RANDOM);

Hopefully this will have a similar effect on other operating systems as well.

On April, 11 2009 the Squid guys released version 3.0Stable14. Changes from the ChangeLog:
More

As can be seen in the download section of Squid-Cache.org, Squid 3.1.0.7 has been released. Changes are essentially the same as in 3.0Stable14:

More