Lighttpd revisited

I wrote a post not too long ago about using lighttpd (lighty) to ease your server load. My setup consisted of one high-end server box (2x Dual-core Xeon's, RAID 5, 6GB RAM) running Windows Server 2003 x64 and Virtual Server 2005. Virtual Server had three VM's running... my CF/IIS server, my database server, and my lighttpd server... All running on Windows 2003 Server Standard.

Now, the lighttpd box always had a cpu load of 30-50%... Not a big deal, considering it was always pushing 6-10mbps of files at any given moment, day or night. I eventually hit up the friendly folks in #lightpd on FreeNode, asking if I could somehow tweak my config file even more, to get performance as such that I could run a single lighttpd thread to handle all the traffic. The general response was "Don't use Windows."

Now, I know you're all thinking that the response was typical for OSS people, but I decided to give it a try. I whipped up another VM, installed SUSE 10.2 on it, and set up lighttpd. I redirected my traffic from the Windows VM to the Linux one (which was easy and instantaneous thanks to my firewall, IPCop, which will be covered in a future post), and low and behold, a single lighttpd thread was handling all the traffic.

Not only was a single thread handling it all, but CPU usage on the linux box was peaking at 2%, with little disk usage and plenty of free memory. Also, as I watched the bandwidth usage over the course of the day, I noticed that the bursts were able to actually burst higher (i.e. push more traffic and/or handle more simultaneous requests) than they were on Windows. This enabled me to take advantage of low-cost rented hardware to do some Round-Robin DNS load balancing.

So, in summary, if you're going to use lighttpd, use it on Linux.