Domain sharding tests in IE7, IE8, Firefox, and Safari

I was curious about the possible performance penalty of sharding domains for browsers that support a high number of connections per hostname, so I decided to set up a few tests. Taking after Steve Souder’s domain sharding demo, I created a page with 20 images, 2 30K stylesheets, and 2 50Kb Javascript files. I then ran a short series of tests on a few different servers with IE7, IE8, Firefox 3.5 and Safari 4.

Domain what?
For the sharding laggards, domain sharding is the practice of spreading page components over multiple hostnames to increase the number of parallel downloads a browser will perform. The number of parallel downloads that browsers allow varies from 2 (IE6/7) to 8 (Opera 9), but in all browsers this number can be multiplied by serving components from one more than one hostname.

For example, if you serve 20 images from bainbridge.com, IE7 (assuming HTTP/1.1) will download the images 2 at a time.  If you serve 10 of those images from dixon.bainbridge.com, IE7 will download the images 4 at a time – 2 in parallel from bainbridge.com and 2 in parallel from dixon.bainbridge.com.  Increasing the number of parallel downloads from 2 to 4 can provide a big speed boost for IE7 users.  But what is the performance effect on browsers like IE8 and Firefox 3.5 that support 6 connections per hostname?  Does performance suffer if you end up with 18+ concurrent connections?

The above chart shows the results of tests run on an Apache server with gzip disabled.  These were the results I had expected: big gains in IE7, particularly in the move from 1 to 2 domains; in the other browsers, small gains when using 2 domains and a slight retreat when using 3 domains.  What I didn’t expect was for Firefox to behave indifferently.

I only tested up to 3 domains because I conducted the tests primarily for practical purposes and couldn’t foresee a scenario in which I would want to serve a large number of assets from more than 3 domains.  But out of curiosity I plan to run more tests to see what happens when the modern browsers are pushed to ~30 simultaneous connections.

In addition to the Apache tests, I ran tests on Nginx and IIS 6.0.  The only salient result was the performance difference in IE7 between 1 and 2 hostnames.  On Nginx, using 2 hostnames produced a 29% faster page load.  On Apache it was 26% faster, and on IIS it was only 20% faster.

A few disclaimers:  (1) Compared to the experiments conducted by Yahoo and Aaron Hopkins, these tests are rather crude jobs.  The number of runs on each browser for each scenario was 20, distributed over 2 days.  Over time I intend to fatten the sample size with more tests.  (2) These tests only measure performance for an individual, isolated user.  In the real world you’d need to consider the effect of hundreds or thousands of users attempting to access resources with 18 or more parallel connections each.  (3) Different tools were used to measure page load time in IE, FF, and Safari.  Relative speed between the different browsers should be disregarded.