The Importance of Benchmarks

I've been running a bunch of benchmarks on our various systems lately. I go through phases...

I plan to upgrade some systems in the near future so I want to make sure that I'm going to get better performance from the new stuff. (Or hopefully marvel at the increase in performance.) On the other hand, some systems are being downsized to some extent. I plan to replace an old Athlon 64 3400+ with an Atom based system similar to what I'm using for offsite NAS. I am curious if I'll be giving up some performance and if so, how much. A part of the new system will be to use USB sticks for the boot drive because I can. I've done a boatload of performance measurements on RAIDed USB drives as well as the variety of drives on other systems. It looks like four RAIDed USB sticks provide performance not too far from spinning drives. I need to sift through all of the data I've collected before I can conclude anything.

Over the holidays our son gave us his retired Dell Vostro 1700 to replace the aging Thinkpad T42 used by SWMBO. For those not familiar, the Vostro is apparently one of those laptops that is really a desktop with a hinged display. It actually has room for two laptop drives. The specs are pretty impressive, falling just a bit short of my Thinkpad T500. I had fun loading it with Ubuntu and running some benchmarks.

One of the areas where the Vostro falls short is that it only supports Wireless G. I'm sold on Wireless N and happy that my netbook (Eee PC 901) and smart phone (Droid X) both support draft 802.11n. (Is it still a draft standard?) I looked up the specs on the Dell website and found that they listed a couple 802.11n cards. I looked around and now have a card upgrade enroute for the princely sum of $15 US.

I want to make sure the card works so I found some benchmarks to run. I started with netperf. It's a nice simple package that installs a server and a client which communicate with each other to measure throughput. It is now installed on all of our systems. I also did some large file copies across the wireless LAN, but that can be affected by the speed of file I/O. And caching. I was surprised that the second time I copied a 340,393,984 byte file from the remote system, it finished in a few seconds with no network activity.

The other surprise was that wireless throughput was better on the Vostro (with 802.11g) than on the Thinkpad T500 which supports 802.11n. :( I investigated and found that the router was set up to support G, not N. I don't know how that happened. It has two radios and at one point I was trying to figure out how to set one up to run G and the other to support N only. It was my hope to get better N throughput by doing that, but I don't think the router is capable of that configuration. I probably misconfigured it while working on that. I now set it to support N as well as G and throughput for the T500 went up considerably. In fact, some tests with netperf reported throughput higher than between two hosts on our wired LAN (100baseT) It also become much more variable. And shortly after the change, it lost the connection between the T500 and the AP to the point where the T500 requested the WPA password. My research also revealed that the firmware that ships with Ubuntu 10.10 for the wireless card in my T500 (Intel 5300) may have problems. The 10.10 install disables 802.11n if it detects an Intel 5300/5100 card. I'm glad I haven't upgraded my T500 to Ubuntu 10.10 yet. I hope N isn't going to be flaky.

Other items of note that I found from benchmarking.

• An Atom processor is not all that far behind a five year old Socket 939 Athlon.
• An Athlon 64 X2 3800+ benchmarks slower than Athlon 64 3400+, at least for single threaded tasks. I think it is because the faster processor is in a system that has all four RAM slots populated which causes the memory to run at a slower speed - DDR333 vs. DDR400.
• Newer hard drives are pretty much faster than older drives. I guess that should be no surprise.
• Four USB sticks on the same USB bus configured as RAID0 (striped) are considerably faster than a single USB stick on the same USB 2.0 bus. I would have thought that (at least for reading) the USB sticks would be close to saturating the bus. Apparently not.

Pending upgrades: An Atom based system (Atom D525, 2GB RAM) has shipped to replace the Athlon 64 3400+. Along with that I have a gigbit Ethernet switch coming. Hopefully this one will last longer than the other one I had which seemed to lose ports during electrical storms. I suppose using it to connect stuff within several feet of each other should help. (Toonsinator, NAS and the rest of the LAN.)