Hardware adventures

I've been assembling computers for over 10 years now, and even built PC based rack mounted servers for a company I was involved in. The advantages of building your own machines are of course the lower costs and the freedom to select and mix the best/cheapest/fastest components available (these requirements rarely go together btw). Besides, it's fun!

Having a fast computer around also allows you to validate any benchmark or vendor claim you can find on the Internet, and especially ETL or database benchmarks have my special attention. Besides, benchmarking is fun too!

My first foray into the benchmarking world started about 3 years ago, running TPC-H queries on a small Windows box. Small even in a literal sense: it was a Shuttle SFF which today still serves as my sons game PC. But, with only 2 GB of Ram and a single 320GB disk, testing was limited to smaller data sets. So, two years ago, I started thinking about a bigger machine with more serious specs. It was delivered around Christmas of 2007 and with a quadcore Intel CPU, 8 GB of Ram, WD Raptor system disks and 4*750GB data disks plugged into a hardware Raid controller set up as Raid 5, the fun could begin. In fact, this is the PC I still use every day to work on (though there's some tough competition from the Macbook Pro I got about a month ago). But still, this wasn't the kind of hardware to be found in corporate IT data centers so this year I started musing about a real killer machine. Anything with a brand on it was way beyond budget since I was looking for the hottest disk technology around: SSD! Not just a single SSD drive, but preferably 6 or 8 of them to squeeze out every bit of bandwidth available.

Looking back, I totally understand why companies buy preconfigured machines from established manufacturers. But then again, buying a dual Xeon machine with 64GB of Ram, 8 large SSD drives and fast controller cards from any of those would have set me back at least $15,000. Not even close to what I can afford, so the search for components started. The good thing is, you can find everything you need at good prices in various online stores. The bad thing however: nobody can guarantee that your dreamed up configuration will actually work. During the process I learned quite a lot more about hardware than I already knew, for instance the difference between quad and dual ranked memory chips. Anyway, I decided to go with my regular hardware vendor since they always offer good prices and a terrific service. On June 26 I ordered all required components (which includes assembling & testing the system), hoping that I could have a working machine early July.

Wrong.

First of all, delivery lead times were pretty long. The Xeon's were particularly hard to get and the selected SuperMicro motherboard also took some time. Then, the first phone call: the E-ATX board didn't fit into the E-ATX case. Wtf?(1). Alternatives were either order a SuperMicro case (they somewhat 'modified' the E-ATX form factor) which was also much more expensive, or order another motherboard. I went for the latter and waited another 5 days after which I got the next phone call. 'Your machine only reports 48 GB of Ram but we installed 64'. Wtf?(2) That's when you learn about dual and quad ranked DDR memory. Bottom line: if you want to use more than 48 GB of Ram, order dual ranked memory. Of course I didn't so I had to wait another 4 days untill the new chips arrived which, by the way, are also more expensive.

Finally, the call I was hoping for: your machine works, we tested it and it's ready for shipping. Hurray! Next day I got home from a trip, unpacked the machine, plugged in a monitor and keyboard and pressed the power button. Yes, something did happen: the machine booted and made it to the Windows boot screen (they apparently used Win2008 to test and left it on the machine). After 40 minutes: still the same screen. Switched off and back on again: same story. Wtf?(3)

Ok, then lets look at the storage configuration: I ordered two Adapted 5805 SAS/SATA controller cards (the fastest available), 2 15K 146GB SAS drives and 8 OCZ Vertex 120 GB SSD's (nearly as fast as the Intel X-25 but about 50% cheaper per GB). Apparently they hooked up all SSD's to a single controller card, and since the disks have a maximum of 250 MB sustained sequential read and the controller card can transfer 1.2 GB that wasn't what I had in mind. I switched the plugs, reconfigured the raid sets (all of them Raid 0: I just want speed) and first tried to install Ubuntu. No way of course: no Adaptec driver available. So I switched to CentOS which has a superior disk configuration utility and recognized the controller. That seemed to work OK, until  it started to format the SSD drives. At about 70%, a loud continuous beep from one of the Adapted cards indicated that something was seriously wrong. Wtf?(4) Rebooting and opening the Raid config utility showed that in one Raid set the system only recognized 3 of the 4 disks, and the other one dropped two. Tried again. Same issue, but now only one raid set got corrupt. Called the vendor who told me everything worked fine with Windows 2008 server, so I started installing that one (you can see how desperate I was getting at that point). And yes, same problem. In fact, I'm not the first and certainly not the only one to experience problems with the Adaptec/OCZ combination. Real shame considering the record breaking results with the same cards and 16 Intel SSD's. I 'only' have 8 disks, but I was hoping to get 1.5 GB in a similar setup.

Now I had two options: switch controllers or switch the disks. That wasn't a very hard choice; although I like the speed on paper of the Adaptec cards, I found their Linux support pretty disappointing. And I didn't feel like spending even more money than I already did on Intel SSD's. Back to the 3ware 9690SA then: a solid performer and the driver is built right into the Linux kernel. The only worry I have is whether that card will work with the Vertex drives. Well, I'll find out next week!