February 8, 2003: When I first conceived of the idea to put a group of Macs and PCs through a series of common pro digital photography performance tests, I thought the results might serve as a useful reference for digital photographers selecting a computer. The speed report, as it was dubbed internally, eventually became In pro digital photography, megahertz matters.
Within days of its publication in early January 2003 the report became the most-widely read article in the history of this site, thanks in part to links on various Macintosh, PC and general technology news sites. A month has passed since the speed report was originally posted and the questions and comments continue to arrive. This article provides answers to some common questions I've received.
Q: How were the processing tasks selected?
The 53 processing tasks that comprise the January 2003 speed report were designed to test different elements of the workflow of a professional digital SLR photographer. The emphasis was on RAW photo processing, in large part because the conversion of RAW files is where my own workflow bogs down, and because I also believe that RAW will continue to grow in popularity among quality-minded pro shooters. My hope was that the range of processing tasks would be broad enough that a photographer reading the report could look at what was important in their workflow and make a reasonable assessment about the speed at which critical daily tasks would complete on a given computer.
Q: How did you choose the four computers?
The report didn't start out as a Mac vs PC contest. My original thought was to take a range of a machines - Mac and PC, laptop and desktop, old and new - run performance tests, and present the results so that a photographer could get a sense of what class of machine would be required to get their work done. Given that was the goal, my main focus wasn't to assemble a collection of Macs and PCs that were "equal," whatever that might actually mean, but rather to try to assemble a small but meaningful cross-section of computers. I opted for two very fast machines from today, and two reasonably fast machines from a generation ago.
I had an inkling that the PCs were going to put up strong performance numbers, based on my own use of both Macs and PCs in the last year or so. But, with the inclusion of the Power Mac G4/1.25GHz dual processor desktop (the quickest Mac available at the time of the report's publication), I thought that that would be plenty of Mac to counter the powerful PC side.
Only until well into the testing did it become clear that even the Power Mac G4/1.25GHz dual processor was lagging badly, and that this was setting the stage for the report to be viewed as another Battle of the Platforms by the Mac faithful.
Q: Do you use Mac or PC?
The Mac is my primary computing platform, though in my work as a digital photography trainer and consultant I need to speak Windows also. My own digital workflow is inextricably tied to the photo browser Photo Mechanic and the photo cataloger iView MediaPro, which only exist on the Mac currently (okay, there is a Windows version of Photo Mechanic (v2.0r5), but it's a pale imitation of Photo Mechanic 3 for Mac). So, despite the test results, the Mac will remain my primary computing platform for now. But I have no fear of Windows, and in fact I really quite like Windows XP, so a switch to PC is not out of the question here once Photo Mechanic and MediaPro are available cross-platform.
You can read more from both sides of the Mac-PC divide in a long thread in the forums on this site.
Q: How should the test results be interpreted?
The battery of tests were designed to gauge computer performance in the workflow of a pro shooter using a digital SLR camera. If this doesn't describe you, then it's quite possible that the test data in the speed report doesn't apply to you.
If you're a pro digital shooter, and the tests included multiple instances of the type of processing tasks that occupy your time each day, then these are just some of the possible ways to interpret the data:
"I know the Mac inside and out, I depend on programs that are only available on the Mac - I'm staying on the Mac. But, the performance gap between the two platforms when running the same program is simply unacceptable. Apple, Canon, Fuji, Kodak, Nikon, Adobe and Extensis are all going to hear from me."
"I shoot RAW and I do a lot of batch processing in Photoshop. I need to switch from Mac to PC right now."
"Sure, speed is important, but there are other factors - including ease of use, reliability, cost and available pro digital photography software - that are more important. That's why I use a Mac now, and will continue to use a Mac in the future."
"Sure, speed is important, but there are other factors - including ease of use, reliability, cost and available pro digital photography software - that are more important. That's why I use a PC now, and will continue to use a PC in the future."
"The Powerbook G4 15" may be kind of slow, but its balance of performance, weight and functionality is ideal for me when working in the field. I don't see anything available for PC that offers the same combination of features."
"The Alienware Area-51m is so darned fast that I don't care how heavy it is. This is one computer that I can use for absolutely everything without having to give up an ounce of performance."
If you're a developer of one of the programs tested in the speed report, then I hope your interpretation of the results is "we have to do better for the Mac folks."
The release version of MacBibble 3.0 can serve as inspiration here: by writing RAW processing code to explicitly take advantage of both multiple processors and the G4 chip's Altivec processing engine, MacBibble developer Eric Hyman was able to bring blazing RAW photo conversion speed to the Mac version of his program. Faster, in fact, than Bibble 3.04 for Windows running on the single-processor Alienware Area-51m in the two Bibble/MacBibble batch conversion tests in the report.
When the speed report was first published, it included test data from a near-final version of MacBibble 3.0. Eyeing the test results, which showed a respectable but not stellar showing for the Mac, Eric Hyman got in touch to say he was confident that he could tune MacBibble to be even faster than it was in its late beta form. Eric Hyman delivered. Thanks to some key RAW processing code being rewritten to take advantage of Altivec, MacBibble 3.0, in its now-released form, is dramatically faster than the version originally tested (which was no slouch already). In fact, MacBibble 3.0 running on the Power Mac G4/1.25GHz dual processor desktop is second to none in RAW photo conversion speed (the speed report has been updated with data from the final version of MacBibble 3.0).
There's no guarantee that other programs and other processing tasks can derive as much benefit from being written expressly for the Mac's processor architecture. But, the RAW processing speed of MacBibble should at least provide a ray of hope to Mac users, while also providing some powerful ammunition to lob in the general direction of other software developers and Apple itself. It's also a compelling reason to check out MacBibble 3.0 to see if it meets your RAW photo processing needs.
Q: Should I select a computer exclusively based on processing speed?
No. Processing speed is only one of several factors in the computer selection process. How much weight you give this factor will depend on how much time you spend converting RAW photos, batch processing in Photoshop and the like.
Q: An article by guest author Chris Russ in the January 24, 2003 Imaging Resource newsletter says that you used tricked-out PCs against a stock Macintosh, and that the tests were heavily dependent on drive performance. Is this true?
Both the Alienware and Dell computers were running stock components. To be sure, the Alienware Area-51m is a fast machine, but neither it nor the Dell Dimension 8200 had been souped up in a way that made them tricked-out or otherwise better-appointed than their Mac counterparts.
The majority of tests were actually designed from the outset to minimize the role of hard drive performance, by utilizing JPEG as the destination file format for the RAW photo processing tasks. JPEGs are much smaller files than, say, TIFF, which means the portion of any given processing task devoted to writing to the hard drive is much smaller with JPEG than with TIFF. For other tasks, JPEGs were used as both the source and destination file format, for the same reason. Setting up the tests this way was not a stretch, since it emulates how I often use these programs.
To confirm that the speed report was not in fact highly drive system dependent, I reran all the tests on the Power Mac G4/1.25GHz dual processor desktop. Instead of reading from and writing to a 20GB partition on the stock Hitachi Deskstar 120GXP 120GB hard drive and internal ATA100 controller, this time I opted for a snazzier Sonnet Tempo ATA133 controller PCI card and a drive suggested to me by Chris Russ, the author of the Imaging Resource piece: the Hitachi Deskstar 180GXP 120GB. It too was configured to use a 20GB partition.
The result? An average improvement of just shy of 1% for all 53 processing tasks in the report. This drive combo is indeed quicker for truly drive-intensive tasks: I measured a 14% speed bump when opening, rotating and saving 100MB TIFFs through Photoshop, and a 16% jump when copying 100MB TIFFs in the Finder. But for the tests in the speed report (which did not include the chugging about of huge files like this), drive performance was, at best, a minor factor.
It's worth noting, however, that if your computing day is spent processing really big photos (ie 100MB+), a faster hard drive and/or drive controller can make an appreciable difference in the speed at which tasks complete. This has been tested and found to be true here on the Mac, and is likely true on a PC too.
This page details the complete test results for the two drive configurations.
Imaging Resource has now published a new article by Chris Russ. In it, he steps back from his position that the speed report was highly dependent on drive speed and acknowledges that it did not use tricked-out PCs.
Q: Has Apple CEO Steve Jobs contacted you since the speed report was published?
Q: Is the Alienware Area-51m laptop really green?
This laptop is available in 6 different colours/finishes, including Cyborg Green. The unit tested was the standard grey two-tone model.