In the wake of last week's outage at Amazon's Simple Storage Service (S3), which left many customers hanging for more than six hours while internal "communications" issues were corrected, it seems of particular importance to clarify what you are paying for when you are purchasing a "cloud-based" service and how to quantify it in terms comparable between cloud providers. It's difficult to comparison shop if you don't know exactly what it is that you are comparing.

Phil Wainewright discusses just this issue, and advances the idea that a standard "compute cycle" may be defined from a certain base hardware setup (he uses the example of Mosso's newly developed but not yet released plan based on a 1.2Ghz server under average load for one month, then sliced accordingly). It's an interesting idea, but it strikes me that what works well for Mosso may not as accurately represent the loading and structure on the back end of many other cloud service providers. Each has their own technique to measure utilization currently (which is exactly the problem); but to what extent those techniques are intimately wed to the methods used to run the service, only those providers can tell.

The upshot of this is that consumers don't have a level playing field to compare and select from among competing cloud service providers.

This isn't exactly a new problem, however. Although one has traditionally been able to compare, say, processor to processor performance specifications on hardware devices, software has never been as easy to compare. Different paradigms of operation, different key features, and sometimes deliberately obfuscated operating characteristics have always plagued CIOs attempting to compare two different software packages with the same putative purpose.

Evaluating cloud services comparatively is no different than comparing Microsoft Exchange and Lotus Notes. You simply have to line up the features and pricing against the particular utilization patterns and needs of your business. Processing portability, a fine goal and one that no doubt will force the industry itself into defining a price-compatible unit of computing power, simply isn't here yet, and until it is there isn't any practical requirement to compare vendors in that light. The combination of pricing and performance for your particular needs are all that are important. While this results in a more difficult job of evaluating providers, it is one which IT departments should be well suited for at this stage.