RSSConcerns over Microsoft’s datacenters
Mary Jo Foley brought up the interesting point yesterday that Microsoft's ongoing problems with the popular XBox Live service might have implications for the company's other Live or hosted software services.
The XBox Live issues have been intermittent since the services debut in 2005 but came to a head this holiday season, when a rush of new subscribers apparently overwhelmed the hosted service, resulting in outages and service disruptions significant enough that someone has already seen fit to file a lawsuit over them. Other complaints and perspectives are offered on Mary Jo's blog, but the gist of it boils down to the fact that an estimated 4 million new users rendered the service all but unusable for many of them for nearly a month.
Microsoft's glut of new data centers have the task of supporting not only these Xbox users but also subscribers to the company's other "Live" branded services, and presumably forthcoming offerings intended to host outsourced corporate data services. This movement toward a generic, utility-model of service provisioning by both Microsoft and other major players in the IT industry has seen considerable discussion recently, but little of that has focused on execution, and perhaps more of it should. It is no small thing to build up an international infrastructure of networked data centers and to write software that will take advantage of such an infrastructure, and it may well be that Microsoft's legacy code base has saddled it with products that are inherently more difficult to adapt to this model than their competitors. As Amazon's Werner Vogels has pointed out, constructing a massive, scalable, reliable software and hardware infrastructure only looks easy in retrospect. Coming out of Cornell with significant theoretical expertise in scalable enterprise systems, Vogels is no light-weight in the field, and points out that it took Amazon ten years to get to the point where it is today able to offer web services with a reasonable Service Level Agreement.
Both Amazon and Google have had to go through this process as a part of delivering their core products, but Microsoft's expertise in scalable systems has largely been relegated to providing in-house IT requirements… at their most demanding, orders of magnitude less massive than outsourced web services must be.
Can Microsoft overcome this deficit in experience to match their Live services and upcoming Exchange hosting to the scale and reliability of an Amazon or Google? Or will it matter in their slightly different "Software+Services" model? The company has proved its adaptability often enough in the past but only time will tell if this is a momentary blip or a significant architectural deficiency.
I think that this is ridiculous how Microsoft can do this to their customers. I have already given them my $50.00 for my year of Xbox Live and right now I am not getting any play time except for maybe one or two hours if I am lucky, and even that play time is very slow and laggy. Xbox keeps posting on their website that Xbox Live is up and running 24/7, while no one can get online. To top it all off, for our patience with them, when they are done, or if ever, they are going to give us a free Xbox Live Arcade Game, which I guarantee you nobody will ever play in a million years. They should at least reimburse us for our month that we have paid for. What they need to do is shut down Xbox Live for 3 or so days and fix the problem. One suggestion that I have is to divide the world up into different servers like Blizzard does it with U.S. East, U.S. West, Asia, etc. Microsoft should at least keep us informed.
I liked Tycho’s characterization of the Live issues at Penny Arcade: “If I asked you to help me move, and you said yeah, but then you realized that you had to do something else, you’d apologize and withdraw the offer. If I pay you money to help me move – if you are a moving company – and then you apologize in lieu of the agreed upon service? I don’t even know how to describe that, and describing things is all I can do.”
Regarding dividing the regions data centers serve by geography, that may work for Xbox Live, but it’s no solution for their other Live services, which have to work globally. They need an infrastructure that can handle this sort of traffic more gracefully or they are going to lose a lot more than a few Xbox players.