The Battle of the Hyper Giants (Part I)
My blog post last month on the rapid growth of Google generated a bit of discussion around Google and its competitors. In particular, this Wired article (“Google’s Traffic Is Giant”) suggests Google’s infrastructure should “frighten the world’s current ISPs” and content distributors (i.e. CDNs like Akamai and Limelight). Going even further, a panicked “EatMoreBeef” Wired reader warned “I’m selling my Akamai stock!”
As Google grows towards 10% of all Internet traffic, will the multi-media search giant squash all competitors under its chrome-plated multi-terabit steamroller?
Or will the global zeitgeist tire of kitten videos and plow YouTube under the treads of hundreds of millions of virtual tractors tending to their farms and social networks on Facebook?
Or will Microsoft’s desktop OS juggernaut link with a growing Azure Cloud and pink phone to form an impenetrable competitive enterprise and consumer road block?
I have no idea.
Given my previous market predictions (“Google will never go above $200!”), I’m not going to try and predict the winners / losers in today’s Hyper Giant fight.
But I do know that that the future of the Internet is being decided today by billions of dollars of investments in data centers, backbone infrastructure and alliances / contracts with other content owners and last-mile providers. And increasingly, Hyper Giant strategies are coalescing around similar infrastructure investments as the giants compete on content, capacity (bandwidth, storage, compute), cost and performance. In other words, Google is not unique in their infrastructure ambitions.
In the next couple of blog posts, I’ll look at several of the “Hyper Giants” to help put all of this in perspective.
The below graphic shows market data and Internet routing and traffic statistics for Google (Alexa #1), Facebook (Alexa #2) and Microsoft (Alexa #5). [To save you from having to go to the Alexa web site, #3 is Yahoo and #4 is Google’s YouTube]. I have also included Akamai — one of the largest Internet infrastructure providers most consumers have never heard of — in the list below.
Famously started in Harvard dorm room in 2005, Facebook has grown well beyond its Ivy League roots to become the daily required Internet stop for hundreds of millions of consumers. Facebook content has also evolved beyond short text updates bragging about last night’s party to include thousands of applications, games and petabytes of pictures and video.
And a lot of Internet traffic.
The below graph shows Facebook as a weighted average percentage of all Internet inter-domain traffic. As in previous blog posts, I’m using data from 110 Internet providers around the world anonymously sharing coarse grain traffic engineering statistics. I have also included MySpace traffic as a point of reference.
Between March of 2007 and April 2010, Facebook grew from zero to more than .5% of all Internet traffic globally — placing the dominant social media site well in the top 50 Internet Hyper Giants. And this number does not include the significant volumes of Facebook CDN traffic.
Given the expense and time required for new Internet scale datacenter construction ($500 million or more), most Internet content companies begin life using colo (e.g. Twitter) or leased wholesale space (e.g. Facebook). Many nascent Internet companies (Facebook included) also start out leveraging third-party distribution infrastructure like LimeLight or Akamai (currently the dominant CDN used by Facebook).
But as computing, storage and distribution demands increase, small differences in capital / operational expense become large competitive differentiators at Internet scale (this 2008 NANOG presentation provides a nice overview of datacenter / colo pricing pressures). As Facebook crosses the 30,000 server mark, the company’s strategy has increasingly shifted to focus on its own proprietary infrastructure. Earlier this year, Facebook began construction of its first datacenter to “deliver a faster, more reliable experience worldwide.” Facebook is rumored to have plans on the drawing books for another four mega-scale Internet data centers.
Also like Google, Facebook has aggressively pursued direct peering with last-mile / consumer networks. As of March 2010, Facebook uses direct peering for more than 25% of its traffic (up from 5% in 2009). Like other content heavy Hyper Giants, Facebook also offers a liberal peering policy with a presence at more than 15 public exchange points.
While Facebook may not yet have the same infrastructure footprint as Google or other larger Hyper Giants, the game is clearly afoot. Leveraging wholesale datacenters, third-party CDNs and a raft of partnerships and alliances, Facebook may yet outgrow competitors with an all encompassing social media cum application platform. As of last month, Facebook reportedly surpassed Google as the most visited site on the Internet.