Posted on Thursday, August 6th, 2009 | Bookmark on del.icio.us

Where Did All the Tweets Go?

by Craig Labovitz

At roughly 9:00am (EDT) this morning, the Twitisphere fell silent (or at least significantly fewer twitters).

And though you could not follow the outage via tweets, Twitter’s blog announced the popular site was under DDoS.

The below graph shows Observatory data from 55 providers around the world to Twitter’s two NTT hosted addresses blocks: 168.143.0.0/16, 128.121.0.0/16.

From the data, Twitter traffic declined abruptly around 9am EDT this morning.

We generally don’t see a lot of data (i.e. it takes thousands of tweets to match the bandwidth of a single video), but 55 ISPs in the Internet Observatory were exchanging roughly 200 Mbps with Twitter before the DDoS. Then traffic dropped to a low of 60 Mbps around 10:40am and began climbing after that. As of 1pm EDT, Twitter traffic was still down by 50% at 150 Mbps (normally we see close to 300 Mbps for this time of day).

From DNS, it looks like Twitter has moved some of their infrastructure to different address blocks as of 2pm EDT.

This entry was posted on Thursday, August 6th, 2009 at 1:43 pm and is filed under Arbor Networks, Twitter. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

19 Responses | Add your own



Comment Post by: Twitter Overwhelmed by Web Attack - Bits Blog - NYTimes.com — August 6th, 2009 @ 3:39 pm EST  Reply

[...] | 12:39 p.m. A chart by Arbor Networks showing traffic to Twtter Thursday [...]

Comment Post by: Additional DDoS info: « Ppl H8 Me On the Internet — August 6th, 2009 @ 3:44 pm EST  Reply

[...] Here’s an interesting article from Arbor Networks about the Twitter DDoS: Where Did All the Tweets Go? by Craig Labovitz [...]

Comment Post by: Apparent DDOS attacks on twitter, facebook and livejournal — August 6th, 2009 @ 4:42 pm EST  Reply

[...] 16:45: Here’s a graph from Arbor Networks showing a dramatic drop in traffic this [...]

Comment Post by: Fenixnordic Group » Blog Archive » Twitter, Facebook Attacks No Surprise to Security Experts — August 6th, 2009 @ 6:26 pm EST  Reply

[...] largest monitoring service, saw traffic to Twitter drop abruptly at 9 a.m. Eastern and saw Twitter move portions of its services to different neighborhoods on the network around 2 [...]

Comment Post by: Captain Democracy — August 6th, 2009 @ 6:39 pm EST  Reply

Go to http://www.CaptainDemocracy.wordpress.com and read about where the attack came from, “Tehran Iran.
“Captain Democracy” North Beach, San Francisco Ca.

Comment Post by: Operation “Silence Cyxymu” Crushes Twitter, Facebook, LiveJournal — August 6th, 2009 @ 9:07 pm EST  Reply

[...] was only mildly affected. Here is a snapshot of the dropoff in traffic to Twitter according to Arbor Networks: The below graph shows Observatory data from 55 providers around the world to Twitter’s two NTT [...]

Comment Post by: Twitter Restores Service And Tries to Regroup After Attack Last Night | YUdez — August 6th, 2009 @ 9:51 pm EST  Reply

[...] | 3:38 p.m. A chart by Arbor Networks showing traffic to Twitter Thursday [...]

Comment Post by: Jeff — August 7th, 2009 @ 4:54 am EST  Reply

Craig,

Can you clarify why traffic would go down under DDoS? Is this graph measuring only legitimate traffic, measuring a host name that wasn’t under attack, or does it imply that the attacks originated from networks not included in the Observatory?

Comment Post by: Craig Labovitz — August 7th, 2009 @ 9:08 am EST  Reply

The short answer is I don’t know the full details of this particular attack so I can only speculate.

But in general, attackers (or at least attack tools) have grown smarter over time.
Instead of “brute force” flooding attacks (i.e. overwhelming a router interface with sheer volume of traffic), many attacks today are smaller and much more targeted.

Examples of low bandwidth DDoS include the decade old TCP Syn attack (usually high pps but comparatively low bps) and more recently, application / service focused attacks. This latter category includes attackers using Bots to bring down a service by exercising expensive SQL queries, Web 2.0 API calls, SIP initiations, attacking DNS etc.

Comment Post by: Gurdip — August 7th, 2009 @ 9:50 am EST  Reply

Thanks for the explanation Craig. Any traffic details on other affected services such as Facebook (who apparently were also attacked)?

Comment Post by: Faisal Khan — August 7th, 2009 @ 10:35 am EST  Reply

Craig,

Any idea on the type of an attack? Or what was the size of the incoming attack? 200-300Mbps would have been very easy for the (alleged) Russian bot-net operators to bring down no? Was this a bandwidth saturation attack or an attack that overwhelmed the servers/routers?

Faisal Khan.

Comment Post by: Silence Cyxymu - Bits & Pieces — August 7th, 2009 @ 10:45 am EST  Reply

[...] is behind this attack, they had significant bandwidth available. Our best guess is that these attacks were done by nationalistic Russian hackers who [...]

Comment Post by: domoaringatoo — August 7th, 2009 @ 11:44 am EST  Reply

Speculation on F-Secure

http://www.f-secure.com/weblog/archives/00001746.html

was that this DDoS orginated from Russian nationalists in order to silence a Georgian blogger. Anyone with additional information that can shore up or refute this theory?

Comment Post by: Kristofferst — August 7th, 2009 @ 1:36 pm EST  Reply

I assume this is enough data to rule out Bill Woodcock’s explanation in The New York Times:
“Rather, he said, at about 10:30 a.m. E.S.T., millions of people worldwide received spam e-mail messages containing links to Twitter and other sites. When recipients clicked on the links, those sites were overwhelmed with requests to access their servers. “It’s a vast increase in traffic that creates the denial of service,” he said. ”
http://www.nytimes.com/2009/08/07/technology/internet/07twitter.html

Comment Post by: Craig Labovitz — August 7th, 2009 @ 1:42 pm EST  Reply

At this point, I believe several of the site owners and their upstream ISPs have a better picture of what happened during the attacks yesterday. But it is up to the site owners to release any of these details.

Comment Post by: Lots of Women on Facebook, Few Kids on Twitter, no Marines Anywhere | eGov Digest — August 7th, 2009 @ 6:19 pm EST  Reply

[...] Chart from Arbor Networks [...]

Comment Post by: Twitter, Facebook Fend Off Dos Attacks - Lets Be Secure | Lets Be Secure — August 16th, 2009 @ 7:20 am EST  Reply

[...] service Twitter fell precipitously, reaching a bandwidth of 60 Mbps by 10:40 a.m. ET, according to Arbor Networks, a networking services firm. Twitter had reached nearly 200 Mbps prior to the [...]

Comment Post by: Jeremie — August 20th, 2009 @ 7:31 pm EST  Reply

Hello Craig,

Thank you for your very interesting post (and the added information provided in your comment replies)…

I’m curious as to the source of the data used to plot the graph. Is the Observatory a publically available source? And if so, how can I access it?

Best regards…

Comment Post by: Jeremie — August 20th, 2009 @ 9:04 pm EST  Reply

Do you happen to have a graph of the number of flows for the same time period and IP blocks?

Leave a Comment