First off, as others have said, disabling port 80 is a great way to handle this, I don't really care that much if I can see/use github the website for a few hours, but I'd be much more upset if I couldn't pull my code.
Secondly, I kind of like when big sites go down when I'm not in desperate need because it means a really nice aftermath write up is on the way. Can't wait to hear more about this one.
> "disabling port 80 is a great way to handle this"
It's been a disaster for a bunch of people I know.
git is distributed by nature so they all had extra remotes they could use.
github's issues and other project metadata wasn't distributed, since that's github's alone.
So all of the friends of mine who are corporate github users who were using git in a distributed style (a minority of their overall customer base, I would suspect) are more screwed by the web app's absence than they would have been by a repository problem.
I suspect github made the right choice for their customer base overall, but I still find the anecdata interesting.
In fact, all HTTP access redirects to HTTPS for just about everything. And most modern browsers (recent versions of Chrome and Safari) that have accessed a website over HTTPS once happen to _prefer_ HTTPS by default for that site.
Like I said, disabling port 80 is a great way to handle this kind of attack. Those would obviously be trickier. I'm no network security expert, but I would assume that they are much more resilient to other attacks, as most protocols aren't as network intensive as TCP.
I heard once about a tire shop that drummed up business by strewing nails along the highway around it. Somewhere outside America; India or Thailand or somewhere else. Anyway, that kind of business practice wouldn't get you far in the Western world, so it seems unlikely.
OTOH, if I had a botnet and I wanted to see how powerfully it could DDOS without drawing a lot of mainstream media attention, github might be good for target practice. They have better infrastructure than most, and they always do detailed write-ups afterwards.
OT: the urban legend about tire shops putting nails in the neighborhoods' is very widespread. I have heard about it in at least three countries, and I recall my school teacher telling me about it in '91.
I sort of expect it wouldn't be an effective practice anywhere.
It's an urban legend until you actually catch a nail and there is a tire shop conveniently located not 100 yards away. True story, Indian reservation in BC, Canada. Who cares if it's a tacky business model or an urban legend. It works.
Low Orbit Ion Cannon (LOIC) is an open source network
stress testing and denial-of-service attack application
[...]
LOIC performs a denial-of-service (DoS) attack (or when
used by multiple individuals, a DDoS attack) on a target
site by flooding the server with TCP packets or UDP
packets with the intention of disrupting the service of a
particular host. People have used LOIC to join voluntary
botnets.
Fortunately since you have an entire copy of the repository it doesn't really matter. Your team can just push the latest copy to another remote like Bitbucket and start using that.
Well, except for Issues, Pull Requests, Wiki, etc, but those aren't part of Git.
Nothing against Github but this probably highlights the real benefit of DVCS: setting up multiple remotes for your repo. Manage it probably and when one service goes down, fall back to Bitbucket or another service.
It would limit the potential damage these attacks could cause, given the reliance dev teams have on pushing code to a central repo. Taking down a site like Github has a fairly clear effect on the productivity of a lot of their users.
Multiple remotes are easy. Three lines in the config to push to Github and Bitbucket. Which is great if you rely on either service. It's everything else - hooks, wiki, issues, etc. - that get overlooked.
If it's one service though there is still always a single point of failure outside of your control. If you want true redundancy you need to use multiple versions with no interconnections and develop your own solution to mitigate partial failure, of course your own solution is also a single point of failure, but at least it's under your control.
That said, Github has never been down long enough for me to justify a fallback procedure and policy for my team. The complexity of the solution outweighs the benefit. We'll cross that bridge when we come to it.
Of course there will be. We are talking about git. Everyone has a full copy of the repository. Github's data center gets hit by a meteor tomorrow and it's a minor setback, nothing more.
I'm reading over the status updates a few hours after the fact, but for some reason the 1:33PM stands out because it is wrapped in quotes. Probably minor and meaningless. Just noted because it's different from the others.
It's because we update the status site from our chat room via Hubot and whoever posted that particular update didn't realize that they didn't need to put quotes around the body of the message.
It's a site most people here use, and it's interesting. It's not about getting things back up faster, but letting people know what's going on and giving them the chance to witness how a large site fails and recovers.
Github status: "We've temporarily disabled service on port 80 while we investigate the source of a connection flood. HTTPS, GIT, and SSH service are unaffected."
Great way to keep the git push/pull workflow unaffected.
It's Likely, but this is someone with a history of claims to this sort of attack and a large following. I was interested in who was running the large attacks and what their motivations might be and no sources had been suggested in the discussion at that point.
Was just able to push/pull okay, overall seems like they're dealing with the outage really well. Their status page is really helpful, they were on twitter quickly, and they tried to restore service to people who might need to push/pull to their repos.
Github status is hosted on Github, d'oh. Their Twitter feed posted at 2037Z: "We are experiencing issues due to a DDOS attack, working hard to restore service..." https://twitter.com/github/status/259029493669310464
Looks like github status is on heroku. It's loading fine for me:
> host status.github.com
status.github.com is an alias for appid129905herokucom-760859479.us-east-1.elb.amazonaws.com.
appid129905herokucom-760859479.us-east-1.elb.amazonaws.com has address 50.19.101.240
appid129905herokucom-760859479.us-east-1.elb.amazonaws.com has address 50.19.101.216
appid129905herokucom-760859479.us-east-1.elb.amazonaws.com has address 107.22.241.165
Secondly, I kind of like when big sites go down when I'm not in desperate need because it means a really nice aftermath write up is on the way. Can't wait to hear more about this one.