CPU mining of scrypt-based cryptocurrency is highly inefficient. Let's do some math.
A cc2.8xlarge is reported to mine at 85 kh/s, so 20 of them would give you 1700 kh/s. That's roughly equivalent to a couple of high-end AMD GPUs (say a couple of overclocked 290x). This hashing power gives you a little over 0.5 LTC per day. It mined for two days, so it gained a little over 1 LTC. Let's call it $40.
That's right, the idiot behind this cost the OP $3000+ for $40 profit. A smarter criminal would have spawn GPU instances on EC2.
Criminals don't care how much money they're costing their victims, only how much money they'll make from their crime. This is no different than someone smashing your car's window ($100+) and destroying its dashboard ($1000+) to remove a stereo head unit that they'll flip for $50.
Correct, but for the same crime and effort, (so just as easy) the criminal could have made a bigger profit. They took the stereo and left behind the laptop. Criminals however are not always the brightest bulbs.
Someone broke into the office of a consulting shop I was working for, passing up dozens of macbook pros and who knows what else for an old-school translucent aqua imac and a $100 Walmart bike.
Many of these opportunistic thieves think about what they can move quickly so if they are caught they no longer have the goods on them. That imac and walmart bike they could have moved the same day for probably $100 profit, where as the macbooks they may have had to shop them around for longer, long enough for theft services to lock them down and track them once they are turned on... Not good news for the crook. They are smarter than you probably think.
The thief probably wasn't bright and hadn't seen it before, thinking that iMac must be so new that he hasn't seen it in the news and therefore the most valuable (after all, Apple is making colored iPhones now).
No it doesn't. cgminer is the "default" that almost everyone uses and the current version doesn't even have CPU mining. The current version doesn't have script either, you gotta download a previous version but that's a different story. You gotta throw in a --script when starting it, otherwise it mines with SHA-256.
If you wanna CPU mine, you gotta download a specific CPU miner.
Makes you wonder why the criminal didn't launch g2.2xlarge instances which would get 185 kH/s per instance. In fact for awhile there litecoin mining was profitable at spot prices.
My guess would be ignorance. I don't think the criminal behind this knows what he is doing.
A smarter criminal would have opted for g2.2xlarge instances as well as mining for a currently more profitable coin. Granted he'd need to be careful not to leave a trail, he could essentially trade these coins for LTC and still obtain more litecoins.
An even smarter criminal would have "mined" the Ripple give away using EC2, there are still people paying for those instances as we speak so surely they would be more profitable than litecoin mining (assuming you sell them right away).
Uh, nope. I "mined" Ripple through that little project for a couple of days (using the g2 instances) and got a couple of dollars worth of Ripple back for over $100 ec2 bill (460 hours of WCG runtime).
I'll boinc on my desktop for science and goodwill but will not waste any more time on Ripple's program or indeed on Ripple itself, as brilliant as the idea is. Ripple's founders are hanging on to their 60%+ outstanding Ripples so tightly that I suspect the project will never truly get off the ground.
As much as people criticize it, I think some variation of Bitcoin's seignorage mining is the only realistic way to bootstrap a successful virtual currency, at least at this point.
I used it to mine dogecoin using those same instructions. I either had to use an older nvidia driver version or an older version of CUDAminer, though. So it wasn't that simple to set up once CUDAminer got the update that broke the newer NVidia software. This happened near the first of the month. http://doges.org/index.php?topic=43.0
My guess: the difficulty is low enough that it's profitable to mine dogecoin and flip it for Bitcoin or Litecoin. I know a lot of altcoin miners do this, and there are some pools (middlecoin) that are geared towards this.
The doge brand is the USP. Most LTC clones are already largely undifferentiated, so this is actually a strong contender given it's technically almost identical to the rest.
Not only that, LTC has been one of the lesser profitable scrypt-coins to mine for a while.
See https://www.multipool.us/ for reference. Although it fluctuates a lot, LTC has been around 4th-5th in profitability among scrypt coins. Often 5th. And wait for Doge to feature in these pools :-)
Actually, it's worse than that. Check http://coinwarz.com and http://coinchoose.com. Multipool is limited to the coins they mine. If you check out the more general list at the sites I mentioned, LTC is not even in the top 25.
This is rough luck, but getting specific servers hacked is more commonplace. In the AWS billing console [0] there is an "alert" option. It walks you through setting up the various types of alarms.
If you're hacked the most likely problem you'll get is a spike in data transfer costs. You can up the alarms to, for example, email you if the bandwidth usage goes above x (cost) over y time period.
I had a perl DOS bot get into a server, took about 2 hours to trigger the alarm. Shame I was fast asleep at the time, but the idea was there...
I think smarter usage of IAM roles would have also helped here. Keys created strictly for S3 access should not have the ability to launch new instances and so on. Limiting keys to their specific purpose is a good security practice even for dev environments.
AWS IAM supports the ability to permit or prevent specific types of instances from being started by a given key; if folks are worried about a key being used to start G2 or CG1 or any other specific instance, take a look at the instructions here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-polic...
I wonder if the author is going to be on the hook for the bill for this.
If he originally received this note from amazon, it makes me also wonder if amazon knew about the fraud while it was happening. I sense that they probably monitor the launch of many of the XXL servers more closely than others.
> I wonder if the author is going to be on the hook for the bill for this.
Almost certainly not. I've seen AWS reverse charges even when it's unequivocally the user's fault rather than someone hacking in.
As an example, I launched two reserved instances outside of a VPC when I meant to launch inside a VPC. At the time, this couldn't be changed, and the docs were very clear on that. After filing a support ticket, they reversed the reservations... and gave me a $100 AWS credit. For my fuck-up.
I can't vouch for AWS, since I've never used it, but if the same principles carry across from their online store then one thing I've always found with Amazon is that they have the most amazing customer service.
I had an issue with a 9 month on SD card splitting, not expecting much I emailed them and they immediately sent out a replacement and told me to return the old one. I've had a couple of things go wrong with orders (sometimes just delayed in post) and they've always been very quick and helpful. Not all companies, online ot brick & mortar, understand this well enough.
I have a support ticket open with Amazon about this. I'm hoping not, but I have to accept a certain amount of responsibility for leaking the key in the first place.
I highly doubt you will have to pay this large bill, they most likely will reverse the charges and you should be free and clear, especially if you detail what you found and that you also have an email from them that your key was exposed.
I for sure would go to your billing report on the AWS console and setup alerts. You can have AWS notify you automatically if your bill goes above any threshold you want to define.
I just checked my EC2 page and have found $45 worth of charges from instances that I shut down weeks ago. Would it have automatically re ordered some if the bidding price dropped below a threshold? When I looked at my management console there were no EC2 instances running. This has turned out to be a very expensive experiment in LTC mining.
They are quite certainly not monitoring their servers closely, I once had a application running on a small instance costing me more than 30,000$ in just a few weeks because it was doing expensive API calls to AWS. When a growth from <10$ per month to 30,000$ in about 2 weeks isn't concerning them why would 3000$?
Ugh, that sucks. Too late to help you now (but perhaps others) on your billing alerts points: check out http://cloudability.com -- alerts, analytics, prediction, suggestions, etc. Free for the most useful stuff.
Another good habit to be in is never checking any kind of credentials into source control; even if it's some private personal project, just don't be tempted to check in your credentials to source control, because at some point you may find some portion of that that's useful that you import into a public project, accidentally preserving full history.
Sorry to the OP, hope that Amazon reverses those charges once you tell them what happened.
We know we shouldn't be committing password or personal key. But shit happens and it does happen very frequently, even top notched people do.
What we need is not to say "don't it", because no shit we shouldn't be doing that. instead we need defense mechanism. It would be helpful and interesting if git or hg has a plugin that detects when some credentials is leaking through and warn users "hey you better check this shit out" before doing a real commit.
The other thing is "don't commit key into a private repository". Don't chef and puppet users usually do that? How are people backing up their keys?
For rails, the Figaro gem is really helpful for managing environment variables, which is a good place to store credentials (the config file for Figaro is then added to .gitignore)
But how do you back up this set of environment variables. Yes, in practice I also use ignore file to prevent sensitive things leak into repository and I usually generate password dynamically on the fly or through some script.
What's the best way to handle api keys without storing them in a repository? A separate configuration file?
I've saved api keys to a repository, but only ones with no payment information attached and read only access. Mainly because I couldn't think of a better way at the time.
This is very convenient on Heroku, but on my own servers, I have to use a post-deploy hook to copy a file that has all of my API keys to the server. That works great, except I now have a separate file to manage outside of source control, it's not under its own source control, and it's difficult to share amongst my team.
I've had some luck with having a separate repo. I'll have a "public" (even if it's not public) with all of the usual source code, and a "private" repo, with passwords, fabric files, run scripts, etc.
Works well on teams, where the subset of developers that don't need to do actual deployments don't need the private repo anyway.
I suggest the OP check out Cloudability[1], which provides realtime cost management for AWS and other cloud providers. We help over 10,000 customers make sure this doesn't happen to them. (disclosure: I work there)
You can (and should) set up an AWS CloudWatch alert on your account that will send you an email or SMS notification when your monthly bill exceeds a set threshold.
I was surprised how incredibly difficult that is to set up. Eventually I dead-ended following the instructions when CloudWatch told me there were 0 metrics to choose from for monitoring...
Shoot, as someone who made the same mistake of leaving my AWS keys in an open source project, I think I narrowly dodged a bullet. I didn't realize this risk was so high. Thanks for this post!
One cc2.8xlarge instance will give you 85 kH/s so 20 instances would give 1700 kH/s which only nets about $18 at current market prices / difficulty. So over 2 days he would have made off with a whopping $36.
Writing a script that generated $38 every time somebody accidentally commits a AWS key sounds like an amazing source of extra cash, depending on your current income.
If he would have used an altcoin that is limited by CPU like Primecoin he could have earned around 8000$ in 2days. Not sure if my calculation is legit through. It's based on the chains/day from the list at http://anty.info/primecoin-calculator/
Scary. I'm sure a lot of universities and servers will see an influx of hacks for coin-mining.
It's important to remember that open-sourcing is generally one-way: once it's out there, it's impossible to completely eliminate all traces. Always audit code, and if there's even a remote possibility that you'll regret it you should check again
Ask a simple question, get a simple answer: it is obviously and unambiguously illegal. You can certainly refer this to the FBI computer crimes folks and your local law enforcement. It is unlikely they will successfully investigate.
What specifically is illegal about that? Someone gave you credentials to an online service. Now you use those credentials. What law did you brake? You probably broke the ToS/EULA/whatever, but that by itself is not illegal.
Just because you have the means to do something does not mean you have the permission. If I drop my car key on the ground and you take it, it's still stealing. I gave you the means (accidentally), not the permission. Especially when using the ill-gotten item will cost me (gas, time, money).
Reminds me of the old phrase: What's the difference between hacking and penetration testing? Permission.
Yes, I see that difference. I also know there are laws protecting computer systems and such. But was wondering if it could be applied to an online service.
statutory minimum is 5k. we work with federal law enforcement somewhat frequently at wepay to deter fraud, and it appears that the informal threshold to get their interest is about 100k of damages
But will any law enforcement department or agency actually look into it? Highly unlikely. There's so much major scale white collar crime/cybercrime going on daily that it's generally too hard to dedicate resources to smaller cases like this.
Now, if they got evidence the perpetrator was doing this with dozens or hundreds of AWS accounts, that would be another story.
Amusingly, I went to a meetup with some feds the other day.
Their statutory minimum to investigate is 5k, but they suggested it's worth getting in touch with them either way. If nothing else, it'll give them an excuse to say "cyber" a few more times.
This. I was defrauded to the tune of $3,500 (run of the mill stolen Paypal account bought a top of the line iMac from me on eBay) and after discovering to my horror that both eBay and Paypal sidestep all fraud for protection... I called the FBI, police, etc.
They listened politely until I said how much it was worth -- one of my favorite quotes was "If you added a zero to the end, we'd knock down their door at 6AM tomorrow. For $3,500 it's simply not worth our time."
Dang, that sucks. So neither eBay OR Paypal helped? Did you send the macbook to the "Paypal verified address", because I thought that was one of the protection requirements?
The FBI wasn't involved in Swartz's prosecution, which was considerably easier for the US Attorney by dint of them actually knowing who was behind the incursions.
or "...with unauthorized account usage on AWS." I get that the unauthorized use was mining, but the mining operation itself isn't unauthorized by Amazon nor by the creator of the currency.
I know there are a few code-quality bots on Github, but is there any service that you can install as a webhook which automatically checks for things like Amazon key pairs (which, IIRC, always start with "AKIA", at least the API keys anyway)?
All AWS keys I've seen start with 'AKIA'. I am assuming that they have bots that search Github and other search engines for access keys. At that point it is easy for them to tie them back to an account and notify the user.
Well, kudos to them for doing that, at least. Of course it's awful that you could be out ~$3k, but imagine how bad it could have been if they hadn't been so proactive.
"Having a poke around confirmed what I had already guessed. The unauthorised user had been mining litecoin with the mining pool pool-x.eu."
Hmmm..you already guessed someone hacked your account to mine litecoin? Astroturfing much? That's the last thing I would have guessed. I would have thought someone was using it as some crazy web server or mail server to generate spam or phony websites for bogus ad clicks.
A cc2.8xlarge is reported to mine at 85 kh/s, so 20 of them would give you 1700 kh/s. That's roughly equivalent to a couple of high-end AMD GPUs (say a couple of overclocked 290x). This hashing power gives you a little over 0.5 LTC per day. It mined for two days, so it gained a little over 1 LTC. Let's call it $40.
That's right, the idiot behind this cost the OP $3000+ for $40 profit. A smarter criminal would have spawn GPU instances on EC2.