I wish _I_ had a business that was successful enough to justify multiple engineers working 7 months on porting our infrastructure from heroku to kubernetes
Knowing the prices and performance of Heroku (as a former customer) the effort probably paid for itself. Heroku is great for getting started but becomes untenably expensive very fast, and it's neither easy nor straightforward to break the vendor lock in when you decide to leave.
I find AWS ECS with fargate to be a nice middle ground. You still have to deal with IAM, networking, etc. but once you get that sorted it’s quite easy to auto-scale a container and make it highly available.
I’ve used kubernetes as well in the past and it certainly can do the job, but ECS is my go-to currently for a new project. Kubernetes may be better for more complex scenarios, but for a new project or startup I think having a need for kubernetes vs. something simpler like ECS would tend to indicate questionable architecture choices.
ECS is far, far far smoother, simpler and stable than anything else out there in cluster orchestration. It just works. Even with EC2 instances it just works. And if you opt for Fargate, then that's far more stable option.
I am saying this after bootstrapping k8s and ECS both.
It works, but the way it’s not part of fargate, and instead some combination of cloudwatch events and rules modifying the ‘desiredCount’ property on the service.
Just feel like it could all be done in a slightly more integrated way.
How does it compare with fly.io? Last I checked, startup time is still in minutes instead of less than a second on fly, but I presume it's more reliable and you get that "nobody ever got fired for using AWS" effect
Fly is really cool and it's definitely an extremely quick way to get a container running in the cloud. I haven't used it in production so I can't speak to reliability, but for me the main thing that stops me from seriously considering it is the lack of an RDS equivalent that runs behind the firewall.
assume a dev is $100k/year... so $200k with taxes, benes, etc. That's 16,666/month, at 1.5 months is 25k. So it'll take 3.5 months to break even. And they'd save around .8 of their pay, or .4 of their total cost a year...
Generally I am hoping my devs are working a good multiplier to their pay for revenue they generate. Not sure I'd use them this way if there was other things to do.
Where are you finding capable DevOps engineers for 100k total comp? It’s hard to find someone with the skills to rebuild a SaaS production stack who’s willing to work for that little around here!
I'm picking a random salary that's not too high that lower comp countries/industries will reject, and not too low that higher comp countries / industries will reject, and then doing math on those.
You can then take my numbers or math and plug in YOUR comp rates. But the TL;DR I've seen is many people never even do napkin math like this on ROI.
Now consider that some places are not in Silly Valley, or not even in USA, and the fully loaded cost of engineer (who, once done with on-prem or at least more "owned" stack, can take on other problems) can be way, way lower
These numbers are actually WAY LOW for silicon valley. If I was doing the ROI for an Amazon employee I'd start with around 350 for an SDE all told for entry level and half a mill for one with a few years experience.
But they're also way high for other places. And just right.
The point is how to do the math not the ballpark. Also that even at 100k for a dev it's maybe a wash depending on your time horizon.
My experience is that a lot of "simpler alternatives" ballooned costs beyond cost of someone to wrangle the more complex solution - and well, after initial pains, the workload drops so you can have them tackle other problems if not at full time.
Or as I said it few times at meetups, Heroku is what I use when I want to go bankrupt (that was before Heroku got sold)
I mean, $7,000 a month isn’t nothing. But it’s not a lot. Certainly not enough to justify a seven month engineering effort plus infinite ongoing maintenance.
This is $7k/mo today. If they are actively growing, and their demand for compute is slated to grow 5x or even 10x in a year, they wanted to get off Heroku fast.
That “main engineering effort” will go on forever. People neglect to note that everything is constantly changing. Just like the roof on your house, if you don’t upgrade your components regularly, eventually you will face a huge rewrite when that thing your ancient home-made infrastructure relies on is no longer supported or is no longer updated to support the latest thing you need for your SaaS.
You can’t avoid this cost. Some people refer to it as technical debt, but I think more accurately it could be called “infrastructure debt”. Platform providers maintain the infrastructure debt for you - this is what you pay them for. And they do it with tremendous economies of scale. Unless your scale is truly enormous - like Meta, for instance - it isn’t worth build your own infrastructure.
(Except for Postgres, since Fly's solution isn't managed)
Heroku's price is a persistent annoyance for every startup that uses it.
Rebuilding Heroku's stack is an attractive problem (evidenced by the graveyard of Heroku clones on Github). There's a clear KPI ($), Salesforce's pricing feels wrong out of principle, and engineering is all about efficiency!
Unfortunately, it's also an iceberg problem. And while infrastructure is not "hard" in the comp-sci sense, custom infra always creates work when your time would be better spent elsewhere.
> Salesforce's pricing feels wrong out of principle
What do you mean exactly? If it takes multiple engineers multiple months to build an alternative on kubernetes, then it sounds like Heroku is worth it to a lot of companies. These costs are very "known" when you start using Heroku too, it's not like Salesforce hides everything from you then jump scares you 18 months down the line.
SF's CRM is also known to be expensive, and yet it's extremely widely used. Something being expensive definitely doesn't always mean it's bad and you should cheap out to avoid it.
Couldn't you move to AWS? They offer managed Postgresql. Heroku already runs on AWS, so there could be a potential saving in running AWS managed service.
I moved our entire stack from Heroku to Render in a day and pay 1/3 less. Render is what Heroku would be if they never stopped innovating. Now I’m thinking of moving to fly as they are even cheaper.
If you use containers. If you're big enough for the cost savings to matter, you're probably also not looking for a service like Render or Fly. If your workload is really "just containers" you can save more with even managed container services from AWS or GCP.
We are talking about moving from Heroku, I don't think being too needy for the likes of Fly is at all a given. (And people will way prematurely think they're too big or needy for x.)
- Your current cloud / PaaS costs are north of $5,000/month
- You have at least two developers who are into the idea of running Kubernetes and their own infrastructure and are willing to spend some time learning how to do so
So you will spend 150k+/year (2 senior full stake eng salaries in EU - can be much higher, esp for people up to the task) to save 60k+/y in infra costs?
Does not compute for me - is the lock-in that bad?
I understand it for very small/simple use cases - but then do you need k8s at all?
It feels like the ones who will benefit the most is orgs who spend much more on cloud costs - but they need SLAs, compliance and a dozen other enterprisy things.
So I struggle to understand who would benefit from this stack reclaim.
The idea that we're implying you need 2 full time engineers is a misunderstanding. We just mean to say that you'll want at least 2 developers to spend enough time digging in to Kubernetes etc to have a good enough idea of what you're doing. I don't think more than 2 month of messing about should be required to reach proficiency.
We currently don't spend more than ~4 days per month total working on platform related stuff (often we spend 0 days, eg. I was on parental leave during 3 months and no one touched the platform during that time).
WRT employee cost, Swedish DevOps engineers cost less than half of what you mentioned on average, but I guess YMMV depending on region.
Not to be a pedantic asshole, but those guidelines don't mention italicizing as emphasis, just that * causes italicizes. In fact the OP should probably say that they believe "HN users use italicization to emphasize," which again, who's "we?" _This_ style of emphasis, as others have mentioned, has been bouncing around IRC and whatnot forever.
