Cloud computing seems to be a winner take all scenario. For example, if you use AWS and need a message broker service then you'll use this. If you use Azure, you'll use their version. Development seems like just hooking up this components. I can't tell if this is a good thing or a bad thing.
It’s a bad thing - whenever you cannot make a choice you become a slave to tyranny. Infra lock-in means you’re a slave to their whims (financial, legal, competitivenes, whatever)
This is justified by “less maintenance, and easier deployment” but the reality of the situation is, it’s not worth giving your freedom up for, and to a lesser degree - if your platform becomes popular, you end up spending the same amount of time tweaking and optimising to match the idiosyncrasies of their implementation anyway.
But the most important part is vendor lock-in, it’s bad.
How is using AWS or Azure's managed RabbitMQ service lock-in? You can easily switch to someone else's managed service, or roll your own, since it's still just good old RabbitMQ.
How many cases are you truely not locked into anything? Even if you host your own RabbitMQ you're still coupled to the software. I'm not sure being coupled to cloud service X is any worse than OSS product Y. For the latter, you tend to need to have more expertise to run it yourself.
At the end of the day, you can still rewrite your code and switch in both cases. You can end up in a tough spot if the OSS community loses interest in the software you've already bet your complicated app on, as well
AWS has sqs & kinesis which are much better queueing options in that scenario where you’re in AWS doing new dev and can pick a technology. This is more likely to do with opening doors for large and complex applications that can’t be rewritten to come into the AWS cloud. Unsexy stuff but there’s some fun engineering to be done in that realm, if you find like puzzles and shitshows to be fun anyways :-)
how are those much better? if you use SQS and you write your code for it, then you are stuck on a proprietary platform. Also, SQS is super-basic and actually requires a bunch of code to do anything beyond trivial - although yes, it seems reliable and well-supported, at least from my experience. I was actually really waiting for AWS to support Rabbit since it seems to hit the right combo of features, usability and platform independence for me, and it looked friendlier than ActiveMQ.
If you're using a decent framework there's a good chance it already does most of the work for you. With Ruby on Rails, there's the ActiveJob abstraction which you can hook up to different backends like SQS or Redis with a few lines of code. In addition, AWS has a lot of out of the box integration with SNS and SQS for other services like Cloudwatch and Lambda (in fact, lambda you don't need any special code).
If you have a Lambda function processing SQS messages they just get dumped in your handler method and it your function runs successfully they get automatically removed from the q. If your lambda fails, the message reappears after the visibility timeout out subject to your redrive policy
I'm referring to things like topics, multiple consumers, routing, etc etc that are not even possible with SQS and once you grow into a need for those, SQS stops being adequate no matter what library you use for it.
They aren't. They're technically more correct but not always the practical best choice.
RabbitMQ is a smart play as Rabbit is very easy to use, understand, and troubleshoot at the low end (which is where I suspect the vast majority of queue systems live).
It also has a feature which is actually really hard to do (and sqs doesn't do). Guaranteed delivery of a message once.
That was THE reason we never migrated to SQS, there are scenarios where SQS can double deliver. Our codebase was built up from nothing over time and couldn't gracefully handle double delivery of messages in all scenarios. We could have refactored, but it wasn't worth the work when we were already doing a half billion in revenue without getting even close to the limitations of rabbit AND were close to selling (which we ultimately did).
AWS is great at selling multiple slight variations of the same product. If you look you can usually find ONE variation that works for you. The real test will be if the billing isn't garbage (garbage billing is why we didn't use their other AMQP service and part of the reason why we don't use things like EKS or Managed SFTP despite having the need).
That flies in the face of my distributed systems knowledge. It's not possible in some failure cases.
If your acknowledgement of a message gets lost (because either server involved or the pipes in-between fail) you've processed the message already but the queue server will think you haven't. It either has to resend it (duplicate delivery) or it ignores acknowledgements all together (drops messages that it sent you, but you didn't process - maybe because your server failed.) So the choice when there is a failure in the system is between at least once or at most once - exactly once cannot be guaranteed.
You are correct, a better description is that their path to 'deliver exactly once to the best of your ability' is clearer.
If I remember correctly SQS is hard limited to a fairly short timeout to requeue messages delivered but not acked. In rabbit it's much more configurable.
Also regular rabbit hosts support the kludge pattern of, 'just run one host and accept if it goes poof you can lose messages,' which is useful if you don't want to bother with the complexity of clustering or are on a shoe string budget.
Lastly you get a nice user interface with the management plugin and you can stand it up locally with docker compose (without depending on AWS for dev or any of the 'aws but on your laptop' solutions).
They’ve been around for quite some time so they have a much wider customer base & bigger teams supporting them. AWS services can be a bit choppy in the beginning so imo, especially with queues, I’d wait for it to bake.
Nope. If only. There are four considerations to look at which are rarely if ever mentioned between the mountains of well organised hype:
The first is application complexity. A lot of real workloads are quite complex but do not need to scale out a lot. The cloud and all the hype is organised around simple workloads that need to scale out easily which is an easy win. So for example Netflix or a SaaS application with a few tens of endpoints and a React front end or something. The wide sprawling real businesses are a terrible fit and tend to get rather expensive rather quickly when you start putting their workloads into the cloud. There is marginal aggregate cost benefit over actually buying hardware ($4m a year SQL server clusters are a reality in the cloud), the real benefit being only agility.
The second is simply "hooking up components" sounds really easy. But it's not. I think perhaps 50% of my time is working out why X won't talk to Y or why Z is broken and finding some opaque abstraction which doesn't allow me to get to the bottom of the problem. It's very very easy to turn your deployment into a complete tangle of chaos and circular dependencies which are very hard to rationalise and automate even with state of the art automation tools (which I will say tend to melt in your hands). This is existing layering on top of the same concerns you had before rather than a different one.
Thirdly we have to work out the difference between mature products and hype. Nearly all solutions are described in little blog snippets that make things look really easy for a specific and narrow use case but realistically things are really fucking complicated and in some cases absolutely awfully described in documentation. In a lot of cases, including AWS, it's actually hard to find someone at the cloud vendor who knows how something works when you break it. And sometimes there are solutions which are just absolutely dire. Again pointing the finger here at Amazon's managed ElasticSearch.
Fourthly, you end up being perpetual bean counter afraid of the rube goldberg machine waking in the middle of the night due to some event you didn't anticipate and drinking the content of your credit card in a few minutes. Some of the cost management and spot instance management software automates this rather nicely into a whole cluster of new failure modes as well just as if the complexity wasn't enough already. A trite version of this is "saving money costs money and sometimes the benefits are less than the costs"
So what you end up doing is trading your original problems for a set of new and shiny ones which are possibly even more complicated.
But at least you only have one vendor to shout at, which is a net win if you've ever tried to get HPE and Cisco to work out what fucked up mess is going on between their two lumps of iron.
I digress but be careful with assumptions about it being magical unicorns. They poop and you have to shovel it.
I beg to differ on your second point - at my company we've fully embraced AWS and putting vendor lock-in issues aside, the end result is focusing more on the application and less on the minutiae of operational issues which is a big win. This makes things much more interesting since you can get there faster and consequently take on more impactful projects in the same timeframe. This in general is a boon for developers in my experience.
Somehow I don't think physical laborers ever complain when they get new and more powerful tools to make their job easier.
It's only software engineers that bemoan their lives getting easier, so they can spend more time working on other problems higher up the abstraction chain.
The 'problems higher up the abstraction chain' are the ones that are closer to labour, or factory work. Mundane and repetitive, relatively speaking easy - requiring less thought and being to die extent trainable as working within a pattern/template.
Longterm isn't this bad? For example, if you can do it with 5 years experience. At 15 years experience you'll likely be too expensive for companies to want to hire you. They'll just hire more junior people.
Nah, they support so many platforms and AWS is only one of them. I also think their Heroku business might actually benefit from this. Now they don't even have to manage the servers themselves -- they can essentially re-sell AWS's service with custom support if they wanted and charge customers to move the cost to Heroku's bill rather than AWS's (you'd be surprised how many do this).
> Reselling AWS's offerings would cost more than running it on bare VM's, I'm sure they would be able to compete on price with them.
True -- I do think passing on the cost and taking a tiny margin with drastically reduced maintenance cost could be an attractive business model at scale though.
> It also doesn't make sense to rewrite their current software which is probably abstracted for multi-cloud to support re-selling.
I have no idea what their current software looks like, do you have any inside knowledge?
If they have abstracted, then they probably have multiple implementations of a similar API -- this is just changing one of them (or maybe even cloning it to reduce possibility of breakage). This might be as simple as just changing the AWS-specific provisioner to call out to AmazonMQ instead of EC2, or changing some code that generates terraform/pulumi scripts.
One thing I think they'd have to deal with is the fact that they support custom plugins that AmazonMQ may not.
From the FAANG, Amazon strikes me as the one using most open source code while at the same time not having much to show as open source - firecracker is a (relative) toy.
They must be laughing their sock off at the tops of FAANG: thousands upon thousands commit millions of hours to open source which they then run for a nice profit.
This is because reliability, ease of setup, and support are actually just as important as what is being run.
If you're not using AWS you're not fully leveraging your software team, because that means you've got people spending time building and supporting these sorts of internal systems. That should only be done when you reach large scale, at which point people have leveraged other AWS synergies making it harder to exit the platform.
I really doubt you have any real world experience with AWS if you try to sell it as a less complex alternative to anything,really. AWS is perhaps the most complex and arcane service provider ever, and is progressively getting worse by the way the service keeps growing and changing.
And no, learning a specific AWS service is not a solid career investment. Tell that to anyone who tried to learn CloudFormation and then SAM and now forget everything because AWS pushes you to use CDK.
> If you're not using AWS you're not fully leveraging your software team, because that means you've got people spending time building and supporting these sorts of internal systems.
This is simply not true at all, and flies in the face of real world usage.
The only concrete and objective selling point of AWS is it's global coverage of data centers, and the infrastructure they have in place aimed at delivering reliable global-scale web services.
The problem is that the companies who actually operate at such a scale and with such tight operational requirements can be counted with your fingers. That count then drops down to a fraction once you start to do a cost/benefit analysis.
The rest of the world is quite honestly engaged in cargo cult software development.
And no, doing AWS is not simpler nor more efficient. You might launch an EC2 instance with a couple of clicks, but to navigate a service designed with global scale and multiple levels of redundancy across the same service and with tight integration and dependency across half a dozen AWS offers which may or may not be redundant or competing... No, that is not simple or allows for any type of time efficiency.
Hell, with AWS you do not learn how to manage or operate Infrastructure. With AWS you learn the AWS dashboard,and learn pavlov reactions to which button you press if you hear an alarm. You never fully grasp the impact or the reaction of pressing a button, and you have absolutely no idea what impact that click will have on your monthly bill.
In contrast, if you need to run microservices chatting through a message broker then your system on OVH or Hetzner or any other barebones system will be comprised of a bunch of nodes where one of them runs RabbitMQ and everyone else points to the RabbitMQ node. You can get everything running from scratch on a cluster managed by Docker Swarm in about 15 to 20 minutes. In the end you have a far simpler service running for a fraction of the cost and ina far more manageable environment.
AWS is resume-driven development fueled by cargo cult development.
Do they need to? The point of FOSS isn't to get freebies from big corporations. It's so that software remains in the hands of the users. If Amazon doesn't contribute back, or if they fork it, that's fine, because RabbitMQ will always be there.
It’s (literally) Google’s crosvm with some changes. AWS forked an existing codebase, made some changes that mostly consisted of removing functionality to tailor the VMM for their use case, and then made a big PR push about “open sourcing” Firecracker as though it was a project they built from scratch. The announcement devoted 1 sentence to the fact that it’s a crosvm fork, and that’s not even the only thing covered in that one sentence.
Toy in the sense that it’s not a significant contribution to the open source community.
Edit: In comparison to the scale of Amazon and the scale of contribution of other similarly-sized tech companies. Firecracker would rank as a more major contribution in my book if it wasn’t a cut-down fork of a pre-existing (and still active!) project.
Define significant? If you're expecting every open source software user to push a million commits before they can be called "contributors" apart from Microsoft/Facebook I dont see any significant contributors.
The context up thread is that Amazon contributes a remarkably small amount to open source compared to the amount of code that Amazon produces and the amount of open source projects that Amazon depends on. Also I find it interesting that you don’t think that Microsoft and Facebook are comparable to Amazon.
After watching a presentation on firecracker from Amazon team it seemed like it is more of a playground for their junior developers than some high profile library. Not trying to be dismissive, just an impression.
Offtopic: we are evaluating rabbit at the moment vs activemq; rabbit won for one detail: we need delayed messages (ex. publish {...} in 2h) and activemq seems to support those clustered while rabbit only on one node, which does not fit the business case (we cannot lose messages). I worked with rabbitmq before and it was great but this seems indeed an issue. Someone here with some insights?
Hopefully I'm not misunderstanding your use case, but I would think you could accomplish this by publishing a durable message to a highly available, durable queue with no consumers, setting a TTL on the message with a policy to publish to a "dead letter exchange" that fronts the queue or queues with your eventual consumers. The durability flags ensure both message and queue survive a restart, while the high availability policy on the queue ensures that each node in the cluster has a copy. And I'm sure there are a few similar patterns that would garner the same desired behavior.
I'm not taking a position on whether this is an acceptable level of complexity for the desired feature, of course, just pointing out how one might accomplish it if Rabbit is otherwise desirable.
Disclosure: I work at AWS and this is my personal opinion.
I've seen growing levels of AWS contribution back to upstream projects over the past four years. Teams start out by operating a piece of software at scale, whether it is Redis, Kubernetes, etc. After they have operated it for a while they discover the bugs or performance issues, or customers of the service complain about something. At that point the team now has enough real world experience with that software to begin to contribute back to upstream.
It takes time: to learn the ins and outs of the software well enough to know where and what improvements should be made, to understand the software's design and history well enough not to make bad suggestions or contributions that were already determined to be dead ends in the past, and to earn the approval of the community and existing maintainers enough to get significant contributions accepted in the first place.
They have quite a number of commits from various authors to the Linux kernel at least:
linux (master=) $ git shortlog -ns --author amazon
79 Arthur Kiyanovski
76 Gal Pressman
59 David Woodhouse
51 Sameeh Jubran
45 Netanel Belgazal
38 KarimAllah Ahmed
35 SeongJae Park
30 Jan H. Schönherr
26 Frank van der Linden
18 Andra Paraschiv
12 Paul Durrant
11 Talel Shenhar
10 Shay Agroskin
...
Is it bold? If a group of people put something together and release the details for free, they have to expect others will profit off of it. This is the fate of all useful open source software that doesn't have some kind of no-commercial clause (which potentially moves it outside the ideals of open source as it restricts the use of the source code).
Not really. This has been going on for decades. Examples: early web hosts running the LAMP stack: Linux/Apache/MySQL/PHP. Eventually people put an admin UI on top (control panels, like Plesk and cPanel.) The core functionality is all free software. The value is not having to configure and manage all that stuff yourself.
What a bizarre post! Seems really strange to me to see such a 'toys out of the pram' reaction piece like that from an organisation as big and corporate as AWS.
That it feels the need to respond like that makes me see it in worse light over the matter rather than better.
That's true, but a lot of software standardised on RabbitMQ specifically, including its client libraries. That means there's still a lot of demand for AMQP 0.9.1 over 1.0. (RabbitMQ does have a plugin for AMQP 1.0, however.)
