The biggest advantage of this is for the llm providers like OAI not application developers.
LLM's are brains with no tools (no hands, legs, etc).
When we use tool calling we use them to empower the brain. But using normal API's the language model providers like OpenAI have no access to those tools.
With MCP they do. The brain they create can now have access to a lot of tools that the community builds directly _from_ llm, not through the apps.
This is here to make ChatGPT/Claude/etc _the gateway_ to AI rather than them just being API providers for other apps.
This looks very neat. In our company, for the past 10 years I've bet heavily on Postgres have not allowed any other databases to be used. Stuff like this makes me hope I can continue to do so as our database is growing.
But, it appears that we are hitting our limits at this point. We have a table with tens of millions of rows and the use case is that it's faceted search.
Something like Zillow, with tends of millions of listings and tens of columns that can be queried. I'm having a tough time scaling it up. Does anyone have experience building something like that in pg?
One end is PaaS like Heroku, where you just git push.
The other end is bare metal hosting.
Every option you mentioned (VPS, Manages K8S, Self Hosted K8S, etc) they all fall somewhere between these two ends of the spectrum.
If, a developer falls into any of these "groups" or has a preference/position on any of these solutions, they are just called juniors.
Where you end up in this spectrum is a matter of cost benefit. Nothing else. And that calculation always changes.
Those options only make sense where the cost of someone else managing it for you for a small premium gets higher than the opportunity/labor cost of you doing it yourself.
So, as a business, you _should_ not have a preference to stick to. You should probably start with PaaS, and as you grow, if PaaS costs get too high, slowly graduate into more self-managed things.
A company like fly.io is a PaaS. Their audience has always been, and will always be application developers who prefer to do nothing low-level. How did they forget this?
> Where you end up in this spectrum is a matter of cost benefit. Nothing else. And that calculation always changes.
This is where I see things too. When you start out, all your value comes from working on your core problem.
eg: You'd be crazy to start a CRM software business by building your own physical datacenter. It makes sense to use a PaaS that abstracts as much away as possible for you so you can focus on the actual thing that generates value.
As you grow, the high abstraction PaaS gets increasingly expensive, and at some point bubbles up to where it's the most valuable thing to work on. This typically means moving down a layer or two. Then you go back to improving your actual software.
You go through this a bunch of times, and over time grow teams dedicated to this work. Given enough time and continuous growth, it should eventually make sense to run your own data centers, or even build your own silicon, but of course very few companies get to that level. Instead most settle somewhere in the vast spectrum of the middle, with a mix of different services/components all done at different levels of abstraction.
You’re correct that it would be absurd to build a DC, but you left out the next-best thing, and the one that is VERY financially attractive: colo’ing. I can rent 1U for around $50-75/month, or if I want HA-ish (same rack in the same DC isn’t exactly HA, but it solves for hardware failure anyway), 5U would probably run $200-250/month or so, and that lets you run two nodes with HAProxy or what-have-you, sharing a virtual IP, fronting three worker nodes running K8s, or a Proxmox cluster, or whatever. The hardware is also stupidly cheap, because you don’t need anything remotely close to new, so for about $200/node, you’ll have more cores and memory than you know what to do with.
The DC will handle physical service for you if something breaks, you just pay for parts and labor.
All of this requires knowledge, of course, but it’s hardly an impossible task. Go look at what the more serious folk in r/homelab (or r/datacenter) are up to; it’ll surprise you.
For $200/month, I can have 2 ALBs, 2 ECS services, 2 CloudWatch log groups, and 2 RDS instances on AWS (one each for dev and prod) and a GitHub Team account with enough included runner minutes to cover most deployments. A colo is going to be more hassle, and I'll have to monitor more things (like system upgrades and intrusion attempts). I'd also have to amortize parts and labor as part of the cost, which is going to push the price up. If I need all that capacity, then the colo is definitely the better bet. But if I don't, and a small shop usually doesn't, then managed infrastructure is going to be preferable.
Pretty much. I don't really see the problem though. You're also getting regular snapshots, which is yet another thing you have to build in that colo setup (and where are your backups going?). This is not for personal projects or shoestring-budget non-profits where you're willing to volunteer time, it's for businesses and decent paying work, where two hundred dollars is one man-hour of totally compensated labor.
A Postgres `db.m6g.large` (the cheapest non-burstable instance) runs $114/month for a single AZ, and that's not counting storage or bandwidth. A `db.t4g.medium` runs $47/month, again, not counting storage or bandwidth. An ALB that somehow only consumed a single LCU per month would run $22. The rest of the mentioned items will vary wildly depending on the application, but between those and bandwidth - not to mention GitHub's fees - I sincerely doubt you'd come in anywhere close to $200. $300 maybe, but as the sibling comment mentioned, the instances you'll have will be puny in comparison.
> I'll have to monitor more things (like system upgrades and intrusion attempts)
You very much should be monitoring / managing those things on AWS as well. For system upgrades, `unattended-upgrades` can keep security patches (or anything else if you'd like, but I wouldn't recommend that unless you have a canary instance) up to date for you. For kernel upgrades, historically it's reboots, though there have been a smattering of live update tools like kSplice, kGraft, and the latest addition from GEICO of all places, tuxtape [0].
> I'd also have to amortize parts and labor as part of the cost, which is going to push the price up.
Given the prices you laid out for AWS, it's not multi-AZ, but even single-AZ can of course failover with downtime. So I'll say you get 2U, with two individual servers, DBs either doing logical replication w/ failover, or something like DRBD [1] to present the two servers' storage as a single block device (you'd still need a failover mechanism for the DBs). So $400 for two 1U servers, and maybe $150/month at most for colo space. Even with the (IMO unrealistically low) $200/month quote for AWS, at 5 months, you're now saving $50/month. Re: parts and labor, luckily, parts for old servers is incredibly cheap. PC3-12800R 16GiB sticks are $10-12. CPUs are also stupidly cheap. Assuming Ivy Bridge era (yes, this is old, yes, it's still plenty fast for nearly any web app), even the fastest available (E5-2697v2) is $50 for a matched pair.
I don't say all of this just guessing; I run 3x Dell R620s along with 2x Supermicros in my homelab. My uptime for services is better than most places I've worked at (of course, I'm the only one doing work, I get that). They run 24/7/365, and in the ~5 years or so I've had these, the only trouble the Dells have given me is one bad PSU (each server has redundant PSUs, so no big deal), and a couple of bad sticks of RAM. One Supermicro has been slightly less reliable but to be fair, a. it has a hodgepodge of parts b. I modded its BIOS to allow NVMe booting, so it's not entirely SM's fault.
EDIT: re: backups in your other comment, run ZFS as your filesystem (for a variety of reasons), periodically snapshot, and then send those off-site to any number of block storage providers. Keep the last few days, with increasing granularity as you approach today, on the servers as well. If you need to roll back, it's incredibly fast to do so.
I upvoted for the details, and I agree that if you try to buy comparable capacity in AWS, it's going to be more expensive. Scaling up in AWS is definitely going to cost more over time too. I don't want to hide these facts.
But you don't need comparable capacity, at least not at first. And when you do, you click some buttons or run terraform plan/apply. Absolutely it's going to cost more measured only by tech specs. But you're not paying primarily for tech specs, you're paying for somebody else to do the work. That's where the cost comparability really needs to be assessed.
Security in AWS is a thorny topic, I'll agree, but the risks are a little different. You need to secure your accounts and users, and lock out unneeded services while monitoring for unexpected service utilization. Honestly, I think for what you're paying, AWS should be doing more for you here (and they are improving albeit slowly). Hence maybe the real point of comparison ought to be against PaaS because then all of that is out of scope too, and I think such offerings are already putting pressure on AWS to offer more value.
> But you don't need comparable capacity, at least not at first.
Agreed.
> But you're not paying primarily for tech specs, you're paying for somebody else to do the work.
...
Honestly, I think for what you're paying, AWS should be doing more for you here
Also agreed, and this is why I don't think the value proposition exists.
We can agree to disagree on which approach is better; I doubt there's an objective truth to be had.
> A Postgres `db.m6g.large` (the cheapest non-burstable instance) runs $114/month for a single AZ, and that's not counting storage or bandwidth. A `db.t4g.medium` runs $47/month, again, not counting storage or bandwidth.
This is why numbers do not stack up in the calculations – the premise that the DB has to be provisioned is not the correct one to start off with.
The right way of cooking RDS in AWS is to go serverless from the start and configure the number of DCU's, e.g. 1 to N. That way it will be even cheaper than the originally quoted $200.
Generally speaking, there is absolutely no need for anything to be provisioned at a fixed compute capacity in AWS unless there is a very specific or an edge case that, likewise, warrants a provisioned instance of something.
> The right way of cooking RDS in AWS is to go serverless from the start
Nitpick, but there is no Serverless for RDS, only Aurora. The two are wildly different in their architecture and performance characteristics. Then there's RDS Multi-AZ Cluster, which is about as confusingly named as they could manage, but I digress.
Let's take your stated Minimum ACU of 1 as an example. That gives you 2 GiB of RAM, with "CPU and networking similar to what is available in provisioned Aurora instances." Since I can't find anything more specific, I'll compare it to a `t4g.small`, which has 2 vCPU (since it's ARM, it's actual cores, not threads), and 0.128 / 5.0 Gbps [0] baseline/burst network bandwidth, which is 8 / 625 MBps. That burst is best-effort, and also only lasts for 5 – 60 minutes [1] "depending on instance size." Since this is tiny, I'm going to assume the low end of that scale. Also, since this is Aurora, we have to account for both [2] client <--> DB and DB-compute (each node, if more than one) <--> DB-storage bandwidth. Aurora Serverless v2 is $0.12/hour, or $87.60/month, plus storage, bandwidth, and I/O costs.
So we have a Postgres-compatible DB with 2 CPUs, 2 GiB of RAM, and 64 Mbps of baseline network bandwidth that's shared between application queries and the cluster volume. Since Aurora doesn't use the OS page cache, its `shared_buffers` will be set to ~75% of RAM, or 1.5 GiB. Memory will also be consumed by the various processes, like the WAL writer, background writer, auto-vacuum daemon, and of course, each connection spawns a process. For the latter reason, unless you're operating at toy scale (single-digit connections at any given time), you need some kind of connection pooler with Postgres. Keeping in the spirit of letting AWS do everything, they have RDS Proxy, which despite the name, also works with Aurora. That's $0.015/ACU-hour, with a minimum 8 ACUs for Aurora Serverless, or $87.60/month.
Now, you could of course just let Aurora scale up in response to network utilization, and skip RDS Proxy. You'll eventually bottleneck / it won't make any financial sense, but you could. I have no idea how to model that pricing, since it depends on so many factors.
I went on about network bandwidth so much because it catches people by surprise, especially with Aurora, and doubly so with Postgres for many services. The reason is its WAL amplification from full page writes [3]. If you have a UUIDv4 (or anything else non-k-sortable) PK, the B+tree is getting thrashed constantly, leading to slower performance on reads and writes. Aurora doesn't suffer from the full page writes problem (that's still worth reading about and understanding), but it does still have the same problems with index thrashing, and it also has the same issues as Postgres with Heap-Only Tuple updates [4]. Unless you've carefully designed your schema around this, it's going to impact you, and you'll have more network traffic than you expected. Add to that dev's love of chucking everything into JSON[B] columns, and the tuples are going to be quite large.
Anyway, I threw together an estimate [5] with just Aurora (1 ACU, no RDS Proxy, modest I/O), 2x ALBs with an absurdly low consumption, and 2x ECS tasks. It came out to $232.52/month.
I don't know much about Aurora, there was/is a little too much magic there for my taste, but I feel like once we start with "Postgres-compatible DB", we can't necessarily reason about how things perform under the hood in terms of ordinary Postgres servers. Is there a detailed breakdown of Aurora and its performance/architecture out there? My experience is that AWS is cagey about the details to maintain competitive advantage.
There are some details available[0], although they are scarce.
I procured my Aurora intel from a lengthy phone conversation with an exceptionally knowledgeable (and excessively talkative) AWS engineer – who had worked on Aurora – several years ago. The engineer provided detailed explanations of Aurora’s architecture, do's, and dont's as part of our engagement with AWS. The engineer was very proud of AWS’ accomplishments (and I concur that their «something serverless» products are remarkable engineering feats as well as significant cost-saving solutions for me and my clients). The engineer was willing to share many non-sensitive technical details. Generally speaking, a sound understanding of distributed architectures and networks should be sufficient to grasp Aurora Serverless. The actual secret sauce lies in the fine-tuning and optimisations.
You can find some re:Invent talks about it, but the level of depth may not be what you want.
The tl;dr is they built a distributed storage system that is split across 3 AZs, each with 2 storage nodes. Storage is allocated in 10 GiB chunks, called protection groups (perhaps borrowing from Ceph’s placement group terminology), with each of these being replicated 6x across the nodes in AZs as mentioned. 4/6 are required for quorum. Since readers are all reading from the same volume, replica lag is typically minimal. Finally, there are fewer / no (not positive honestly; I have more experience with MySQL-compatible Aurora) checkpoints and full page writes.
If you’ve used a networked file system with synchronous writes, you’ll know that it’s slow. This is of course exacerbated with a distributed system requiring 4/6 nodes to ack. To work around this, Aurora has “temporary local storage” on each node, which is a fixed size proportional to the instance size. This is used for sorts that spill to disk, and building secondary indices. This has the nasty side effect that if your table is too large for the local storage, you can’t build new indices, period. AWS will tell you “upsize the instance,” but IMO it’s extremely disingenuous to tout the ability for 128 TiB volumes without mentioning that if a single table gets too big, your schema becomes essentially fixed in place.
Similarly, MySQL normally has something called a change buffer that it uses for updating secondary indices during writes. Can’t have that with Aurora’s architecture, so Aurora MySQL has to write through to the cluster volume, which is slow.
AWS claims that Aurora is anywhere from 3-5x faster than the vanilla versions of the respective DBs, but I have never found this to be true. I’ve also had the goalposts shifted when arguing this point, with them saying “it’s faster under heavy write contention,” but again, I have not found this to be true in practice. You can’t get around data locality. EBS is already networked storage; requiring 4/6 quorum across 3 physically distant AZs makes it even worse.
The 64 TiB limit of RDS is completely arbitrary AFAIK, and is purely to differentiate Aurora. Also, if you have a DB where you need that, and you don’t have a DB expert on staff, you’re gonna have a bad time.
Thanks for the correction, it is Aurora, indeed, although it can be found under the «RDS» console in AWS.
Aurora is actually not a database but is a scalable storage layer that operates over the network and is decoupled from the query engine (compute). The architecture has been used to implement vastly different query engines on top of it (PgSQL, MySQL, DocumentDB – a MongoDB alternative, and Neptune – a property graph database / triple store).
The closest abstraction I can think of to describe Aurora is a VAX/VMS cluster – where the consumer sees a single entity, regardless of size, whilst the scaling (out or back in) remains entirely opaque.
Aurora does not support RDS Proxy for PostgreSQL or its equivalents for other query engine types because it addresses cluster access through cluster endpoints. There are two types of endpoints: one for read-only queries («reader endpoints» in Aurora parlance) and one for read-mutate queries («writer endpoint»). Aurora supports up to 15 reader endpoints, but there can be only one writer endpoint.
Reader endpoints improve the performance of non-mutating queries by distributing the load across read replicas. The default Aurora cluster endpoint always points to the writer instance. Consumers can either default to the writer endpoint for all queries or segregate non-mutating queries to reader endpoints for faster execution.
This behaviour is consistent across all supported query engines, such as PostgreSQL, Neptune, and DocumentDB.
I do not think it is correct to state that Aurora does not use the OS page cache – it does, as there is still a server with an operating system somewhere, despite the «serverless» moniker. In fact, due to its layered distributed architecture, there is now more than one OS page cache, as described in [0].
Since Aurora is only accessible over the network, it introduces unique peculiarities where the standard provisions of storage being local do not apply.
Now, onto the subject of costs. A couple of years ago, an internal client who ran provisioned RDS clusters in three environments (dev, uat, and prod) reached out to me with a request to create infrastructure clones of all three clusters. After analysing their data access patterns, peak times, and other relevant performance metrics, I figured that they did not need provisioned RDS and would benefit from Aurora Serverless instead – which is exactly what they got (unbeknownst to them, which I consider another net positive for Aurora). The dev and uat environments were configured with lower upper ACU's, whilst production had a higher upper ACU configuration, as expected.
Switching to Aurora Serverless resulted in a 30% reduction in the monthly bill for the dev and uat environments right off the bat and nearly a 50% reduction in production costs compared to a provisioned RDS cluster of the same capacity (if we use the upper ACU value as the ceiling). No code changes were required, and the transition was seamless.
Ironically, I have discovered that the AWS cost calculator consistently overestimates the projected costs, and the actual monthly costs are consistently lower. The cost calculator provides a rough estimate, which is highly useful for presenting the solution cost estimate to FinOps or executives. Unintentionally, it also offers an opportunity to revisit the same individuals later and inform them that the actual costs are lower. It is quite amusing.
> Aurora is actually not a database but is a scalable storage layer that operates over the network and is decoupled from the query engine (compute).
They call it [0] a database engine, and go on to say "Aurora includes a high-performance storage subsystem.":
> "Amazon Aurora (Aurora) is a fully managed relational database engine that's compatible with MySQL and PostgreSQL."
To your point re: part of RDS, though, they do say that it's "part of RDS."
> The architecture has been used to implement vastly different query engines on top of it (PgSQL, MySQL, DocumentDB – a MongoDB alternative, and Neptune – a property graph database / triple store).
Do you have a source for this? That's new information to me.
> Aurora does not support RDS Proxy for PostgreSQL
Yes it does [1].
> I do not think it is correct to state that Aurora does not use the OS page cache – it does
It does not [2]:
> "Conversely, in Amazon Aurora PostgreSQL, the default value [for shared_buffers] is derived from the formula SUM(DBInstanceClassMemory/12038, -50003). This difference stems from the fact that Amazon Aurora PostgreSQL does not depend on the operating system for data caching." [emphasis mine]
Even without that explicit statement, you could infer it from the fact that the default value for `effective_cache_size` in Aurora Postgres is the same as that of `shared_buffers`, the formula given above.
> Switching to Aurora Serverless resulted in a 30% reduction in the monthly bill for the dev and uat environments right off the bat
Agreed, for lower-traffic clusters you can probably realize savings by doing this. However, it's also likely that for Dev/Stage/UAT environments, you could achieve the same or greater via an EventBridge rule that starts/stops the cluster such that it isn't running overnight (assuming the company doesn't have a globally distributed workforce).
What bothers me most about Aurora's pricing model is charging for I/O. And yes, I know they have an alternative pricing model that doesn't do so (but the baseline is of course higher); it's the principal of the thing. The amortized cost of wear to disks should be baked into the price. It would be difficult for a skilled DBA with plenty of Linux experience to accurately estimate how many I/O a given query might take. In a vacuum for a cold cache, it's not that bad: estimate or look up statistics for row sizes, determine if any predicates can use an index (and if so, the correlation of the column[s]), estimate index selectivity, if any, confirm expected disk block size vs. Postgres page size, and make an educated guess. If you add any concurrent queries that may be altering the tuples you're viewing, it's now much harder. If you then add a distributed storage layer, which I assume attempts to boxcar data blocks for transmission much like EBS does, it's nearly impossible. Now try doing that if you're a "cloud native" type who hasn't the faintest idea what blktrace [3] is.
The heartburn I have is how this stifles innovation for some. The cost of experimentation is high if every person wanting to try a new idea goes down this road.
My personal AWS account is stuffed with globally distributed multi-region, multi-az, fault tolerant, hugely scalable things that rarely get used. By “rarely” I mean requests per hour or minute, not second.
The sum total CPU utilization would be negligible. And if I ran instances across the 30+ AZs I’d be broke.
The service based approach (aka event driven) has some real magic at the low end of usage where experimentation and innovation happens.
This is news to us. Our primary DX is a CLI. One of our defining features is hardware isolation. To use us, you have to manage Dockerfiles. Have you had the experience of teaching hundreds of Heroku refugees how to maintain a Dockerfile? We have had that experience. Have you ever successfully explained the distinction between "automated" Postgres and "managed" Postgres? We have not.
You're not wrong that there's a PaaS/public-cloud dividing line, and that we're at an odd place between those two things. But I mean, no, it is not the case that our audience is strictly developers who do nothing low-level. I spent months of my life getting _UDP_ working for Fly apps!
> Their audience has always been, and will always be application developers who prefer to do nothing low-level. How did they forget this?
to this:
Their audience has always been, and will always be application developers who prefer to do nothing except to build their main product.
> Our primary DX is a CLI. One of our defining features is hardware isolation. To use us, you have to manage Dockerfiles. Have you had the experience of teaching hundreds of Heroku refugees how to maintain a Dockerfile? We have had that experience. Have you ever successfully explained the distinction between "automated" Postgres and "managed" Postgres? We have not.
I'm pretty much sure an application developer in this day and age has to know all of them, yes. Just like git.
No. It is definitely not the case that the modal developer today needs to know Docker. If only! It's a huge pain point for us with the PaaS customer cohort.
Agreed. I have been amazed at how Little most developers seem to know docker now. It seemed like more people understood docker 5 to 10 years ago than they do now. I'm not sure why this regression, but it is definitely been my experience
How would you find good lawyers that have experience dealing with this stuff?
I had to sign stuff like this and the lawyers I was sent to seemed they didn't really have much relevant experience and I'm deeply uncomfortable with their assessments given how confused they looked.
You can't just use your random family lawyer. You need one who specialises in corporate law, VCs, RSUs and so on. Who that will be depends entirely on your ___location. I know a company that does this in London, but that's no use at all to this poster.
> Perhaps more importantly, this is also a strategically worrisome direction to go in, because it plays into the narrative that Big Tech is more powerful than sovereign nations.
I don't buy it. I think it can be a fantastic win for them. Announce a leave date unless the government backs down and advertise it heavily on TV and everywhere:
Your government wants us to spy on you. We are not going to comply. Call your parliament members to <blah>.
This would make them champions of privacy and every other government will know how deep their commitment is.
Yes, the request itself is required to be kept secret in a similar way to National Security Letters (NSL) in the US (when used under the Patriot Act disclosure provisions).
This assumes that most Britons oppose their government having such powers.
Which maybe they do in abstract, but when the government reframes this as, "Apple prevents us from catching child abusers", I don't think Apple would win that match.
That still presumes that as an entire voting population, Britons would choose freedom over convenience or perception of propriety. The current government there doesn't appear to reflect the former.
I don't think "people" are going to have a say in this at all. Their privacy is now between Apple and the UK government, two entities with less-than-favorable track records protecting their constituents.
The UK could have followed the EU's lead in forcing Apple to diversify their service offerings. They could have written their own consumer protection laws or leveled lawsuits about Apple's dubious privacy considerations whenever they wanted. But instead they waited until it was too late, because everyone thought; it would never happen to me, right?
I know people will call this a fatalist reaction, but Apple has fought hard to control their fate. Now you have to trust that Apple will do the right thing, while Tim Cook stands shoulder-to-shoulder with America's most radical leader in half a century. If this matters to Britons, they should vote with their wallets and ditch Apple themselves.
The odds will be massively in favour to Apple, the British government is broke, they can't afford the consequence of Apple leaving UK market, it will be a fatal blow to the chanceller's pro-business policy.
Rechat is the leading AI-powered operating platform for real estate brokers and agents. Think Shopify for real estate—a mobile-first super app with marketing automation, workflow management, and everything agents and brokers need to run their business in one place. We’re a small, sharp team building a big, sophisticated product without the usual corporate nonsense.
Why You’ll Like It Here:
Remote-first with flexible hours
Minimal meetings—we focus on real work
No tech hype-chasing—we use the right tools, not the latest trends
Flat team structure—no layers of bureaucracy
Who We’re Looking For:
Senior Frontend Engineer (TypeScript, React) – Own the UI and craft intuitive experiences
Senior Backend Engineer (Node.js, PostgreSQL) – Build and refine our robust, high-performance backend
If you thrive in small teams solving big technical challenges, reach out to [email protected].
We've been on Heroku Enterprise for 8 years now. I read your comment and clicked on the link with so much enthusiasm.
Duh. You guys have completely forgotten who your audience is. Your audience is _application developers_. I have no idea what all that mambo jambo means in that article _and thats why I pay Heroku_.
I'm on Heroku because I don't want to know about cloud native and fir and open telemtry are. You tell me I should get excited on Heroku? How about improving your autoscaling options so the narrow response-time-based scaling is not the only option?
All that article tells me is that you guys are maybe improving your underlying infrastructure. Meh. Good for you. From a customer (AKA Application Developer) perspective nothing has changed.
The blog post is one of three published recently. It's from Terence Lee, an architect and former maintainer of Bundler (Ruby package manager). He co-founded the Cloud Native Buildpack project and was part of open sourcing the original Ruby buildpack. He gets pretty into the weeds with the tech.
The other article that's not been linked yet is this one https://blog.heroku.com/next-generation-heroku-platform. It's still not exactly what you're asking for "give me the application features" but it is a little less lingo heavy.
One thing kinda hidden in there:
> and AWS Graviton into the platform.
That means ARM support. Already the Heroku supported buildpacks work with both AMD (x86) and ARM (aarch64). Think mac Intel versus M(1/2/3/4) chips.
> From a customer (AKA Application Developer) perspective
I posted another comment earlier. Local debugging with CNBs is pretty neat. But I also agree, I see this still as an "investment" phase. This is the start of the work, that gets us more fun stuff later.
> How about improving your autoscaling options so the narrow response-time-based scaling is not the only option?
This is not my team, so I'm speaking not from first-hand experience. It's my understanding that Kubernetes has a pretty rich ecosystem for autoscaling. On our current platform, if someone wants to try out an autoscaler, it's a bespoke solution and difficult to try in the wild. Moving to a more standards based and community backed infrastructure means what's easy to do on Kubernetes, should also be easy for us to do on Heroku on the new platform.
I hear that you care about autoscaling options and I'll pass that along. Anything specific in that area?
Something that has been baffling me is how Uber is not in panic mode and it's market price not affected by Waymo?
I've always been skeptical [of self driving] but at this point (after having a few rides with Waymo) it appears that the folks at Alphabet have figured it out already and are coming for Uber. Why is not that affecting Uber's price?
One question I would ask is how fast is Waymo growing? At this stage isn't it too early to comment on the viability? Concerns around safety may take a decade of safe riding/data to come down.
Also, when you talk about Uber's stock price - Uber may not only be restricting itself to retail use cases but UberFreight could be a big market as well. In the logistics space too, adoption of self-driving may be slower than in retail is what I suspect.
Also, on an apples-to-apples basis, ~46% of Uber's revenue comes from markets outside the US where self-driving is yet to go through its own motions, regulatory approvals, local dynamics, learning (see how autos drive in subcontinent), etc.
And illegal human driven taxis are very different from robot driven unoccupied taxis. First one was existing phenomenon not in big scale that occasionally got handled.
Waymo has zero presence in the vast majority of the United States and nowhere internationally. Meanwhile, "uber" is a verb and noun like Kleenex. And even if in ten years Waymo displaces Uber for transit, Uber will be dominant in food and grocery delivery.
Like if you don't live in the Bay Area or one of the very few places outside it where Waymo has been testing, people have never heard of it.
Basically, Uber is everywhere and Waymo is nowhere. Uber achieved dominance by rapidly scaling and being able to fight regulation with venture capital. Waymo hasn't been able to do that anywhere.
Google is risk-averse. There is no real incentive for its management to be aggressive. Even with the Gemini AI, they only acted after being pushed around by OpenAI.
You need someone ruthless and aggressive at the top, like Travis Kalanick, Jeff Bezos, Elon Musk or Steve Jobs who basically stop at nothing. Folks who act fast and can take big decisions. MBA business types often get caught up in endless meetings, prevaricating over inessential things.
The time horizon on Waymo getting there is quite long. Uber is in every city because they don't have to build and maintain their own fleet. Waymo will have to. That's not only a massive capital cost, but a substantial scaling problem.
That'd be another reason for market to react? If Uber becomes a front end to Cruise/Waymo, their price should still be massively impacted as those are the ones with the moat and Uber won't be getting nowhere near the same cut from each ride?
LLM's are brains with no tools (no hands, legs, etc).
When we use tool calling we use them to empower the brain. But using normal API's the language model providers like OpenAI have no access to those tools.
With MCP they do. The brain they create can now have access to a lot of tools that the community builds directly _from_ llm, not through the apps.
This is here to make ChatGPT/Claude/etc _the gateway_ to AI rather than them just being API providers for other apps.