Minor quibble: i.e. stands for id est, or “that is.” It’s a re-statement or clarification. Exempli gratia (e.g.) means for example. In the diagramme, every i.e. should be an e.g.. Put another way, if you can’t put an et cetera at the end, it’s an i.e..
Wouldn’t ordinarily rise to the level of commenting. But it’s an Andreessen post, and so I’m perhaps primed for the purpose being promotion versus informing.
there is a literary device where you say important things twice, once accurately by description and then again by example, it helps folks orient and synthesize the idea
I argue that the examples--preferably more than one--should always precede the idea, principle, or definition. A fitting audience, then, will arrive at/or could smell the actual idea themselves, before it's presented.
(It's also my plea to mathematicians/academics: please, give us more examples before you hit us with your formula-heavy artillery).
That really depends on the audience. On this topic there are broadly-speaking two types of people: those who want the conclusion first and the details later ("managers", if you will), and those who want the details first and the conclusion later ("investigators", perhaps). Knowing which class your audience falls into is incredibly valuable.
I've seen this described as a pyramid. The first group wants to see the precipice first and work their way down to the foundation. The second group doesn't want to look at any stone before understanding what it's resting on.
> one of the world’s top companies, dedicated to communication
Is there another Andreessen Horowitz that I don't know about? Because otherwise a $35B fund that invests in tech startups is very far from that description.
I suspect the OP might be alluding to the fact that a16z is basically a content machine, that happens to also invest. Their content (or more charitably: their highly visible external communication) is a large part of what launched them onto the scene so quickly and had them in consideration as a top tier firm despite having no track record as a fund (GPs reputation as founders notwithstanding)
> content (or more charitably: their highly visible external communication) is a large part of what launched them onto the scene so quickly and had them in consideration as a top tier firm
The VC predates the blog. It used to be a serious name, being an early backer of Twitter and Y Combinator. It started to go dogshit about ten years ago and criminal after 2019. They remain a fundraising powerhouse. Though the market seems to be noticing their flagging performance, with secondary funds trading at strong double-digit discounts to NAV.
If Andreesen-Horowitz disappeared overnight, would it really make any difference? The capital would flow through other companies. What has it actually contributed that only it could do? Are you sure its prominence is really any different from that of, say, the Kardashians? I.e. a socially driven expression of a power law?
Most likely this time A16z got invested in some bottom-of-the-heap Dev tools SAAS businesses.
> And a giant hairball of queues and retry logic is too brittle. To deal with this, we’ve seen, over the last few years, the emergence of new solutions that bring sanity back to transactional logic. They can be roughly categorized as either workflow-centric approaches or database-centric approaches.
Ok, using databases and queues is bad. So what is good then:
> we call this the application logic transactional platform (ALTP) approach because, ultimately, it extends OLTP transactions into the application
Really how much I will love to get few of my brain farts funded on being brilliant insights for typical enterprise application.
To be fair, GraphQL manages to reign in the ludicrussness that is exporting a read only SQL connection to levels that are merely unpredictable instead of outright dangerous.
GraphQL does not prevent you or the GraphQL→DB translation layer from doing stupid things. Depending on the capabilities exposed by the GraphQL schema and the implementation of the translation layer, you can write a GraphQL query that is translated to 20 JOINs or N+1 hell or other bad things.
I'm sorry but there was a lot of jargon thrown around and circumnavigating standard problems in data consistency but no real takeaway or upshot. Felt like a chatgpt article
The takeaway seems to be "you must pay for every query and for every bit and byte we hold and transact for you", here's a list of companies you probably never heard of before (Upstash, Momento, Stellate, Xata), some with unsearchable names (Convex, Temporal, Trigger), pay them to have your database locked arbitrarily or whenever they somehow reach unicorn status and will be too big to reply to your emails.
Accessing the website of any of the listed companies, the Pricing page in the header is predictably there: everyone is looking for a rent. Just to be a contrarian, because I just can't believe all these companies are making significant money, perhaps it's time for a return to the pay-for-binary model in such an Everything as a (payable) Service world: here's the blob for a nice database (or whatever), pay $1,000 (or maybe even $10,000, $100,000) and the current version is yours forever.
I think you're mostly right, but I will drop an asterisk: Temporal is self-hostable and is incredibly good. Like, I am pretty convinced that for the classes of problems it addresses around offline/the-customer-isn't-waiting workflow management, it is the Correct Way To Do Things. I'd never have them white-label AWS to host my stuff, but it's great.
But if you don't get your well-funded company to pony up for these SaaSes, highly paid dev positions at companies like these are going to dwindle, reducing labour market demand, and dev wages will fall!
It's so hard to read this VC babble as anything other than lukewarm also-ran puff.
The massive disclaimer at the end includes a write-off that the whole thing "may not be reliable" and "we haven't actually checked any of this", and "this might just be an advert" and if anything is wrong with it, well it's not a16z's views, it's just one of our ICs, we stand by none of it.
All presented as authoritative "industry" commentary. Only to be revealed as diluted waffle, regurgitating some investment thesis as a thinly veiled advert.
Their target audience isn't engineers but decision makers at large organizations. They want ALDC (or whatever acronym they are pushing) to become a standard part of the stack so that everyone will have to buy ALDC Ops Service or ALDC Security service.
And what do you know A16Z has investments in companies that offer those SaaS products.
Also plays into the trend of "lets hire fullstack devs AKA frontend devs that knows how to call APIs and have a vague understanding of transactions and its pitfalls" and offload all backend work to SaaS products.
If the startup takes off? Great! If it doesn't? At least those SaaS products offered free trials.
I'm guessing the authors are making an internal pitch that a16z should invest in upstash[1]? I'm not feeling it personally - the whole hypothesis seems definitely not aligned with what I've seen.
I've never seen people trying to build transaction guarantees into their whole microservices stack using Fancy New Startup Tech[tm]. What I have seen a lot is people building transaction guarantees into their stack using something like event sourcing and cqrs. You pay some complexity but what you get in return is a transaction guarantee when events are processed which guarantees consistency when you need it (eg financial transactions etc), and the view part is eventually consistent but is always accurate as of a point in time.
> I'm guessing the authors are making an internal pitch that a16z should invest in upstash
thats not how it works, they already made their bets long before writing something like this (+ validated by talking to a ton of pple than can fit into this post... i know bc i've spent time with the authors haha)
> What I have seen a lot is people building transaction guarantees into their stack using something like event sourcing and cqrs. You pay some complexity but what you get in return is a transaction guarantee when events are processed which guarantees consistency when you need it (eg financial transactions etc), and the view part is eventually consistent but is always accurate as of a point in time.
correct. what's happening here is the rise of well tested frameworks that hopefully reduce the cost of implementation while preserving/extending the benefits (which can be very large if you go as far as temporal have done)
> thats not how it works, they already made their bets long before writing something like this
Correct. Andreessen is unusual in that its partners have a habit of plugging their books irrespective of the underlying companies’ health. So when e.g. PG says “look, I funded this company and they’re being badass,” people listen. When these guys write this tripe, it makes me wonder who’s running out of cash on that list.
I will give you a warning from BigCo land, having watched this pattern play out in various spaces/tech/industries over the past decade.
When folks run out of ways to make money, they productize smaller and smaller chunks of the overall stack to attempt to extract more value/rent/payment out of their customer base.
They will bring in ever more increasingly crazy sounding architecture experts to justify this (often times these folks get planted as FTEs in the very orgs they are selling to!).
When you find yourself asking 'is this complexity for complexities sake', know that you are being sold vaporware :)
I figured a modern stack would involve modern tools and not be a head first dive into enterprise SaaS contracts and integration hell, but what do I know.
At this point I’m diversified enough in my investments to just start agreeing. “Yes, you’re right, we’ll all be drinking our meals and everyone who uses Postgres will be fired”
Isn't the whole problem area of distributed transactions a bit of a conceptional issue?
Looking at it from an event driven, eventually consistent view it seems to me that one can achieve considerable complexity gains by explicitly avoiding distributed transactions.
In the past I had often the discussion that "Oh, no! Our business requirement X forces us to have atomic transactions across these 74 services". I cannot talk in absolutes but all cases I have personally seen actually didn't need distributed transactions and could (in most cases easily) be translated into event driven architectures. In the not easy cases it usually amounted to services being split badly or the IT org being too strongly present in the deployed architecture.
I implemented something similar to the approach in this article. I was building a payment transaction system and ran into the issue of having locks and transactions in the database causing higher and higher latency as the queue got larger.
I never coined a term, but I implemented an “ALTP” using in memory cache library in elixir. The library is called nebulex. It was so simple to implement. Since the library supported distribution out of the box. I had a distributed in memory transactional guarantee that worked well in a short amount of time.
The cool thing was I also wrapped the database lock under the distributed in memory lock so in the case of a net split the database’s lock’s got my back. In normal circumstances it made things scalable because the in memory lock would prevent the client from connecting to the database until the in memory lock was let go.
In elixir doing these things are really intuitive and enjoyable and without requiring an external dependency. I think it took me only a couple of days work design / implement and solved the problem.
> The cool thing was I also wrapped the database lock under the distributed in memory lock so in the case of a net split the database’s lock’s got my back. In normal circumstances it made things scalable because the in memory lock would prevent the client from connecting to the database until the in memory lock was let go.
how did you coordinate this, since both are different systems altogether?
edit: I went through the video at 1.5x speed and I understand it now. This is the grand idea:
1. Use a separate distributed lock manager, get lock on the wallet. This lock is in-memory
2. Start transaction on DB and make changes
3. Release the lock
4. Other transactions wait at #1, since they can't get the lock. Thus this reduces the overall load on the DB.
I watched the beginning of the video but it's over an hour and relatively slow going.
I don't know Elixer or frameworks around it.
But I'm guessing you basically just used something like Zookeeper to do distributed locks at the API call level. Only when they got that lock would they then be given DB access, which then ensured the DB itself wouldn't grind to a halt as many calls were all locking tables at the same time.
So what was the solution to returning status to the caller? Did you just give them a ticket which they could then poll status later?
Meanwhile, the individual servers just put the requests on a local queue and spin in a thread till they get the lock from Zookeeper (or similar) and can access the DB?
Yeah. Couple of days. To be fair Elixir and Nebulex library did most of the heavy lifting. I had been using Nebulex for a long time before implementing this solution. But as our system scaled we ran into more and more problems so I decided to try it.
It's a difficult subject, and it seems to me that the authors got confused / overreached themselves in the second half. For example, one of their diagrams shows Temporal/Cadence being used together with Zeebe/Camunda in the same backend. This would be an extremely complicated and redundant bad idea and I believe no-one with expertise in the area would consider it for a moment.
> However, microservices and other modern application architectures introduced new complexities into application design: Developers needed to manage data across different services and ensure consistency between them, which forced them to build complex data synchronization and processing mechanisms in-house.
I always wonder how many of these "modern" deployments could be replaced by a monolith + DB running on a ~single server. What are we up to these days for off-the-shelf hardware? 128 cores, 6TB+ of RAM, x NVMe SSDs in RAID? Application on one server, cache-backed DB on another. Failover to a cloned setup.
Is anybody here running something like this on latest hardware and can provide rough performance metrics?
The article was too vague for me to really understand, but I think it's suggesting something like Brandur's deeply-integrated idempotency keys design [1] except that the infra is provided by a saas company? Also somehow it supports rollbacks?
Does anyone know of any other more concrete explanation of how these kinds of systems could work?
I see the word 'distributed' being thrown around way too much in TFA without any justification as to why apps should be so distributed in the first place.
This is trading architectural complexity with integration complexity.
You can learn to architecture better, but you can never get better at duck taping together the capricious and arbitrary APIs between hottest new techs.
Wow, so now we need to do what we might call "distributed transaction processing", we need some kind of "extended architecture". Wonder if there's a snappy abbreviation for that?
The prediction of the future is tough. Basically no-one predicted the AI iphone moment, everyone was suprised, the landscape changed forever in a very quick way.
I usually look at the authors, because if a partner at a VC had great tech insights, they would found a startup themselves, and what I see from my coachees, most VCs just follow the herd ;-) (yes, some few have) One of the authors was a CTO though.
seeing a lot of ad hominem comments on a16z and think that's a pretty low quality response - let's try to respect HN guidelines and discuss the substance of the article? this is an emerging space and we all lack the language to adequately describe it, so cut them a little slack there if the chosen terminology doesnt connect at first.
> "We’re seeing the emergence of systems that extend strong transactional guarantees beyond the database, into the distributed apps themselves."
yes absolutely. but its not just transactions, but also the ability to route/distribute work, and "self provision" (https://www.swyx.io/self-provisioning-runtime) queues and timers and new application states (unconstrained by preset DAG) as you code
> a giant hairball of queues and retry logic is too brittle
> (temporal) will ensure the code runs to completion, even during a failure.
i'd correct to "transient failure" (eg due to network or uptime issues). retries cant solve application logic issues and mostly fatten the tail of timeout failures that still need to be dealt with. fault tolerant code is a much better default, but there's no totally-free lunch here.
> The downside is that it (temporal) often requires a lot of integration work and wrapper code to expose useful and safe interfaces to application developers.
> The other approach, which is more popular with application developers (particularly Typescript/Javascript) is for the workflow engine to serve as an orchestrator of async functions (e.g. Inngest, Defer, and Trigger)... (truncated)
lest i sound like a pure temporal shill, i also agree that this is an attractive model for fullstack web developers (as opposed to say backend/systems devs). this is why i got excited about Defer enough to review it (https://www.youtube.com/watch?v=iGccpHaB1hA)
> Database-centric approaches start with a database, but extend it to support arbitrary code execution to allow for workflows alongside data management.
i've always used the analogy that workflow systems are "stored procedures on steroids". glad to see some signals that Convex (and Supabase?) have plans to make this available to their users. i think there'll be happy users of either approach, but personally am doubtful that the DB-centric approach will take significant share simply because 1) this stuff is not existential for them while it is for the workflow companies, so they will probably lag behind in features, investment, product/GTM focus, and 2), workflow systems easily bridge multiple systems and databases while the db-centric approach is necessarily limited to the system that db lives in. if you have a real microservices setup instead of a distributed monolith, its pretty obvious which approach will win here
Thanks swyx again for sharing your insight in this space.
I closely followed your work while you were at Temporal. "React for Backend" resonated with what I experienced while building couple of systems in the backend!
It's been a year or two since then, and I'm frankly surprised that Temoporal-like services don't appear that mainstream. At least in my small circle friends, orchestration framework is still in the "early adopter" stage of innovation.
Any recent take on the adoption of Temporal-like frameworks? Do you still believe it will have a React like moment in the future?
> I'm frankly surprised that Temoporal-like services don't appear that mainstream. At least in my small circle friends, orchestration framework is still in the "early adopter" stage of innovation.
yup and the problem is in both demand and supply. it is still too difficult to get up and running (whether with temporal or orkes or other) and also not enough developers realize the business value of/(difficulty of doing well) doing long running async work (i tried to communicate that in all my temporal intros, but getting devs to care about what code has abnormal biz value is surprisingly difficult, only a minority look at code like that).
further - choreography (as referenced in my article) will always be simpler to start with than orchestration, so many people will start there first and basically stop there. so the activation energy to feel the pain is nonzero.
that said, we have evidence that app developers will eventually be convinced to overcome learning curves and shift work left, with typescript released in 2012 but only "winning" in 2018-19. there was no React "moment", it grew and grew through the work and advocacy of a small group of believers and the fortunate self immolation of Angular 2
Thanks - just reading the article that you linked to above on your website. It seems like a very similar concept to state machines; can you help me wrap my head around how this might play with something like xstate or that concept?
TLDR: ALTP is backend-as-a-service. Write your front-end application and consume from a BaaS that provides durability and strong consistency as well as business logic.
Wouldn’t ordinarily rise to the level of commenting. But it’s an Andreessen post, and so I’m perhaps primed for the purpose being promotion versus informing.