Managed DBs have tremendous lock in. Just try migrating off RDS with zero downtime. You can't, because they've "managed" your ability to configure external replicas. Then built a whole brittle data migration service that probably won't work for your DB.
I'm having a really difficult time migrating my managed postgresql instance in Google Cloud just to get it to a newer version. I'm down to my last option: shut down everything, export, re-import somewhere else again.
I did briefly have logical replication working, and so had my near zero downtime solution in hand. But the replication broke after a few days because I delayed cutting over, and then I couldn't get it going again. I couldn't find out how to debug this either.
The lesson I've learned here is that I'll be avoiding managed DB products whenever I can. It takes away control that turns out I sometimes really need.
they've already had to migrate twice due to questionable technology decisions. having to migrate again due to needing to change cloud providers seems a lot less likely than having to migrate again because their outside the box technology choice didn't pan out.
"Questionable technology decisions". You're trying to dunk, but I don't think you understand where the hoop is. Their technology decisions have panned out Tailscale well. We should all be so fortunate. This isn't Twitter with the "fail whale"; the only reason you know about any of this stuff is because they wrote about it. They ran their entire service with a JSON file backend for 18 months, and switched from it to etcd without you even knowing about it.
i really don't know a ton about this product or team but it sounds like if they had used aurora mysql or aurora postgres in the first place then there would be nothing to write a blog post about because it would've just worked and kept working. they say they want to avoid vendor lock-in but if the vendor became a real issue they'd be doing their first migration instead of being on v3 already. additionally, their bespoke solution relies on s3, which is also a vendor-specific technology, so it seems like they haven't avoided vendor lock in? i've seen many cases of developers doing more work to avoid vendor lock-in than it would take to replatform if it ended up being a necessity, and this really feels like that looking at it from the outside. i'd understand this better if mysql or postgres couldn't solve their problem, but that is not the case here, and i can't wrap my head around a company who is ok with their devs reinventing a very good wheel 3 times when the obvious choice would've worked fine the first time. it seems like they are successful in spite of these decisions, not because of these decisions.
You don't know what this database is --- upthread, you said you don't even know what the product is. All you appear to know is that they should be using something like RDS. Isn't that a weird position to take?
it's also kinda weird to write a blog post about a database being used for what is apparently a very specific use case without mentioning what that use case is. the burden of proof is on tailscale to explain why they need to deviate from industry norm here and clearly from the entire comment thread of people wondering the same thing as me they haven't done that. this blog post might actually be valuable if they included more context so that people could learn when something like this might be a good idea, especially since it's not a good idea >99% of the time. as is, no one should be surprised by this response.
This is one of several comments you've written where you've acknowledged you don't know what the use case is. But you've stridently insisted that they should have used Aurora Postgres or Aurora MySQL. You get how strange this take is, right?
One thing that would have helped your writing on this thread: a lot more question marks. It's OK not to understand something! Asking questions helps everybody.
at no point in the blog post or this thread has anyone actually explained what their super special use case is that makes doing what almost everyone else does a bad choice and this replicated sqlite thing a good choice. it's pretty clear you're an investor in this company or have some other vested interest in shilling for them, good luck with that.
The thing about startup decisions is: most of them are "wrong". Or they start right and become wrong later. Successful startups aren't successful in spite of their wrong decisions, they're successful because they can change them very quickly (then write a blog post about it and get more customers).
There's also a strong correlation between people who look at things from weird angles and also build good products. Why are you surprised the people who invented an entirely new way of doing VPNs also don't cargo cult database storages?
Sounds like you need to do a lot more research before commenting on this? It's pretty easy to find in the Litestream docs that it replicates to S3, Google Cloud Storage, Azure, and other options even including SFTP. In fact by off-loading the storage integration details to Litestream, the Tailscale people now get seamless storage vendor independence almost for free.
This is what a really smart and future-proof solution looks like.
Again: this is an infrastructure component, running on many machines internally. It's not a Rails app. We use SQLite in very similar circumstances across hundreds of machines. Having all those machines schlepping all their reads back to RDS would not only be untenably slow, but it would also make the whole system less stable. I don't think you've really thought through the design at all, and you should before making comments like these.
I'd put money on this not being the last database change too! But not for the same reasons you would.
Unless I'm misunderstanding something, it seems to me that Tailscale's controller, discussed in the previous blog post [1], is like a typical CRUD API server, currently only running on one machine (they're not using Litestream's work-in-progress live read replicas), not something widely distributed like Fly's service discovery infrastructure.
I'm not joining the "just use RDS" crowd, but it looks to me like using a managed DB service like RDS would have been a reasonable decision for this application.
Choosing a technology that supports your business needs and growth, and that also lets you easily migrate to something else once it stops being suitable (or your requirements change, or you discover a way in which the tech doesn't work for you, or whatever), doesn't sound like a questionable technology decision, it sounds like a great one.
