> Honest question: what do the sqlite authors think of litestream?
Litestream author here. Dr Hipp and his team reached out when I first released Litestream and we had a video call together. They were fantastic. Really friendly and down to earth. I explained how I put together Litestream and we went back and forth on different approaches. They were really helpful with understanding some of the shared memory stuff that's what makes the new read replication work. They ended the call by asking me if there's anything they could help with or anything I needed.
> I guess I'm wondering if it's a recommended/supported mode of operation in sqlite.
I don't think the SQLite team endorses any tooling outside of what they build AFAIK. So in that sense, no, it's not officially supported. However, the API for maintaining checkpointing is publicly available and there are docs for controlling it from outside processes (which is what Litestream does).
> What are the failure cases? How much data can you lose?
Litestream is designed so that it keeps retrying in the event that you can't connect to a replication destination. S3 is pretty reliable so it may be a network outage that could cut you off. You can also enable Prometheus metrics to be reported out of Litestream if you want to add monitoring and alerting.
As far as data loss, by default it's setup to bundle database changes together every 1 second, compress them, and upload them to somewhere like S3. So your window for data loss is 1 second unless S3 goes down.
You can also run regular backups with the SQLite CLI and upload those as a fallback. That's what I typically do since it's really cheap and easy to setup an hourly cronjob and I'm overly paranoid. :)
We didn’t discuss the session extension. That extension solves a different problem of having multiple primaries that need to merge their data. I haven’t used it personally but it looks great for that purpose.
Litestream author here. Dr Hipp and his team reached out when I first released Litestream and we had a video call together. They were fantastic. Really friendly and down to earth. I explained how I put together Litestream and we went back and forth on different approaches. They were really helpful with understanding some of the shared memory stuff that's what makes the new read replication work. They ended the call by asking me if there's anything they could help with or anything I needed.
> I guess I'm wondering if it's a recommended/supported mode of operation in sqlite.
I don't think the SQLite team endorses any tooling outside of what they build AFAIK. So in that sense, no, it's not officially supported. However, the API for maintaining checkpointing is publicly available and there are docs for controlling it from outside processes (which is what Litestream does).
> What are the failure cases? How much data can you lose?
Litestream is designed so that it keeps retrying in the event that you can't connect to a replication destination. S3 is pretty reliable so it may be a network outage that could cut you off. You can also enable Prometheus metrics to be reported out of Litestream if you want to add monitoring and alerting.
As far as data loss, by default it's setup to bundle database changes together every 1 second, compress them, and upload them to somewhere like S3. So your window for data loss is 1 second unless S3 goes down.
You can also run regular backups with the SQLite CLI and upload those as a fallback. That's what I typically do since it's really cheap and easy to setup an hourly cronjob and I'm overly paranoid. :)