I'm a developer who started a cloud based pos app from the ground up. The last two years of scaling have been a master class in sql and thinking about data.
I blame a lot of performance and scaling problems on ORMS which are fundamentally a bad abstraction for sql and enable devs to brute force something that "works" instead of dinging an elegant and high performance solution to a problem. Most devs don't really understand how to use case statement or aggregate queries for instance. When i was building out analytics pipeline, my original vision was to use some sort of elastic search. We're scrappy and bootstrapping though so instead I came up with a set of triggers that populated special tables that pre-crunch certain bits of data from the original source. Between that and judiculous use of indexing, we are able to do realtime queries with no delay. If you know how to layout your information correctly, the database can be more than fast enough for most use cases.
I think everyone working on database abstraction libraries should look at Ecto. It doesn't try to hide sql. instead it embraces the semantics of sql and extends it to make it more ergonomic. It become easy to escape hatch into raw sql for things the libary doesn't support and even better, I don't have some werid under the hood transformation going on that messes with my intuition for the sql being generated.
tldr: stop treating sql as a glorified k/v store. you can do a lot of cool stuff inside the db and benefit from a crazy speed boost
I blame a lot of performance and scaling problems on ORMS which are fundamentally a bad abstraction for sql and enable devs to brute force something that "works" instead of dinging an elegant and high performance solution to a problem. Most devs don't really understand how to use case statement or aggregate queries for instance. When i was building out analytics pipeline, my original vision was to use some sort of elastic search. We're scrappy and bootstrapping though so instead I came up with a set of triggers that populated special tables that pre-crunch certain bits of data from the original source. Between that and judiculous use of indexing, we are able to do realtime queries with no delay. If you know how to layout your information correctly, the database can be more than fast enough for most use cases.
I think everyone working on database abstraction libraries should look at Ecto. It doesn't try to hide sql. instead it embraces the semantics of sql and extends it to make it more ergonomic. It become easy to escape hatch into raw sql for things the libary doesn't support and even better, I don't have some werid under the hood transformation going on that messes with my intuition for the sql being generated.
tldr: stop treating sql as a glorified k/v store. you can do a lot of cool stuff inside the db and benefit from a crazy speed boost