Datalog is much better but sadly its recursion model requires thinking harder so most people bail too early. SQL is easier to understand so it sticks.
SQL was designed with an English-like syntax so it could be used by non-programmers. Before SQL, business people would write in English what they wanted and a programmer had to write a program to execute the pseudo-query.
The pound of flesh: poor composability, ugly recursion (CTE), limited abstractions, and a "nightmare to parse".
Is not that hard to come with a better alternative, but just "nicer SQL" is not good enough. We need a "Rust"-like moment for this.
My bet (https://tablam.org) is that we need algebraic data type support, real GROUP-by, a low-level language (where for example, you encode that "a = b" is in fact calling a BTree) and high-level porcelain.
The low-level language (like webassembly+relational) will unlock a necessary steps where is possible to compile "normal" SQL, that is, the thing that allows typescript to be a viable alternative to JS.
There are more details, but the above is the key. Like, you can create a proxy using the postgres wire protocol that take the SQL/Nicer lang then compiles then do the magic and is the enabler for adoption IMHO.
BTW: This is what I wanna do, so happy to join forces!
OLAP and batch analytics is just getting so much more common. Thinking back 30 years ago, column stores were rare and SQL was all about OLTP CRUD. It still does all that, but it's also become mainstream as the data science and analytics becomes ever more routine and approachable too.
And the dialects of the language itself, SQL keeps getting more relaxed and interoperable and forgiving. With WITH and CTEs and things it keeps getting easier and cleaner to express things, so it's going steadily in the right direction. There are still a few slight differences in the syntax for window functions between bigquery and duckdb, for an example I fight often, but they are all a lot closer to each other today than they used to be back 30 years ago when you had to use SQL differently and construct complicated queries differently just to run on Oracle vs MySQL vs Postgres.
The query languages that aren't SQL keep popping up and they're honestly nicer to read/write in application code in many contexts but none have the ecosystem/maturity or "every BI tool speaks it" advantage. Until there's a single non SQL language that runs efficiently everywhere from my laptop to the datacenter and every tool between SQL wins.
SQL forever? Maybe but please let me stop writing it as strings in my actual application code. Sqlc is a hope for me. It is almost like treating your DB as an RPC.
There is already a way to completely avoid writing SQL strings in your application code, and it’s not even an ORM.
In your SQL database create lots of views and functions. Don’t be shy. There is no limit to how many you can make. Every single time your application needs to do a parameterized query, have it call an SQL function instead.
This method probably increases how much SQL you have to write overall. But it allows you to completely separate the SQL from whatever other programming language you are using. All the SQL exists in one area with whatever framework you have for handling schema migrations. Your application code now interacts with the database via an API of functions you have designed and never actually builds a query.
Frankly whenever people show me some "new SQL killer" it often looks like "SQL with extra steps". Sure, everyone is afraid of joins, but you can often engineer yourself further away from that. Document-oriented database systems are fine, but have downsides.
One of the main reasons I won't be dropping SQL unless I have to is schema. I *love* that data stored in SQL has an inherent schema. As anyone who has dealt with raw JSON will know, it can get pretty ugly when you make changes and either have to migrate everything, or complicate your parsing logic to handle corner cases. As long as I have a sane SQL schema, I can use migrations to make sure all data is in line with a single schema, and that makes reasoning and writing business logic easier. I have worked places where they haven't been as strict about schema as they probably should have been with DDB's and its ugly.
I like when my code is stupid simple because I am dumb and make tonnes of dumb mistakes. Having a database with a strict schema means it is simple to reason about. DDB may seem easy because you can just slap objects in a database, but you are actually just moving the complexity and schema job to YOU the developer. We all make stupid mistakes from time to time, so best to pick tools that makes our job easy, right?
I'm happy to replace SQL but I haven't found anything that gives me the qualities I want yet.
SQL was designed with an English-like syntax so it could be used by non-programmers. Before SQL, business people would write in English what they wanted and a programmer had to write a program to execute the pseudo-query.
The pound of flesh: poor composability, ugly recursion (CTE), limited abstractions, and a "nightmare to parse".
-- SQL engine developer