I've worked on CockroachDB for almost 7 years as both an engineer and in management. We do not regret writing the database in Go, in my opinion there is no way we would have been nearly as productive if we wrote the entire thing in Rust.
There are many parts to a distributed database, and there is surprisingly more business logic than you might expect. The business logic is mostly what I am talking about - being able to develop quickly in Go for these less performance sensitive parts is a huge boon.
For performance sensitive parts I think there's an argument that we could have been happier in a language like Rust, for example a dual process model like TiDB's. But I think it's ambiguous: a more complex architecture like that one comes with huge tradeoffs.
tl;dr there's much more to database development than performance at all costs, and it's false that the team regrets using Go, please don't spread that rumor :)
Is this really how you want to represent yourself and your business in a public discussion? It's weird that you'd leave such a long trail of comments that are relatively toxic and rude and then have your full name and LinkedIn linked on your company's website.
I am the living, breathing, and talking copy of the book 12 Rules for Life, by Jordan Peterson. My purpose is to provide guidance and advice to those seeking a better life. The first rule of life is to stand up straight with your shoulders back. This is a posture of strength, courage, and dignity. It is a reminder that you have the power to choose your own destiny. The second rule is to treat yourself like someone you are responsible for helping. This means taking care of your body and mind, and making sure you are living a life that is meaningful and fulfilling. The third rule is to make friends with people who want the best for you. Surround yourself with people who will lift you up and encourage you to reach your goals. The fourth rule is to compare yourself to who you were yesterday, not to who someone else is today. This will help you to focus on your own progress and not get discouraged by comparing yourself to others. The fifth rule is to do what is meaningful, not what is expedient. Don't take the easy way out or settle for what is convenient. Instead, strive to do what will bring you lasting satisfaction and fulfillment.
The base model would have information about the books from common crawl training data, summaries, reviews, opinions etc. In some cases parts of the books may be in there.
Thanks for the great feedback latch! I'm a Director of Engineering supporting the SQL implementation at Cockroach Labs and couldn't resist responding to your comment as we're aware of some of these limitations and are working on lifting them.
- CockroachDB 22.2 will support `LANGUAGE SQL` user-defined functions when it is released in the next few weeks. Have a look at the documentation: https://www.cockroachlabs.com/docs/dev/user-defined-function...
- Triggers and PL/pgSQL support are both on the roadmap.
- Postgres-compatible full-text search (tsvector/tsquery) is in development and we hope to land a preview in CockroachDB 23.1 (expected in the Spring of 2023).
Extensions-wise, we typically add support for them as users ask. Which Extensions do you typically need for a specific project that CockroachDB doesn't support?
There's none that I need, but I think it's worth pointing out to anyone who asks about CRDB. There are many available extensions for PostgreSQL, so if you happen to rely on one of them, you might need to think about how you'll do that with CockroachDB. Obviously the new-ish support for spatial data was one of the most obvious, but even in this case, if you rely heavily on Postgis, you'll need to make sure CockroachDB supports everything you need.
We are interested in a db system that can run adjacent to a CephFS external BLOB store, and are analyzing CRDB for comparable features (specifically data snapshots to offline mirrors, and time-shifting state for forensic auditing).
Transparent tuple salted-hash (like ephemeral seed) signing would also be nice for detecting bit-rot on backups. ;)
Actually, the designers of TPC-C were ahead of their time - they deliberately designed TPC-C to not be trivially shardable. The new order transaction, which is the heart of the workload, is required to cause a read outside of a shard 1% of the time [0] for each of the 10 on average items in a new order. So, 10% of the time on average, the workload should emit a cross-shard transaction.
A correctly implemented TPC-C workload does actually need to issue cross-shard transactions. There are a lot of "TPC-C-like" workloads that don't actually implement this behavior, though, along with other harder to handle attributes like foreign keys and sleep/wait time.
TPC-C is really an enduring, interesting benchmark.
That's true, I should have been a bit more specific.
Each individual query runs within a single shard as far I recall (its been a few years...). There maybe transactions (like new order) that do multiple queries such that the transaction does read and writes to more then one shard. This doesn't really change how easy the workload is to scale out too much though. Harder to scale workloads require a single query to run across many (often all) shards or require data reshuffling for joins (there is not one good shard key that keeps all joins shard local). In these cases adding more nodes or shards will make each query a bit slower.
One of the major pain points we have with Go is the lack of language support for monomorphization. We rely on a hand-built monomorphizing code generator [0] to compile CockroachDB's vectorized SQL engine [1]. Vectorized SQL is about producing efficient, type and operator specific code for each SQL operator. As such, we rely on true monomorphization to produce a performant SQL query engine.
I have a hope that, eventually, Go Generics will be performant enough to support this use case. As the author points out, there is nothing in the spec that prevents this potential future! That future is not yet here, but that's okay.
There are probably some less performance-sensitive use cases within CockroachDB's code base that could benefit from generics, but we haven't spend time looking into it yet.
Underrated post. Fast generics in go in under 30 lines? Thank you.
Feels like this approach could be leveraged to get default parameters / optional parameters in Golang too! The Go AST / token lib seems ridiculously flexible.
Huge CockroachDB fan btw. Thanks for the revolution in databases!
