this post was submitted on 03 Jan 2024
11 points (92.3% liked)

DevOps

1649 readers
1 users here now

DevOps integrates and automates the work of software development (Dev) and IT operations (Ops) as a means for improving and shortening the systems development life cycle.

Rules:

Icon base by Lorc under CC BY 3.0 with modifications to add a gradient

founded 1 year ago
MODERATORS
 

What is the industry/production grade solutions or if you have already any experience please share it. Thanks

top 8 comments
sorted by: hot top controversial new old
[–] [email protected] 3 points 9 months ago

In AWS, you can use Aurora to scale reads access over multiple instances. It's easy to setup and you can use scaling policies to follow your load. You need to check your billing, it's not the cheapest service.

[–] [email protected] 3 points 8 months ago (1 children)

Scaling horizontally is never trivial, can you share more about the requirements?

Usually application level partitioning, but it's hardly repeatable. Cloud providers offer solutions like Spanner, that cost quite a bit. You have new SQL DBs like yugabyte or cockroach, which are not exactly MySQL, and are not really battle tested. And noSQL like MongoDB or Cassandra, which are, well, not even SQL, but are proven technologies and scale quite effortlessly.

[–] [email protected] 3 points 8 months ago

I work for another distributed database company. I can say that it's much harder to convert cockroachDB customers than Yugabyte customers. Given that, I'd think that CockroachDB is likely the more vetted solution. Sure it's new (2017), but it's not THAT new.

[–] [email protected] 3 points 8 months ago

IIRC, MySQL (and PosgreSQL) is pretty much limitted to a write/read-replica sort of horizontal scaling. Other SQL engines have better support for multi-master configurations.

However, these types of configurations are usually tied to licensing - especially for Microsoft SQL server and OracleDB.

As another commenter suggested, there is Yugabyte and CockroachDB as well - of those two I think CockroachDB is the more mature product. And they're one of the fiercest competitors for the company I work for too.

I cannot speak to "Battle Tested-ness" of CockroachDB, but given it's been around for a few years now, I don't think it's quite as risky as other comments have indicated. Also, they're doing something right if we haven't been able to convert many CockroachDB customers.

[–] [email protected] 2 points 7 months ago (1 children)

If you truly need read/write to scale, multi-masrer clustering with MariaDB Galera is probably the best way to do it. They (MariaDB) also sell a load balancer/query proxy Max scale that can do a lot of surprisingly complex stuff (like publish new data to Kafka or centrally ship binlogs from one place to multiple read replicas).

However generally my advice is that if you're finding yourself trying to build a big relational database and writing to it a significant amount of times, it might be time to consider a different or at least modified architecture. Especially if your use case starts to scale to more than what Galera can handle. At some point, all these solutions become eventually consistent the more you scale. And if you are willing to accept eventual consistency, there are some clever ways to do storage using things like queueing, batching, caching etc that can scale horizontally much further than any relational database can.

[–] [email protected] 1 points 7 months ago (1 children)

Wow, thanks 👍👍👍 this will help me a lot.

[–] [email protected] 2 points 6 months ago

I hope so! Looking forward to the writeup at the end!

[–] netburnr 2 points 9 months ago

For on-prem we use a Galara cluster. It's managed by ClusterControl because we needed something with enterprise support.