rqlite is a lightweight, open-source, distributed relational database written in Go, which uses SQLite as its storage engine.
Recently I helped Replicated replace their use of PostgreSQL with rqlite. This brought much easier operation and high-availability to their Application Manager.
To quote from their blog post:
“Because the database is now highly available, the application manager no longer requires distributed storage (like Rook or Longhorn) when running in a multi-node Kubernetes installer cluster. The application manager now uses rqlite instead of Postgres to store things like version information. Unlike Postgres, rqlite replicates data itself, so it doesn’t require the infrastructure to replicate its data to any node in the cluster. This eliminates the requirement that vendors use distributed storage like Rook or Longhorn in their Kubernetes installer specs when deploying multi-node clusters, and it enables the use of lighter weight storage like OpenEBS Local PV.”
This is great to see as rqlite’s main design goals are lightweight operation and really easy clustering — and it met its goals in this case.
The key change Replicated made was to their KOTS framework, a central part of the delivery platform. They swapped out PostgreSQL with rqlite, automating the migration as part of the upgrade process.
As part of working with Replicated developer Salah Al Saleh I made quite a few improvements to rqlite, meaning it now runs even smoother on Kubernetes.
Much improved Kubernetes configuration
The recommended configuration for running rqlite on Kubernetes has been much improved. The latest configuration includes both a Headless Service for rqlite autoclustering, and a more standard service so clients can communicate easily with the rqlite cluster.
Kubernetes is quickly becoming the best way to run headache-free, highly-available, rqlite clusters.
As of release 7.10.0 rqlite is even easier to operate.
- Backup-and-restore can now take place using any node in the cluster. Prior to this you had to perform these operations by directly contacting the Leader node.
- A new join-read-only permission, meaning you can allow read-only nodes to join your cluster, get database updates, but be sure those nodes can’t make any changes to your data.
- Docker improvements including adding even more configuration operations when deploying rqlite as a container.
An important bug related to Named Parameters was also fixed, allowing NULL to be used as a parameter. SQLite was also upgraded to 3.39.4.
Many thanks to Replicated
The work of Salah and the team at Replicated have made a big difference recently to rqlite. Salah has also become a maintainer of the rqlite Go client library.
As a result of their feedback and input rqlite runs even better, and is even easier to operate. I’m really looking forward to seeing what else they build on rqlite.
7 thoughts on “Replacing Postgres with rqlite”
What about the performance? Sqlite isn’t the fastest db. If you want to have HA postgres you just need a crunchydata or do not run database in Kubernetes at all. Why nowadays is very popular to use tools because they are fancy? Kubernetes main goal is to host stateless apps and services. Top much green hairs working in IT and now we have such diarhea in the business. Too much devops is not good for your company.
Actually SQLite is a very fast database, especially when you consider it runs in process (no network latency). In any event it’s not the bottleneck here, it the Raft system.
how is rqlite comparable to cockroach db, which also claims the distributed nature of replicating the data by itself ?
CockroachDB replicates for performance and reliability. rqlite replicates for reliability.
100% agree, Great DevOps job security is how I see it normally.
…and how does Rqlite compare to Zookeeper? If I understand it correctly, Rqlite is the same to Sqlite as Zookeeper is to Kafka, for example?
rqlite is a bit like Zookeeper, in the sense that you can store data in a reliable, fault-tolerant sense. Any comparison to Kafka doesn’t make sense.