Tag Archives: aws

rqlite 7.16.0 – restore your system from S3

rqlite is a lightweight, open-source, distributed relational database written in Go, utilizing SQLite as its storage engine.

Version 7.16.0 has been released and now includes support for restoring your rqlite system automatically from AWS S3. With this in place you can now automatically recover your system from any backup which has been previously uploaded to Cloud-based storage.

You can download the release from GitHub.

Adding automatic S3 backups to rqlite

rqlite is a lightweight, open-source, distributed relational database written in Go, utilizing SQLite as its storage engine.

Version 7.15.0 has been released and now includes support for automated backups to Amazon Web Services (AWS) S3. This enhancement offers increased data protection and expands your recovery options in case of emergencies.

You can download the release from GitHub.

Building a cluster discovery service with AWS Lambda and DynamoDB

AWS LambdaIn this post I describe how I built a serverless cluster discovery service for rqlite, the distributed relational database built on SQLite.

Built using the AWS API Gateway service, AWS Lambda, and DynamoDB, it means rqlite nodes no longer need to be passed the network address of an existing node in a cluster, and can instead connect automatically.

Continue reading Building a cluster discovery service with AWS Lambda and DynamoDB

rqlite v3: Globally replicating SQLite

rqlite on GitHubrqlite is an open-source distributed relational database, which uses SQLite as its storage engine. rqlite is written in Go and uses Raft to achieve consensus across a set of SQLite databases. It gracefully handles leader election, and can tolerate machine failure.

With the v3 release series, rqlite can now replicate SQLite databases on a global scale, with very little effort. Let’s see it in action using the AWS EC2 cloud.

Continue reading rqlite v3: Globally replicating SQLite

InfluxDB and Grafana HOWTO

This blog describes working with InfluxDB 0.8. InfluxDB 0.8 is no longer supported, and has been superseded by the 1.0 release.

grafanaI recently came across InfluxDB — it’s a time-series database built on LevelDB. It’s designed to support horizontal as well as vertical scaling and, best of all, it’s not written in Java — it’s written in Go. I was intrigued to say the least.

Continue reading InfluxDB and Grafana HOWTO

Infrastructure at Scale: Apache Kafka, Twitter Storm and elasticsearch

storm_logoAWS have posted the video online of Jim Nisbet’s and my talk at AWS:reinvent 2013. In it, Jim and I describe the system we built at Loggly, which uses Apache Kafka, Twitter Storm, and elasticseach, to build a high-performance log aggregation and analytics SaaS solution, running on AWS EC2.

Continue reading Infrastructure at Scale: Apache Kafka, Twitter Storm and elasticsearch

Speaking at AWS re:Invent 2013

amazon.com_web_servicesThis past week I had the opportunity to speak, with my colleague Jim Nisbet, at AWS re:Invent 2013. Titled “Unmeltable Infrastructure at Scale: Using Apache Kafka, Twitter Storm, and Elastic Search on AWS“, Jim and I described the architecture of Loggly’s next-generation log aggregation and analytics Infrastructure, which went live 3 months ago, and runs on AWS EC2.

Continue reading Speaking at AWS re:Invent 2013