Avoiding elasticsearch split-brain

elasticsearchLoggly recently held an elasticsearch meetup, which was a great success. One question that was repeatedly asked was how to ensure elasticsearch does not suffer a partition — known as a split-brain.

This can be a particular problem in AWS EC2, where the network is subject to interruptions. It can also happen if the elasticsearch master node performs long garbage collection cycles.

Continue reading Avoiding elasticsearch split-brain

Loggly Generation 2 Released!

logglyAfter 14 months of hard work, the next generation of Loggly has been released. It’s been a great time to be part of the Software Infrastructure team at Loggly and we have put together a superb log aggregation & real-time analytics platform.

We used a combination of custom log Collectors, Apache Kafka, Twitter StormElasticSearch, and lots of secret sauce. You can find more details about the technology stack from my Loggly blog post.