This diagram is a high-level view of Loggly‘s next-generation Loggly’s Log aggregation and analytics data pipeline. I was the lead developer of the team that designed, developed, and implemented, this high-performance data pipeline. Built using much custom software, Apache Kafka, and Apache Storm, the pipeline went live in September 2013.