Another interesting paper came my way, thanks to the Morning Paper mailing list. Nines are Not Enough:Meaningful Metrics for Clouds discusses a topic that I deal with regularly in my role at Google.
SLIs, SLOs, and SLA are easy to discuss in a general sense, but surprisingly subtle to put into practise. This paper, authored by Google engineers, explores why this is so, and offers a new framework for thinking about them.
As an Engineering Manager at Google, I get a lot of email — everyone does. Google — at least my group — doesn’t make heavy use of IM-like tools internally, and I’m happy about that. Combined with traffic from the internal system, it all adds up to a lot in my Inbox.
So I was forced to really think about how I handle it all — and not miss anything important.
I recently came across a new paper co-authored by John Ousterhout, one of the original authors of the Raft protocol. In it John, and his co-author, describe an approach which can double the throughput of some popular replicated distributed key-value stores.
Sometime ago I was asked where to begin to learn data engineering. It was a broad question, and it took some to understand what exactly I was being asked.
The batching of data or computation amortizing a fixed cost over multiple units — is a very common pattern in many computers systems. It’s particularly prevalent in networking and CPU memory accesses.
But the implementation of batching includes many subtleties — in particular when to wait for more data, and when to transmit what you have.
I’m not old enough to remember Jimmy Carter in office, but I did see him speak once in 2013. With a B.Sc., and some training in nuclear power, his background was always somewhat interesting to me — particularly how someone with a technical education approached politics at the highest levels.
Continue reading Jimmy Carter and the Failures of Management
I recently had a chance to speak about rqlite, the distributed, lightweight database built on SQLite, at the University of Pittsburgh Computer Science Club. It was a good evening as I spoke about distributed systems, the problems they solve, and how rqlite uses Raft to replicate SQLite.
You can find the presentation here.
Go remains one of the languages I’m most productive in. Its combination of the rigour of static typing, but fluidity of Python, makes it both robust and easy to code in.
It’s also got some innovative features that help you catch those tough-to-find issues, particularly when they only occur in production. An example is the Go Race Detector.
I’ve moved continuous testing of rqlite to CircleCI 2.0. The initial work I did with hraftd was helpful, though rqlite was definitely more involved.
Testing is significantly quicker with the new, container-based, version of CircleCI, which should help noticeably with development.
Since I recently joined Google Cloud Platform (GCP), I thought it’s time to get some practical experience with the platform. As a result I’m going to migrate this blog from Rackspace to GCP — specifically I’ll use GCE for WordPress, and Cloud SQL for the persistent database storage.
CircleCI, which I used for much of my open-source integration testing, has released version 2.0. Support for 1.0 is finishing in August 2018, so it’s time to migrate my projects.
I’ve started with hraftd. It was pretty easy, but I find the documents for 2.0 are not great.
Monitoring — the measurement of your system, the gathering of telemetry, and alerting when it behaves anomalously — is key to running large-scale, modern computer systems. But what many developers today don’t realise is that monitoring can be a key part of your design cycle too.
Hashicorp
recently released version 1.0 of their Raft consensus package. The Hashicorp implementation, along with SQLite, forms the core of rqlite. rqlite has now been ported to release 1.0 and will be a key change in the upcoming release of rqlite 5.0.
After almost 20 years in the San Francisco Bay Area, I am moving to Pittsburgh, PA, to accept a management role with Google. I am very much looking forward to working at a world-class software company.
It was a great two years at Percolate, and I wish all my old colleagues there the very best.
2018 sees the ninth year of this blog. Happy New Year to one and all.
I am planning major work on rqlite, as well as more client libraries. I also hope to continue to write on software engineering leadership and management.
