If you work in the logging, monitoring — or even Observability — space long enough, you eventually end up on team that tries to build a system that handles both logs and time series in a high-performant and cost-efficient manner.

Well, it’s a lot harder than it sounds — because logs and time-series are not the same.

This may seem obvious to many — one is text, and one is numbers. Then why I have been involved with multiple attempts to store the same types of data in a single system? And why has every one of those systems ended with up two distinct storage technologies under the covers?

I think it’s worth taking a step back and looking at why logs and time series are so different.

Log data

Log data is often much richer than time series. It can be both structured and unstructured, usually employing full-text search technologies. This means inverted indexes, rich query languages, and large data storage overheads. And the tolerance for data loss is very low – a single log message could be critical to understanding what has happened to a system:

May  5 17:01:43 owl sshd[9024]: pam_unix(sshd:session): session opened for user root by (uid=0)

The kind of queries one runs on a log system are also characteristic. They are usually “needle in a haystack” queries, where one is attempting to find a single event.

Time series

Compare this to time series data.

It always has some sort of structure, and that structure is usually pretty simple. It’s tolerant of data loss — lose some data and you’ll probably still understand what is happening with your system. Time series can be down-sampled too, but compelling cases for down sampling log data are not as obvious. Time series data can also compressed really effectively, without any loss of information. Techniques such as run-length encoding and delta compression (and even delta-delta compression) work very well.

Finally — and this key — one is almost never interested in a single data point. One almost always runs range-queries over time series data, which are often transformed via aggregations, averages, and other mathematical operations.

Time series are pointers, but logs tell you what happened

Both types of data are critical to running modern computer infrastructure. I used to wonder, half-jokingly, if logs and time series were like Space and Time — different sides of the same underlying thing known as Spacetime.

But I’ve decided they are fundamentally different, and this is reflected clearly in the different technologies, systems, and products used to process each.

Now this doesn’t mean a SaaS can’t be built that does a great job working with both types of data. But if someone tells you their database supports both cases really well, be sceptical.

Because there is no Mikowski for logs and time series, and when it comes to processing this data, there is no magic.