I recently came across InfluxDB — it’s a time-series database built on LevelDB. It’s designed to support horizontal as well as vertical scaling and, best of all, it’s not written in Java — it’s written in Go. I was intrigued to say the least.
InfluxDB was very easy to set up, but it took me an evening to get it fully integrated with Grafana, so I could view pretty graphs. It did not take long by any means, but because both InfluxDB and Grafana are new projects, the sequence of steps to integrate the two systems was not clearly laid out in a single place. This blog post describes the steps I took. I actually scripted most of process, so if you run my script, you’ll be up and running in minutes, with a sine waveform being written to InfluxDB, and being rendered by Grafana.
Using AWS EC2
This HOWTO describes the steps I took on a brand new EC2 Instance. That way I could blow away the EC2 Instance if I wanted to start over, and meant I didn’t have to worry about affecting the software on my own machine, until I was sure I understood the process. It also means you should be able to reproduce my steps exactly. This post does not describe how to launch EC2 Instances. It’s easy to do, but you’ll need to learn it elsewhere.
I launched the Instance in the US Northern California Region, using ami-ee4f77ab, which gives you a running 64-bit Ubuntu Server 14.04 LTS virtual machine. While the script and steps I outline should run on most recent Ubuntu releases, your mileage may vary. The rest of this HOWTO assumes you’ve launched a suitable EC2 Instance (I tested on a 64-bit m3.medium), and that you’ve granted access to your Instance from your IP address.
Installation and Configuration
I have posted a bash script on github that executes the following steps:
- Downloads and installs, the latest version of InfluxDB. It then starts InfluxDB and creates the test database test1.
- Downloads, installs, and configures, Grafana 1.54. Specifically it downloads this gist and replaces PUBLIC_HOSTNAME with the hostname of the EC2 Instance.
- Installs nginx, and configures it so that it serves up Grafana.
- Downloads a simple sine waveform generation program.
Note that this script executes some operations using sudo. If sudo privileges are not available, the script will fail.
Execute the script like so:
Examine the script if you want to understand exactly what steps it executes. If the script has been successful, it will finish with output like so (the actual hostnames will be different):
Creating Influxdb database test1.
Downloading sine wave generation program.
Configuration complete. You can find InfluxDB and Grafana at the URLs below.
Influxdb URL: http://ec2-5-1-44-92.compute.amazonaws.com:8083
Grafana URL: http://ec2-5-1-44-92.compute.amazonaws.com
Note the actual InfluxDB URL and the Grafana URL — you’ll need both in a moment. At this stage both InfluxDB and nginx are running.
Generating the sine waveform time-series
A simple program, written in Python, simply sends the time-series generated by a sine function into InfluxDB. It injects 1 point per second. Start the program, which was download by the bash script, like so:
Every 10 seconds this program will print the number of points inserted into InfluxDB. Let it inject a few hundered points for the full effect.
Sanity-checking the data in InfluxDB
InfluxDB comes with its own simple Administration and Data Exploration interface. Before you build the dashboard in Grafana, it’s best to check that the data made it into InfluxDB using this interface. To access this interface, copy and paste the InfluxDB URL into your browser.
Once the InfluxDB page appears, click Connect. (You don’t need to enter anything.)
Then click Explore Data on the next page.
Finally, enter the query SELECT * from sin, click Execute Query, and a graph should appear, as shown below.
Building the Grafana Dashboard
Once you’ve verified the data is in InfluxDB, it’s time for the fun part — viewing the data in Grafana. I initially found it a little non-obvious how to do this.
Start by editing the default Grafana dashboard by clicking on the title Grafana test and then clicking Edit.
Then add the query as shown in screenshot below, removing any other queries pre-created by Grafana. If everything is correct — and it’s important to complete the query input boxes exactly as shown — you’ll then see the sine graph appear. You’ll almost certainly have to adjust the time-range in the top right-hand corner, to achieve a better view. Alternatively you can simply zoom in on the graph using your mouse.
This is the only the start of what can be done with this system. Obviously the time-series data is synthetic, and the integration simple, but it serves as a useful proof-of-concept. It shows the 3 systems — InfluxDB, Grafana, and nginx — working together correctly and ready for more experimentation.
Many thanks to the developers of InfluxDB and Grafana — both very interesting pieces of software.