Writing a Syslog Collector in Go

gopherI’ve started coding in Go (golang), and I received some advice recently from Robert Griesemer, whom I was fortunate enough to sit beside at a recent Go Meetup. To learn Go, Robert suggested that I code a solution in Go for a problem I had previously solved in a different language.

So I decided to code an RFC5424-compliant Syslog Collector, a problem I had to solve while at Loggly — and had to solve well. At that time I solved it using C++ and the Boost ASIO library. At a high-level the goals of the Collector are to receive log messages over TCP or UDP, perform some basic parsing, and then stream those messages to Apache Kafka.

Coding the Collector is Go was a very interesting experience. I am impressed with Go — I find that the code I write behaves in the manner I expect. The Go net package also had some nice features that helped enormously to deal with some common corner cases. For example, you can easily trigger timeouts if data has not been received on a socket after a certain period of time. Go also comes with basic packages for collecting statistics within the program itself. Features like this show me that the Go creators really did have system software in mind when creating the language.

I also really enjoy that I don’t have to write code using a heavyweight IDE, which is almost required when coding in Java. I coded the Collector using vim, though I took the time to install the vim-go plugin. I do plan a more detailed post in the future describing my Go development setup.

I hope to develop this project further. You can find the code on github.

3 thoughts on “Writing a Syslog Collector in Go”

  1. Hi, I get this error trying to compile the syslog-gollector tool.

    user@box:/home/syslog-gollector# go install github.com/otoolep/syslog-gollector
    src/github.com/otoolep/sarama/snappy.go:5:2: cannot find package “code.google.com/p/snappy-go/snappy” in any of:
    /usr/lib/go/src/pkg/code.google.com/p/snappy-go/snappy (from $GOROOT)
    /home/syslog-gollector/src/code.google.com/p/snappy-go/snappy (from $GOPATH)

Leave a Reply

Your email address will not be published. Required fields are marked *