Realtime stats with StatsD / Graphite

Who I am

• Matthias Lüdtke
• Engineer with Dynport, Hamburg
• Heavy Java past, now doing Ruby on Rails
• Working on simfy.de
• More the backend kind of guy

• xing.to/mat
• github.com/mat
• twitter.com/mmmmat

What I want to show you

• Draw application metrics like cache hits/misses, API performance, external response times, ...

• in near realtime
• without impacting runtime performance

Features

• handles high volume data (i.e. measure anything you need)
• near realtime (10s delay)
• counters (+1 or +n)
• timers (47.11ms)
• sampling, if needed
• flexible, add new metrics on the fly

Ingredients

• Your App
• Sends metric data to StatsD
• StatsD
• Acts as buffering, aggregating proxy
• Talks UDP (incoming)
• Graphite
• Storage backend
• Uses round robin database to store metrics with decreasing granularity over time
• Provides graphing capabilities

Architecture

Architecture

Sending a UDP packet from Ruby

  
$ irb
>> require 'socket'
>> UDPSocket.new.send(message="foo:5000|ms", flags=0, host="graphite", port=8125)
  
  

Hammer time

What it can't do (well)

• handle non continuous data
• discerning absolute values
• measuring gauges (current_user_count(n), ..., current_user_count(m))

What it's good for

• quick, near realtime feedback on code changes
• hypothesis checking (How often is this code getting invoked? How fast is it?)
• recognizing trends
• visualizing irregularities
• relative comparisons (last week, ...)

Further reading

• Initial idea: http://code.flickr.com/blog/2008/10/27/counting-timing/
• nodejs implementation anno 2011: http://codeascraft.etsy.com/2011/02/15/measure-anything-measure-everything/
• Graphite URL API: http://readthedocs.org/docs/graphite/en/latest/url-api.html/
• Setup: https://gist.github.com/1142828