We now support, in standard, application counters using an UDP listener embedded inside Knock Daemon backed by public SDKs.
Official SDKs:
We're not UDP gurus. So why UDP? Let's start by some history.
During last past years, we implemented several version of application counters. One of the latest resides in pythonsol MeterManager.
It was working with file exchange (applications were writing a supervision file, embedding all counters, every minute; supervision daemon was reading, parsing and pushing these counters). This was fine.
But we were using advanced applications, acting as daemons, with threads and/or co-routines available.
With much more simple applications, like php or scripts langages, this cannot or hardly be done (no threads, no async stuff, no persistence accross requests).
For similar reasons, TCP was not an option (locking application while sending counters was not possible in a synchronous world).
We ended up using UDP. It's not reliable as TCP, but it will not lock applications, is fire-and-forget, works locally (/var/run/knockdaemon.udp.socket).
Yes, we may lost counters somewhere. But our UDP listener is able to sustain several thousands UDP counters per seconds; and your application will not be locked.
A simple counter, storing a current instant value. Last value provided wins.
For instance, it can be used to graph current baskets on your web site.
SDK python sample:from knock import Knock
def get_basket_count_from_mysql():
# Doing mysql request...
# Get result
basket_count = result["basket_count"]
# Push
Knock.gauge("current.mysql.basket.count", basket_count)
# Commit
Knock.commit()
Gauge visual sample (current http request average response time):
A simple counter, storing cumulative values and graphing them as delta per second.
For instance, it can be used to graph current application requests per second.
SDK python sample:from knock import Knock
def handle_dummy_request():
# One request more
Knock.increment("http.request.dummy.in", 1)
try:
# Do something
do_something()
except:
# One request failed more
Knock.increment("http.request.dummy.ko", 1)
finally:
# Commit
Knock.commit()
Increment visual sample (current http request processed per second):
A simple counter, storing execution time and graphing them by execution time interval.
For instance, it can be used to graph current application request execution times.
SDK python sample:
from knock import Knock
def handle_dummy_request():
# Request start now
t1 = Knock.start_delay("http.request.dummy.ms")
# Do something
do_something()
# Request end now
Knock.stop_delay(t1)
# Commit
Knock.commit()
Delay-to-count visual sample (http response time, graphed as request per second, by time intervals):
You must explicitly call Knock.commit()
.
Call Knock.commit()
at the end of one HTTP request or, better, at regular interval - 30 sec or one minute - if you can implement this.
This commit() call will flush UDP pending counters toward KnockDaemon.
If you call it too often, you will spam UDP and will loose performance at client side.
If you don't call it, you will loose counters.
At UDP Listener end, the maximum binary size of one UDP packet is 61440 bytes.
Use ASCII characters for key naming.
There is a hard limit on item string key length you can use, do not abuse of huge keys.
Do not dynamically create item keys for each request (for instance, creating a different key for each request will be totally useless, you will have one value per key, so a graph with a single data...).
Think to correctly name your keys, these are the unique identifiers of your application counters.
Think to prefix keys by a component name. Prefixing with "http."
or "cron."
or similar is highly recommended.
"."
and "_"
delimiters are welcome.
Do not engage counters in massive for loop, you may loose performance at client side and overload UDP channel.
Instead, populate counters after for loop - for instance by doing a Knock.increment("http.myforloop.itemprocessed", 1000000)
.