Stupid Systems Monitoring Tricks

Yes, we are all supposed to have moved into the fourth age of Web 7.0 systems monitoring (I think the cool kids are calling it observability today?), but sometimes you need to know the state of a machine without a collection endpoint on it. Or you are offended by the thought of loading 68MB of HTML and Javascript when a simple shell command will do.

Print all sockets matching a certain PID:

~> while true; do sockstat -4 | egrep $PID; sleep 2; echo '-----'; done

Print number of file descriptors in use:

~> while true; do sysctl -a | egrep openfiles; sleep 2; echo '-----'; done

And as a personal reminder, don't forget to check ulimit to see what your session allows when you invariably run out of memory/filedescriptors/whatever:

~> ulimit -a
-t: cpu time (seconds)              unlimited
-f: file size (blocks)              unlimited
-d: data seg size (kbytes)          33554432
-s: stack size (kbytes)             524288
-c: core file size (blocks)         unlimited
-m: resident set size (kbytes)      unlimited
-l: locked-in-memory size (kbytes)  64
-u: processes                       8499
-n: file descriptors                116946
-b: socket buffer size (bytes)      unlimited
-v: virtual memory size (kbytes)    unlimited
-p: pseudo-terminals                unlimited
-w: swap size (kbytes)              unlimited
-k: kqueues                         unlimited
-N 14:                              unlimited