Traceloop for systemd and Kubernetes + Inspektor Gadget (asg2019)

Presenting [traceloop](https://github.com/kinvolk/traceloop), a “time travel” tracing tool to trace system calls in cgroups using BPF and overwritable ring buffers. Many people use the “strace” tool to synchronously trace system calls using ptrace. [Traceloop](https://github.com/kinvolk/traceloop) similarly traces system calls but asynchronously in the background, using BPF and tracing per cgroup. I’ll show how it can be integrated with systemd and with Kubernetes via [Inspektor Gadget](https://github.com/kinvolk/inspektor-gadget). Traceloop's traces are recorded in a fast, in-memory, overwritable ring buffer like a flight recorder. As opposed to “strace”, the tracing could be permanently enabled on systemd services or Kubernetes pods and inspected in case of a crash. This is like a always-on “strace in the past”. Traceloop uses BPF through the gobpf library. Several new features have been added in gobpf for the needs of traceloop: support for overwritable ring buffers and swapping buffers when the userspace utility dumps the buffer. https://github.com/kinvolk/traceloop https://github.com/kinvolk/inspektor-gadget https://github.com/iovisor/gobpf Slides: https://docs.google.com/presentation/d/1zIZUrTrD7FkS9pHnWz87ZmoLTrO1g9-J_lDMD7E5kdo/edit about this event: https://cfp.all-systems-go.io/ASG2019/talk/98A9LW/