On the infrastructure side I’ve been experimenting on running continues profiling on the hosts along with the node. This allows to see in which function the node is spending it’s time. During an active DoS bug/attack, we can look what code paths are causing this. The data is stored for a few days and allows us to go back and can be inspected a for a certain time-range too. At the moment I’m using https://parca.dev for this as it integrates well with Grafana. I think GitHub - anakryiko/wprof: High-performance system-wide BPF-based workload tracer with Perfetto-backed trace visualization. · GitHub is also an option. It doesn’t integrate with Grafana AFAIK, but stores tracing data as https://perfetto.dev/ files and can be stored and analyzed later. Yet another option would be to roll our own callstack-extractor only hooking into Bitcoin Core (not system wide) as described in call-stack extractor: In which function is `bitcoind` spending it's time? · Issue #391 · peer-observer/peer-observer · GitHub - more work on our side, but can be specialized for Bitcoin Core.
The current Grafana-based parca flamegraph looks similar to this:
