Pre calculated Top-K counters
complete
F
FastNetMon
Our default Top-k dashboad is very CPU intensive and we need to provide better alternative for it.
We can pre-calculate top-K hosts (per direction, by bytes/s, packets/s, flow/s) on FastNetMon side and export only them.
It will reduce load on Clickhouse and even can be implemented for InfluxDB
F
FastNetMon
complete
F
FastNetMon
Done, 2.0.356.
Performance comparison.
Old:
7600 rows in set. Elapsed: 1.438 sec. Processed 388.31 million rows, 1.62 GB (270.00 million rows/s., 1.13 GB/s.)
New:
1802 rows in set. Elapsed: 0.016 sec. Processed 96.16 thousand rows, 779.96 KB (6.05 million rows/s., 49.05 MB/s.)
Dashboard changes: https://github.com/FastNetMon/fastnetmon-grafana-dashboards/commit/530aa5ab959018f63da701eba4d9bb3786cb84b2
FastNetMon changes needed:
sudo fcli set main clickhouse_metrics_export_top_hosts true