Expose per-module and per-container resource metrics

[gsanchietti]

When a node is under load or resources run low, administrator require a way to tell which module (or container inside one) is consuming most CPU/memory/disk on a node.

Proposed solution

Collect per-module/per-container CPU and memory metrics, and expose them inside Prometheus.
Create also a Grafana dashboard to expose the new metrics.

Alternative solutions

• Rely on podman stats / manual SSH inspection per node
• Use node-level-only metrics

[gsanchietti]

Testing images:

• core:3.22.0-dev.8
• metrics:1.3.2-dev.2

Test case 1

• Install on a clean machine
• Verify the following:
  • refresh-volume-metrics timer should be enabled, use systemctl list-timers
  • refresh-container-metrics service should be enabled and running, use systemctl status refresh-container-metrics
  • execute once systemctl start refresh-volume-metrics timer.service to gather storage metrics (it's executed once a day)
• Enable and access Grafana, verify there is a new "Container" dashboard; open the dashboard and verify there are data inside

Test case 2

• Update an existing machine with some modules
• Verify the same points from test case 1