/

Kubernetes Node Metrics

Analyzing node metrics collected by the kubelet


The nodeMetrics analyzer is available to analyze node metrics data collected by kubelet and served via kubernetes API server. The metrics are collected by the nodeMetrics collector. The analyzer can be used in support bundles or preflights that need to report on checks such as pvc usage capacity violations.

This analyzer's when outcome clause compares the condition specified with the resources present such as a pvc.

This analyzer also supports a filters property. If provided, the resources analyzed are filtered to any resource that matches the filters specified. If no filters are specified, all collected metrics are inspected by the analyzer.

Available Filters

Filters can be used to limit what resources are analyzed. Filters are usually used in conjunction with a few other outcome fields.

Filter NameDescription
pvc.namespaceThe namespace the PVC has been deployed. Used to filter down PVC resources to analyze. It is used in conjunction with pvcUsedPercentage outcome.
pvc.nameRegexA regular expression of the PVC name. Used to filter down PVC resources to analyze. It is used in conjunction with pvcUsedPercentage outcome.

Outcomes

The when value in an outcome of this analyzer contains scalar quantities such as percentages. They are compared with generated values that are generated from various values in the raw metrics. Comparisions are done using available logical operators.

The conditional in the when clause can accept the following fields:

FieldDescription
pvcUsedPercentagePercentage value to compare with the available space remaining in a PVC. The formula used is (available space / capacity ) * 100

The message field can contain strings represting go text templates. The analyzer supports the following template placeholders:

FieldDescription
PVC.ConcatenatedNamesComma separated concatenated list of PVC names that matched the defined when clause from the filtered list of resources
PVC.NamesList of PVC names that matched the defined when clause from the filtered list of resources

Example Analyzer Definitions

apiVersion: troubleshoot.sh/v1beta2
kind: Preflight
spec:
  analyzers:
    - nodeMetrics:
        checkName: Check for PVCs using more than 80% storage space in the entire cluster
        outcomes:
          - fail:
              when: "pvcUsedPercentage >= 80"
              message: "There are PVCs using more than 80% of storage: {{ .PVC.ConcatenatedNames }}"
          - pass:
              message: "No PVCs are using more than 80% of storage"

Example of filtering the PVC resources to analyze:

apiVersion: troubleshoot.sh/v1beta2
kind: Preflight
spec:
  analyzers:
    - nodeMetrics:
        checkName: Check if minio pvc storage usage is less than 80%
        filters:
          pvc:
            nameRegex: "minio-data-ha-minio.*"
            namespace: "minio"
        outcomes:
          - fail:
              when: "pvcUsedPercentage >= 80"
              message: ""There are {{ len .PVC.Names }} PVCs using more than 80% of storage""
          - pass:
              message: "No PVCs are using more than 80% of storage"
Edit on GitHub