Commit Graph
9 Commits
Author SHA1 Message Date
Arunprasad Rajkumar c4e43dc412 assets: regenerate
Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2022-04-28 10:40:06 +05:30
Arunprasad Rajkumar b54ad2ea71 Adjust NodeFilesystemSpaceFillingUp thresholds according default kubelet GC behavior
Previously[1] we attempted to do the same, but there was a
misunderstanding about the GC behavior and it caused the alert to be
fired even before GC comes into play.

According to[2][3] kubelet GC kicks in only when `imageGCHighThresholdPercent` is hit which is set to 85% by default. However `NodeFilesystemSpaceFillingUp` is set to fire as soon as 80% usage is hit.

This commit changes the `fsSpaceFillingUpWarningThreshold` to 15% so
that we give ample time to GC to reclaim unwanted images. This commit
also changes `fsSpaceFillingUpCriticalThreshold` to 10% which gives more time to admins to react to warning before sending critical alert.

[1] https://github.com/prometheus-operator/kube-prometheus/pull/1357
[2] https://docs.openshift.com/container-platform/4.10/nodes/nodes/nodes-nodes-garbage-collection.html#nodes-nodes-garbage-collection-images_nodes-nodes-configuring
[3] https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/

Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
(cherry picked from commit 6ff8bfbb02)
2022-04-27 08:03:57 +05:30
Arunprasad Rajkumar 1ff5273f87 Deprecate thanosSelector and keep until 0.10.0
Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2021-12-13 16:23:37 +05:30
Arunprasad Rajkumar 1fb864ec21 Remove thanosSelector and expose mixin._config.thanos config variable for thanos sidecar
This commit removes existing `thanosSelector` and exposes a single config variable `mixin._config.thanos` to customize thanos sidecar mixins. It follows same structure as https://github.com/thanos-io/thanos/blob/d2d74dac9878f633f1fb0e253bcc0b1cdea376d3/mixin/config.libsonnet

Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2021-12-10 08:56:44 +05:30
Arunprasad Rajkumar 19347ed662 Fix typo in jsonnet dependency upgrade GH actions
Replace `commiter` to `committer`.

Refer https://github.com/peter-evans/create-pull-request#action-inputs for more
info about supported input properties for `peter-evans/create-pull-request@v3`.

Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2021-10-19 16:03:20 +05:30
Arunprasad Rajkumar c5d265a14e thanos: bump to latest and add thanosPrometheusCommonDimensions
This commit pulls latest changes from thanos mixins and sets `thanosPrometheusCommonDimensions`
to `namespace, pod` for k8s use case.

Refer https://github.com/thanos-io/thanos/pull/4508 for more details.

Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2021-09-27 12:07:08 +05:30
Arunprasad Rajkumar 4de44139ec add comments to reason fsSpaceFilling threshold adjustment
Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2021-09-02 17:38:02 +05:30
Arunprasad Rajkumar 16ef18ec54 assets: regenerate
Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2021-09-01 13:33:41 +05:30
Arunprasad Rajkumar 03471fd86f Adjust threshold for SpaceFillingUp warning alert
Reduce threshold of NodeFilesystemSpaceFillingUp warning alert to 20% space available, instead of 40% (default).

This will align the threshold according to default kubelet GC values
below[1],

"imageMinimumGCAge": "2m0s",
"imageGCHighThresholdPercent": 85,
"imageGCLowThresholdPercent": 80,

[1] https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/

Signed-off-by: Arunprasad Rajkumar <arajkuma@redhat.com>
2021-09-01 13:29:36 +05:30