This patch introduces a new Custom Resource Definition to the
Prometheus Operator - the Rule CRD. It addresses two main
needs:
1. Prometheus (alerting and recording) Rule validation during creation time
via Kubernetes Custom Resource Definition validation.
2. Life-cycle management of Prometheus application Rules alongside the
application itself, inside the applications Kubernetes namespace, not
necessarily the namespace of the scraping Prometheus instance.
A user defines Prometheus alerting and recording Rules via a Kubernetes
Custom Resource Definition. These Custom Resource Definitions can be
fully validated by the Kubernetes API server during creation time via
automatically generated OpenAPI specifications. Instead of the
restriction of a Prometheus instance to only select Rule definitions
inside its own namespace, the Prometheus specification is extended to
also specify namespaces to look for Rule Custom Resource Definitions
outside its own namespace.
---
Dependent technical changes:
- prometheus: Use github.com/jimmidyson/configmap-reload to reload rules
- prometheus: Remove Prometheus Statefulset deletion function. Starting
with K8s >=1.8 this is handled via OwnerReferences.
- prometheus: Do not add rule files checksum to Prometheus configuration
secret
- prometheus: Update StatefulSet only on relevant changes. Instead of
updating the Prometheus StatefulSet on every `sync()` run, only update
it if the input parameters to `makeStatefulSet` change. Enforce this
via a checksum of the parameters which is saved inside the annotations
of the statefulset.
- e2e/prometheus: Check how often resources (Secret, ConfigMap,
Prometheus CRD, Service) are updated to enforce that Prometheus Operator
only updated created resources if necessary.
- contrib/prometheus-config-reloader: Remove logic to retriev K8s
ConfigMaps. These are mounted into the pod right away now.
According to the man pages of `rm` and the `-f` option:
> Attempt to remove the files without prompting for confirma- tion,
> regardless of the file's permissions. If the file does not exist, do
> not display a diagnostic message or modify the exit status to reflect an
> error. The -f option overrides any previous -i options.
This patch prevents the `build.sh` script from failing if files do not
exist when attempting to delete them.
* kube-prometheus: fix alert rule K8SManyNodesNotReady
* fix alert "K8SManyNodesNotReady" in helm templates & make generate
* Use sync_kube_prometheus.py to make rules in helm in sync
The addon resizer was referring to an incorrect role binding and was unable to
access the API. This changes the rolebinding name to refer to the correctly
defined rolebinding.
With 1d00e0ab ("ServiceMonitor: Enable Prometheus to select ServMon
outside own ns") merged: To emphasize the fact, that ServiceMonitors can
be deployed alongside their targets, this patch moves the Alertmanager,
Kube-State-Metrics, Node-Exporter and Prometheus Operator
Service-Monitor into the respective target folder in the kube-prometheus
project.
In kubeadm 1.10 (and perhaps earlier versions, not sure), ctrl-mgr and scheduler pods' label key is `component=` rather than `k8s-app=`, which is the selector key used for kube-prometheus exporters for these components.
* charts: Add Core DNS Support
With CoreDNS becoming a first class citizen it should be an optional part
of the kube-prometheus deployment.
Fixes: #1174
* added kube-prometheus service and service monitor
In order to work directly with a cluster created by bootkube used port 9153,
and created a service exposing metrics since bootkube does does not. Also in
the helm chart changed the default port to 9153 since that is the coredns
plugins default port and to be consistent throughout the repo.
The current example-app setup in the kube-prometheus project is able to
discover scraping targets in the default namespace. It is not able to
discover the configured Alertmanager in the monitoring namespace.
This patch adds an alertmanager-discovery rbac role, to permit the
above described action. In addition it does the following cleanups:
- Remove kubeconfig configuration in deploy and teardown script. kubectl
chooses .kube/config whenever KUBECONFIG is not set by default
- Remove namespace specification option via NAMESPACE env var. In most
of the manifests the metadata/namespace was hardcoded anyways, in
addition in the promtheus frontend role binding the service account
namespace is hardcoded to default as well.
- Instead of `kubectl {apply,delete}` individual manifests, the deploy
and teardown shell scripts {apply,delete} on the entire folder.
In some cases we have been using `alert-rules` in some cases
`prometheus-rulefiles`. This led to confusion [1]. Instead, unify the
Prometheus rules configmap labels to:
```yaml
labels:
role: alert-rules
```
[1] https://github.com/coreos/prometheus-operator/issues/1102
The `prometheus-frontend` role of the example app kubeprometheus section
is scoped to the default namespace. Thereby the frontend Prometheus
instance is not able to discover anything outside of the default
namespace. We might as well restrict the front end service monitor to
the default namespace too.
