easylinux/kube-prometheus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #74 from goll/readme-fixes

Browse Source 
README fixes, Dockerfile bump
This commit is contained in:
Frederic Branczyk
2019-04-18 11:14:54 +02:00
committed by GitHub
parent 4d91bed740 0e27a3db68
commit d0fa86d3af
4 changed files with 188 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

96
README.md
View File

@@ -58,17 +58,17 @@ This adapter is an Extension API Server and Kubernetes needs to be have this fea
### minikube ### minikube
In order to just try out this stack, start minikube with the following command: In order to just try out this stack, start [minikube](https://github.com/kubernetes/minikube) with the following command:
``` ```shell
$ minikube delete && minikube start --kubernetes-version=v1.13.2 --memory=4096 --bootstrapper=kubeadm --extra-config=kubelet.authentication-token-webhook=true --extra-config=kubelet.authorization-mode=Webhook --extra-config=scheduler.address=0.0.0.0 --extra-config=controller-manager.address=0.0.0.0 $ minikube delete && minikube start --kubernetes-version=v1.13.5 --memory=4096 --bootstrapper=kubeadm --extra-config=kubelet.authentication-token-webhook=true --extra-config=kubelet.authorization-mode=Webhook --extra-config=scheduler.address=0.0.0.0 --extra-config=controller-manager.address=0.0.0.0
``` ```
> The kube-prometheus stack includes a resource metrics API server, like the metrics-server does. So ensure the metrics-server plugin is disabled on minikube: The kube-prometheus stack includes a resource metrics API server, like the metrics-server does. So ensure the metrics-server plugin is disabled on minikube:
>
> ``` ```shell
> minikube addons disable metrics-server $ minikube addons disable metrics-server
> ``` ```
## Quickstart ## Quickstart
@@ -76,7 +76,7 @@ This project is intended to be used as a library (i.e. the intent is not for you
Though for a quickstart a compiled version of the Kubernetes [manifests](manifests) generated with this library (specifically with `example.jsonnet`) is checked into this repository in order to try the content out quickly. To try out the stack un-customized run: Though for a quickstart a compiled version of the Kubernetes [manifests](manifests) generated with this library (specifically with `example.jsonnet`) is checked into this repository in order to try the content out quickly. To try out the stack un-customized run:
* Simply create the stack: * Simply create the stack:
``` ```shell
$ kubectl create -f manifests/ $ kubectl create -f manifests/
# It can take a few seconds for the above 'create manifests' command to fully create the following resources, so verify the resources are ready before proceeding. # It can take a few seconds for the above 'create manifests' command to fully create the following resources, so verify the resources are ready before proceeding.
@@ -87,7 +87,7 @@ $ kubectl apply -f manifests/ # This command sometimes may need to be done twice
``` ```
* And to teardown the stack: * And to teardown the stack:
``` ```shell
$ kubectl delete -f manifests/ $ kubectl delete -f manifests/
``` ```
@@ -100,7 +100,7 @@ Prometheus, Grafana, and Alertmanager dashboards can be accessed quickly using `
Prometheus Prometheus
```shell ```shell
kubectl --namespace monitoring port-forward svc/prometheus-k8s 9090 $ kubectl --namespace monitoring port-forward svc/prometheus-k8s 9090
``` ```
Then access via [http://localhost:9090](http://localhost:9090) Then access via [http://localhost:9090](http://localhost:9090)
@@ -108,7 +108,7 @@ Then access via [http://localhost:9090](http://localhost:9090)
Grafana Grafana
```shell ```shell
kubectl --namespace monitoring port-forward svc/grafana 3000 $ kubectl --namespace monitoring port-forward svc/grafana 3000
``` ```
Then access via [http://localhost:3000](http://localhost:3000) and use the default grafana user:password of `admin:admin`. Then access via [http://localhost:3000](http://localhost:3000) and use the default grafana user:password of `admin:admin`.
@@ -116,7 +116,7 @@ Then access via [http://localhost:3000](http://localhost:3000) and use the defau
Alert Manager Alert Manager
```shell ```shell
kubectl --namespace monitoring port-forward svc/alertmanager-main 9093 $ kubectl --namespace monitoring port-forward svc/alertmanager-main 9093
``` ```
Then access via [http://localhost:9093](http://localhost:9093) Then access via [http://localhost:9093](http://localhost:9093)
@@ -132,7 +132,7 @@ This section:
The content of this project consists of a set of [jsonnet](http://jsonnet.org/) files making up a library to be consumed. The content of this project consists of a set of [jsonnet](http://jsonnet.org/) files making up a library to be consumed.
Install this library in your own project with [jsonnet-bundler](https://github.com/jsonnet-bundler/jsonnet-bundler#install) (the jsonnet package manager): Install this library in your own project with [jsonnet-bundler](https://github.com/jsonnet-bundler/jsonnet-bundler#install) (the jsonnet package manager):
``` ```shell
$ mkdir my-kube-prometheus; cd my-kube-prometheus $ mkdir my-kube-prometheus; cd my-kube-prometheus
$ jb init # Creates the initial/empty `jsonnetfile.json` $ jb init # Creates the initial/empty `jsonnetfile.json`
# Install the kube-prometheus dependency # Install the kube-prometheus dependency
@@ -144,7 +144,9 @@ $ jb install github.com/coreos/kube-prometheus/jsonnet/kube-prometheus # Creates
> An e.g. of how to install a given version of this library: `jb install github.com/coreos/prometheus-operator/contrib/kube-prometheus/jsonnet/kube-prometheus/@v0.22.0` > An e.g. of how to install a given version of this library: `jb install github.com/coreos/prometheus-operator/contrib/kube-prometheus/jsonnet/kube-prometheus/@v0.22.0`
In order to update the kube-prometheus dependency, simply use the jsonnet-bundler update functionality: In order to update the kube-prometheus dependency, simply use the jsonnet-bundler update functionality:
`$ jb update` ```shell
$ jb update
```
### Compiling ### Compiling
@@ -201,54 +203,41 @@ This script runs the jsonnet code, then reads each key of the generated json and
### Apply the kube-prometheus stack ### Apply the kube-prometheus stack
The previous steps (compilation) has created a bunch of manifest files in the manifest/ folder. The previous steps (compilation) has created a bunch of manifest files in the manifest/ folder.
Now simply use kubectl to install Prometheus and Grafana as per your configuration: Now simply use `kubectl` to install Prometheus and Grafana as per your configuration:
`kubectl apply -f manifests/` ```shell
$ kubectl apply -f manifests/
```
Check the monitoring namespace (or the namespace you have specific in `namespace: `) and make sure the pods are running. Prometheus and Grafana should be up and running soon. Check the monitoring namespace (or the namespace you have specific in `namespace: `) and make sure the pods are running. Prometheus and Grafana should be up and running soon.
### Containerized Installing and Compiling ### Containerized Installing and Compiling
If you don't care to have `jb` nor `jsonnet` nor `gojsontoyaml` installed, then build the `po-jsonnet` Docker image (this is something you'll need a copy of this repository for). Do the following from this `kube-prometheus` directory: If you don't care to have `jb` nor `jsonnet` nor `gojsontoyaml` installed, then build the `po-jsonnet` Docker image (this is something you'll need a copy of this repository for). Do the following from this `kube-prometheus` directory:
``` ```shell
$ make ../../hack/jsonnet-docker-image $ make hack/jsonnet-docker-image
``` ```
Then you can do commands such as the following: Then you can do commands such as the following:
``` ```shell
docker run \ $ docker run --rm -v $(pwd):$(pwd) --workdir $(pwd) po-jsonnet jb update
--rm \ $ docker run --rm -v $(pwd):$(pwd) --workdir $(pwd) po-jsonnet ./build.sh example.jsonnet
-v `pwd`:`pwd` \
--workdir `pwd` \
po-jsonnet jb init
docker run \
--rm \
-v `pwd`:`pwd` \
--workdir `pwd` \
po-jsonnet jb install github.com/coreos/prometheus-operator/contrib/kube-prometheus/jsonnet/kube-prometheus
docker run \
--rm \
-v `pwd`:`pwd` \
--workdir `pwd` \
po-jsonnet ./build.sh example.jsonnet
``` ```
## Update from upstream project ## Update from upstream project
You may wish to fetch changes made on this project so they are available to you. You may wish to fetch changes made on this project so they are available to you.
### Update jb ### Update jb
jb may have been updated so it's a good idea to get the latest version of this binary `jb` may have been updated so it's a good idea to get the latest version of this binary:
``` ```shell
go get -u github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb $ go get -u github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb
``` ```
### Update kube-prometheus ### Update kube-prometheus
The command below will sync with upstream project. The command below will sync with upstream project:
``` ```shell
jb update $ jb update
``` ```
### Compile the manifests and apply ### Compile the manifests and apply
@@ -266,13 +255,13 @@ These are the available fields with their respective default values:
namespace: "default", namespace: "default",
versions+:: { versions+:: {
alertmanager: "v0.16.1", alertmanager: "v0.16.2",
nodeExporter: "v0.17.0", nodeExporter: "v0.17.0",
kubeStateMetrics: "v1.5.0", kubeStateMetrics: "v1.5.0",
kubeRbacProxy: "v0.4.1", kubeRbacProxy: "v0.4.1",
addonResizer: "1.8.4", addonResizer: "1.8.4",
prometheusOperator: "v0.29.0", prometheusOperator: "v0.29.0",
prometheus: "v2.5.0", prometheus: "v2.7.2",
}, },
imageRepos+:: { imageRepos+:: {
@@ -408,12 +397,12 @@ To produce the `docker pull/tag/push` commands that will synchronize upstream im
```shell ```shell
$ jsonnet -J vendor -S --tla-str repository=internal-registry.com/organization sync-to-internal-registry.jsonnet $ jsonnet -J vendor -S --tla-str repository=internal-registry.com/organization sync-to-internal-registry.jsonnet
docker pull k8s.gcr.io/addon-resizer:1.8.4 $ docker pull k8s.gcr.io/addon-resizer:1.8.4
docker tag k8s.gcr.io/addon-resizer:1.8.4 internal-registry.com/organization/addon-resizer:1.8.4 $ docker tag k8s.gcr.io/addon-resizer:1.8.4 internal-registry.com/organization/addon-resizer:1.8.4
docker push internal-registry.com/organization/addon-resizer:1.8.4 $ docker push internal-registry.com/organization/addon-resizer:1.8.4
docker pull quay.io/prometheus/alertmanager:v0.16.1 $ docker pull quay.io/prometheus/alertmanager:v0.16.2
docker tag quay.io/prometheus/alertmanager:v0.16.1 internal-registry.com/organization/alertmanager:v0.16.1 $ docker tag quay.io/prometheus/alertmanager:v0.16.2 internal-registry.com/organization/alertmanager:v0.16.2
docker push internal-registry.com/organization/alertmanager:v0.16.1 $ docker push internal-registry.com/organization/alertmanager:v0.16.2
... ...
``` ```
@@ -626,7 +615,6 @@ the following process:
1. Make your changes in the respective `*.jsonnet` file. 1. Make your changes in the respective `*.jsonnet` file.
2. Commit your changes (This is currently necessary due to our vendoring 2. Commit your changes (This is currently necessary due to our vendoring
process. This is likely to change in the future). process. This is likely to change in the future).
3. Update the pinned kube-prometheus dependency in `jsonnetfile.lock.json`: `jb 3. Update the pinned kube-prometheus dependency in `jsonnetfile.lock.json`: `jb update`
update`. 3. Generate dependent `*.yaml` files: `make generate-in-docker`
3. Generate dependent `*.yaml` files: `make generate-in-docker`.
4. Commit the generated changes. 4. Commit the generated changes.

8
jsonnetfile.lock.json
View File

@@ -8,7 +8,7 @@
"subdir": "jsonnet/kube-prometheus" "subdir": "jsonnet/kube-prometheus"
} }
}, },
"version": "cb4b913c32d053cddb5d4a2d572eaf11b55d8971" "version": "4d91bed7405bcc6929f71452d7cc3735284da69d"
}, },
{ {
"name": "ksonnet", "name": "ksonnet",
@@ -28,7 +28,7 @@
"subdir": "" "subdir": ""
} }
}, },
"version": "c4acc6e194632ca134fe56de7e4e4e9c7cef2221" "version": "6187d7dbbf4bf0b7d34d08e6773dc0e101676c58"
}, },
{ {
"name": "grafonnet", "name": "grafonnet",
@@ -48,7 +48,7 @@
"subdir": "grafana-builder" "subdir": "grafana-builder"
} }
}, },
"version": "11d1552f120f2d9e8d932a671faf67beae91503a" "version": "233757a0a9047eb74993c977cd5b1be0fb54e8d6"
}, },
{ {
"name": "grafana", "name": "grafana",
@@ -78,7 +78,7 @@
"subdir": "Documentation/etcd-mixin" "subdir": "Documentation/etcd-mixin"
} }
}, },
"version": "f29b1ada19713544b698dab8c94c97cfa1e83dac" "version": "cd7ffbe2270d5290e708f55edfbfae840ec879bb"
} }
] ]
} }

142
manifests/grafana-dashboardDefinitions.yaml
View File

@@ -7299,6 +7299,13 @@ items:
"intervalFactor": 2, "intervalFactor": 2,
"legendFormat": "load 15m", "legendFormat": "load 15m",
"refId": "C" "refId": "C"
},
{
"expr": "count(node_cpu_seconds_total{cluster=\"$cluster\", job=\"node-exporter\", instance=\"$instance\", mode=\"user\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "logical cores",
"refId": "D"
} }
], ],
"thresholds": [ "thresholds": [
@@ -8947,7 +8954,20 @@ items:
], ],
"annotations": { "annotations": {
"list": [ "list": [
{
"builtIn": 1,
"datasource": "$datasource",
"enable": true,
"expr": "time() == BOOL timestamp(rate(kube_pod_container_status_restarts_total{job=\"kube-state-metrics\", cluster=\"$cluster\", namespace=\"$namespace\", pod=\"$pod\"}[2m]) > 0)",
"hide": false,
"iconColor": "rgba(215, 44, 44, 1)",
"name": "Restarts",
"showIn": 0,
"tags": [
"restart"
],
"type": "rows"
}
] ]
}, },
"editable": false, "editable": false,
@@ -9027,6 +9047,13 @@ items:
"intervalFactor": 2, "intervalFactor": 2,
"legendFormat": "Limit: {{ container }}", "legendFormat": "Limit: {{ container }}",
"refId": "C" "refId": "C"
},
{
"expr": "sum by(container_name) (container_memory_cache{job=\"kubelet\", namespace=\"$namespace\", pod_name=~\"$pod\", container_name=~\"$container\", container_name!=\"POD\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "Cache: {{ container_name }}",
"refId": "D"
} }
], ],
"thresholds": [ "thresholds": [
@@ -9247,8 +9274,15 @@ items:
"expr": "sort_desc(sum by (pod_name) (rate(container_network_receive_bytes_total{job=\"kubelet\", cluster=\"$cluster\", namespace=\"$namespace\", pod_name=\"$pod\"}[1m])))", "expr": "sort_desc(sum by (pod_name) (rate(container_network_receive_bytes_total{job=\"kubelet\", cluster=\"$cluster\", namespace=\"$namespace\", pod_name=\"$pod\"}[1m])))",
"format": "time_series", "format": "time_series",
"intervalFactor": 2, "intervalFactor": 2,
"legendFormat": "{{ pod_name }}", "legendFormat": "RX: {{ pod_name }}",
"refId": "A" "refId": "A"
},
{
"expr": "sort_desc(sum by (pod_name) (rate(container_network_transmit_bytes_total{job=\"kubelet\", cluster=\"$cluster\", namespace=\"$namespace\", pod_name=\"$pod\"}[1m])))",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "TX: {{ pod_name }}",
"refId": "B"
} }
], ],
"thresholds": [ "thresholds": [
@@ -9299,6 +9333,110 @@ items:
"title": "Dashboard Row", "title": "Dashboard Row",
"titleSize": "h6", "titleSize": "h6",
"type": "row" "type": "row"
},
{
"collapse": false,
"collapsed": false,
"panels": [
{
"aliasColors": {
},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "$datasource",
"fill": 1,
"gridPos": {
},
"id": 5,
"legend": {
"alignAsTable": true,
"avg": true,
"current": true,
"max": false,
"min": false,
"rightSide": true,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"links": [
],
"nullPointMode": "null",
"percentage": false,
"pointradius": 5,
"points": false,
"renderer": "flot",
"repeat": null,
"seriesOverrides": [
],
"spaceLength": 10,
"span": 12,
"stack": false,
"steppedLine": false,
"targets": [
{
"expr": "max by (container) (kube_pod_container_status_restarts_total{job=\"kube-state-metrics\", cluster=\"$cluster\", namespace=\"$namespace\", pod=\"$pod\", container=~\"$container\"})",
"format": "time_series",
"intervalFactor": 2,
"legendFormat": "Restarts: {{ container }}",
"refId": "A"
}
],
"thresholds": [
],
"timeFrom": null,
"timeShift": null,
"title": "Total Restarts Per Container",
"tooltip": {
"shared": false,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": [
]
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": 0,
"show": true
}
]
}
],
"repeat": null,
"repeatIteration": null,
"repeatRowId": null,
"showTitle": false,
"title": "Dashboard Row",
"titleSize": "h6",
"type": "row"
} }
], ],
"schemaVersion": 14, "schemaVersion": 14,

5
scripts/jsonnet/Dockerfile
View File

@@ -1,9 +1,8 @@
FROM golang:1.11-stretch FROM golang:1.12-stretch
ENV JSONNET_VERSION 0.12.1 ENV JSONNET_VERSION 0.12.1
RUN apt-get update -y && apt-get install -y g++ make git jq && \ RUN apt-get update -y && apt-get install -y jq && rm -rf /var/lib/apt/lists/*
rm -rf /var/lib/apt/lists/*
RUN curl -Lso - https://github.com/google/jsonnet/archive/v${JSONNET_VERSION}.tar.gz | \ RUN curl -Lso - https://github.com/google/jsonnet/archive/v${JSONNET_VERSION}.tar.gz | \
tar xfz - -C /tmp && \ tar xfz - -C /tmp && \
cd /tmp/jsonnet-${JSONNET_VERSION} && \ cd /tmp/jsonnet-${JSONNET_VERSION} && \