By Philipp Haidenbauer in medium — Apr 22, 2018

Kubernets with Prometheus and Grafana on HetznerCloud

Recently I started to mess around with kubernetes. I did it many times in the last year, but actually never build anything bigger with it. As I’m writing this I’m just starting out with the whole thing. I think I will make an article-series out of this, as it might be too much for one article. Some parts of this article are based on the work of xetys (David Steiman) (obviously we will be using his CLI) and on his article at his blog.

Let’s get started

First, you will need an account at https://www.hetzner.com/cloud. Thereafter you have to create a Project via “Add Project” inside the CloudConsole we will name it demo. Then you need to create an API-Token through “Access” > “API-Tokens” > “Generate API Token”, store it somewhere safe, as it’s only displayed once.

hetzner-cube configuration

Now let’s configure the hetzner-cube cli (currently you need go installed to use it).

Open the shell of you favor (I’m using the Windows PowerShell, as I’m doing the setup on Windows).

https://gist.github.com/af8e0b8b3c01cf38396ce2d1f8927bdc

After that you need to add an ssh-key:

https://gist.github.com/6ab9b20d1f4a1f1baa0e08fb1566c78e

Now we are ready to spin up the virtual machines and install the needed dependencies. The best thing about it: hetzner-cube does this for us:

https://gist.github.com/42bacc11324c7cb09f732958080a9406

This will create a simple cluster with one master and one worker. Of course, this is not the high-availability setup-method, but It’s the cheapest one (each instance costs 0,005 EUR per hour if you delete them you pay only the used hours). This will take a few minutes. When it’s done you will see to servers under the “Servers” Tab on the Dashboard:

demo-master-01
demo-worker-01

We will configure our local kubectl to work with the created cluster:

https://gist.github.com/899e0232d2fee08cc0ac8b0600c9b3f7

deploying the dashboard

Since the cluster is up and running and our local kubectl is running we can deploy the kubernets-dashboard to see what's going on. If you don’t need the dashboard you can skip to the next step. (Based on this GitHub issue, thanks to radutopala)

https://gist.github.com/c963e759ae3e81c11ae35752ac714255

To confirm it worked run:

https://gist.github.com/e05ce4a519b4158af53c14326b7fc4c5

The second entry is the one we are after. Time to create a user for it: Create a YAML file (“admin-user.yaml”) with the following contents:

https://gist.github.com/e850e16e18835bfbc2fb529f05468045

and execute it:

https://gist.github.com/4360a82f6b6513243bf60e2eabf3ddd4

We need to bind the ServiceAccount to the ClusterRole “cluster-admin”, so we need another file (“admin-user-cluster-role.yaml”) (you could also use the existing one as this are one-time commands):

https://gist.github.com/626c17e8bef13ab7432931618bbf2e47

great. The last step is to create the login token:

https://gist.github.com/2b8b9ec35224be04207c9d6cc36feda5

As with the Hetzner-API-Token, store this token somewhere safe, as it’s the login-token for the kubernetes-dashboard.

Now we can open a local proxy to the dashboard through the kubectl command:

https://gist.github.com/2a0b069e2b53cc97eec14d33ccea84b1

If you open 127.0.0.1:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login in your browser you should see the dashboard login. Select token and enter the previously created one.

deploying rook for storage

To deploy the rook storage we can use hetzner-cube again:

https://gist.github.com/cb0d97f1469b47121f8aac81df426fe3

This should complete in a few seconds. In the dashboard under “Storage Classes” you should seek “rook-block”, and there should be a “rook”-namespace under “Namespaces”.

deploying prometheus

Of course, kubernetes has its own package-manager called helm. So it’s pretty straight forward to deploy prometheus. First, install the helm-addon:

https://gist.github.com/2c0bea131954c7e2261a8e0e2c3d2d88

After that, you need to get a local copy of the prometheus-chart as we need to modify some values.

Edit the “values.yaml” file, and change the following: Inside the “server”-block is a “persistentVolume”-block which holds the “storageClass”-property. We uncomment it and change it to “rook-block”, as we are using “rook” as our storage provider. Makes sense, doesn’t it?

https://gist.github.com/d849d8a746af58cb6b37597526b0afa5

Now let’s deploy prometheus with the help of helm:

https://gist.github.com/5a06083e56572d574bd4030964d5f76e

This should be pretty fast, you can check if it worked by visiting the dashboard. You should have a “prometheus” namespace to select inside the “Namespace” dropdown. Also, there should be two “Persistent Volumes” under the appropriate menu-point.

deploying grafana

To our luck, there is also a chart for grafana. And again we are going to edit the “values.yaml” file:

https://gist.github.com/cba099fd27dd5ee85aedabe5f13e5fb6

Under the “persistence”-nlock we set the following:

https://gist.github.com/42b6240c2a11ec69243bc9d30864bfc8

Let’s deploy grafana with the help of helm:

https://gist.github.com/62f9a3fafb57eda93af5bd40ef612a02

Now we need to port-forward the grafana-dashboard to our local machine, for that we need the instance name of grafana, we can get it through:

https://gist.github.com/3e8191175327b07577dcb1c811285b2c

and then when can open the proxy:

https://gist.github.com/09926e1c6f039e8d4d45ff2df7c42d78

Now we can navigate to http://127.0.0.1:3000/ and we should see the grafana UI.

configuring grafana

In the dashboard go to “Add data source” and enter the following:

Click “Save & Test”.

importing the default prometheus dashboard

Select the freshly created Kubernetes Datasource and go to the “Dashboard”-Tab. Click on Import on the desired dashboards (ex. “Prometheus 2.0 Stats”). If you go back to the Dashboards-Site and click on the dropdown labeled “Home” you can now select “Prometheus 2.0 Stats”. This dashboard shows stats about prometheus itself, we want to see the kubernetes metrics.

To import them we click on “+”-sign an select “Import”. We are going to use a dashboard hosted hat the garfana-dashboard store called Kubernetes All Nodes. Just paste the URL inside the input and you should get a new interface.

You could change the name if you want. In the “prometheus”-input select our created “Kubernetes” Datasource and click “Import”.

And that's it. You can now play around with the whole cluster. If you would like to delete the whole thing and don’t pay any more just run the following command:

https://gist.github.com/a6450a7c2e542485d2e604b2aa265dc4

finally

While I was writing this the cluster costs me about 0,03 EUR. Thanks for reading and till next time!

Table of Contents