- Published on
Deploy a Monitoring Stack on Docker Swarm with Grafana and Prometheus
- Authors
- Name
- Ruan Bekker
- @ruanbekker
In this tutorial we will deploy a monitoring stack to docker swarm, that includes Grafana, Prometheus, Node-Exporter, cAdvisor and Alertmanager.
If you are looking for more information on Prometheus, have a look at my other Prometheus and Monitoring blog posts.
What you will get out of this
Once you deployed the stacks, you will have the following:
- Access Grafana through Traefik reverse proxy
- Node-Exporter to expose node level metrics
- cAdvisor to expose container level metrics
- Prometheus to scrape the exposed entpoints and ingest it into Prometheus
- Prometheus for your Timeseries Database
- Alertmanager for firing alerts on configured rules
The compose file that I will provide will have pre-populated dashboards
Deploy Traefik
Get the traefik stack sources:
$ git clone https://github.com/bekkerstacks/traefik
$ pushd traefik
Have a look at HTTPS Mode if you want to deploy traefik on HTTPS, as I will use HTTP in this demonstration.
Set your domain and deploy the stack:
$ DOMAIN=localhost PROTOCOL=http bash deploy.sh
Username for Traefik UI: demo
Password for Traefik UI:
deploying traefik stack in http mode
Creating network public
Creating config proxy_traefik_htpasswd
Creating service proxy_traefik
Traefik UI is available at:
- http://traefik.localhost
Your traefik service should be running:
$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
0wga71zbx1pe proxy_traefik replicated 1/1 traefik:1.7.14 *:80->80/tcp
Switch back to the previous directory:
$ popd
Deploy the Monitoring Stack
Get the sources:
$ git clone https://github.com/bekkerstacks/monitoring-cpang
$ pushd monitoring-cpang
If you want to deploy the stack with no pre-configured dashboards, you would need to use ./docker-compose.html, but in this case we will deploy the stack with pre-configured dashboards.
Set the domain and deploy the stack:
$ docker stack deploy -c alt_versions/docker-compose_http_with_dashboards.yml mon
Creating network private
Creating config mon_grafana_config_datasource
Creating config mon_grafana_dashboard_prometheus
Creating config mon_grafana_dashboard_docker
Creating config mon_grafana_dashboard_nodes
Creating config mon_grafana_dashboard_blackbox
Creating config mon_alertmanager_config
Creating config mon_prometheus_config
Creating config mon_prometheus_rules
Creating service mon_blackbox-exporter
Creating service mon_alertmanager
Creating service mon_prometheus
Creating service mon_grafana
Creating service mon_cadvisor
Creating service mon_node-exporter
The endpoints is configured as ${service_name}.${DOMAIN} so you will be able to access grafana on http://grafana.localhost as showed in my use-case.
Use docker stack services mon to see if all the tasks has checked into its desired count then access grafana on http://grafana.${DOMAIN}
Accessing Grafana
Access Grafana on http://grafana.${DOMAIN} and logon with the user admin and the password admin:
You will be asked to reset the password:
You will then be directed to the ui:
From the top, when you list dashboards, you will see the 3 dashboards that was pre-configured:
When looking at the Swarm Nodes Dashboard:
The Swarm Services Dashboard:
Exploring Metrics in Prometheus
Access prometheus on http://prometheus.${DOMAIN} and from the search input, you can start exploring though all the metrics that is available in prometheus:
If we search for node_load15 and select graph, we can have a quick look on how the 15 minute load average looks like for the node where the stack is running on:
Having a look at the alerts section:
Resources
For more information and configuration on the stack that we use, have a look at the wiki:
The github repository:
Thank You
Thanks for reading, if you like my content, feel free to check out my website, and subscribe to my newsletter or follow me at @ruanbekker on Twitter.
- Linktree: https://go.ruan.dev/links
- Patreon: https://go.ruan.dev/patreon
