Monitor Docker Containers with Prometheus and Grafana

In this article, Monitor Docker Containers with Prometheus and Grafana | explore how to enhance Docker container monitoring using Prometheus and Grafana. Learn how to leverage these powerful tools to gain insights into container performance metrics efficiently.

What is Grafana?

Grafana is an open source tool for performing data analytics, retrieving metrics that make sense of large amounts of data, and monitoring our apps using nice configurable dashboards.

Grafana integrates with a wide range of data sources, including Graphite, Prometheus, Influx DB, ElasticSearch, MySQL, PostgreSQL, and others. When connected to supported data sources, it provides web-based charts, graphs, and alerts.

What is Prometheus?

Prometheus is an open-source monitoring and alerting toolkit designed for reliability and scalability in modern, dynamic environments. Developed by the Cloud Native Computing Foundation, Prometheus excels at collecting and storing time-series data, allowing users to gain valuable insights into the performance and health of their applications and infrastructure.

With its powerful query language and support for multi-dimensional data, Prometheus has become a popular choice for monitoring systems within cloud-native ecosystems.

What is Grafana Vizualization Dashboard?

A Grafana Visualization Dashboard is a user interface tool that displays data from various sources in visually appealing and customizable formats. It allows users to create interactive charts, graphs, and other visualizations to analyze and monitor data in real-time.

Prerequisites

To follow this tutorial, you will need:

• AWS Ubuntu 22.04 LTS Instance.
• User with sudo access (see the Initial Server Setup with Ubuntu 22.04 tutorial for details).
• Install Docker on Ubuntu Server with required Permission

Monitor Docker Containers with Prometheus and Grafana

Step#1:Run Grafana using Docker Container

• Update Package Repositories:

sudo apt update

• Install Docker:

sudo apt install docker.io

• Adjust Docker Permissions:

sudo chmod 666 /var/run/docker.sock

• To run the latest stable version of Grafana, run the following command

docker run -d -p 3000:3000 --name=grafana grafana/grafana

Where:

1. docker run: This initiates the process of running a Docker container.
2. -d: It runs the container in detached mode, meaning it runs in the background.
3. -p 3000:3000: This option maps port 3000 from the host machine to port 3000 in the container. Grafana typically uses port 3000 for its web interface.
4. –name=grafana: This assigns the name “grafana” to the running container.
5. grafana/grafana: This specifies the Docker image to be used for creating the container. In this case, it’s the official Grafana image from the Docker Hub.

• Edit the inbound rules of the Instance

If you are using virtual machine/ec2-instance then you need to change the security group inbound rules of that instance:

• Access Grafana Login Page on Browser

To access the Grafana login you need to type in url your ip address with port no which we are currently using is 3000.

http://your_ip_address:3000/login 

http://3.110.29.81:3000/login

You can login by entering Username and Password as admin then the Grafana Home page will be loaded

• Stop the Grafana container

To display the lists Docker containers, run the following command:docker ps

To stop the Grafana container, run the following command:docker stop grafana

To start the Grafana container, run the following command:docker start grafana

To see the docker images, run the following command:docker images

Step#2:Run Prometheus using Docker Container

• Create Prometheus Configuration Directory:

sudo mkdir prometheus

• Configure Prometheus: Edit the prometheus.yml file:

sudo nano prometheus.yml

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    target_groups:
      - targets: ['YOUR_SERVER_IP:9090']

• Create Dockerfile for Prometheus:

sudo nano Dockerfile

FROM prom/prometheus
ADD prometheus.yml /etc/prometheus/

• Build Prometheus Docker Image:

docker build -t my-prometheus .

• Run Prometheus Container:

docker run -p 9090:9090 my-prometheus

• Configure Security Group: Ensure that port 9090 is enabled in your instance’s security group settings.

• Access Prometheus Page through Browser:

https://Your_IP_Address:9090

Step#3:Integrate the Docker containers with Grafana

• Locate the configuration file of the Docker daemon, daemon.json. The file is usually found in the /etc/docker/ directory.

• If the daemon.json file is not present in the specified location, create a new file named daemon.json.

sudo nano /etc/docker/daemon.json

• Open the daemon.json file using a text editor.

• Add the following code snippet to the daemon.json file:

{
        "metrics-addr" : "0.0.0.0:9323",
        "experimental" : true
}

#if this code not working then you can try following code 

{
        "metrics-addr" : "127.0.0.1:9323",
        "experimental" : true
}

• For apply this change you have to restart docker service sudo systemctl restart docker
• The containers will be paused state we need to start running it type the below command to start it

docker start <container id>

• Now wait some time and then check logs on
  • http://<public-ip>:9323/metrics
• You should get docker logs on webpage like following.

Step#4:Add new target in prometheus.yaml located in prometheus docker container

• Display all container created

docker ps

• Edit the prometheus.yml located in my-prometheus container

docker exec -it 1672dfed842a /bin/sh

cd /etc/prometheus

 vi  prometheus.yml

• Add Docker job in prometheus.yaml file.

- job_name: 'docker'
scrape_interval: 5s
static_configs:
- targets: ['<Your _IP>:9323']

• Exit after saving

exit

• After above step you have to restart Prometheus docker container

 docker restart <container id>

• The Target of docker will added in prometheus targets section.

Step#5:Check the logs and create docker dashboard on Grafana UI

• To create Dashboard in Grafana you have to login with admin admin user and password on http://<public-ip>:3000
• After login click on Add your first data source.

Step#6:Monitoring Docker Containers in Grafana

• Click on Explore View

• In the “Metrics browser” selection, choose engine_daemmon_engine_cpus_cpus.
• Select the “Job” and set the job name as “docker.”
• Click on “Run queries,” and Grafana will fetch the number of Engine CPU metrics.

By following these steps, you will have a Grafana dashboard with four panels showing different metrics related to the Docker containers, providing you with valuable insights into their performance and status.

Conclusion:

Monitoring Docker container performance with Prometheus and Grafana facilitates real-time visibility and scalability, empowering users to customize dashboards and enable proactive responses to ensure optimal containerized environment performance. This integration offers a comprehensive solution for monitoring and managing Docker deployments with efficiency and precision.

Related Articles:

How to Install Prometheus on Ubuntu 22.04 LTS

Secure Grafana with Nginx, Reverse Proxy and Certbot

Reference:

Grafana Installation official page