Introduction
In today’s digital era, efficient system monitoring is imperative for ensuring the smooth operation and performance of IT infrastructures. Linux servers, being the backbone of many organizations, require robust monitoring solutions to track vital metrics and detect potential issues promptly. Prometheus, alongside Grafana, offers a potent combination for system monitoring, enabling administrators to collect, visualize, and analyze metrics effectively. This report provides a detailed walkthrough of setting up Linux server monitoring using Prometheus and Grafana, highlighting each step from installation to configuration.
Prerequisite:
Good internet connectivity
Security Groups Configured properly
Any web Browser
Security Groups Configured on EC2 Instances:
Port 9090 — Prometheus Server
Port 9100 — Prometheus Node Exporter
Port 3000 — Grafana
Prometheus Architecture:
Grafana: Grafana is a free and open source visualization tool mostly used with Prometheus to which monitor metrics. It provides various dashboards, charts, graphs, alerts for the particular data source. It allows us to query, visualize, explore metrics and set alerts for the data source which can be a system, server, nodes, cluster, etc.
Node Exporter: Node exporter is one of the Prometheus exporters which is used to expose servers or system OS metrics. With the help of Node exporter we can expose various resources of the system like RAM, CPU utilization, Memory Utilization, disk space.
Prometheus: Prometheus is a open source Linux Server Monitoring tool mainly used for metrics monitoring, event monitoring, alert management, etc.It has changed the way of monitoring systems and that is why it has become the Top-Level project of Cloud Native Computing Foundation (CNCF).
Steps:
1. Creating Prometheus System Users and Directory
We will have to create a Prometheus user named Prometheus and a Prometheus directory named as Prometheus.
Using below commands we can create a user and directory.
2. Update Prometheus user
As user groups and directories are created successfully which store the Prometheus data and files.
Now we will have to update the group and user ownership on the newly created directories.
By using the below command we update the ownership.
3. Download Prometheus Binary File on Ubuntu
Now we will download the latest version of Prometheus. We can copy the download link as per our Operating System from Prometheus download page
Using the below command we can download Prometheus, here we are downloading Prometheus 2.31.1 version, you use the above link to download specific version.
4. Install Prometheus and Grafana on Ubuntu 20.04 LTS:
Extract the files using tar command :
5. Move the configuration file and set the owner to the Prometheus user:
6. Update Prometheus user ownership on Binaries:
Now we will update the user and group ownership on the binaries of Prometheus.
Using following commands we will update the user and group ownership.
7. Check Prometheus Version
Now the Prometheus is successfully installed on our system. We will check the version of Prometheus and to configure it.
Follow the commands to verify prometheus version.
8. Prometheus configuration file
We have already copied /tmp/prometheus-2.31.1.linux-amd64/prometheus.yml file /etc/prometheus directory, verify if it present and should look like below and modify it as per your requirement.
9. Creating Prometheus Systemd file
Now we will create a system service file in /etc/systemd/system location.
After adding the program save the file with Ctrl+O and exit with Ctrl+X.
To use the newly created service we will have to reload the daemon services, Use the below command to reload daemon services.
Prometheus installation and configuration is set up, We can see status Active: active(running)
10. Accessing Prometheus on Browser
Now as Prometheus installation and configuration is set up and it is ready to use we can access its services via web interface.Also check weather port 9090 is UP in firewall.
Use below command to enable prometheus service in firewall
As we can see the Prometheus dashboards, we can also check the target.As we can observe Current state is UP and we can also see the last scrape.
11. Install Grafana on Ubuntu
Download the Grafana GPG key with wget, then pipe the output to apt-key. This will add the ky to your APT installation’s list of trusted keys, which will allow you to download and verify the GPG-signed Grafana package:
In this command, the option -q turns off the status update message for wget, and -O outputs the file that you downloaded to the terminal. These two options ensure that only the contents of the downloaded file are pipelined to apt-key.
You will receive output similar to this:
It is always a good practice to change your login credentials.
Provide your New Password and click on Change Password
Now here you can see Home Dashboard page of Grafana
12. Configure Prometheus as Grafana Data Source
Once you logged into Grafana Now first Navigate to Settings Icon ->> Configuration ->> data sources
Now lets click on Add Data sources and select Prometheus
Now configure Prometheus data source by providing Prometheus URL
As per your requirement you can do other changes or you can also keep remaining configuration as default.
Now click on Save & test so it will prompt a message Data Source is working.
13. Install Node Exporter on Ubuntu 20.04 LTS
Node Exporter collects the metrics of your system such as Memory usage, CPU usage, RAM, disk space, etc.
To install Node Exporter first navigate to Prometheus official download page, Scroll down and you will get node_exporter section and then select Linux OS for amd64.
Now right click on node exporter and copy link address
14. Create a node_exporter service file in the /etc/systemd/system directory
We have covered How to Install Prometheus and Grafana on Ubuntu 20.04 LTS with Node Exporter.
15. Configure the Node Exporter as a Prometheus target
Now to scrape the node_exporter lets instruct the Prometheus by making a minor change in prometheus.yml file
16. Creating Grafana Dashboard to Monitor Linux Server
Now lets build a dashboard in Grafana so then it will able to reflect the metrics of the Linux system.So we will use 14513 to import Grafana.com, Lets come to
Grafana Home page and you can see a “+” icon. Click on that and select “Import”
Now provide the Grafana.com Dashboard ID which is 14513 and click on Load
Now provide the name and select the Prometheus Datasource and click on Import.
There you are done with the setup. Now your Dashboard is running up!.
Final Output:
Conclusion:
In conclusion, the implementation of Prometheus and Grafana for Linux server monitoring offers organizations a comprehensive solution to track system performance and health effectively. By leveraging the capabilities of Prometheus, administrators can collect a wide range of metrics, including CPU usage, memory utilization, disk space, and more. Grafana complements Prometheus by providing intuitive dashboards, visualization tools, and alerting mechanisms, enabling administrators to gain actionable insights into system behavior. With proper setup and configuration, organizations can ensure the reliability, availability, and performance of their Linux servers, thereby enhancing overall operational efficiency and user satisfaction.