In this article, we learn how to Send Apache2 Logs to Elastic Stack and Filebeat | How to monitor Apache2 Logs with Elastic Stack and Filebeat on Ubuntu 24.04. Elasticsearch, Kibana, and Filebeat provide a powerful stack for collecting, storing, and visualizing real-time logs. When running a web server like Apache, monitoring logs is essential for tracking errors, traffic patterns, and security incidents. This guide will set up the Elastic stack (Elasticsearch, Kibana, and Filebeat) on Ubuntu 24.04 and configure Filebeat to collect Apache2 logs. By the end, you will have a dashboard in Kibana displaying Apache logs.
Table of Contents
Prerequisites
- AWS Account with Ubuntu 24.04 LTS EC2 Instance.
- At least 2 CPU cores and 4 GB of RAM for smooth performance.
- Java and Apache2 installed.
Step #1:Set Up Ubuntu EC2 Instance
Update the Package List to ensure you have the latest versions.
sudo apt update
Elasticsearch requires Java, so we need to install OpenJDK 11.
sudo apt install -y openjdk-11-jdk
Install the Apache web server.
sudo apt install apache2
Check the status of the Apache service to ensure it is running.
sudo systemctl status apache2
Open your browser and navigate to http://<your-server-ip>. You should see the default Apache welcome page.
Step #2:Install Elasticsearch on Ubuntu 24.04 LTS
Import the Elasticsearch GPG key.
curl -fsSL https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo gpg --dearmor -o /usr/share/keyrings/elasticsearch-keyring.gpg
Add the Elasticsearch repository.
echo "deb [signed-by=/usr/share/keyrings/elasticsearch-keyring.gpg] https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee /etc/apt/sources.list.d/elastic-8.x.list
Now lets update the package list again. The repository is added to the system’s package sources.
sudo apt update
Install Elasticsearch.
sudo apt install -y elasticsearch
Enable and start Elasticsearch.
sudo systemctl enable elasticsearch sudo systemctl start elasticsearch
Check the status of the elasticsearch to ensure it is running.
sudo systemctl status elasticsearch
Modify Elasticsearch configuration for remote access.
sudo nano /etc/elasticsearch/elasticsearch.yml
Find the network.host setting, uncomment it, and set it to 0.0.0.0 to bind to all available IP addresses and uncomment the discovery section to specify the initial nodes for cluster formation discovery.seed_hosts: []
For a basic setup (not recommended for production), disable security features.
xpack.security.enabled: false
Save and exit the editor.
Restart Elasticsearch to apply the changes.
sudo systemctl restart elasticsearch
Send a GET request to check if Elasticsearch is running and responding. If successful, you should see a JSON response with cluster information.
curl -X GET "localhost:9200"
You can access it using browser with your Public IP address:9200 port which is a default port for Elasticsearch.
Step #3:Install Kibana on Ubuntu 24.04 LTS
Kibana provides visualization for Elasticsearch data. Install Kibana on the system.
sudo apt install -y kibana
Enable and start Kibana.
sudo systemctl enable kibana
sudo systemctl start kibana
Checks the status of Kibana.
sudo systemctl status kibana
Open the Kibana configuration file for editing.
sudo nano /etc/kibana/kibana.yml
Uncomment and adjust the following lines to bind Kibana to all IP addresses and connect it to Elasticsearch.
server.port: 5601 server.host: "0.0.0.0" elasticsearch.hosts: ["http://localhost:9200"]
Restart Kibana to apply the changes.
sudo systemctl restart kibana
Access the Kibana interface by navigating to http://<your-server-ip>:5601 in your web browser. This will open the Kibana dashboard where you can start exploring your data.
You can start by adding integrations or Explore on my own.
Step #4:Install Filebeat on Ubuntu 24.04 LTS
Filebeat collects and forwards log data to Elasticsearch or Logstash. Install Filebeat on the system.
sudo apt install -y filebeat
No need to edit the filebeat configuration as by default it is configured to send logs to Elasticsearch.
Enable the Apache module in Filebeat.
sudo filebeat modules enable apache
Configure the Apache module.
sudo nano /etc/filebeat/modules.d/apache.yml
Ensure the following configuration is enabled to send Apache logs.
- module: apache
access:
enabled: true
var.paths: ["/var/log/apache2/access.log*"]
error:
enabled: true
var.paths: ["/var/log/apache2/error.log*"]
Save and exit the file.
Test the configuration.
sudo filebeat test config
Apply Filebeat setup changes.
sudo filebeat setup
Start and enable the Filebeat service.
sudo systemctl enable filebeat
sudo systemctl start filebeat
Checks the status of filebeat.
sudo systemctl status filebeat
Ensure Elasticsearch is receiving data from Filebeat by checking the indices.
curl -XGET "localhost:9200/_cat/indices?v"You should see output indicating the presence of indices created by Filebeat.
Step #5:Verify Apache2 Logs in Kibana
Now go back to Kibana. Scroll down and click on the Logs option in Obeservability in the left-hand navigation menu. If the menu is collapsed, click the Expand icon at the bottom left to reveal the options.
Kibana displays Apache2 logs data from the last 15 minutes, visualized as a histogram along with individual log messages below. (You may need to adjust the time range.)
Step #6:Generating a 404 Error in Apache2 for Testing
To generate a 404 Not Found error and see it in Kibana, access the following page on browser.
http://<public-ip-address>/this-page-does-not-existThis request will be logged in Apache’s access log and should be visible in Kibana.
Now refresh the kibana logs page.
You can even see the details of the your Apache2 logs. You can see the details of our Cloud provider also some other details.
Conclusion:
In this guide, we successfully installed Elasticsearch, Kibana, and Filebeat to monitor Apache2 logs on Ubuntu 24.04. We configured Filebeat to collect access and error logs, and we verified Apache2 logs ingestion in Kibana. With this setup, you can efficiently monitor Apache2 logs in real time, helping you track errors, analyze traffic, and enhance server security.
Related Articles:
How to Install Elastic Stack on Ubuntu 24.04 LTS
Install Elastic Stack on Amazon Linux 2
Set Up ELK Stack (Elasticsearch, Logstash and Kibana) On Windows
Reference:
