Instrumentation of .Net App Metrics Using OpenTelemetry, Prometheus and Grafana

In this article, we will guide you through the process of instrumenting a .NET application to monitor its metrics using OpenTelemetry, Prometheus, and Grafana.

This setup will help you gain insights into your application’s performance by collecting, storing, and visualizing key metrics. We will cover the steps to instrument your .NET application, use Prometheus to collect the metrics, and Grafana to create insightful dashboards.

Prerequisites

Before you start, ensure you have the following installed on your system.

• AWS Account with Ubuntu 24.04 LTS EC2 Instance.
• .NET SDK, Docker, Prometheus and Grafana installed.

Step #1:Set Up the .NET Application

If you don’t have SDK, Docker and Prometheus installed on your system you can install it by using following commands.

First Update the package list.

sudo apt update

To install the .NET SDK on your Ubuntu server, use the following command.

sudo apt install -y dotnet-sdk-8.0

For this example, we’ll use a sample .NET application from our GitHub repository.

Clone the below our Opentelemetry GitHub repository using following command.

git clone https://github.com/devopshint/opentelemetry-prometheus-for-dotnet-app.git

Navigate to the project directory.

cd opentelemetry-prometheus-for-dotnet-app

Open the Program.cs file

nano Program.cs

Add the following line to the top of your Program.cs file.

using System.Globalization; and using Microsoft.AspNetCore.Mvc;

using System.Globalization;
using System.Diagnostics.Metrics;
using Microsoft.AspNetCore.Mvc;
using OpenTelemetry.Logs;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using OpenTelemetry.Instrumentation.AspNetCore;
using OpenTelemetry.Instrumentation.Http;
using Prometheus;


var builder = WebApplication.CreateBuilder(args);

const string serviceName = "roll-dice";


// Add services to the container.
builder.Services.AddControllers();

// Create a custom meter for the API
var meter = new Meter("roll-dice.Metrics", "1.0");


var httpRequestCounter = meter.CreateCounter<long>("http_requests_total", description: "Total number of HTTP requests");


builder.Logging.AddOpenTelemetry(options =>
{
    options
        .SetResourceBuilder(
            ResourceBuilder.CreateDefault()
                .AddService(serviceName))
        .AddConsoleExporter();
});

// Configure OpenTelemetry
builder.Services.AddOpenTelemetry()
    .WithTracing(tracerProviderBuilder =>
    {
        tracerProviderBuilder
            .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("roll-dice"))
            .AddAspNetCoreInstrumentation() // Tracks incoming HTTP requests
            .AddHttpClientInstrumentation()
            .AddConsoleExporter(); // Optional: For debugging
    })
    .WithMetrics(meterProviderBuilder =>
    {
        meterProviderBuilder
            .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("roll-dice"))
            .AddAspNetCoreInstrumentation() // Tracks incoming HTTP request metrics
            .AddHttpClientInstrumentation()
            .AddPrometheusExporter(); // Expose metrics to Prometheus
    });

// Add middleware to count HTTP requests
var app = builder.Build();

// Use the Prometheus middleware to expose the /metrics endpoint.
app.UseRouting();

app.UseHttpMetrics(); // Middleware for collecting HTTP metrics

app.UseEndpoints(endpoints =>
{
    endpoints.MapControllers();
    endpoints.MapMetrics(); // Expose the /metrics endpoint for Prometheus scraping
});



string HandleRollDice([FromServices]ILogger<Program> logger, string? player)
{
    var result = RollDice();

    if (string.IsNullOrEmpty(player))
    {
        logger.LogInformation("Anonymous player is rolling the dice: {result}", result);
    }
    else
    {
        logger.LogInformation("{player} is rolling the dice: {result}", player, result);
    }

    return result.ToString(CultureInfo.InvariantCulture);
}

int RollDice()
{
    return Random.Shared.Next(1, 7);
}

app.MapGet("/rolldice/{player?}", HandleRollDice);

app.Run();

Explanation of the OpenTelemetry Instrumentation code for .Net Application

1. Imports

These lines import the necessary namespaces, including those for OpenTelemetry, ASP.NET Core, metrics, and Prometheus. Each namespace provides classes and methods for building the application’s telemetry and HTTP functionalities.

2. Creating the Web Application

WebApplication.CreateBuilder(args) initializes the ASP.NET Core application builder.serviceName defines the name of the service, which is used later in telemetry to identify the source of traces and metrics.

3. Adding Services to the Container

Adds support for controllers to the application, enabling the MVC framework and allowing you to handle HTTP requests using controller methods.

4. Creating a Custom Meter for Metrics

Creates a custom Meter named roll-dice.Metrics to track metrics for this application.

A counter metric called http_requests_total is defined to track the total number of HTTP requests.

5. Configuring OpenTelemetry for Logging

Configures OpenTelemetry to handle logging with the service name roll-dice.AddConsoleExporter() sets up console logging to output telemetry data, which is useful for debugging.

6. Configuring OpenTelemetry for Tracing and Metrics

Configures OpenTelemetry tracing and http metrics:

• Tracing: Instruments HTTP requests, tracks HTTP client interactions, and exports trace data to the console.
• Metrics: Instruments HTTP request metrics and exposes them to Prometheus using the AddPrometheusExporter() method.

7. Building the Application and Configuring Middleware

app.UseRouting() sets up request routing.app.UseHttpMetrics() integrates Prometheus HTTP metrics, which automatically tracks details about incoming HTTP requests.

8. Exposing the /metrics Endpoint

Maps endpoints for the controllers.MapMetrics() exposes the /metrics endpoint, allowing Prometheus to scrape metric data from the application.

9. Handling the /rolldice Endpoint

HandleRollDice method:

• Handles requests to the /rolldice endpoint.
• Rolls a dice using the RollDice() method and logs the result.
• If a player name is provided, it logs the name; otherwise, it logs that an anonymous player is rolling the dice.

RollDice method:

• Generates a random number between 1 and 6, simulating a dice roll.

app.MapGet("/rolldice/{player?}", HandleRollDice);: Maps the /rolldice endpoint to the HandleRollDice method, making the dice-rolling logic accessible via HTTP requests.

10. Running the Application

Starts the web application and begins listening for incoming HTTP requests.

Next open the MyPrometheusApp.csproj file.

nano MyPrometheusApp.csproj

Update the packages to the latest versions.

“OpenTelemetry.Instrumentation.AspNetCore” Version=”1.7.0″ and “OpenTelemetry.Instrumentation.Http” Version=”1.7.0″

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.8" />
    <PackageReference Include="OpenTelemetry.Exporter.Console" Version="1.9.0" />
    <PackageReference Include="OpenTelemetry.Exporter.Prometheus.AspNetCore" Version="1.9.0-beta.2" />
    <PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.6.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.7.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.7.0" />
    <PackageReference Include="prometheus-net.AspNetCore" Version="6.0.0" />
    <PackageReference Include="Swashbuckle.AspNetCore" Version="6.4.0" />
  </ItemGroup>

</Project>

Explanation of the code:

This project file sets up the .NET web application to leverage modern observability tools and web standards.

1. Project Definition: This part specifies that the project uses the Microsoft.NET.Sdk.Web SDK, which is specifically designed for building web applications using ASP.NET Core. It includes all the necessary tools and dependencies for web projects.
2. PropertyGroup Section:
   • TargetFramework: This property specifies the .NET version the application is targeting, which in this case is .NET 8.0. It defines the version of the runtime and libraries that the application will use.
   • Nullable: Setting this to enable enables nullable reference types in C#. It ensures that the code explicitly handles null values, reducing null reference errors and increasing code safety.
   • ImplicitUsings: When set to enable, this feature automatically includes common using directives, so you don’t need to write them at the top of each file. This simplifies the code by implicitly including commonly used namespaces.
3. ItemGroup Section: This section lists all the external dependencies (NuGet packages) that the project requires.

Step #2:Set Up Docker on Ubuntu

To install the Docker on your Ubuntu server, use the following command.

sudo apt install -y docker.io

start and enable the docker service.

sudo systemctl start docker
sudo systemctl enable docker

Step #3:Set Up Prometheus Using Docker

Now let’s get the prometheus using Docker.

sudo docker pull prom/prometheus

Next open the prometheus.yml

nano prometheus.yml

change the targets. Make sure to give your instance public IP address.

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'MyPrometheusApp'
    scrape_interval: 5s
    metrics_path: '/metrics'
    static_configs:
      - targets: ['<EC2-instance-IP>:8080']

Step #4:Build and Run the Application

To compile and run your application, use the following commands.

Build the application.

dotnet build

Run the application.

dotnet run

This will start the application, and it will expose the /metrics endpoint for  Prometheus to scrape.

You can test your application by accessing the endpoint that returns the http metrics (e.g., http://<EC2-Instance-IP>:8080/metrics).

http://<EC2-Instance-IP>:8080/metrics

You can run application rolldice by using http://<EC2-Instance-IP>:8080/rolldice.

This endpoint would simulate a dice roll and return the result of that roll.

Navigate to the project directory.

cd opentelemetry-prometheus-for-dotnet-app

Run the Prometheus container.

sudo docker run -p 9090:9090 -v ~/opentelemetry-prometheus-for-dotnet-app/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

Once the container is running, you can access the Prometheus web interface by visiting http://<EC2-Instance-IP>:9090.

To check if Prometheus is correctly scraping metrics from your .NET application, navigate to the “Targets” page in the Prometheus UI. Click on “Status” in the menu bar and then “Targets. “Ensure that your job MyPrometheusApp is listed and marked as “UP.”

Step #5:Set Up Grafana Using Docker

Run the following command to download the Grafana Docker image.

sudo docker pull grafana/grafana

Now lets run the Grafana container.

sudo docker run -d -p 3000:3000 --name=grafana grafana/grafana

Grafana will be accessible at http://<EC2-Instance-IP>:3000.

The default login credentials for Grafana are Username: admin and Password: admin

You will be prompted to change the password after your first login. You can change it or skip it.

Step #6:Configure Prometheus as a Data Source in Grafana

After logging in to Grafana, you’ll need to add Prometheus as a data source.

• Click on “Connections”
• Click on the “Data source” button.

Search for prometheus in search bar and select it as a data source. Then click on Add new data source.

Set the URL to: http://<EC2-Instance-IP>:9090. Replace <EC2-Instance-IP> with the IP address of your EC2 instance. Leave other settings at their default values.

Click the “Save & Test” button to ensure that Grafana can connect to Prometheus.

Click on the “Plus” icon in the sidebar and select “Dashboard.”

Click on “Add visualization”

In the “Query” section, select the Prometheus data source. Enter a Prometheus query to retrieve metrics from your .NET application, such as

http_requests_received_total

Choose a visualization type (e.g., “Time series,” “Graph”, “Gauge”) that best represents your data.

Here we have used “Gauge.”

Save the entire dashboard by clicking “Save Dashboard” at the top. Give the appropriate title

Conclusion:

In this article, we successfully instrumented a .NET application to collect HTTP metrics using OpenTelemetry. We set up Prometheus to scrape these metrics and configured Grafana to visualize them in an intuitive dashboard. This setup helps you monitor your application’s health and performance effectively, allowing you to identify and resolve issues quickly.

Related Articles:

Export OpenTelemetry Metrics to Prometheus for .NET Application

Reference:

Official Opentelemetry Page