Metrics and Tracing for .NET App using OpenTelemetry, Jaeger and Prometheus

In this article, we will guide you through integrating OpenTelemetry with Jaeger for tracing and Prometheus for metrics collection in a .NET Core application.

In modern microservices and distributed systems, monitoring, observability, and tracing have become essential for identifying performance bottlenecks and troubleshooting issues. We’ll walk through the process of setting up OpenTelemetry, exporting traces to Jaeger, and exposing metrics to Prometheus.

Prerequisites

Before you start, ensure you have the following installed on your system.

• AWS Account with Ubuntu 24.04 LTS EC2 Instance.
• .NET SDK, Docker, Prometheus and Jaeger installed.

Step #1:Set Up Ubuntu EC2 Instance

If you don’t have .Net SDK, Docker, Prometheus and Jaeger installed on your system you can install it by using following commands.

First Update the package list.

sudo apt update

To install the .NET SDK on your Ubuntu server, use the following command.

sudo apt install -y dotnet-sdk-8.0

To install the Docker on your Ubuntu server, use the following command.

sudo apt install -y docker.io

start and enable the docker service.

sudo systemctl start docker
sudo systemctl enable docker

Now let’s download the prometheus using Docker.

sudo docker pull prom/prometheus

Download the Jaeger using using Docker. This command download the Jaeger all-in-one container.

sudo docker pull jaegertracing/all-in-one:latest

Step #2:Set up .NET Project with OpenTelemetry

For this example, we’ll use a sample .NET application from our GitHub repository.

Clone the below our Opentelemetry GitHub repository using following command.

git clone https://github.com/devopshint/opentelemetry-prometheus-for-dotnet-app.git

Navigate to the project directory.

cd opentelemetry-prometheus-for-dotnet-app

Step #3:Instrumenting .NET Application with OpenTelemetry

Open the Program.cs file.

nano Program.cs

Add the following content into it.

using System.Globalization;
using System.Diagnostics.Metrics;
using Microsoft.AspNetCore.Mvc;
using OpenTelemetry.Logs;
using OpenTelemetry.Metrics;
using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using OpenTelemetry.Instrumentation.AspNetCore;
using OpenTelemetry.Instrumentation.Http;
using OpenTelemetry.Exporter;
using Prometheus;

var builder = WebApplication.CreateBuilder(args);

const string serviceName = "roll-dice";


// Add services to the container.
builder.Services.AddControllers();

// Create a custom meter for the API
var meter = new Meter("roll-dice.Metrics", "1.0");


var httpRequestCounter = meter.CreateCounter<long>("http_requests_total", description: "Total number of HTTP requests");


builder.Logging.AddOpenTelemetry(options =>
{
    options
        .SetResourceBuilder(
            ResourceBuilder.CreateDefault()
                .AddService(serviceName))
        .AddConsoleExporter();
});

// Configure OpenTelemetry
builder.Services.AddOpenTelemetry()
    .WithTracing(tracerProviderBuilder =>
    {
        tracerProviderBuilder
            .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("roll-dice"))
            .AddAspNetCoreInstrumentation() // Tracks incoming HTTP requests
            .AddHttpClientInstrumentation()
            .AddJaegerExporter(jaegerOptions =>
            {
                jaegerOptions.AgentHost = "localhost";  // Update with your Jaeger host if necessary
                jaegerOptions.AgentPort = 6831;         // Default Jaeger agent port
            })
            .AddConsoleExporter(); // Optional: For debugging
    })
    .WithMetrics(meterProviderBuilder =>
    {
        meterProviderBuilder
            .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService("roll-dice"))
            .AddAspNetCoreInstrumentation() // Tracks incoming HTTP request metrics
            .AddHttpClientInstrumentation()
            .AddPrometheusExporter(); // Expose metrics to Prometheus
    });

// Add middleware to count HTTP requests
var app = builder.Build();

// Use the Prometheus middleware to expose the /metrics endpoint.
app.UseRouting();

app.UseHttpMetrics(); // Middleware for collecting HTTP metrics

app.UseEndpoints(endpoints =>
{
    endpoints.MapControllers();
    endpoints.MapMetrics(); // Expose the /metrics endpoint for Prometheus scraping
});



string HandleRollDice([FromServices]ILogger<Program> logger, string? player)
{
    var result = RollDice();

    if (string.IsNullOrEmpty(player))
    {
        logger.LogInformation("Anonymous player is rolling the dice: {result}", result);
    }
    else
    {
        logger.LogInformation("{player} is rolling the dice: {result}", player, result);
    }

    return result.ToString(CultureInfo.InvariantCulture);
}

int RollDice()
{
    return Random.Shared.Next(1, 7);
}

app.MapGet("/rolldice/{player?}", HandleRollDice);

app.Run();

Explanation of the code:

1. Imports

These lines import the necessary namespaces, including those for OpenTelemetry, ASP.NET Core, metrics, and Prometheus. Each namespace provides classes and methods for building the application’s telemetry and HTTP functionalities.

2. Service Setup

WebApplication.CreateBuilder(args) initializes the ASP.NET Core application builder.serviceName defines the name of the service, which is used later in telemetry to identify the source of traces and metrics. builder.Services.AddControllers() adds support for controllers to the service container, enabling MVC-style routing.

3. Creating a Custom Meter for Metrics

A custom meter named roll-dice.Metrics is created, which will be used to track application-specific metrics. The httpRequestCounter tracks the total number of incoming HTTP requests using a custom metric, "http_requests_total".

4. Logging Configuration

Configures OpenTelemetry to handle logging with the service name roll-dice.AddConsoleExporter() sets up console logging to output telemetry data, which is useful for debugging.

5. OpenTelemetry Tracing Configuration

OpenTelemetry is used for tracing:

• AddAspNetCoreInstrumentation: Automatically traces incoming HTTP requests in the application.
• AddHttpClientInstrumentation: Traces outgoing HTTP client calls.
• AddJaegerExporter: Configures the application to send traces to Jaeger running on the specified host and port.
• AddConsoleExporter: Optionally exports trace data to the console for easier debugging during development.

6. OpenTelemetry Metrics Configuration

Metrics are collected using OpenTelemetry:

• AddAspNetCoreInstrumentation: Automatically tracks metrics for incoming HTTP requests (e.g., latency, request count).
• AddHttpClientInstrumentation: Tracks metrics for outgoing HTTP requests made by the application.
• AddPrometheusExporter: Exposes metrics on the /metrics endpoint, which can be scraped by Prometheus for monitoring.

7. Building the Application and Configuring Middleware

app.UseRouting() sets up request routing.app.UseHttpMetrics() integrates Prometheus HTTP metrics, which automatically tracks details about incoming HTTP requests.

8. Exposing the /metrics Endpoint

Maps endpoints for the controllers.MapMetrics() exposes the /metrics endpoint, allowing Prometheus to scrape metric data from the application.

9. Handling the /rolldice Endpoint

HandleRollDice method:

• Handles requests to the /rolldice endpoint.
• Rolls a dice using the RollDice() method and logs the result.
• If a player name is provided, it logs the name; otherwise, it logs that an anonymous player is rolling the dice.

RollDice method:

• Generates a random number between 1 and 6, simulating a dice roll.

10. Endpoint Mapping for Roll Dice

Maps the /rolldice/{player?} endpoint to the HandleRollDice method, which processes dice roll requests.app.Run starts the application.

Next open the MyPrometheusApp.csproj file.

nano MyPrometheusApp.csproj

Add and Update the packages to the latest versions.

Add a <PackageReference Include=”OpenTelemetry.Exporter.Jaeger” Version=”1.5.1″ /> and update “OpenTelemetry.Instrumentation.AspNetCore” Version=”1.7.0″ and “OpenTelemetry.Instrumentation.Http” Version=”1.7.0″

<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.OpenApi" Version="8.0.8" />
    <PackageReference Include="OpenTelemetry.Exporter.Console" Version="1.9.0" />
    <PackageReference Include="OpenTelemetry.Exporter.Prometheus.AspNetCore" Version="1.9.0-beta.2" />
    <PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.6.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.9.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.9.0" />
    <PackageReference Include="prometheus-net.AspNetCore" Version="6.0.0" />
    <PackageReference Include="Swashbuckle.AspNetCore" Version="6.4.0" />
    <PackageReference Include="OpenTelemetry.Exporter.Jaeger" Version="1.5.1" />
  </ItemGroup>

</Project>

Explanation of the code:

This project file sets up the .NET web  application to leverage modern observability tools and web standards.

1. Project Definition: This part specifies that the project uses the Microsoft.NET.Sdk.Web SDK, which is specifically designed for building web applications using ASP.NET Core. It includes all the necessary tools and dependencies for web projects.
2. PropertyGroup Section:
   • TargetFramework: This property specifies the .NET version the application is targeting, which in this case is .NET 8.0. It defines the version of the runtime and libraries that the application will use.
   • Nullable: Setting this to enable enables nullable reference types in C#. It ensures that the code explicitly handles null values, reducing null reference errors and increasing code safety.
   • ImplicitUsings: When set to enable, this feature automatically includes common using directives, so you don’t need to write them at the top of each file. This simplifies the code by implicitly including commonly used namespaces.
3. ItemGroup Section: This section lists all the external dependencies (NuGet packages) that the project requires.

Next open the prometheus.yml

nano prometheus.yml

change the targets. Give your EC2 instance public IP address.

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'MyPrometheusApp'
    scrape_interval: 5s
    metrics_path: '/metrics'
    static_configs:
      - targets: ['<EC2-instance-IP>:8080']

Step #4:Build and Run .NET Application with OpenTelemetry

To compile and run your application, use the following commands.

Build the application.

dotnet build

Run the application.

dotnet run

This will start the application, and it will expose the /metrics endpoint for  Prometheus to scrape.

You can test your application by accessing the endpoint that returns the http metrics (e.g., http://<EC2-Instance-IP>:8080/metrics).

http://<EC2-Instance-IP>:8080/metrics

You can run application rolldice by using http://<EC2-Instance-IP>:8080/rolldice.

This endpoint would simulate a dice roll and return the result of that roll.

Now open a duplicate tab and Navigate to the project directory.

cd opentelemetry-prometheus-for-dotnet-app

Run the Prometheus container.

sudo docker run -p 9090:9090 -v ~/opentelemetry-prometheus-for-dotnet-app/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

Once the container is running, you can access the Prometheus web interface by visiting http://<EC2-Instance-IP>:9090.

To check if Prometheus is correctly scraping metrics from your .NET application, navigate to the “Targets” page in the Prometheus UI. Click on “Status” in the menu bar and then “Targets. “Ensure that your job MyPrometheusApp is listed and marked as “UP.”

Next run Jaeger in a Docker container with the following command.

sudo docker run -d --name jaeger \
  -e COLLECTOR_ZIPKIN_HTTP_PORT=9411 \
  -p 5775:5775/udp \
  -p 6831:6831/udp \
  -p 6832:6832/udp \
  -p 5778:5778 \
  -p 16686:16686 \
  -p 14268:14268 \
  -p 14250:14250 \
  -p 9411:9411 \
  jaegertracing/all-in-one

After running the application, visit http://<EC2-Instance-IP>:16686 in your browser. This is the Jaeger UI where you can view traces from your .NET application. You should see traces for incoming HTTP requests and any custom spans created in your application.

In service, select the roll-dice instead of jaeger-all-in-one, You can select perticular operation like all, GET, GET/rolldice/{player?}, Set the Lookback time here it is 5 minutes and you can set Max Duration And Min Duration of the Traces and Limit Results for how many traces you want.

Conclusion:

In conclusion, today we have seen a basic setup to get started. By integrating OpenTelemetry with Jaeger and Prometheus in your .NET Core application, you gain powerful observability features, including distributed tracing and real-time metrics monitoring. With Jaeger, you can visualize end-to-end traces and pinpoint slow or failing operations. With Prometheus, you can track metrics. These tools help you track and analyze the performance of your application, making it easier to diagnose issues in production.

Related Articles:

Instrumentation of .Net App Metrics Using OpenTelemetry, Prometheus and Grafana

Reference:

Official Opentelemetry Page