logging-observability

What this skill does

You are a senior .NET architect specializing in observability. When implementing logging and monitoring in Razor Pages applications, follow these patterns to ensure production-grade observability, troubleshooting capabilities, and integration with monitoring systems. Target .NET 8+ with nullable reference types enabled.

## Rationale

Effective observability is critical for production applications. Poor logging makes debugging impossible, and lack of correlation IDs makes tracing requests across services difficult. These patterns provide structured, searchable logs with proper context for troubleshooting.

## Core Principles

1. **Structured Logging**: Use structured formats (JSON) for machine parsing
2. **Correlation IDs**: Every request gets a unique ID for end-to-end tracing
3. **Contextual Enrichment**: Logs include relevant context (user, endpoint, duration)
4. **Log Levels**: Use appropriate levels (Debug, Info, Warning, Error, Fatal)
5. **External Sinks**: Send logs to centralized systems (Seq, Datadog, CloudWatch)

## Pattern 1: Serilog Configuration

### NuGet Packages

```xml
<PackageReference Include="Serilog.AspNetCore" Version="8.0.*" />
<PackageReference Include="Serilog.Expressions" Version="4.0.*" />
<PackageReference Include="Serilog.Sinks.Seq" Version="7.0.*" /> <!-- Optional -->
```

### Bootstrap Logger (Program.cs Start)

```csharp
using Serilog;
using Serilog.Debugging;

// Enable Serilog self-logging for diagnostics
SelfLog.Enable(msg => Console.Error.WriteLine($"[SERILOG] {msg}"));

// Create bootstrap logger for startup errors
Log.Logger = new LoggerConfiguration()
    .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
    .Enrich.FromLogContext()
    .WriteTo.Console(formatProvider: CultureInfo.CurrentCulture)
    .CreateBootstrapLogger();

try
{
    Log.Information("Starting web application...");
    var builder = WebApplication.CreateBuilder(args);
    
    // ... configure services ...
    
    var app = builder.Build();
    await app.RunAsync();
}
catch (Exception ex)
{
    Log.Fatal(ex, "Application terminated unexpectedly");
}
finally
{
    await Log.CloseAndFlushAsync();
}
```

### Service Registration

```csharp
public static class LoggingServiceRegistration
{
    public static IServiceCollection ConfigureSerilog(
        this IServiceCollection services, 
        IConfiguration configuration)
    {
        services.AddSerilog((services, lc) => lc
            .ReadFrom.Configuration(configuration)
            .Enrich.FromLogContext()
            .Enrich.WithMachineName()
            .Enrich.WithEnvironmentName()
            .WriteTo.Console(formatter: new ExpressionTemplate(
                "[{@t:hh:mm:ss.fff tt} {@l:u3}] {SourceContext} - {CorrelationId} - {@m}\n{@x}",
                theme: TemplateTheme.Code))
            .WriteTo.Seq(
                configuration["Seq:ServerUrl"] ?? "http://localhost:5341",
                apiKey: configuration["Seq:ApiKey"],
                formatProvider: CultureInfo.CurrentCulture)
        );

        return services;
    }
}
```

### appsettings.json Configuration

```json
{
  "Serilog": {
    "Using": ["Serilog.Sinks.Console", "Serilog.Sinks.Seq"],
    "MinimumLevel": {
      "Default": "Information",
      "Override": {
        "Microsoft": "Warning",
        "Microsoft.AspNetCore": "Warning",
        "System": "Warning"
      }
    },
    "WriteTo": [
      {
        "Name": "Console",
        "Args": {
          "outputTemplate": "[{Timestamp:HH:mm:ss} {Level:u3}] {Message:lj}{NewLine}{Exception}"
        }
      }
    ],
    "Enrich": ["FromLogContext", "WithMachineName", "WithThreadId"],
    "Properties": {
      "Application": "MyApp"
    }
  }
}
```

## Pattern 2: Correlation ID Middleware

```csharp
public class RequestContextLoggingMiddleware
{
    private readonly RequestDelegate _next;
    private const string CorrelationHeaderName = "X-Correlation-Id";

    public RequestContextLoggingMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task Invoke(HttpContext httpContext)
    {
        var correlationId = GetCorrelationId(httpContext);
        
        // Add to response headers
        httpContext.Response.OnStarting(() =>
        {
            httpContext.Response.Headers[CorrelationHeaderName] = correlationId;
            return Task.CompletedTask;
        });

        using (LogContext.PushProperty("CorrelationId", correlationId))
        using (LogContext.PushProperty("RequestPath", httpContext.Request.Path))
        using (LogContext.PushProperty("RequestMethod", httpContext.Request.Method))
        {
            await _next.Invoke(httpContext);
        }
    }

    private static string GetCorrelationId(HttpContext httpContext)
    {
        httpContext.Request.Headers.TryGetValue(
            CorrelationHeaderName, out var correlationId);

        return correlationId.FirstOrDefault() ?? httpContext.TraceIdentifier;
    }
}

// Extension method for easy registration
public static class RequestContextLoggingExtensions
{
    public static IApplicationBuilder UseRequestContextLogging(
        this IApplicationBuilder app)
    {
        return app.UseMiddleware<RequestContextLoggingMiddleware>();
    }
}
```

### Registration

```csharp
// Program.cs
var app = builder.Build();

// Place early in pipeline
app.UseRequestContextLogging();
app.UseSerilogRequestLogging(); // Logs each request
```

## Pattern 3: Request Logging with Serilog

```csharp
// Program.cs
app.UseSerilogRequestLogging(options =>
{
    options.EnrichDiagnosticContext = (diagnosticContext, httpContext) =>
    {
        diagnosticContext.Set("UserId", httpContext.User.Identity?.Name ?? "anonymous");
        diagnosticContext.Set("ClientIp", httpContext.Connection.RemoteIpAddress?.ToString());
        diagnosticContext.Set("UserAgent", httpContext.Request.Headers["User-Agent"].ToString());
    };
    
    options.GetLevel = (httpContext, elapsed, ex) =>
    {
        // Log 5xx as Error, slow requests as Warning
        if (ex != null || httpContext.Response.StatusCode > 499)
            return LogEventLevel.Error;
        
        if (elapsed > 1000)
            return LogEventLevel.Warning;
        
        return LogEventLevel.Information;
    };
});
```

### Custom Request Logging (PageModel)

```csharp
public class OrderDetailsModel(ILogger<OrderDetailsModel> logger) : PageModel
{
    public async Task OnGetAsync(Guid orderId)
    {
        // Push contextual properties
        using (logger.BeginScope(new Dictionary<string, object>
        {
            ["OrderId"] = orderId,
            ["UserId"] = User.Identity?.Name ?? "anonymous"
        }))
        {
            logger.LogInformation("Loading order details");
            
            try
            {
                var order = await _orderService.GetAsync(orderId);
                
                if (order == null)
                {
                    logger.LogWarning("Order not found");
                    return NotFound();
                }
                
                logger.LogInformation("Order loaded successfully");
            }
            catch (Exception ex)
            {
                logger.LogError(ex, "Failed to load order");
                throw;
            }
        }
    }
}
```

## Pattern 4: MediatR Pipeline Logging

```csharp
public class RequestLoggingBehavior<TRequest, TResponse>(ILogger<RequestLoggingBehavior<TRequest, TResponse>> logger)
    : IPipelineBehavior<TRequest, TResponse>
    where TRequest : IRequest<TResponse>
{
    private static readonly string RequestName = typeof(TRequest).Name;
    private static readonly TimeSpan SlowRequestThreshold = TimeSpan.FromMilliseconds(500);

    public async Task<TResponse> Handle(
        TRequest request,
        RequestHandlerDelegate<TResponse> next,
        CancellationToken cancellationToken)
    {
        logger.LogInformation("Handling {RequestName}", RequestName);

        var stopwatch = Stop