Skip to content

Debugging

This page covers how to debug the Cargonerds backend and frontends locally. There are three common setups — the full Aspire graph, a single .NET host pointed at a deployed environment's neighbors, and the frontends (Blazor WebAssembly and the Next.js realtime app) — followed by how to read logs, traces and metrics in the Aspire dashboard, a set of common issues, and the SQL query-performance notes (SET ARITHABORT ON, OPTION (RECOMPILE)) that are specific to this codebase.

If you have not run the stack before, start with Running locally and the Aspire integration deep-dive; this page assumes the AppHost and containers already come up.

How debugging fits together

The local stack is orchestrated by .NET Aspire. The Cargonerds.AppHost project (src/Cargonerds.AppHost/Program.cs) is a distributed-application host that starts every backend service, every infrastructure container (SQL Server, Redis, RabbitMQ), the frontends and a docs container, and exposes a single dashboard where their logs/traces/metrics are aggregated over OpenTelemetry.

flowchart TD
    IDE["IDE debugger\n(VS / Rider / VS Code)"]
    AppHost["Cargonerds.AppHost\n(Aspire orchestrator)"]
    Dash["Aspire dashboard\nhttps://localhost:17176"]

    subgraph svc["Child processes (own OS processes)"]
        Auth["Cargonerds.AuthServer\n:44345"]
        Api["Cargonerds.HttpApi.Host\n:44354"]
        Admin["Cargonerds.Blazor (admin)\n:44381"]
        Mig["Cargonerds.DbMigrator"]
        FE["frontend/realtime (Next.js)\n:4200"]
    end

    subgraph infra["Containers"]
        Sql[("spark-db\nSQL Server")]
        Redis[("spark-redis")]
        Rabbit[("spark-rabbitmq")]
    end

    IDE -- F5 launches --> AppHost
    AppHost -- starts --> svc
    AppHost -- starts --> infra
    AppHost -- OTLP --> Dash
    svc -- logs / traces / metrics (OTLP) --> Dash
    IDE -. attach to .-> svc

The AppHost starts child processes

Each service runs as its own OS process that Aspire launches. The debugger you attach to the AppHost process does not automatically step into those children — you attach to the child you care about, or debug a single host standalone (see below). Visual Studio and Rider can be configured to attach to the spawned children automatically; otherwise attach by hand.

Debug the full stack via Aspire

Run or debug the Cargonerds.AppHost project:

Set Cargonerds.AppHost as the startup project and press F5 (Visual Studio / Rider) or use the https launch profile.

dotnet run --project src/Cargonerds.AppHost

Aspire launches every service and infrastructure container and opens the Aspire dashboard. The AppHost's own endpoints come from src/Cargonerds.AppHost/Properties/launchSettings.json:

Endpoint URL (https profile)
Dashboard / app URL https://localhost:17176 (http: http://localhost:15198)
Dashboard OTLP ingest (DOTNET_DASHBOARD_OTLP_ENDPOINT_URL) https://localhost:21236
Resource service (DOTNET_RESOURCE_SERVICE_ENDPOINT_URL) https://localhost:22044

Docker must be running first

The last line of Program.cs is builder.Build().EnsureDockerRunningIfLocalDebug().Run();. EnsureDockerRunningIfLocalDebug() is a Nextended.Aspire helper that fails fast with a clear message if Docker Desktop is not running during local debug. If the AppHost exits immediately complaining about Docker, start Docker Desktop and retry.

Once it is up, the dashboard lists each resource with its live URL, health status, structured logs, distributed traces and metrics — see The Aspire dashboard below.

Attaching to a single child service

When you only need to step through one service (e.g. the API) but still want the rest of the graph running:

  1. Start the AppHost (F5 or dotnet run).
  2. Attach your debugger to the child process — in Visual Studio/Rider use Attach to Process and pick the executable (e.g. Cargonerds.HttpApi.Host), or in the Aspire dashboard open the resource's details to find its PID.
  3. Set breakpoints in that project as normal. The child already has the right configuration because the AppHost set USE_ASPIRE_CONFIG=true on it (see the gotcha below).

This keeps real service discovery, Redis, RabbitMQ and the database wired exactly as in a normal run, while you only pay the debugger cost on one process.

Debug a single .NET host standalone

To step through one service while it talks to a deployed environment's neighbors (no local Aspire graph, no containers), set AZURE_ENVIRONMENT and start just that host. The Azure config layer then supplies both the backend connections and the Services block, so the service-discovery tokens ({api}, {auth}, {realtime}, {web}, {admin}) resolve to the public URLs.

# e.g. debug the API against the dev environment's auth/realtime/etc.
AZURE_ENVIRONMENT=dev ASPNETCORE_ENVIRONMENT=Development \
  dotnet run --project src/Cargonerds.HttpApi.Host

How the config resolves in this mode is implemented in src/Cargonerds.ServiceDefaults/HostApplicationBuilderExtensions.cs (AddExtraConfigFiles → AddAzureJsonFiles). See the Configuration Reference for the full recipe and the important caveats.

Never debug against prod

AZURE_ENVIRONMENT=prod (or any prod.* value) resolves to production connection strings — those credentials are real and committed in src/Cargonerds.ServiceDefaults/appsettings.azure.*.json. Use dev/test. If you must run local code against a real Azure DB at all, use the AppHost's dedicated "spark DB" mode (APPHOST_AZURE_ENVIRONMENT + APPHOST_AZURE_DATABASE_COPY_SUFFIX), which forces a renamed copy for prod.* so you cannot mutate the live database — see Aspire integration and Running locally.

Fixed run-mode HTTPS ports (from each host's launchSettings.json and src/Cargonerds.AppHost/RunModeServicePorts.cs):

Host URL Constant
Cargonerds.AuthServer https://localhost:44345 RunModeServicePorts.AuthServerHttpsPort
Cargonerds.HttpApi.Host https://localhost:44354 RunModeServicePorts.ApiHttpsPort
Cargonerds.Blazor (admin) https://localhost:44381 RunModeServicePorts.AdminHttpsPort
Cargonerds.Web.Public https://localhost:44332 —
frontend/realtime (Next.js, http) http://localhost:4200 RunModeServicePorts.RealtimeHttpPort
SQL container (un-proxied TCP) localhost:14330 (DbPort = 53938 for the proxied endpoint) RunModeServicePorts.DbPort

USE_ASPIRE_CONFIG is mandatory under the AppHost — and must be set for standalone Aspire-style runs

The AppHost sets USE_ASPIRE_CONFIG=true on each child via AddProjectWithDefaults (src/Cargonerds.AppHost/Extensions/ResourceBuilderExtensions.cs). Per EnvVars.UseAspireConfig: "If not set the default ABP configuration will be used which is not compatible with the Aspire AppHost." When you launch a host directly (IIS Express / plain dotnet run) it stays false and the host falls back to the static appsettings.json — which is fine for the standalone-against-Azure recipe above, but means you are not using the Aspire appsettings.aspire.json token wiring.

Per-developer overrides (appsettings.spark.local.user.json)

When debugging against SPARK_ENVIRONMENT=local, the per-developer override file appsettings.spark.<env>.user.json (e.g. appsettings.spark.local.user.json) is layered on top with reloadOnChange: true (loaded in AddExtraConfigFiles). It is git-ignored and is the right place to point a host at your own local Hub DB / ports without touching committed config. See the Configuration Reference and appsettings reference.

Debugging Blazor WebAssembly

The Blazor admin (Cargonerds.Blazor) launch profiles set inspectUri for WASM debugging in src/Cargonerds.Blazor/Properties/launchSettings.json (both the IIS Express and Cargonerds.Blazor profiles):

"inspectUri": "{wsProtocol}://{url.hostname}:{url.port}/_framework/debug/ws-proxy?browser={browserInspectUri}"

Launch the Cargonerds.Blazor profile (or attach to it when running under the AppHost) and debug .razor / client code in your IDE or the browser dev tools. The admin server listens on https://localhost:44381. See Blazor admin UI.

Debugging the Next.js frontend (realtime)

Run it standalone with source maps:

cd frontend/realtime
npm run dev        # next dev --turbopack -p 4200

Use the browser dev tools / React DevTools. TanStack Query devtools are wired in frontend/realtime/src/app/providers.tsx. Point the app at a running API/auth via the env vars in frontend/realtime/src/types/envVars.ts. Under the AppHost these env vars are injected for you by WithNextAuth (src/Cargonerds.AppHost/Extensions/NodeAppExtensions.cs); standalone you supply them yourself. See Realtime Frontend.

The Aspire dashboard

The dashboard (default https://localhost:17176) is the central place to observe the running graph. Telemetry reaches it over OpenTelemetry (OTLP); the dashboard's OTLP ingest endpoint is published to the children via DOTNET_DASHBOARD_OTLP_ENDPOINT_URL.

Dashboard tab What you get Where it comes from
Resources Every service/container with state, endpoints, env vars, and per-resource commands. The resource graph in Program.cs.
Console logs Raw stdout/stderr per resource. The child process console.
Structured logs Searchable structured log records with scopes/levels. Serilog → WriteTo.OpenTelemetry() (see below) plus OTel logging.
Traces Distributed traces across HTTP calls between services. ASP.NET Core + HttpClient instrumentation.
Metrics Runtime / ASP.NET Core / HttpClient meters. AddRuntimeInstrumentation() + ASP.NET Core/HttpClient meters.

How telemetry is produced

AddServiceDefaults() (called first in every host's Program.cs, before the ABP module bootstrap) configures OpenTelemetry in src/Cargonerds.ServiceDefaults/HostApplicationBuilderExtensions.cs:

public static TBuilder ConfigureOpenTelemetry<TBuilder>(this TBuilder builder)
    where TBuilder : IHostApplicationBuilder
{
    builder.Logging.AddOpenTelemetry(logging =>
    {
        logging.IncludeFormattedMessage = true;
        logging.IncludeScopes = true;
    });

    builder.Services.AddOpenTelemetry()
        .WithMetrics(metrics =>
        {
            metrics.AddAspNetCoreInstrumentation().AddHttpClientInstrumentation().AddRuntimeInstrumentation();
        })
        .WithTracing(tracing =>
        {
            tracing
                .AddSource(builder.Environment.ApplicationName)
                .AddAspNetCoreInstrumentation()
                .AddHttpClientInstrumentation();
        });

    builder.AddOpenTelemetryExporters();
    return builder;
}

The exporter is chosen by environment variable (AddOpenTelemetryExporters):

// OTLP (the Aspire dashboard) — enabled when the endpoint is present
if (!string.IsNullOrWhiteSpace(builder.Configuration["OTEL_EXPORTER_OTLP_ENDPOINT"]))
    builder.Services.AddOpenTelemetry().UseOtlpExporter();

// Azure Monitor / Application Insights — enabled when the connection string is present
var connectionString = builder.Configuration["APPLICATIONINSIGHTS_CONNECTION_STRING"];
if (!string.IsNullOrEmpty(connectionString))
    builder.Services.AddOpenTelemetry().UseAzureMonitor(o => o.ConnectionString = connectionString);

Both exporters can be active

OTEL_EXPORTER_OTLP_ENDPOINT and APPLICATIONINSIGHTS_CONNECTION_STRING are independent — locally Aspire sets the OTLP endpoint so logs/traces flow to the dashboard; in Azure the connection string additionally ships telemetry to Application Insights.

Serilog → dashboard

ABP hosts log through Serilog. The bridge that makes Serilog records show up under the dashboard's Structured logs is the OpenTelemetry sink. From src/Cargonerds.HttpApi.Host/Program.cs:

loggerConfiguration
    .MinimumLevel.Override("Microsoft", LogEventLevel.Information)
    .MinimumLevel.Override("Microsoft.EntityFrameworkCore", LogEventLevel.Warning)
    .Enrich.FromLogContext()
    .WriteTo.Async(c => c.Console())
    .WriteTo.Async(c => c.OpenTelemetry());   // <-- exported to the Aspire dashboard / OTLP

EF Core SQL is filtered by default

Microsoft.EntityFrameworkCore is overridden to Warning, so generated SQL is not logged at the default level. To see the SQL EF emits while debugging, lower that override (e.g. to Information) or use the Hub CommandLoggingInterceptor path — see Entity Framework. Note #if DEBUG already sets the global minimum to Debug in Debug builds.

Speeding up startup while debugging

The AppHost honors two env vars (defined in src/Cargonerds.AppHost/EnvVars.cs). Both are checked for the literal value "1" — any other value is treated as off:

Variable Effect Accessor
SKIP_WAITING_IN_BACKEND=1 Skip inter-service WaitFor / health / migrator-completion gating, so backends start without waiting on each other. EnvVars.SkipWaitingInBackend
EXPLICIT_FRONTEND_START=1 Don't auto-start the frontends; start them manually from the dashboard. EnvVars.ExplicitFrontendStart

These feed the WaitForIf / WaitForCompletionIf / WithExplicitStartIf calls in Program.cs. The migrator-related VALIDATE_MIGRATIONS_ONLY toggle (validate, don't apply) is documented under Migrations and the Aspire integration page.

Common issues

Symptom Likely cause / fix
AppHost exits immediately with a Docker message Docker Desktop isn't running; EnsureDockerRunningIfLocalDebug() aborts early. Start Docker and retry.
Service throws InvalidOperationException listing "Available: …" services at startup An appsettings.aspire.json value contains a {name} service-discovery token with no matching services:* endpoint / ConnectionStrings:* entry. Wire the resource in the AppHost, or remove the token. Resolution is fail-fast in ServiceDiscoveryConfigurationProvider.
Standalone host uses the wrong (static) config / can't find {auth} etc. USE_ASPIRE_CONFIG not set. Under the AppHost it's set automatically; for standalone Aspire-style runs set USE_ASPIRE_CONFIG=true, or use the AZURE_ENVIRONMENT=dev recipe instead.
Port already in use (44345/44354/44381/4200) or SQL conflict on 14330 The run-mode ports are fixed (RunModeServicePorts.cs) and the SQL container's TCP endpoint is un-proxied (14330). A stray local SQL Server / leftover host on that port collides — stop it.
Startup crashes on a SPARK_ENVIRONMENT typo SparkEnvironment.FromName returns Dev for null/empty but throws on an unknown name. Fix the value (local/dev/test/prod/prod-read-only).
WHITE_LABEL_SETTINGS_PATH / white-label file not found when running a host directly Aspire runs each process with its project directory as cwd (not the build output), so the AppHost passes the absolute path to appsettings.rohlig.json via WithWhiteLabelSettingsPath. Standalone, ensure the file is reachable from the working directory.
Permission/DDL errors only during migration The DbMigrator uses admin DB credentials (appsettings.azure.migrator.<env>.json), distinct from the app's runtime credentials. See Migrations.
OData groupby/aggregate ($apply) widget request returns NullReferenceException The live TagWith(recompile) + $apply interaction — see Query performance below and OData filtering.

Query performance (ARITHABORT and OPTION (RECOMPILE))

A recurring class of "the app is slower than SSMS/Rider for the same query" issues comes down to SQL Server plan-cache behaviour. Two EF Core interceptors in modules/hub/src/Hub.EntityFrameworkCore/EntityFrameworkCore/ address it. Both are registered in HubEntityFrameworkCoreModule for the pooled HubDbContext:

opts.AddInterceptors(
    sp.GetRequiredService<SecondLevelCacheInterceptor>(),
    sp.GetRequiredService<CommandLoggingInterceptor>(),
    sp.GetRequiredService<FacetJoinInterceptor>(),
    sp.GetRequiredService<ArithAbortConnectionInterceptor>(),
    sp.GetRequiredService<RecompileCommandInterceptor>()
);

(The Default CargonerdsDbContext also gets ArithAbortConnectionInterceptor — added in CargonerdsEntityFrameworkCoreModule, configured in the same options.Configure(...) as the SQL Server provider; splitting that into a second Configure overwrites the provider and yields "No database provider configured". See Entity Framework and Layered architecture.)

ArithAbortConnectionInterceptor — same plans as your SQL tool

.NET SqlClient connects with ARITHABORT OFF while SSMS/Rider use ON. Differing SET options land in separate plan-cache slots, so the app can get its own — often worse — execution plan than the one you see when you run the identical query in your SQL tool. The interceptor re-applies the setting on every connection open (not once), because connection pooling resets SET options via sp_reset_connection:

// modules/hub/src/Hub.EntityFrameworkCore/EntityFrameworkCore/ArithAbortConnectionInterceptor.cs
private const string SetCommand = "SET ARITHABORT ON;";

public override void ConnectionOpened(DbConnection connection, ConnectionEndEventData eventData)
{
    using var cmd = connection.CreateCommand();
    cmd.CommandText = SetCommand;
    cmd.ExecuteNonQuery();
}
// (ConnectionOpenedAsync does the same for async opens)

Reproducing an app plan in SSMS/Rider

Because the app now runs with ARITHABORT ON, your SQL tool's default ON matches it. If you are still chasing a discrepancy, confirm the other SET options and parameter values match — EF's parameterization (and .NET 10/EF 10 collection-parameter translation, UseParameterizedCollectionMode) affects the cached plan too.

RecompileCommandInterceptor — opt-in OPTION (RECOMPILE)

For queries whose optimal plan depends heavily on the specific parameter values (parameter sniffing), you can force a fresh plan per execution. The interceptor appends OPTION (RECOMPILE) only when the command text contains the literal -- recompile marker, which EF adds via TagWith(RecompileCommandInterceptor.Tag):

// modules/hub/src/Hub.EntityFrameworkCore/EntityFrameworkCore/RecompileCommandInterceptor.cs
public const string Tag = "recompile";
private const string TagMarker = "-- " + Tag;   // EF renders TagWith("recompile") as "-- recompile"
private const string Hint = "\nOPTION (RECOMPILE)";

private static void ApplyHint(DbCommand command)
{
    var text = command.CommandText;
    if (text.Contains(TagMarker, StringComparison.Ordinal)
        && !text.Contains("OPTION (RECOMPILE)", StringComparison.Ordinal))
    {
        command.CommandText = text + Hint;
    }
}

Use it from a query like:

query = query.TagWith(RecompileCommandInterceptor.Tag);

Trade-off: recompile costs CPU

OPTION (RECOMPILE) recompiles the statement on every execution. It avoids a bad cached plan but spends optimizer CPU each time and removes the query from the plan cache — apply it to specific problem queries, not broadly.

Live TagWith(recompile) + OData $apply causes a NullReferenceException

ODataControllerBase.Get (modules/hub/src/Hub.HttpApi/Controllers/ODataControllerBase.cs) calls res = res.TagWith(RecompileCommandInterceptor.Tag); unconditionally and returns the query to [EnableQuery]. When the request is an OData $apply aggregation (groupby / aggregate), OData rewrites the query into GroupBy(...).Select(new GroupByWrapper/AggregationWrapper{...}), and the tag on that shape triggers a NullReferenceException — this hit POC-report groupby/aggregate widgets. The known fix is to apply the tag only when $apply is absent:

if (string.IsNullOrEmpty(Request.Query["$apply"]))
{
    res = res.TagWith(RecompileCommandInterceptor.Tag);
}

See OData, Filtering & Facets for the full controller pipeline, the cn.facets annotation and this gotcha.

Other Hub interceptors worth knowing while debugging

  • SecondLevelCacheInterceptor (EFCoreSecondLevelCache) — caches all queries for a 1-minute absolute TTL (CacheAllQueries(CacheExpirationMode.Absolute, TimeSpan.FromMinutes(1))). A query that "doesn't reflect a write you just made" within a minute is usually this cache, not a bug.
  • FacetJoinInterceptor — text-replaces LEFT JOIN → INNER JOIN for commands tagged with the facet prefix.
  • CommandLoggingInterceptor — scoped command logging (CommandLoggingScope<HubDbContext>).

All four are described in detail on Entity Framework and Caching.