CrispyWaffle.BackgroundJobs

CrispyWaffle.BackgroundJobs runs fire-and-forget and delayed jobs inside an ASP.NET Core application. You register typed handlers under a name, enqueue a job with that name and a payload, and a hosted worker picks it up, runs the handler in its own DI scope, and retries it with exponential backoff when it fails. Jobs are kept in an in-memory store by default, or in a database through Entity Framework Core.

The package is standalone: it does not reference the core CrispyWaffle package. It is built on Microsoft.Extensions.Hosting (BackgroundService) and Microsoft.Extensions.DependencyInjection. It is a different thing from the core package's cron scheduler: that one runs in-process actions on cron expressions through JobManager, while this package queues persisted, retryable jobs that carry a payload. The CronJobService base class in this package does not parse cron expressions (see Recurring work).

How it works

Registration

AddCrispyBackgroundJobs registers, all as singletons unless noted:

IJobStore and BackgroundJobQueue are added with TryAdd, so a store you register before the call wins.

AddJobHandler<THandler, TData>(name) registers THandler as scoped and adds an IStartupFilter that calls IJobHandlerRegistry.Register<THandler, TData>(name) when the ASP.NET Core request pipeline is built. This means AddJobHandler only populates the registry in an ASP.NET Core host. In a generic host (worker service) the filter never runs; register the handler type in DI yourself and call Register on the registry after building the host (see Use a generic host).

Enqueueing

JobDispatcher.EnqueueAsync (or IJobScheduler.EnqueueAsync) creates a JobEntity with Status = Pending, the handler name, the priority, MaxAttempt, and the payload serialized with System.Text.Json default options, then saves it with IJobStore.SaveAsync. ScheduleAsync does the same with ScheduledAt = UtcNow + delay; the job stays in the store until it is due.

Only when no IJobStore is registered does the dispatcher fall back to the BackgroundJobQueue: EnqueueAsync pushes the job onto the queue, and ScheduleAsync waits for the delay in a background Task and then pushes it. With AddCrispyBackgroundJobs a store is always registered, so this path is not used.

Worker loop

BackgroundWorker starts JobOptions.WorkerCount loops (default: one per processor). Each loop:

1. Calls IJobStore.FetchNextAsync, which returns the oldest due Pending job with the highest priority (High before Normal before Low, then by CreatedAt) and marks it Processing. If nothing is due, the loop sleeps for JobOptions.PollingInterval (default 2 seconds).
2. If the job's ScheduledAt is still in the future, puts it back (MarkRetryAsync with the same time) and moves on.
3. Creates a DI scope, looks up the handler type and payload type in IJobHandlerRegistry, deserializes the payload into the payload type, resolves the handler from the scope, and calls its HandleAsync.
4. Records the outcome in the store (see below).

An unexpected error in the loop itself is logged and the loop waits 1 second before continuing.

The handler is invoked by reflection: the worker looks up a public method named HandleAsync on the concrete handler type. Implement IBackgroundJobHandler<TData>.HandleAsync as a normal public method, not as an explicit interface implementation, and do not add other HandleAsync overloads; otherwise the call throws and the job is retried as described below.

Lifecycle and retries

The backoff is min(2^min(Attempt, 10), 300) seconds, where Attempt is the already incremented count: 2, 4, 8, 16, 32, 64, 128, 256, then 300 seconds. The cap is hard-coded; JobOptions.MaxBackoffSeconds is not read. JobEntity.RetryDelaySeconds is also not read.

MaxAttempt counts total executions. With the default of 3, a handler that keeps returning Fail(..., retry: true) runs three times and is then marked Failed.

On shutdown, a job interrupted by the stopping token is normally caught by the "handler throws" path and rescheduled with backoff. The JobStatus.Dead value is never assigned by the library.

Job stores

In-memory (default). InMemoryJobStore keeps jobs in a ConcurrentDictionary. Jobs are lost when the process stops, and completed and failed jobs are never removed. FetchNextAsync selects and marks a job in two separate steps without a lock, so with more than one worker the same job can occasionally be picked by two loops. Set WorkerCount = 1 if handlers must not run twice.

EF Core. Pass useEfCoreStore: true and register JobDbContext yourself with your database provider, for example services.AddDbContext<JobDbContext>(o => o.UseSqlServer(connectionString)). You must also create the Jobs table, for example with db.Database.EnsureCreated() or with migrations generated against JobDbContext (migrations need a MigrationsAssembly in your project, since the context lives in this package).

The EF Core mode has wiring problems you need to work around:

• JobDispatcher takes a BackgroundJobQueue constructor parameter, but the queue is only registered in in-memory mode. Without services.AddSingleton<BackgroundJobQueue>(), resolving JobDispatcher or IJobScheduler fails.
• EfCoreJobStore is scoped, but the singleton BackgroundWorker and JobDispatcher resolve it from the root provider. With scope validation enabled (the default in the Development environment) the host fails to start with a "Cannot consume scoped service" error. With validation disabled, a single JobDbContext instance is shared by every worker loop and every caller of the dispatcher. DbContext is not thread-safe, so keep WorkerCount = 1 and avoid enqueueing while the worker is running, or register your own IJobStore that creates a context per call.
• No concurrency token is configured on JobEntity, so FetchNextAsync does not prevent two processes from claiming the same job.

Treat the EF Core store as experimental until these are fixed.

Monitoring

JobMetrics is a singleton dictionary of named counters. The library registers it but does not increment any counter itself, so you record the values you care about, for example from your handlers. MonitoringMiddleware serves JobMetrics.Snapshot() as a JSON object ({"counter": value}) for any request whose path starts with the segment /jobs/metrics, and passes every other request on. There is no UseXxx extension; add it with app.UseMiddleware<MonitoringMiddleware>(). The endpoint has no authentication of its own.

Recurring work

CronJobService is an abstract BackgroundService that calls ExecuteOnce in a loop with a fixed one-minute delay after each run. Despite the file name (CronJobScheduler.cs), it does not accept or parse a cron expression, and it is not registered by AddCrispyBackgroundJobs. Register a subclass with AddHostedService. For real cron expressions, use the core scheduler, or enqueue jobs from your subclass.

Getting started

using CrispyWaffle.BackgroundJobs.Abstractions;
using CrispyWaffle.BackgroundJobs.Core;
using CrispyWaffle.BackgroundJobs.Extensions;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddCrispyBackgroundJobs(o => o.WorkerCount = 2);
builder.Services.AddJobHandler<SendEmailHandler, SendEmailData>("send-email");

var app = builder.Build();

app.MapPost(
    "/signup",
    async (string email, IJobScheduler jobs) =>
    {
        await jobs.EnqueueAsync("send-email", new SendEmailData(email, "Welcome!"));
        return Results.Accepted();
    }
);

app.Run();

public sealed record SendEmailData(string To, string Subject);

public sealed class SendEmailHandler(ILogger<SendEmailHandler> logger)
    : IBackgroundJobHandler<SendEmailData>
{
    public async Task<JobResult> HandleAsync(SendEmailData data, CancellationToken cancellationToken)
    {
        try
        {
            logger.LogInformation("Sending '{Subject}' to {To}", data.Subject, data.To);
            await Task.Delay(100, cancellationToken); // send the email here
            return JobResult.Ok();
        }
        catch (HttpRequestException ex)
        {
            return JobResult.Fail(ex.Message, retry: true);
        }
    }
}

Reference

ServiceCollectionExtensions

public static class ServiceCollectionExtensions — registration entry points (namespace CrispyWaffle.BackgroundJobs.Extensions).

IJobScheduler

public interface IJobScheduler — the injectable API for adding jobs (namespace CrispyWaffle.BackgroundJobs.Abstractions).

Because maxAttempts defaults to 3 here and is always passed on, JobOptions.DefaultMaxAttempt has no effect for jobs added through IJobScheduler.

DelayedJobScheduler

public class DelayedJobScheduler : IJobScheduler — default IJobScheduler; forwards both methods to JobDispatcher (namespace CrispyWaffle.BackgroundJobs.Scheduling).

JobDispatcher

public class JobDispatcher — creates JobEntity records and hands them to the store or the queue (namespace CrispyWaffle.BackgroundJobs.Core).

IBackgroundJobHandler<TData>

public interface IBackgroundJobHandler<TData> — implement this to handle one kind of job (namespace CrispyWaffle.BackgroundJobs.Abstractions). Handlers are resolved from a new DI scope per job, so they can depend on scoped services.

IBackgroundJob

public interface IBackgroundJob — a payload-less job abstraction (namespace CrispyWaffle.BackgroundJobs.Abstractions). Nothing in the package executes it; the worker only runs IBackgroundJobHandler<TData> handlers.

JobResult

public sealed record JobResult(bool Success, bool Retry = false, string? ErrorMessage = null) — the outcome returned by a handler (namespace CrispyWaffle.BackgroundJobs.Core).

IJobHandlerRegistry

public interface IJobHandlerRegistry — maps handler names to handler and payload types (namespace CrispyWaffle.BackgroundJobs.Core).

JobHandlerRegistry

public class JobHandlerRegistry : IJobHandlerRegistry — thread-safe default registry backed by a ConcurrentDictionary.

IJobStore

public interface IJobStore — persistence for jobs (namespace CrispyWaffle.BackgroundJobs.Abstractions). Implement it to use another storage; register your implementation before calling AddCrispyBackgroundJobs.

InMemoryJobStore

public class InMemoryJobStore : IJobStore — default store, process-local (namespace CrispyWaffle.BackgroundJobs.Persistence). Implements every IJobStore member; all methods complete synchronously and ignore the token. Unknown ids are ignored. See Job stores for its limits.

EfCoreJobStore

public class EfCoreJobStore : IJobStore — store backed by JobDbContext (namespace CrispyWaffle.BackgroundJobs.Persistence).

JobDbContext

public class JobDbContext : DbContext — EF Core context for the EfCoreJobStore (namespace CrispyWaffle.BackgroundJobs.Persistence).

BackgroundWorker

public class BackgroundWorker : BackgroundService — the hosted service that runs jobs (namespace CrispyWaffle.BackgroundJobs.Core). Registered by AddCrispyBackgroundJobs; you do not use it directly. See Worker loop and Lifecycle and retries.

BackgroundJobQueue

public class BackgroundJobQueue — in-memory priority queue used only when no IJobStore is registered (namespace CrispyWaffle.BackgroundJobs.Core).

When the worker uses this queue, a job that is not yet due is dequeued and enqueued again in a tight loop until it is due.

JobOptions

public class JobOptions — worker settings, configured through AddCrispyBackgroundJobs (namespace CrispyWaffle.BackgroundJobs.Core). Registered as a plain singleton, not through IOptions<T>.

JobEntity

public class JobEntity — a stored job (namespace CrispyWaffle.BackgroundJobs.Abstractions).

JobPriority

public enum JobPriority — lower values run first.

JobStatus

public enum JobStatus — job lifecycle state.

JobMetrics

public class JobMetrics — thread-safe named counters, registered as a singleton (namespace CrispyWaffle.BackgroundJobs.Monitoring). Nothing in the package increments them.

MonitoringMiddleware

public class MonitoringMiddleware — ASP.NET Core middleware that exposes JobMetrics (namespace CrispyWaffle.BackgroundJobs.Monitoring).

CronJobService

public abstract class CronJobService : BackgroundService — base class for work that repeats every minute (namespace CrispyWaffle.BackgroundJobs.Scheduling). See Recurring work.

JobFailedException

public class JobFailedException : Exception — namespace CrispyWaffle.BackgroundJobs.Exceptions. Has one constructor, JobFailedException(string message). The package never throws it; if a handler throws it, it is treated like any other exception (logged and retried).

Examples

Schedule a delayed, high-priority job

JobDispatcher returns the job id; IJobScheduler does not.

using CrispyWaffle.BackgroundJobs.Abstractions;
using CrispyWaffle.BackgroundJobs.Core;

public sealed class ReminderService(JobDispatcher dispatcher)
{
    public Task<Guid> RemindNowAsync(string email) =>
        dispatcher.EnqueueAsync("send-email", new SendEmailData(email, "Reminder"));

    public Task RemindLaterAsync(string email) =>
        dispatcher.ScheduleAsync(
            "send-email",
            new SendEmailData(email, "Your trial ends tomorrow"),
            TimeSpan.FromHours(1),
            maxAttempts: 5,
            priority: JobPriority.High
        );
}

Expose metrics

Increment counters from your handlers and map the middleware.

using CrispyWaffle.BackgroundJobs.Abstractions;
using CrispyWaffle.BackgroundJobs.Core;
using CrispyWaffle.BackgroundJobs.Extensions;
using CrispyWaffle.BackgroundJobs.Monitoring;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddCrispyBackgroundJobs();
builder.Services.AddJobHandler<ResizeImageHandler, ResizeImageData>("resize-image");

var app = builder.Build();
app.UseMiddleware<MonitoringMiddleware>(); // GET /jobs/metrics -> {"resize-image.ok":12}
app.Run();

public sealed record ResizeImageData(string Path, int Width);

public sealed class ResizeImageHandler(JobMetrics metrics) : IBackgroundJobHandler<ResizeImageData>
{
    public Task<JobResult> HandleAsync(ResizeImageData data, CancellationToken cancellationToken)
    {
        // resize data.Path to data.Width here
        metrics.Increment("resize-image.ok");
        return Task.FromResult(JobResult.Ok());
    }
}

Use the EF Core store

Register the context, the queue that JobDispatcher needs, and create the table at startup. Keep one worker because the store's DbContext is shared (see Job stores), and disable scope validation, which otherwise rejects the scoped store inside singletons.

using CrispyWaffle.BackgroundJobs.Core;
using CrispyWaffle.BackgroundJobs.Extensions;
using CrispyWaffle.BackgroundJobs.Persistence;
using Microsoft.EntityFrameworkCore;

var builder = WebApplication.CreateBuilder(args);
builder.Host.UseDefaultServiceProvider(o =>
{
    o.ValidateScopes = false;
    o.ValidateOnBuild = false;
});

builder.Services.AddDbContext<JobDbContext>(o =>
    o.UseSqlServer(builder.Configuration.GetConnectionString("Jobs"))
);
builder.Services.AddSingleton<BackgroundJobQueue>();
builder.Services.AddCrispyBackgroundJobs(o => o.WorkerCount = 1, useEfCoreStore: true);
builder.Services.AddJobHandler<SendEmailHandler, SendEmailData>("send-email");

var app = builder.Build();
using (var scope = app.Services.CreateScope())
{
    scope.ServiceProvider.GetRequiredService<JobDbContext>().Database.EnsureCreated();
}
app.Run();

UseSqlServer comes from the Microsoft.EntityFrameworkCore.SqlServer package; use the provider for your database.

Use a generic host

AddJobHandler relies on an ASP.NET Core startup filter. In a worker service, register the handler in DI and in the registry yourself before the host starts.

using CrispyWaffle.BackgroundJobs.Core;
using CrispyWaffle.BackgroundJobs.Extensions;

var builder = Host.CreateApplicationBuilder(args);
builder.Services.AddCrispyBackgroundJobs();
builder.Services.AddScoped<SendEmailHandler>();

var host = builder.Build();
host.Services.GetRequiredService<IJobHandlerRegistry>()
    .Register<SendEmailHandler, SendEmailData>("send-email");

host.Run();

Run work every minute

using CrispyWaffle.BackgroundJobs.Abstractions;
using CrispyWaffle.BackgroundJobs.Scheduling;

public sealed class CleanupService(ILogger<CleanupService> logger, IJobScheduler jobs)
    : CronJobService(logger)
{
    protected override Task ExecuteOnce(CancellationToken stoppingToken) =>
        jobs.EnqueueAsync("cleanup", new { OlderThanDays = 30 });
}

// builder.Services.AddHostedService<CleanupService>();

The anonymous payload is serialized as {"OlderThanDays":30}, so the cleanup handler's payload type needs a matching OlderThanDays property.

See also

• Diagrams
• Scheduler for cron-based scheduling in the core package
• HTTP client, the other standalone Microsoft.Extensions based package