Concurrent Async Cache

A .NET library for caching values — particularly values computed by asynchronous factories, such as database queries or API calls.

It's thread-safe and provides the expected controls: expirations, timeouts, cancellation, and error-caching. But its central feature is stampede prevention: when several callers request the same uncached value concurrently, only one factory runs; the rest wait on it and share its result. Without that, a popular key expiring under load sends every concurrent request to the source at once.

It also has a concept of collections — specialized caches that know an item belongs to a collection, and maintain both views of the same data at once. Cache a whole collection and every item in it becomes individually retrievable; update a single item and the cached collection reflects the change. HasCollection() reports separately whether the complete collection is cached, so "I have some of these items" and "I have all of them" stay distinct questions.

What it does

• Coalesces concurrent callers for one key onto a single factory run — on expired keys, not just cold ones.
• Caches nulls as "this key resolves to nothing", on their own clock, so repeated lookups of a missing record don't hammer the source.
• Caches failures briefly, so a broken source isn't retried on every request.
• Serves stale data instead of throwing when a refresh fails (opt-in, per call).
• Forces a reload on demand — unconditionally, or only for entries older than a given instant.
• Bounds each worker with a timeout, and honours caller cancellation.
• Item, single-item, collection, and multi-collection wrappers over the core cache.

What it isn't

• Not distributed — single process, in-memory.
• Not size-bounded: no eviction or LRU. Expiration governs staleness, not footprint; entries live until removed or cleared.
• No built-in retry or worker throttling yet.

var cache = new Cache<string>();

// Produce the value with an async factory, abandoning it if it takes more than 2 seconds.
// Concurrent callers asking for "mycalculation" while this runs will share its result.
var result = await cache.GetOrAddAsync("mycalculation",
    async token =>
    {
        await Task.Delay(100, token);
        return 42;
    },
    timeout: TimeSpan.FromSeconds(2));

Writing a factory

A factory may still be running after the call that started it has returned. Coalescing works by having callers await a shared promise rather than their own factory task, so a caller can be handed a result someone else produced — by a concurrent TryAdd/AddOrUpdate, by a Clear/TryRemove discarding the entry, or by an AddOrUpdateAsync superseding the refresh — and return while its own factory is still in flight. AddOrUpdateAsync is no different: forcing a reload buys no protection.

Two rules follow:

• Don't capture anything the caller owns. A request-scoped DbContext, a pooled connection, a transaction, a DI scope — if the caller disposes it on the way out, a factory still running is left holding a disposed resource. Have the factory acquire its own.
• Thread the token all the way down. Every path above cancels the factory's token before resolving, so a factory that passes its CancellationToken into the calls it makes unwinds promptly. One that ignores it runs to completion regardless — cancellation is cooperative, and the cache can only ask.

// Good: owns its scope, and the token reaches the query.
var user = await cache.GetOrAddAsync(userId, async token =>
{
    await using var scope = services.CreateAsyncScope();
    var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
    return await db.Users.FindAsync([userId], token);
});

Specialized generic caches

CollectionCache<TItemKey, TItem>

MultiCollectionCache<TItemKey, TCollectionKey, TItem>

ItemCache<TItemKey, TItem>

SingleItemCache