Caching ¶

cache.Backend is the context-aware integration boundary for replaceable byte caches. A miss returns (nil, false, nil); an outage returns an error. Callers must explicitly choose whether to fall back to their authoritative data source. Errors remain discoverable with errors.Is. Do not use cache contents for session revocation, authorization decisions or distributed locking.

NewMemoryBackend(capacity, maxValueBytes) supplies a concurrent, bounded local implementation. It copies values, expires entries at the TTL boundary, and evicts the oldest written entry when full. The existing Memory and Store APIs remain compatible. Local instances do not coordinate across application processes.

backend, err := cache.NewMemoryBackend(1000, 64*1024)
if err != nil { return err }
registry, err := cache.NewRegistry(map[string]cache.Alias{
    "articles": {Backend: backend, Namespace: "production:blog", Version: "v1"},
})
if err != nil { return err }
articles, err := registry.Lookup("articles")
if err != nil { return err }
err = articles.Set(ctx, "public:42", []byte("cached payload"), time.Minute)

Construct a registry at startup and inject clients into applications. Alias maps are copied. Each alias includes its name, deployment/application namespace and version in a length-framed SHA-256 key. Raw keys are not sent to the backend; keys must contain 1–1024 bytes and TTLs must be positive. Hashing does not encrypt values. Explicitly include every tenant, locale, permission or representation variation in the logical key where applicable.

Delete invalidates one logical key; deleting a missing key succeeds. Changing Version selects a fresh key space, leaving old entries to expire. All processes must adopt that version to agree on the active generation. In-flight readers or writers can race with deletion, so invalidation does not provide transactional consistency. There is deliberately no global flush operation.

Backends must be concurrent, preserve byte ownership, implement positive TTLs, and report operational errors. Cancellation can race with a completed mutation; an error is not evidence of rollback. The built-in adapter checks cancellation before entering its short in-memory operation. Network adapters should pass the context through to their transport.

The built-in backend is local to one process. Use a suitable shared backend when multiple instances must share entries.

Ask your agent ¶

Read https://cicadel.fauzaanu.com/en/dev/topics/cache/index.md.
Cache [expensive read] in this application. Include tenant and permission
variation in the key, set a TTL, define invalidation after writes, and keep
the database authoritative. Test hits, misses and backend failures.

Coalescing expensive loads ¶

Create a reusable cache.NewLoader(client) and call loader.Get(ctx, key, ttl, load), where load accepts a context and returns ([]byte, error). Hits bypass the function. Concurrent misses for the same key share the first caller’s function and TTL; different keys proceed independently. The cache is rechecked before loading to close the miss/join race. Successful loads populate the cache; load and backend failures propagate and can be retried. Each result is copied independently.

Caller cancellation stops only that caller’s wait. The load retains the first caller’s context values, but its lifetime ends when all waiters cancel or the operation finishes. Load functions must honor cancellation and must not recurse into the same loader/key. Use keys that include all authorization and tenant variations, and use one consistent load policy per loader. Coordination is local to a loader instance, not across processes. Deletion and concurrent writes still have the ordinary best-effort cache consistency semantics described above.

Bounded load execution ¶

For request-serving applications, use cache.NewBoundedLoader(client, 32, 5*time.Second) to cap concurrent distinct loads and give each shared operation a deadline. Both limits must be positive. Hits remain available at capacity; callers joining an existing flight share its slot. New misses receive cache.ErrLoaderBusy immediately, discoverable through errors.Is; choose an application-specific retry or fallback policy.

The deadline covers backend rechecking, loading and writing. Functions and backends must cooperate with cancellation. An abandoned operation retains its execution slot until it actually returns, preventing repeated canceled requests from creating unlimited replacement goroutines. A noncooperative function can therefore exhaust capacity; Go cannot forcibly terminate it. Existing NewLoader behavior remains compatible and unbounded. Neither constructor provides distributed admission control or transactional invalidation.

Race-tested evidence: TestBoundedLoaderAdmissionAndAbandonedExecution verifies capacity rejection, abandoned-write prevention and slot reuse; TestBoundedLoaderDeadline verifies deadline propagation and limit validation.