Fieldnotes / engineering

Cache-aside

Explanation

Read the source

The application needs a runbook. The database is the source of record: it holds the saved revision.

A normal source read asks the database for the current record. The animation uses illustrative times.

  1. 0s The application needs a runbook. The database stores the saved record.
  2. 7s It sends a read to the database. The database looks up the key.
  3. 14s The database returns revision 1. The application receives the record.
  4. 21s Every read can repeat that work. A cache can keep a reusable copy.

A cache hit

A cache is a separate store for reusable copies. In cache-aside, the application checks it before reading the source.

A hit means the requested key was found in the cache. It does not establish that the copy is fresh.

  1. 0s A cache holds a reusable copy. The application checks it first.
  2. 7s The requested key is present. This lookup is a cache hit.
  3. 14s The cache returns revision 1. No database read is needed here.
  4. 21s A hit avoids this source read. But is the copy still current?

A cache miss

When the key is absent, the application reads the database and fills the cache. The cache does not fetch the data by itself.

A miss is an absent key, not a failed database. Application code performs the source read and cache fill.

  1. 0s The application checks the cache. The key is absent: a miss.
  2. 7s The application reads the source. The database has revision 1.
  3. 14s The database returns the record. The application receives it next.
  4. 21s The application fills the cache. A later lookup can find this copy.

A stale copy

An editor updates the database, but the cached key is not deleted. A later hit can still return the old runbook.

Source writes and cache changes are separate operations. A successful hit can still return stale data.

  1. 0s Both stores begin at revision 1. The copies agree for now.
  2. 7s An editor saves revision 2. The cached copy is unchanged.
  3. 14s The next lookup checks the cache. It still finds revision 1.
  4. 21s The old copy returns successfully. A cache hit does not mean fresh.

Invalidate and refill

Invalidation removes the cached key. The next reader misses, reads revision 2 from the source, and stores it.

This repairs the illustrated stale key. Failed deletions and concurrent readers or writers need additional handling.

  1. 0s The source is now at revision 2. Remove its stale cached key.
  2. 7s The next cache lookup misses. Read the source again.
  3. 14s The source returns revision 2. The application uses the new record.
  4. 21s Store the new copy in the cache. Concurrent writes still need care.
Read the guide

Helpful first: Replication and consistency.

The idea

The application owns the lookup, source read, and cache fill. An update path can commit to the source and then delete the cached entry. These operations are separate: another writer, a failed deletion, or an overlapping reader can leave outdated data available. This chapter concerns exact-key data caching behind an agent tool, not model prompt caching or similarity-based answer reuse.

Where it helps

Use this when repeated tool reads create measurable source load. Delegation makes a stale result consequential: an agent may use an old runbook to propose a deployment. First decide which decisions tolerate old information; adding a cache is not a prerequisite for adopting agents.

Engineers building agent tools need this when repeated retrieval makes source capacity or latency a practical problem. Keep it as a supporting chapter because most teams should establish trustworthy tool results and review boundaries before optimizing retrieval.

Illustrative scenario: a release-planning agent calls readRunbook(team, service). The cached guide is revision 1. An editor publishes revision 2 with a changed rollback step, but misses invalidation. The tool returns revision 1 again. A recovery operation deletes the key; the following lookup fetches revision 2 and caches it. No deployment executes in this example.

What you will be able to check

  • Trace a hit and a miss, identifying which component reads the source and populates the cache.
  • Reproduce a stale fill after invalidation with a controlled concurrent reader.
  • Specify key scope, freshness checks, and source-load limits for one tool.

What changes in the team

Today
An engineer opens the authoritative runbook, checks its revision, and writes a release plan for review.
With an agent
An agent requests the runbook through a tool that returns its source revision and retrieval time, then drafts the plan with those references.
Still your responsibility
The engineer owns access rules, the acceptable freshness budget, and approval of the proposed operational steps.

Work through an example

  1. Define the contract. Use a mock runbook containing revision and rollbackStep. For this exercise, key the result by team, service, and output-schema version. The team and service come from validated request context; knowing a key never grants access.
  2. Implement the read path. After authorization, look up the key. On a miss, read the source and cache the returned object. For Redis, SET key value EX 300 attaches expiration in the same command. EXPIRE sets a key’s timeout; Redis removes the key after that timeout expires, and a plain GET does not renew its timeout. Cache lifetime is not proof of source freshness.
  3. Implement the update path. Record a successful source commit before deleting the key. Surface deletion failures for recovery. Return the source revision to the caller so a reviewer can compare it against the authoritative revision before relying on operational instructions.
  4. Challenge the happy path. Pause an in-flight old read until after deletion, then release it. A version label alone cannot prevent stale refill. For an action requiring current instructions, make an authoritative recheck part of its execution contract and define what happens if that check fails. A check establishes freshness only at the time it reads the authority; if a later action requires the same revision, the execution system must enforce that revision precondition or revalidate under an appropriate concurrency contract.

The engineering decision

Start with measured repeated reads and an explicit freshness budget. Treat five minutes as a test setting, not a recommendation. Choose keys for the result’s full scope, and keep authorization in application code on every request, including hits. Persistent instructions cannot enforce permissions. Decide whether a failed fresh read should block the proposed action.

  • Caching a permission decision alongside content and reusing it after access changes: check authorization for every tool call.
  • Treating expiration as a consistency guarantee: an old in-flight read can populate a newly expiring entry after a write.
  • Letting simultaneous misses or a cache outage flood the source: test with caching disabled, coalesce duplicate work where appropriate, and cap downstream concurrency.

Try it with your team

Make a stale runbook visible

  1. Build an in-memory source and cache with a controllable pause between source read and cache fill.
  2. Perform one cold read followed by two warm reads and record source-read counts; change the source without deletion and observe the old revision.
  3. Delete the key and read again, checking that the new revision appears.
  4. Run the paused-reader race and capture the stale result; add an authoritative revision check before accepting the plan.

You have an answer when…

  • An initial cold read followed by two warm reads produces exactly one source fetch; deletion causes the next read to fetch again.
  • The race is reproducible, and the acceptance check rejects a plan whose cited revision differs from the current source.

Read further

Can you pause a reader after it fetches revision 1, commit revision 2 and invalidate, then resume the reader and observe revision 1 reappear?

Original video comparisons