Keep the hot path off the database: short TTLs, cached misses, instant off

We run a trade relay. A source account publishes the positions it holds, and follower accounts pull them to open the same trades. Each follower pulls twice a second. With 100 followers, the server answers 200 requests a second, on a 2 GB machine that also runs the database. This article shows how we keep the database from feeling that load.

Examples use Redis 7 and Go, trimmed from the real system.

The principle: separate the hot path from the database

Every system has a hot path, the part called hundreds of times more often than the rest. Here it is the endpoint followers pull positions from. Each call needs to know two things:

1. Which account the follower’s token belongs to, and whether that account still allows following
2. What that account holds right now

If both ask the database every time, 200 requests a second become 200 queries a second, all the time the market is open.

Principle: the hot path must be answered from cache; the database is asked only when the cache expires.

State that disappears when the source goes quiet

Item 2 does not need the database at all. The source keeps publishing its positions, and we store them in Redis with a 60-second lifetime:

SET copy:state:a:42 '{"positions":[...]}' EX 60

If the source stops publishing, because the machine is off or the connection dropped, the state disappears on its own within 60 seconds. Followers never open trades from stale data left behind, and no clean-up job is needed; the key’s lifetime does that.

Caching the token: short, with both values together

Item 1 does need the database, but only once, remembered for 30 seconds:

func tokenLookup(ctx context.Context, tok string) (accountID int64, enabled, ok bool) {
    key := "copy:tok:" + tok
    if v, err := rdb.Get(ctx, key).Result(); err == nil {
        if v == "-" {
            return 0, false, false // already known: no such token
        }
        return parse(v) // "42:1" = account 42, following allowed
    }
    acct, err := db.AccountByCopyToken(ctx, tok)
    if errors.Is(err, pgx.ErrNoRows) {
        rdb.Set(ctx, key, "-", 30*time.Second) // remember the miss too
        return 0, false, false
    }
    if err != nil {
        return 0, false, false // database trouble: do NOT remember a miss
    }
    rdb.Set(ctx, key, fmt.Sprintf("%d:%d", acct.ID, b2i(acct.Enabled)), 30*time.Second)
    return acct.ID, acct.Enabled, true
}

Three things to notice:

• The cached value holds both the id and the on/off flag, because the hot path needs both. Cache only the id and you still ask the database about the flag every time
• Misses are remembered ("-"). Someone hammering random tokens, or a follower whose token was revoked and keeps pulling, ends at Redis and never reaches the database
• A database error is never remembered as a miss. If it were, a valid token would fail for 30 seconds through no fault of its own

The load, before and after

	No cache	30-second cache
Requests in	200 per second	200 per second
Database queries (100 followers, one token)	200 per second	1 per 30 seconds

200 queries a second become one every 30 seconds, 6,000 times fewer. Redis answers the rest from memory.

The price: changes arrive up to 30 seconds late

A cache is not free. If the account owner rotates the token to cut every follower off, or switches following off, the old value can live in the cache for up to 30 more seconds, and followers keep pulling during that time.

Some things can wait. But a kill switch that takes half a minute is not a kill switch, so the system deletes the cache itself at two moments:

// when following is switched off
rdb.Del(ctx, "copy:tok:"+tok)
// when the token is rotated: drop the old one before issuing the new
rdb.Del(ctx, "copy:tok:"+oldTok)

A TTL is the longest you accept stale data. For anything that cannot be stale at all, delete the cache when it changes; do not wait for it to expire.

Limiting writes with SET NX

Some hot paths must also write, for example to know whether each follower is still online. Recording every pull takes us back to 200 writes a second, so Redis acts as a gate allowing at most one write per follower per minute:

fresh, err := rdb.SetNX(ctx, "copy:log:42:"+login, "1", 60*time.Second).Result()
if err == nil && fresh {
    db.TouchFollower(ctx, 42, login) // one database write per minute per follower
}

SET NX succeeds only when the key does not exist yet. For the next 60 seconds, every following request gets false and writes nothing. A follower pulling twice a second causes one write a minute instead of 120.

Note that a Redis error also means no write, the same as “too soon”. If a broken gate let everything through, every request would hit the database at once, at the very moment the system is already in trouble. Losing a minute of online records beats taking the database down with it.

Configure Redis as a real cache

redis-server --maxmemory 128mb --maxmemory-policy allkeys-lru --save "" --appendonly no

• A 128MB memory cap, on a 2 GB machine shared with the database and the app
• allkeys-lru: when full, evict the least recently used key instead of erroring
• No disk persistence: everything in our Redis can be rebuilt, the real data is in the database, so no I/O is spent writing to disk

Common mistakes

• A cache with no TTL, so wrong data stays forever until someone notices
• Not deleting the cache when data changes, for things that must take effect at once
• Remembering an error as “not found”: the database hiccups for a few seconds and users are locked out for the whole TTL
• Caching an incomplete value, so the database still has to be asked for the rest
• Keeping data you cannot lose in a Redis configured as a throwaway cache