Define the production target for rate limiting

You own a multi-tenant RAG answer API at POST /v1/answers. The goal is to protect shared model and vector-search capacity without punishing normal interactive users. The measurable target is a 60 requests/minute user bucket with a burst of 10, a 1,200 requests/minute tenant ceiling, and a 4,000 requests/minute global guard; rejected requests return HTTP 429 in under 25 ms with Retry-After. The known production tension is strict per-user fairness prevents a noisy neighbor but can block legitimate batch imports; local fallback buckets improve availability but permit bounded overshoot during Redis partitions.

Start with the request or operator journey, not a tool choice. For Rate limiting, draw POST /v1/answers entering a multi-tenant RAG answer API, then mark every place where work can queue, fail, or return a misleading success. Label the actor who experiences each outcome. The target is to protect shared model and vector-search capacity without punishing normal interactive users. Rewrite that sentence as an observable promise and list two non-goals so a later engineer cannot quietly expand the scope.

The acceptance contract is: a 60 requests/minute user bucket with a burst of 10, a 1,200 requests/minute tenant ceiling, and a 4,000 requests/minute global guard; rejected requests return HTTP 429 in under 25 ms with Retry-After. Break it into a table in your notes with columns for indicator, threshold, window, data source, and owner. Ratios require an eligible-event definition; latency targets require a start and stop boundary; recovery objectives require durable checkpoints. State whether retries, synthetic probes, client cancellation, fallbacks, and maintenance are included. If eligibility is ambiguous, two correct implementations can report incompatible reliability.

Use this incident as the concrete threat model: tenant acme-importer shipped a zero-jitter retry loop, consumed 74% of the global model budget, and pushed interactive p95 from 1.8 s to 6.4 s. Separate the trigger, contributing conditions, user impact, and delayed detection. Then rank three failure modes by likelihood and impact. The planned drill is run a retry storm from one tenant at 180 requests/second while Redis latency is injected at 250 ms; verify the global guard remains available and the limiter fails closed only for expensive generation. Explain why that fault is representative and why its scope is safe enough for staging.

The essential signals are rate_limit_decisions_total{scope,outcome}, rate_limit_redis_seconds, http_requests_total{status="429"}, and allowed request p95 latency. For each, write its unit, expected baseline, threshold, and maximum observation delay. Counters should be monotonic; histograms need buckets around the objective; gauges need an interpretation when traffic is zero. Do not put prompts, document text, raw user IDs, or other unbounded values in labels.

Treat the following as a design proposal, not accepted production truth:

limits:
  user: { rate_per_minute: 60, burst: 10 }
  tenant: { rate_per_minute: 1200, burst: 100 }
  global: { rate_per_minute: 4000, burst: 250 }
backend:
  redis_timeout_ms: 40
  on_timeout: local_bucket
response:
  status: 429
  retry_after: true

Review every number against demand and consequence. strict per-user fairness prevents a noisy neighbor but can block legitimate batch imports; local fallback buckets improve availability but permit bounded overshoot during Redis partitions. The lab owner must document which side of that tradeoff is preferred, when the preference expires, and what evidence would justify changing it. Add an abort condition for cost, data integrity, unrelated tenants, and failed rollback.

Use this command only to inspect or establish the initial baseline:

curl -i -X POST http://localhost:8080/v1/answers \
  -H 'Authorization: Bearer user-42' -H 'X-Tenant: acme' \
  -d '{"question":"How do refunds work?"}'

Record UTC time, environment, build or policy revision, expected output, and actual output. A screenshot without query text and timestamp is not durable evidence. The page is complete when another engineer can explain what is protected, what is deliberately not protected, and how success can be independently calculated.