AI-200 Azure Managed Redis: Caching, Expiration, Invalidation and Vector Search

Azure Managed Redis is Microsoft's fully managed, in-memory Redis service, and in an AI application it plays three roles: a cache in front of slower stores such as Azure Database for PostgreSQL or Azure Cosmos DB, a vector store for similarity search over embeddings (through the RediSearch module), and a semantic cache that reuses large language model (LLM) answers for prompts that mean the same thing.

Three choices made when you create an instance decide what it can do later:

The creation-time rule has a practical consequence: if a running cache was created without RediSearch (or with the OSS policy), the way to add vector search is a new instance created with RediSearch and the Enterprise policy, with the AI feature pointed at it. Clients connect over TLS on port 10000, the Azure Managed Redis default.

To connect without access keys, the app gets a Microsoft Entra token for the scope https://redis.azure.com/.default and sends it with AUTH: the token is the password and the identity's object ID (not its client ID) is the user name. The identity must also be granted data access on the cache, and access keys can then be disabled.

In Python, the redis-entraid package wraps this into a credential provider for redis-py:

import redis
from redis_entraid.cred_provider import create_from_default_azure_credential

provider = create_from_default_azure_credential(
    scopes=(REDIS_SCOPE,)       # the Azure Managed Redis scope above
)
client = redis.Redis(
    host=CACHE_HOST, port=10000, ssl=True,
    credential_provider=provider, decode_responses=True,
)

The part that is easy to get wrong is token lifetime. Entra tokens have a limited lifetime, so connections must re-authenticate with a fresh token before the current one expires, or they are disrupted. A long-lived connection therefore sends a new token with a new AUTH ahead of expiry; Microsoft recommends doing so at least three minutes before the token expires, with some jitter so many clients don't refresh at once. The credential providers (redis-entraid for Python, Microsoft.Azure.StackExchangeRedis for .NET) refresh and re-authenticate for you.

Every Azure service has its own token audience, and a Redis data connection accepts only tokens issued for the Redis scope. Two prerequisites also apply: Entra authentication works over TLS connections only, and Microsoft Entra groups aren't supported, so access is granted to individual users, service principals or managed identities.

The default pattern for an AI app's cache is cache-aside: the reading code checks Redis first, and on a miss it reads the database and writes the result to Redis with a TTL, so later reads hit. Writers don't change at all, and the TTL caps how stale an entry can get.

def get_description(product_id):
    key = f"desc:{product_id}"
    cached = r.get(key)
    if cached is not None:
        return cached
    row = db.fetch_description(product_id)   # PostgreSQL
    r.set(key, row, ex=900)                  # value + TTL in one call
    return row

Caching misses (negative caching)

Plain cache-aside caches only hits, so requests for keys that don't exist in the database, such as a scraper walking through invalid product IDs, always fall through to the database. Storing a short-lived not-found marker under the key absorbs those repeats. Because nothing else removes the marker when the item is later created (unless the writer is changed to delete it), its TTL must be shorter than the time within which newly created items have to become visible.

A TTL (time to live) is set by passing an expiry option to SET or by calling EXPIRE on an existing key; Redis deletes the key when it runs out. Setting the value and the TTL in one SET call is preferred, because a separate EXPIRE leaves a moment when the key exists with no expiry (and none at all if the second call fails).

Other SET flags don't affect expiry: NX writes only if the key is missing, XX only if it exists, and GET returns the old value.

Which writes clear a TTL

Redis treats some commands as replacing the key and others as changing its value in place, and only the replacements clear the timeout. DEL, SET (without KEEPTTL), GETSET and the *STORE commands clear it. Commands that alter the value without replacing the key, such as INCR/INCRBY, HSET or LPUSH, leave it in place.

So a key first written with an expiry and later overwritten with a plain SET loses that expiry: it never leaves, and TTL returns -1. To overwrite and keep the original deadline, pass KEEPTTL on the overwrite. Passing an expiry option again instead restarts the clock on every write (a sliding expiry), which is a different behaviour.

A TTL only bounds staleness to the TTL's length. When edits must be visible almost at once, the writer deletes the cache key after the database write succeeds, and the normal cache-aside read path reloads it once on the next request. That keeps database load at about one reload per edit.

• Delete after the write, not before. If the key is deleted first, a concurrent reader can miss, read the old row before the write commits, and cache the old value again for a full TTL.
• Only removing or replacing the key changes what readers see. Changing a key's expiry, or shortening every TTL, just trades staleness for database load; a very short TTL sends most reads to the database.

Deleting very large keys: UNLINK

Redis executes commands on one thread, so DEL on a key with millions of elements (a huge hash, for example) blocks every other client while the memory is freed. UNLINK removes the key from the keyspace immediately, so the invalidation is just as instant, and frees the memory on a background thread.

Invalidating a group of keys: hash slots

Clustered Redis spreads keys across 16,384 hash slots. With the OSS clustering policy, every key in a multi-key command or a MULTI/EXEC transaction must be in the same slot, or Redis returns a CROSSSLOT error. To keep a group of keys together (for example everything cached for one user), give them a shared hash tag: Redis hashes only the text inside the first pair of braces, so {user:4821}:cart and {user:4821}:recs land in the same slot and can be changed or deleted together in one atomic operation.

The Enterprise clustering policy lets a short list of multi-key commands span slots: DEL, MSET, MGET, EXISTS, UNLINK and TOUCH. Other multi-key operations still expect keys in one slot, so hash tags remain the portable design.

When an Azure Managed Redis database reaches its memory limit, its eviction policy decides which keys are removed to make room, and the default is volatile-lru: evict the least recently used key among keys that have a TTL. Keys without a TTL are never evicted under it.

Two consequences come straight from the default:

• A TTL is also an eviction marker. Under the default policy, the keys that carry a TTL are the ones Redis may evict, so putting TTLs only on data the app can regenerate and leaving them off data that must persist controls eviction without any setting change. An allkeys policy makes every key evictable, whether it has a TTL or not.
• Keys without a TTL can fill memory. If a service writes everything without a TTL, the volatile policies eventually find nothing to evict and writes fail as if the policy were NoEviction. The real fix is to set TTLs; when that code can't change yet, switching the database to AllKeysLRU makes every key a candidate. The policy can be changed on an existing database (for example az redisenterprise database update --eviction-policy AllKeysLRU).

A cache stampede happens when many popular keys expire at the same moment and hundreds of requests miss together and query the database for the same rows. It is typical of caches warmed in one batch: keys written at startup with the same TTL all expire in the same second, every TTL period.

Two standard fixes tackle the two halves of the problem:

• TTL jitter spreads the expirations: add a random offset to each key's TTL when it is written, for example ex=3600 + random.randint(0, 300).
• A rebuild lock stops concurrent rebuilds of one key: on a miss, the request that wins SET lock:<key> <token> NX EX 10 rebuilds the value, and the others wait briefly and read the cache again. The EX on the lock means a crashed worker can't hold it forever.

if r.set(f"lock:{key}", token, nx=True, ex=10):
    value = load_from_db()
    r.set(key, value, ex=3600 + random.randint(0, 300))
    r.delete(f"lock:{key}")   # production code checks the token first
else:
    time.sleep(0.05)          # then read the key again

Vector indexing in Azure Managed Redis means creating a RediSearch index (FT.CREATE) with a VECTOR field, so that embeddings stored in hashes or JSON documents become searchable by similarity. The index watches keys with a given prefix and indexes each document as it is written. It requires an instance created with the RediSearch module and the Enterprise or Non-clustered policy.

FT.CREATE idx:chunks ON HASH PREFIX 1 chunk:
  SCHEMA
    content  TEXT
    tenant   TAG
    category TAG
    embedding VECTOR HNSW 6
      TYPE FLOAT32 DIM 1536 DISTANCE_METRIC COSINE

The number after the algorithm (6) is the count of attribute arguments that follow (three name-value pairs). Three attributes are required:

• TYPE – element type, such as FLOAT32 (the usual choice for embeddings) or FLOAT64.
• DIM – the number of dimensions; it must equal the embedding model's output size (for example 1,536). A field declared with another DIM can't index those vectors.
• DISTANCE_METRIC – COSINE, IP (inner product) or L2 (Euclidean). Use the metric the embedding model is designed for; cosine is the common choice for text embeddings.

Metadata used to narrow searches goes in ordinary fields beside the vector. A TAG field indexes whole values (tenant IDs, categories, source names) without splitting them into words or stemming them, which is what exact-match filters need. TEXT is for full-text search, NUMERIC for ranges and GEO for locations. To add a field later, FT.ALTER … SCHEMA ADD extends the schema; it can't change an existing field's parameters.

HASH versus JSON storage

A document whose vector doesn't match the schema isn't indexed: the key exists, but queries never return it and FT.INFO counts it under hash_indexing_failures. The classic cause is NumPy's default: np.array(list_of_floats) is float64 (8 bytes per value), so tobytes() writes 12,288 bytes to a field that expects 6,144. Fix the data at the source with dtype=np.float32, since DIM describes the model's output and must not change. Only the two forms in the table are indexed as vectors; a vector stored as text in any encoding is not.

Choosing the algorithm: FLAT versus HNSW

RediSearch offers two vector index algorithms: FLAT compares the query with every vector and returns exact nearest neighbours, while HNSW walks a graph and returns approximate neighbours much faster on large collections.

If a requirement says results must be exact, never approximate, the answer is FLAT; tuning HNSW raises recall but never makes it exact. For tens of thousands of vectors, FLAT's latency is usually acceptable.

When an HNSW index misses too many true neighbours, the fix that needs no rebuild is a higher EF_RUNTIME, set in the index definition or per query ([KNN 10 @embedding $vec EF_RUNTIME $ef]); it raises recall at the cost of some latency. M and EF_CONSTRUCTION are fixed when the field is built, so changing them means re-creating the index and reloading.

A similarity search is an FT.SEARCH with a KNN clause: pass the query embedding as a parameter, ask for the k nearest neighbours, sort by the returned distance and set LIMIT to k.

FT.SEARCH idx:chunks
  "(@category:{handbook})=>[KNN 8 @embedding $vec AS score]"
  PARAMS 2 vec <float32 bytes>
  SORTBY score
  RETURN 2 content score
  LIMIT 0 8
  DIALECT 2

• PARAMS and DIALECT 2. The vector is passed as a binary parameter ($vec), and vector queries need query dialect 2 or later.
• LIMIT. FT.SEARCH returns LIMIT 0 10 by default. A KNN 25 clause finds 25 neighbours, but only 10 come back unless you add LIMIT 0 25; the number returned is set by LIMIT alone.
• Scores are distances. RediSearch reports a distance for every metric, so lower means more similar. For COSINE the distance is 1 − cosine similarity (range 0 to 2); for IP it is 1 − inner product. A minimum cosine similarity of 0.75 becomes a maximum distance of 0.25, and results sort ascending. Code ported from a store that returns similarity (higher is better) keeps the worst matches if it isn't converted.

Filtering: prefilter, not post-filter

A hybrid query puts a filter before the KNN clause: (filter)=>[KNN k @field $vec]. Redis searches only among documents that match the filter and returns that subset's k nearest neighbours, so a partition (a tenant, a category) with at least k documents always gets k results, and the isolation is enforced inside Redis. Filtering the output of an unfiltered KNN search can leave fewer than k, because the global neighbours may belong to other partitions. TAG values in a query go in braces, and punctuation such as a hyphen is escaped: @category:{garden\-tools}.

A range query (@embedding:[VECTOR_RANGE $radius $vec]) is the other option: it returns every document within a distance radius, however many that is, so it suits "anything close enough" checks rather than "exactly the top k". Range queries on HNSW have their own EPSILON attribute, which widens the search boundary.

A semantic cache stores LLM answers keyed by the meaning of the prompt: each prompt is embedded, and a new prompt whose embedding is close enough to a cached one gets the stored answer instead of a model call. That catches rephrasings ("How do I reset my password?" and "I forgot my login, what now?") that exact-key caching misses, because hashing or normalising prompt text only helps when users type the same words, and keyword search depends on shared words.

Microsoft's tutorial for Azure Managed Redis uses RedisVL, whose SemanticCache stores the prompt embedding, response and metadata in a RediSearch index and checks new prompts with a vector query:

cache = SemanticCache(
    name="support-cache-v3",          # include template/model version
    redis_url=REDIS_URL,
    vectorizer=vectorizer,            # same embedding model every time
    distance_threshold=0.1,
    ttl=3600,
    filterable_fields=[{"name": "tenant", "type": "tag"}],
)
hit = cache.check(prompt=question,
                  filter_expression=Tag("tenant") == tenant_id)
if not hit:
    answer = call_llm(question)
    cache.store(prompt=question, response=answer,
                filters={"tenant": tenant_id})

The settings that matter

• distance_threshold is a Redis cosine distance (0 to 2): lower is stricter. Too loose a threshold returns the answer to a different question with similar wording ("upgrade my plan" versus "downgrade my plan"). Microsoft's guidance is to start strict, evaluate with a test set, and loosen only while the cached answers stay correct.
• ttl controls how long answers live, which matters for freshness and memory but not for match quality.
• Partition by tenant. Similarity compares only the prompt, not who asked it, so the same question from two tenants matches whatever the threshold. Use a separate index or a filterable field (tenant, user group, subscription) and filter every check.
• Version by model and prompt template. An answer produced under an old system prompt or model still matches new prompts. Put the model ID, prompt template version and policy version in the cache's index name, metadata or key prefix so a change starts a fresh cache.