Retrieving context

Zep provides three methods to retrieve context from a user graph. Each method gives you a different level of control.

Choosing a retrieval method

Zep’s Context Block

Zep’s Context Block is an automatically assembled string for your agent. Smart Context Assembly uses auto search to build the string.

Auto search combines semantic search, full-text search, and graph search. It uses the four most recent messages from the specified thread as its query.

The thread.get_user_context() method returns the Context Block. The results can include context from any thread in the user’s graph.

The Context Block has a latency of less than 200 ms at the 95th percentile.

Retrieving the Context Block

1
# Get context for the thread
2
user_context = client.thread.get_user_context(thread_id=thread_id)
3
4
# Access the context block for use as untrusted model input
5
context_block = user_context.context
6
print(context_block)

Context Block format

The Context Block returns available context types in a structured format. This example contains a user summary and facts:

# This is the user summary
<USER_SUMMARY>
Emily Painter is a user with account ID Emily0e62 who uses digital art tools for creative work. She maintains an active account with the service, though has recently experienced technical issues with the Magic Pen Tool. Emily values reliable payment processing and seeks prompt resolution for account-related issues. She expects clear communication and efficient support when troubleshooting technical problems.
</USER_SUMMARY>
# These are the most relevant facts and their valid date ranges
# format: FACT (Date range: from - to)
<FACTS>
  - Emily is experiencing issues with logging in. (2024-11-14 02:13:19+00:00 - present)
  - User account Emily0e62 has a suspended status due to payment failure. (2024-11-14 02:03:58+00:00 - present)
  - user has the id of Emily0e62 (2024-11-14 02:03:54 - present)
  - The failed transaction used a card with last four digits 1234. (2024-09-15 00:00:00+00:00 - present)
  - The reason for the transaction failure was 'Card expired'. (2024-09-15 00:00:00+00:00 - present)
  - user has the name of Emily Painter (2024-11-14 02:03:54 - present)
  - Account Emily0e62 made a failed transaction of 99.99. (2024-07-30 00:00:00+00:00 - 2024-08-30 00:00:00+00:00)
</FACTS>

The default Context Block can include a user summary, facts, entities, episodes, observations, and thread summaries. Smart Context Assembly selects relevant context types.

The user summary appears when the account enables user summaries and the user node has a summary. Use a context template to specify types or limits.

Get the Context Block sooner

You can get the Context Block sooner by passing in the return_context=True flag to the thread.add_messages() method. Read more about this in our performance guide.

Custom context templates

You can customize the format of the Context Block by using context templates. Templates allow you to define how context data is structured and presented while keeping Zep’s automatic relevance detection.

To use a template, pass the template_id parameter when retrieving context:

1
from zep_cloud import Zep
2
3
client = Zep(api_key="YOUR_API_KEY")
4
5
# Create a custom template
6
client.context.create_context_template(
7
    template_id="customer-support",
8
    template="""# CUSTOMER PROFILE
9
%{user_summary}
10
11
# FACTS
12
%{edges limit=10}
13
14
# KEY ENTITIES
15
%{entities limit=5}"""
16
)
17
18
# Use the template to retrieve context
19
user_context = client.thread.get_user_context(
20
    thread_id="thread_id",
21
    template_id="customer-support"
22
)
23
context_block = user_context.context

See the Context Templates guide to learn how to create and manage templates.

Advanced Context Block construction

Use Advanced Context Block construction to control the query, parameters, and format. This method uses graph search results.

Using context

Once you retrieve the Context Block, pass it to your model as data. The Context Block can contain end-user or third-party content.

The Context Block can contain text that came from end users, documents, tools, or other external sources. A privileged message gives that text higher instruction priority than ordinary input. Keep the Context Block out of system messages, developer messages, and other privileged instruction channels.

Follow your model provider’s documented method for separating instructions from data:

• For the OpenAI Responses API, send preloaded context through ordinary input or a user message. Use function_call_output only for the result of an actual function call.
• For the Anthropic Messages API, design retrieval as a tool call when context can contain third-party data. Return the context in a tool_result block linked to the original tool_use_id.
• For other providers, use the documented untrusted-data channel. If the provider does not define one, use an ordinary user-level message with explicit data framing.

OpenAI with preloaded context

Place the Context Block after the conversation history and before the latest user request. Everything before the block stays unchanged between turns, so this order preserves the cacheable prefix that prompt caching needs. Replace the previous turn’s block instead of appending a second one.

If the model requests memory through a function, return the Context Block as function_call_output linked to the original call_id.

OpenAI Chat Completions with tool-retrieved context

Anthropic with tool-retrieved context

Do not create a tool message for preloaded context unless the provider documents that pattern. A tool-result type must remain linked to the model’s actual tool request.

Read Memory security best practices for provider-specific mappings, write controls, action authorization, and recovery guidance.

Provide the last four to six messages

Include the last four to six thread messages when you call your LLM provider. Zep ingestion can take a few minutes. The Context Block can omit information from recent messages during that time.

The Context Block provides long-term context. The recent messages provide raw, short-term context. Keep both forms of context out of privileged instruction channels.