Skip to navigation

Ingest

Add business data, documents, JSON, and conversations to Context Graphs

Zep builds Context Graphs from the data you provide. First select the graph that owns the context. Then choose a method by the kind of write: a live conversation turn, a real-time write from your application, or an import of test data, historical data, or other bulk data.

This is the Ingest stage of Working with Context. Before designing any import, review Prepare Data for Ingestion to preserve entity identity, source context, and event time.

Choose an ingestion path

What you are addingUse
A message in a live conversation, as your agent sends and receives itthread.add_messages in the Zep SDK
Test data, or historical data you are loading for the first time: documents, transcripts, email, Slack exports, or past conversationszep-ingest or the Batch API
A recurring bulk import (hourly or nightly dump, ETL output)zep-ingest or the Batch API
An individual document, API response, business event, or webhook payload that your application receives in real timegraph.episode.add with the graph_uuid of a user graph or a graph

Use thread.add_messages and graph.episode.add only for real-time ingestion in production. For example, use them when your agent receives a message or your application receives a live event. These methods are for one episode or a small number of episodes at a time. To load test data or to backfill data of any size, use the Batch API, directly or through zep-ingest. The Batch API ingests many episodes faster.

Live conversation turns: use the SDK

Add each message to Zep as your agent sends and receives it, using thread.add_messages. The SDK already runs inside the service handling the conversation, and a chat turn needs no preparation beyond the message itself. See Adding messages.

Test data, backfills, and bulk imports: use zep-ingest or the Batch API

zep-ingest is the recommended path for test data, a one-time backfill of your history, and a recurring import job. It submits the records through the Batch API when the Batch API is available. You put the data in the form the package requires. Built-in loaders take a path or glob. The package prepares, previews, submits in order, and monitors.

You do not have to use zep-ingest. It is a convenience layer over the Zep SDK, so calling the Batch API directly is fully supported. Do not write your own loop of graph.episode.add or thread.add_messages calls to load test data or to backfill data. If your deployment does not serve the Batch API, zep-ingest falls back to sequential calls for you.

Real-time individual writes: use graph.episode.add

When your application receives the data in real time: a webhook body, an API response, a single document: call graph.episode.add directly. Built-in zep-ingest loaders do not accept in-memory payloads today, so routing those through the package means writing a custom loader yourself; for most event-driven paths the SDK call is simpler. See Adding business data.

When a chunk reads better against earlier chunks of the same source, most often to resolve a pronoun, pass a document_id so Zep groups those chunks for prior-episode context and summarization. Do not share an ID across independent records.

Available methods

MethodData TypeBest For
Create an Ingestion PipelineMultiple source typesBackfills and bulk imports, with preparation, preview, and monitoring
Adding messagesChat messagesLive conversation turns from your agent
Adding business dataText, JSON, or message formatIndividual documents, API responses, emails, or other business data that arrive in real time
Batch ingestionMany episodes and/or messages in one batchTest data, backfills, and other imports, directly or through zep-ingest

Next: shape the graph

After ingesting data, Zep processes it to build Context Graphs. Customize extraction and summaries under Shape the Graph.