Concurrent Ingestion
This page is for developers who import a large number of episodes into one graph. It tells how to turn on concurrent ingestion, how to configure it, and what its limits are.
Serial and concurrent ingestion
In serial ingestion (the default), each episode of one graph (group_id) resolves its entities and facts against the graph that the earlier episodes made. Serial ingestion does not put overlapping calls in order. Await each add_episode call for one graph before you start the next call for that graph. A large import into one graph thus takes the sum of the time of each episode.
Concurrent ingestion is an opt-in mode. It runs the extraction and the resolution of up to window episodes of one graph at the same time. Each episode then commits in the order of its call. Each add_episode or add_episode_bulk call is one unit, and each unit goes through four phases:
- Admit. The unit gets a sequence number in call order, and Graphiti saves the episode node.
- Analyze. The unit extracts and resolves its entities and facts against the graph. The unit writes no entity and no fact to the graph in this phase. Up to
windowunits of one graph analyze at the same time. - Commit. The units commit one at a time, in sequence order. At the commit, Graphiti compares the result of the unit with the units that committed after its analysis started. Graphiti repairs only the items that collide: a duplicate node, a duplicate fact, or a fact that a new fact contradicts. A repair can make LLM calls.
- Enrich. Graphiti updates the attributes and the summaries of the nodes of the unit.
The mode gives a shorter total time only when your code starts more than one call for one graph at the same time. With one call at a time, the mode is slower than serial ingestion.
Before you start
- Concurrent ingestion is available on the Neo4j and FalkorDB drivers. The tests of the mode ran on an in-memory graph and on embedded FalkorDB. The mode is not tested on Neo4j or Amazon Neptune.
- A driver with a
graph_operations_interfacemust implementepisodic_node_save_bulk,node_save_bulk,episodic_edge_save_bulk,edge_save_bulk,node_update_bulk, andedge_update_bulk. If an operation is missing, the call raisesValueErrorwith the names of the missing operations and writes nothing to the graph. - If more than one process ingests into one graph, set a time limit for writes on the database. On FalkorDB, set the server setting
TIMEOUT_MAXto a value that is not zero. WithTIMEOUT_MAX0 (the FalkorDB default) and on Neptune, a write can complete after its lease expires and replace a newer value of a stored fact. Neo4j applies the time limit of the write. - Each unit can send up to
SEMAPHORE_LIMITLLM calls at one time. The mode can thus send up towindowxSEMAPHORE_LIMITLLM calls at one time. Make sure that the rate limits of your LLM provider permit this number. Refer to LLM Configuration for the provider settings.
Turn on concurrent ingestion
Create the configuration
Create a ConcurrentIngestionConfig object. window has no default value. Start with a small value, for example 4.
The signatures and the result types of add_episode and add_episode_bulk do not change. AddEpisodeResults.nodes has the nodes after enrichment. AddEpisodeResults.edges has the created edges and the stored edges that the commit changed.
Configuration
The defaults of lease_seconds, write_margin_seconds, poll_interval_seconds, index_lag_seconds, and max_apply_attempts have no measurement behind them.
State stores
With a shared store and more than one process, each process has its own window. Graphiti.close() closes the state store.
Select graphs for the mode
Use enabled_for_group to turn on the mode for selected graphs only:
Do not use serial ingestion and concurrent ingestion for one graph at the same time. Before you change the result of enabled_for_group for a graph, wait until no call for that graph is in progress.
Identity properties and bulk ingestion
add_episodeandadd_episode_bulkacceptidentity_propertiesin this mode. Give the sameidentity_propertiesvalue in each call for one graph. A unit that a later call recovers is repaired with theidentity_propertiesvalue of the later call.- One
add_episode_bulkcall is one unit. Its episodes commit together. update_communities=Trueis not available in this mode, and the call raisesValueError. Callbuild_communitiesafter the ingestion.
Failures and retries
Limits
- The graph of a window larger than 1 can be different from the graph of serial ingestion. The LLM sees the episodes in a different context, and the repair prompt has only the candidates that the analysis did not see. The mode does not guarantee an equal graph.
- The repairs run in the commit phase, one unit at a time. When most episodes of a graph use the same entities and facts, the time gain is small.
- The mode does not give read-your-writes. A search can return a record of a unit only after the commit of that unit.
add_triplet,remove_episode,build_communities, and the node and edge CRUD methods do not use the sequence. Run these methods when no unit of the graph is in progress. Do not callremove_episodefor an episode whose unit is in progress.
Telemetry
The add_episode and add_episode_bulk spans get the attributes ingest.mode='concurrent', ingest.seq, ingest.window, and ingest.unit_id. Each phase is a child span: ingest.admit, ingest.analyze, ingest.commit, and ingest.enrich. The ingest.commit span has the counts of the collisions and the repairs, for example node.collided, node.merged, edge.deduplicated, edge.ended, and repair.llm_calls. Refer to Telemetry for the tracing setup.
Your own orchestration
An application with its own queues and workers can call the phase functions admit_episode, analyze_episode, commit_unit, and enrich_unit from graphiti_core.ingestion and not use add_episode. The concurrent ingestion specification gives the phase functions, the state store contract, and the full list of limits.