Skip to navigation

Concurrent Ingestion

Ingest more than one episode of one graph at the same time

This page is for developers who import a large number of episodes into one graph. It tells how to turn on concurrent ingestion, how to configure it, and what its limits are.

Serial and concurrent ingestion

In serial ingestion (the default), each episode of one graph (group_id) resolves its entities and facts against the graph that the earlier episodes made. Serial ingestion does not put overlapping calls in order. Await each add_episode call for one graph before you start the next call for that graph. A large import into one graph thus takes the sum of the time of each episode.

Concurrent ingestion is an opt-in mode. It runs the extraction and the resolution of up to window episodes of one graph at the same time. Each episode then commits in the order of its call. Each add_episode or add_episode_bulk call is one unit, and each unit goes through four phases:

  1. Admit. The unit gets a sequence number in call order, and Graphiti saves the episode node.
  2. Analyze. The unit extracts and resolves its entities and facts against the graph. The unit writes no entity and no fact to the graph in this phase. Up to window units of one graph analyze at the same time.
  3. Commit. The units commit one at a time, in sequence order. At the commit, Graphiti compares the result of the unit with the units that committed after its analysis started. Graphiti repairs only the items that collide: a duplicate node, a duplicate fact, or a fact that a new fact contradicts. A repair can make LLM calls.
  4. Enrich. Graphiti updates the attributes and the summaries of the nodes of the unit.

The mode gives a shorter total time only when your code starts more than one call for one graph at the same time. With one call at a time, the mode is slower than serial ingestion.

Before you start

  • Concurrent ingestion is available on the Neo4j and FalkorDB drivers. The tests of the mode ran on an in-memory graph and on embedded FalkorDB. The mode is not tested on Neo4j or Amazon Neptune.
  • A driver with a graph_operations_interface must implement episodic_node_save_bulk, node_save_bulk, episodic_edge_save_bulk, edge_save_bulk, node_update_bulk, and edge_update_bulk. If an operation is missing, the call raises ValueError with the names of the missing operations and writes nothing to the graph.
  • If more than one process ingests into one graph, set a time limit for writes on the database. On FalkorDB, set the server setting TIMEOUT_MAX to a value that is not zero. With TIMEOUT_MAX 0 (the FalkorDB default) and on Neptune, a write can complete after its lease expires and replace a newer value of a stored fact. Neo4j applies the time limit of the write.
  • Each unit can send up to SEMAPHORE_LIMIT LLM calls at one time. The mode can thus send up to window x SEMAPHORE_LIMIT LLM calls at one time. Make sure that the rate limits of your LLM provider permit this number. Refer to LLM Configuration for the provider settings.

Turn on concurrent ingestion

1

Create the configuration

Create a ConcurrentIngestionConfig object. window has no default value. Start with a small value, for example 4.

2

Give the configuration to Graphiti

Pass the object as concurrent_ingestion when you create Graphiti. When concurrent_ingestion is None (the default), each call uses serial ingestion.

3

Start the calls in the order of the episodes

Start the add_episode calls for one graph at the same time, for example with asyncio.gather. When one task starts the calls in order, the calls get their sequence numbers in that order. Each call returns after the commit and the enrichment of its unit.

import asyncio
from datetime import datetime, timezone
from graphiti_core import Graphiti
from graphiti_core.ingestion import ConcurrentIngestionConfig
from graphiti_core.nodes import EpisodeType
episodes = [
{
'name': 'note-1',
'content': 'Maria Lopez joined Northwind Robotics in March 2024.',
'reference_time': datetime(2024, 3, 4, 9, 0, tzinfo=timezone.utc),
},
{
'name': 'note-2',
'content': 'Maria Lopez now leads the support team at Northwind Robotics.',
'reference_time': datetime(2024, 9, 2, 9, 0, tzinfo=timezone.utc),
},
]
async def main():
graphiti = Graphiti(
'bolt://localhost:7687',
'neo4j',
'password',
concurrent_ingestion=ConcurrentIngestionConfig(window=4),
)
try:
results = await asyncio.gather(
*[
graphiti.add_episode(
name=episode['name'],
episode_body=episode['content'],
source=EpisodeType.text,
source_description='support notes',
reference_time=episode['reference_time'],
group_id='support-team',
)
for episode in episodes
]
)
finally:
await graphiti.close()
asyncio.run(main())

The signatures and the result types of add_episode and add_episode_bulk do not change. AddEpisodeResults.nodes has the nodes after enrichment. AddEpisodeResults.edges has the created edges and the stored edges that the commit changed.

Configuration

FieldDefaultDescription
windowNo default. The value must be 1 or more.The largest number of units of one graph that run at one time in one process. window=1 uses the order of serial ingestion.
state_storeA new InMemoryIngestionStateStoreThe store of the sequence numbers, the change sets, and the enrichment order.
enabled_for_groupNoneA function that gets a group_id. When the function returns False, the call for that graph uses serial ingestion. With None, each graph uses the mode.
lease_seconds60.0The time after which a unit with no heartbeat is expired.
write_margin_seconds5.0The time before the deadline of a unit at which a graph write must be complete. The value must be less than lease_seconds / 3.
poll_interval_seconds0.25The wait time between two store calls when a unit waits for a unit of a different process.
index_lag_seconds300.0The longest time between a completed graph write and the time when a full-text search returns the record. Set 0.0 only when you verified that the full-text searches of your database return a completed write immediately.
max_apply_attempts5The number of attempts to write the final change set of one unit.
before_commit_writeNoneAn async function that gets the ChangeSet of a unit after the repair and returns the change set to write. It can raise ChangeSetRejected.
before_enrichment_writeNoneAn async function that gets the enriched nodes and the unit before each enrichment write. A node that the function does not return is not written.

The defaults of lease_seconds, write_margin_seconds, poll_interval_seconds, index_lag_seconds, and max_apply_attempts have no measurement behind them.

State stores

StoreUse
InMemoryIngestionStateStoreOne process. This store is the default.
SQLiteIngestionStateStore(path)More than one process on one machine. Give each process the same file path.
Your implementation of IngestionStateStoreA store on the database of your application, for more than one machine.

With a shared store and more than one process, each process has its own window. Graphiti.close() closes the state store.

Select graphs for the mode

Use enabled_for_group to turn on the mode for selected graphs only:

large_graphs = {'support-team', 'crm-import'}
config = ConcurrentIngestionConfig(
window=4,
enabled_for_group=lambda group_id: group_id in large_graphs,
)

Do not use serial ingestion and concurrent ingestion for one graph at the same time. Before you change the result of enabled_for_group for a graph, wait until no call for that graph is in progress.

Identity properties and bulk ingestion

  • add_episode and add_episode_bulk accept identity_properties in this mode. Give the same identity_properties value in each call for one graph. A unit that a later call recovers is repaired with the identity_properties value of the later call.
  • One add_episode_bulk call is one unit. Its episodes commit together.
  • update_communities=True is not available in this mode, and the call raises ValueError. Call build_communities after the ingestion.

Failures and retries

EventResultWhat to do
An exception or a cancellation before the commit stores the final change setThe episode node stays in the graph with no entities. The exception goes to the caller.Call add_episode again with the same uuid.
A graph write fails after the final change set is storedGraphiti writes the same final change set again, up to max_apply_attempts times, with no LLM call.No action.
All write attempts failThe unit has the status apply_failed. A part of the change set can be in the graph.Remove the episode with remove_episode, and ingest the episode again.
A process stops after the final change set is stored, with SQLiteIngestionStateStore or your own storeAfter lease_seconds, the next unit of the graph writes the stored change set.No action.
A process stops, with InMemoryIngestionStateStoreThe stored change sets of the process are lost. A part of the change set can be in the graph.Remove the episode with remove_episode, and ingest the episode again.
A retry names a unit that is in progressThe call raises UnitInProgressError and writes nothing.Try again after lease_seconds.
A unit calls the store after its deadlineThe store raises UnitExpiredError or StaleCommitError, and the unit stops.Call add_episode again with the same uuid.

Limits

  • The graph of a window larger than 1 can be different from the graph of serial ingestion. The LLM sees the episodes in a different context, and the repair prompt has only the candidates that the analysis did not see. The mode does not guarantee an equal graph.
  • The repairs run in the commit phase, one unit at a time. When most episodes of a graph use the same entities and facts, the time gain is small.
  • The mode does not give read-your-writes. A search can return a record of a unit only after the commit of that unit.
  • add_triplet, remove_episode, build_communities, and the node and edge CRUD methods do not use the sequence. Run these methods when no unit of the graph is in progress. Do not call remove_episode for an episode whose unit is in progress.

Telemetry

The add_episode and add_episode_bulk spans get the attributes ingest.mode='concurrent', ingest.seq, ingest.window, and ingest.unit_id. Each phase is a child span: ingest.admit, ingest.analyze, ingest.commit, and ingest.enrich. The ingest.commit span has the counts of the collisions and the repairs, for example node.collided, node.merged, edge.deduplicated, edge.ended, and repair.llm_calls. Refer to Telemetry for the tracing setup.

Your own orchestration

An application with its own queues and workers can call the phase functions admit_episode, analyze_episode, commit_unit, and enrich_unit from graphiti_core.ingestion and not use add_episode. The concurrent ingestion specification gives the phase functions, the state store contract, and the full list of limits.