Autogen Memory Example

Autogen is a library created by Microsoft for building multi-agent applications. This example demonstrates using Zep for Autogen long-term memory.

NOTE This example does not include all code required to run the Autogen Agents.

A complete Notebook example of using Zep for Autogen long-term memory may be found in the Zep Python SDK Repository.

This example walks through how to build an Autogen Agent with long-term memory. Zep builds a knowledge graph from user interactions with the agent, enabling the agent to recall relevant facts from previous conversations or user interactions.

In this example we will:

Create an Autogen Agent class that extends ConversableAgent by adding long-term memory
Create a Mental Health Assistant Agent, CareBot, that acts as a counselor and coach.
Create a user Agent, Cathy, who stands in for our expected user.
Demonstrate preloading chat history into Zep.
Demonstrate the agents in conversation, with CareBot recalling facts from previous conversations with Cathy.
Inspect Facts within Zep, and demonstrate how to use Zep’s Fact Ratings to improve the quality of returned facts.

Install dependencies

$ pip install autogen zep-cloud

Import Autogen and configure define a `config_list`

1 import os
2 from dotenv import load_dotenv
3 import uuid
4 from typing import Union, Dict
5 from autogen import ConversableAgent, Agent
6 
7 load_dotenv()
8 
9 config_list = [
10     {
11         "model": "gpt-4o-mini",
12         "api_key": os.environ.get("OPENAI_API_KEY"),
13         "max_tokens": 1024,
14     }
15 ]

initiualize the Zep Client

You can sign up for a Zep account here: https://www.getzep.com/

1 from zep_cloud.client import AsyncZep
2 from zep_cloud import Message, FactRatingExamples, FactRatingInstruction
3 
4 MIN_FACT_RATING = 0.3
5 
6 # Configure Zep
7 zep = AsyncZep(api_key=os.environ.get("ZEP_API_KEY"))

ZepConversableAgent

The ZepConversableAgent is a custom implementation of the Autogen ConversableAgent that integrates with Zep for long-term memory management. This class extends the functionality of the base ConversableAgent by adding Zep-specific features for persisting and retrieving facts from long-term memory.

1 class ZepConversableAgent(ConversableAgent):
2     """
3     A custom ConversableAgent that integrates with Zep for long-term memory.
4     """
5     def __init__(
6         self,
7         name: str,
8         system_message: str,
9         llm_config: dict,
10         function_map: dict,
11         human_input_mode: str,
12         zep_session_id: str,
13     ):
14         super().__init__(
15             name=name,
16             system_message=system_message,
17             llm_config=llm_config,
18             function_map=function_map,
19             human_input_mode=human_input_mode,
20         )
21         self.zep_session_id = zep_session_id
22         # store the original system message as we will update it with relevant facts from Zep
23         self.original_system_message = system_message
24         self.register_hook(
25             "a_process_last_received_message", self.persist_user_messages
26         )
27         self.register_hook(
28             "a_process_message_before_send", self.persist_assistant_messages
29         )
30 
31     async def persist_assistant_messages(
32         self, sender: Agent, message: Union[Dict, str], recipient: Agent, silent: bool
33     ):
34         """Agent sends a message to the user. Add the message to Zep."""
35 
36         # Assume message is a string
37         zep_messages = convert_to_zep_messages(
38             [{"role": "assistant", "name": self.name, "content": message}]
39         )
40         await zep.memory.add(session_id=self.zep_session_id, messages=zep_messages)
41 
42         return message
43 
44     async def persist_user_messages(self, messages: list[dict[str, str]] | str):
45         """
46         User sends a message to the agent. Add the message to Zep and
47         update the system message with relevant facts from Zep.
48         """
49         # Assume messages is a string
50         zep_messages = convert_to_zep_messages([{"role": "user", "content": messages}])
51         await zep.memory.add(session_id=self.zep_session_id, messages=zep_messages)
52 
53         memory = await zep.memory.get(self.zep_session_id, min_rating=MIN_FACT_RATING)
54 
55         # Update the system message with the relevant facts retrieved from Zep
56         self.update_system_message(
57             self.original_system_message
58             + f"\n\nRelevant facts about the user and their prior conversation:\n{memory.relevant_facts}"
59         )
60 
61         return messages

Zep User and Session Management

Zep User

A Zep User represents an individual interacting with your application. Each User can have multiple Sessions associated with them, allowing you to track and manage interactions over time. The unique identifier for each user is their UserID, which can be any string value (e.g., username, email address, or UUID).

Zep Session

A Zep Session represents a conversation and can be associated with Users in a one-to-many relationship. Chat messages are added to Sessions, with each session having many messages.

Fact Rating

Fact Rating is a feature in Zep that allows you to rate the importance or relevance of facts extracted from conversations. This helps in prioritizing and filtering information when retrieving memory artifacts. Here, we rate facts based on poignancy. We provide a definition of poignancy and several examples of highly poignant and low-poignancy facts. When retrieving memory, you can use the min_rating parameter to filter facts based on their importance.

Fact Rating helps ensure the most relevant information, especially in long or complex conversations, is used to ground the agent.

1 bot_name = "CareBot"
2 user_name = "Cathy"
3 
4 user_id = user_name + str(uuid.uuid4())[:4]
5 session_id = str(uuid.uuid4())
6 
7 
8 await zep.user.add(user_id=user_id)
9 
10 fact_rating_instruction = """Rate the facts by poignancy. Highly poignant
11 facts have a significant emotional impact or relevance to the user.
12 Low poignant facts are minimally relevant or of little emotional
13 significance."""
14 fact_rating_examples = FactRatingExamples(
15     high="The user received news of a family member's serious illness.",
16     medium="The user completed a challenging marathon.",
17     low="The user bought a new brand of toothpaste.",
18 )
19 
20 await zep.memory.add_session(
21     user_id=user_id,
22     session_id=session_id,
23     fact_rating_instruction=FactRatingInstruction(
24         instruction=fact_rating_instruction,
25         examples=fact_rating_examples,
26     ),
27 )

Preload a prior conversation into Zep

We’ll load a prior conversation into long-term memory. We’ll use facts derived from this conversation when Cathy restarts the conversation with CareBot, ensuring Carebot has context.

1 chat_history = [
2     {
3         "role": "assistant",
4         "name": "carebot",
5         "content": "Hi Cathy, how are you doing today?",
6     },
7     {
8         "role": "user",
9         "name": "Cathy",
10         "content": "To be honest, I've been feeling a bit down and demotivated lately. It's been tough.",
11     },
12     {
13         "role": "assistant",
14         "name": "CareBot",
15         "content": "I'm sorry to hear that you're feeling down and demotivated, Cathy. It's understandable given the challenges you're facing. Can you tell me more about what's been going on?",
16     },
17     {
18         "role": "user",
19         "name": "Cathy",
20         "content": "Well, I'm really struggling to process the passing of my mother.",
21     },
22     {
23         "role": "assistant",
24         "name": "CareBot",
25         "content": "I'm deeply sorry for your loss, Cathy. Losing a parent is incredibly difficult. It's normal to struggle with grief, and there's no 'right' way to process it. Would you like to talk about your mother or how you're coping?",
26     },
27     {
28         "role": "user",
29         "name": "Cathy",
30         "content": "Yes, I'd like to talk about my mother. She was a kind and loving person.",
31     },
32 ]
33 
34 # Convert chat history to Zep messages
35 zep_messages = convert_to_zep_messages(chat_history)
36 
37 await zep.memory.add(session_id=session_id, messages=zep_messages)

Review all facts in Zep

We query all session facts for this user session. Only facts that meet the MIN_FACT_RATING threshold are returned.

1 response = await zep.memory.get_session_facts(session_id=session_id, min_rating=MIN_FACT_RATING)
2 
3 for r in response.facts:
4     print(r)

created_at='2024-10-07T20:04:11.98139Z' fact='Cathy has been feeling down and demotivated lately.' rating=0.5 uuid_='17183c18-381b-45d2-82ea-0c06317acf6f'
created_at='2024-10-07T20:04:11.98139Z' fact='Cathy describes her mother as a kind and loving person.' rating=0.5 uuid_='cd6b2e6d-b287-4d92-9de5-d4ee6e82111e'
created_at='2024-10-07T20:04:11.98139Z' fact='Cathy is struggling to process the passing of her mother.' rating=0.75 uuid_='bb2f100c-2f12-4976-9026-b322c29e457e'

Create the Autogen agent, CareBot, an instance of `ZepConversableAgent`

We pass in the current session_id into the CareBot agent which allows it to retrieve relevant facts related to the conversation with Cathy.

1 carebot_system_message = """
2 You are a compassionate mental health bot and caregiver. Review information about the user and their prior conversation below and respond accordingly.
3 Keep responses empathetic and supportive. And remember, always prioritize the user's well-being and mental health. Keep your responses very concise and to the point.
4 """
5 
6 agent = ZepConversableAgent(
7     bot_name,
8     system_message=carebot_system_message,
9     llm_config={"config_list": config_list},
10     function_map=None,  # No registered functions, by default it is None.
11     human_input_mode="NEVER",  # Never ask for human input.
12     zep_session_id=session_id,
13 )

Create the Autogen agent, Cathy

Cathy is a stand-in for a human. When building a production application, you’d replace Cathy with a human-in-the-loop pattern.

Note that we’re instructing Cathy to start the conversation with CareBit by asking about her previous session. This is an opportunity for us to test whether fact retrieval from Zep’s long-term memory is working.

1 cathy = ConversableAgent(
2     user_name,
3     system_message="You are a helpful mental health bot. You are seeking counsel from a mental health bot. Ask the bot about your previous conversation.",
4     llm_config={"config_list": config_list},
5     human_input_mode="NEVER",  # Never ask for human input.
6 )

Start the conversation

We use Autogen’s a_initiate_chat method to get the two agents conversing. CareBot is the primary agent.

NOTE how Carebot is able to recall the past conversation about Cathy’s mother in detail, having had relevant facts from Zep added to its system prompt.

1 result = await agent.a_initiate_chat(
2     cathy,
3     message="Hi Cathy, nice to see you again. How are you doing today?",
4     max_turns=3,
5 )

CareBot (to Cathy):
Hi Cathy, nice to see you again. How are you doing today?
--------------------------------------------------------------------------------
Cathy (to CareBot):
Hello CareBot! I'm doing well, thank you for asking. I wanted to reflect on our previous conversation—do you remember what we discussed? It would be helpful to revisit that topic and explore it further if you’re up for it!
--------------------------------------------------------------------------------
CareBot (to Cathy):
Of course, Cathy. We talked about your difficulty in processing your mother’s passing and how you've been feeling down and demotivated lately. It's completely natural to have these feelings, especially after such a significant loss. What would you like to explore further?
--------------------------------------------------------------------------------
Cathy (to CareBot):
Cathy: Thank you for reminding me, CareBot. Yes, I’ve been struggling with those feelings, and it’s been tough to navigate. I’d like to explore some coping strategies to help me process my grief. Are there any techniques or practices you would recommend?
--------------------------------------------------------------------------------
CareBot (to Cathy):
Absolutely, Cathy. Here are some coping strategies to help you navigate your grief:
1. **Journaling**: Writing your thoughts and feelings can be a great outlet and help you process your emotions.
2. **Talk to someone**: Sharing your feelings with a trusted friend or therapist can provide support and understanding.
3. **Mindfulness and meditation**: These practices can help ground you and create a sense of calm amid emotional turmoil.
4. **Create a tribute**: Honoring your mother through a scrapbook, writing letters, or lighting a candle can foster connection and memory.
5. **Physical activity**: Engaging in exercise can boost your mood and help alleviate stress.
Remember, it's important to be gentle with yourself as you navigate this process. What resonates with you?
--------------------------------------------------------------------------------
Cathy (to CareBot):
Cathy: Thank you for those suggestions, CareBot. I really like the idea of journaling and creating a tribute for my mother; it sounds like a meaningful way to express my feelings. I also think mindfulness could help me find some peace amidst the sadness. Do you have any specific tips on how to start journaling or practicing mindfulness?
--------------------------------------------------------------------------------

Review current facts in Zep

Let’s see how the facts have evolved as the conversation has progressed.

1 response = await zep.memory.get_session_facts(session_id, min_rating=MIN_FACT_RATING)
2 
3 for r in response.facts:
4     print(r)

created_at='2024-10-07T20:04:28.397184Z' fact="Cathy wants to reflect on a previous conversation about her mother and explore the topic of her mother's passing further." rating=0.75 uuid_='56488eeb-d8ac-4b2f-8acc-75f71b56ad76'
created_at='2024-10-07T20:04:28.397184Z' fact='Cathy is struggling to process the passing of her mother and has been feeling down and demotivated lately.' rating=0.75 uuid_='0fea3f05-ed1a-4e39-a092-c91f8af9e501'
created_at='2024-10-07T20:04:28.397184Z' fact='Cathy describes her mother as a kind and loving person.' rating=0.5 uuid_='131de203-2984-4cba-9aef-e500611f06d9'

Search over Facts in Zep’s long-term memory

In addition to the memory.get method which uses the current conversation to retrieve facts, we can also search Zep with our own keywords. Here, we retrieve facts using a query. Again, we use fact ratings to limit the returned facts to only those with a high poignancy rating.

The memory.search_sessions API may be used as an Agent tool, enabling an agent to search across user memory for relevant facts.

1 response = await zep.memory.search_sessions(
2     text="What do you know about Cathy's family?",
3     user_id=user_id,
4     search_scope="facts",
5     min_fact_rating=MIN_FACT_RATING,
6 )
7 
8 for r in response.results:
9     print(r.fact)

created_at='2024-10-07T20:04:28.397184Z' fact="Cathy wants to reflect on a previous conversation about her mother and explore the topic of her mother's passing further." rating=0.75 uuid_='56488eeb-d8ac-4b2f-8acc-75f71b56ad76'
created_at='2024-10-07T20:04:28.397184Z' fact='Cathy is struggling to process the passing of her mother and has been feeling down and demotivated lately.' rating=0.75 uuid_='0fea3f05-ed1a-4e39-a092-c91f8af9e501'
created_at='2024-10-07T20:04:28.397184Z' fact='Cathy describes her mother as a kind and loving person.' rating=0.5 uuid_='131de203-2984-4cba-9aef-e500611f06d9'

1	import os
2	from dotenv import load_dotenv
3	import uuid
4	from typing import Union, Dict
5	from autogen import ConversableAgent, Agent
6
7	load_dotenv()
8
9	config_list = [
10	{
11	"model": "gpt-4o-mini",
12	"api_key": os.environ.get("OPENAI_API_KEY"),
13	"max_tokens": 1024,
14	}
15	]

1	from zep_cloud.client import AsyncZep
2	from zep_cloud import Message, FactRatingExamples, FactRatingInstruction
3
4	MIN_FACT_RATING = 0.3
5
6	# Configure Zep
7	zep = AsyncZep(api_key=os.environ.get("ZEP_API_KEY"))

1	class ZepConversableAgent(ConversableAgent):
2	"""
3	A custom ConversableAgent that integrates with Zep for long-term memory.
4	"""
5	def __init__(
6	self,
7	name: str,
8	system_message: str,
9	llm_config: dict,
10	function_map: dict,
11	human_input_mode: str,
12	zep_session_id: str,
13	):
14	super().__init__(
15	name=name,
16	system_message=system_message,
17	llm_config=llm_config,
18	function_map=function_map,
19	human_input_mode=human_input_mode,
20	)
21	self.zep_session_id = zep_session_id
22	# store the original system message as we will update it with relevant facts from Zep
23	self.original_system_message = system_message
24	self.register_hook(
25	"a_process_last_received_message", self.persist_user_messages
26	)
27	self.register_hook(
28	"a_process_message_before_send", self.persist_assistant_messages
29	)
30
31	async def persist_assistant_messages(
32	self, sender: Agent, message: Union[Dict, str], recipient: Agent, silent: bool
33	):
34	"""Agent sends a message to the user. Add the message to Zep."""
35
36	# Assume message is a string
37	zep_messages = convert_to_zep_messages(
38	[{"role": "assistant", "name": self.name, "content": message}]
39	)
40	await zep.memory.add(session_id=self.zep_session_id, messages=zep_messages)
41
42	return message
43
44	async def persist_user_messages(self, messages: list[dict[str, str]] \| str):
45	"""
46	User sends a message to the agent. Add the message to Zep and
47	update the system message with relevant facts from Zep.
48	"""
49	# Assume messages is a string
50	zep_messages = convert_to_zep_messages([{"role": "user", "content": messages}])
51	await zep.memory.add(session_id=self.zep_session_id, messages=zep_messages)
52
53	memory = await zep.memory.get(self.zep_session_id, min_rating=MIN_FACT_RATING)
54
55	# Update the system message with the relevant facts retrieved from Zep
56	self.update_system_message(
57	self.original_system_message
58	+ f"\n\nRelevant facts about the user and their prior conversation:\n{memory.relevant_facts}"
59	)
60
61	return messages

1	bot_name = "CareBot"
2	user_name = "Cathy"
3
4	user_id = user_name + str(uuid.uuid4())[:4]
5	session_id = str(uuid.uuid4())
6
7
8	await zep.user.add(user_id=user_id)
9
10	fact_rating_instruction = """Rate the facts by poignancy. Highly poignant
11	facts have a significant emotional impact or relevance to the user.
12	Low poignant facts are minimally relevant or of little emotional
13	significance."""
14	fact_rating_examples = FactRatingExamples(
15	high="The user received news of a family member's serious illness.",
16	medium="The user completed a challenging marathon.",
17	low="The user bought a new brand of toothpaste.",
18	)
19
20	await zep.memory.add_session(
21	user_id=user_id,
22	session_id=session_id,
23	fact_rating_instruction=FactRatingInstruction(
24	instruction=fact_rating_instruction,
25	examples=fact_rating_examples,
26	),
27	)

1	chat_history = [
2	{
3	"role": "assistant",
4	"name": "carebot",
5	"content": "Hi Cathy, how are you doing today?",
6	},
7	{
8	"role": "user",
9	"name": "Cathy",
10	"content": "To be honest, I've been feeling a bit down and demotivated lately. It's been tough.",
11	},
12	{
13	"role": "assistant",
14	"name": "CareBot",
15	"content": "I'm sorry to hear that you're feeling down and demotivated, Cathy. It's understandable given the challenges you're facing. Can you tell me more about what's been going on?",
16	},
17	{
18	"role": "user",
19	"name": "Cathy",
20	"content": "Well, I'm really struggling to process the passing of my mother.",
21	},
22	{
23	"role": "assistant",
24	"name": "CareBot",
25	"content": "I'm deeply sorry for your loss, Cathy. Losing a parent is incredibly difficult. It's normal to struggle with grief, and there's no 'right' way to process it. Would you like to talk about your mother or how you're coping?",
26	},
27	{
28	"role": "user",
29	"name": "Cathy",
30	"content": "Yes, I'd like to talk about my mother. She was a kind and loving person.",
31	},
32	]
33
34	# Convert chat history to Zep messages
35	zep_messages = convert_to_zep_messages(chat_history)
36
37	await zep.memory.add(session_id=session_id, messages=zep_messages)

1	response = await zep.memory.get_session_facts(session_id=session_id, min_rating=MIN_FACT_RATING)
2
3	for r in response.facts:
4	print(r)

1	carebot_system_message = """
2	You are a compassionate mental health bot and caregiver. Review information about the user and their prior conversation below and respond accordingly.
3	Keep responses empathetic and supportive. And remember, always prioritize the user's well-being and mental health. Keep your responses very concise and to the point.
4	"""
5
6	agent = ZepConversableAgent(
7	bot_name,
8	system_message=carebot_system_message,
9	llm_config={"config_list": config_list},
10	function_map=None, # No registered functions, by default it is None.
11	human_input_mode="NEVER", # Never ask for human input.
12	zep_session_id=session_id,
13	)

1	cathy = ConversableAgent(
2	user_name,
3	system_message="You are a helpful mental health bot. You are seeking counsel from a mental health bot. Ask the bot about your previous conversation.",
4	llm_config={"config_list": config_list},
5	human_input_mode="NEVER", # Never ask for human input.
6	)

1	result = await agent.a_initiate_chat(
2	cathy,
3	message="Hi Cathy, nice to see you again. How are you doing today?",
4	max_turns=3,
5	)

1	response = await zep.memory.search_sessions(
2	text="What do you know about Cathy's family?",
3	user_id=user_id,
4	search_scope="facts",
5	min_fact_rating=MIN_FACT_RATING,
6	)
7
8	for r in response.results:
9	print(r.fact)