AIOS is the AI Agent Operating System, which embeds large language model (LLM) into the operating system and facilitates the development and deployment of LLM-based AI Agents. AIOS is designed to address problems (e.g., scheduling, context switch, memory management, storage management, tool management, Agent SDK management, etc.) during the development and deployment of LLM-based agents, towards a better AIOS-Agent ecosystem for agent developers and agent users. AIOS includes the AIOS Kernel (the AIOS repository) and the AIOS SDK (this Cerebrum repository). AIOS supports both Web UI and Terminal UI.
The AIOS-Agent SDK is designed for agent users and developers, enabling them to build and run agent applications by interacting with the AIOS kernel.
- [2024-11-26] π₯ Cerebrum is available for public release on PyPI!
-
Clone Repo
git clone https://github.com/agiresearch/Cerebrum.git cd Cerebrum
-
Create Virtual Environment
conda create -n cerebrum-env python=3.10
or
conda create -n cerebrum-env python=3.11
or
# Windows (cmd) python -m venv cerebrum-env # Linux/MacOS python3 -m venv cerebrum-env
-
Activate the environment
conda activate myenv
or
# Windows (cmd) cd cerebrum-env cd Scripts activate.bat cd .. cd .. # Linux/MacOS source cerebrum-env/bin/activate
-
Install the package
pip install -e .
-
Verify installation
python -c "import cerebrum; from cerebrum.client import Cerebrum; print(Cerebrum)"
Tip
Please see our documentation for more information.
-
Start the AIOS Kernel π See here.
-
Run the AIOS Client
Run an agent using the client
run-agent --llm_name gpt-4o-mini --llm_backend openai --agent_name_or_path <agent name or agent path> --task <task that agent needs to complete>
For the agent_name_or_path argument, you can either pass the agent names (in the format of author_name/agent_name) that are available on the agenthub, or pass the absolute path for the agent folder in your computer.
For example, you can run a demo agent avaiable on the agenthub using the following command:
run-agent --llm_name gpt-4o-mini --llm_backend openai --agent_name_or_path demo_author/demo_agent --task "Tell me what is core idea of AIOS" --aios_kernel_url http://localhost:8000
or you can run the demo agent by passing the absolute path of the agent folder in your computer
run-agent --llm_name gpt-4o-mini --llm_backend openai --agent_name_or_path <your_local_folder_of_cerebrum>/cerebrum/example/agents/demo_agent --task "Tell me what is core idea of AIOS" --aios_kernel_url "http://localhost:8000"
Code file is located at
cerebrum/example/run_agent.py
Let's walk through how to set up and customize your client to work with the AIOS kernel. We'll break this down into simple steps.
First, let's create your client instance:
from cerebrum import config
from cerebrum.client import Cerebrum
aios_kernel_url = "http://localhost:8000"
client = Cerebrum(base_url = aios_kernel_url)
config.global_client = client
The AIOS kernel offers five core modules you can customize:
- LLM (Language Model)
- Memory
- Storage
- Tools
- Scheduler
Here's how to add these layers to your client:
from cerebrum.llm.layer import LLMLayer
from cerebrum.memory.layer import MemoryLayer
from cerebrum.overrides.layer import OverridesLayer
from cerebrum.storage.layer import StorageLayer
from cerebrum.tool.layer import ToolLayer
client.add_llm_layer(
LLMLayer(llm_name="gpt-4o-mini", llm_backend="openai") # Configure your LLM
).add_storage_layer(
StorageLayer(root_dir="root") # Set storage directory
).add_memory_layer(
MemoryLayer(memory_limit=104857600) # Set memory per agent
).add_tool_layer(
ToolLayer() # Add tool capabilities
).override_scheduler(
OverridesLayer(max_workers=32) # Configure scheduling
)
Now you can run agents and get their results:
try:
# Connect to the client
client.connect()
# Execute your agent
agent_path = "demo_author/demo_agent" # Your agent's name or path
task = "Tell me what is core idea of AIOS" # Your task description
result = client.execute(agent_path, {"task": task})
# Get the results
final_result = client.poll_agent(
result["execution_id"],
timeout=300
)
print("π Task result:", final_result)
print("β
Task completed")
except TimeoutError:
print("β Task timed out")
except Exception as e:
print(f"β Failed to execute task: {str(e)}")
finally:
client.cleanup()
You can find all these agents in the example agents folder. If you would like to customize and develop your new agents, you can check out the guides on Developing New Agents and Developing New Tools.
This guide will walk you through creating and publishing your own agents for AIOS.
First, let's look at how to organize your agent's files. Every agent needs three essential components:
author/
βββ agent_name/
βββ entry.py # Your agent's main logic
βββ config.json # Configuration and metadata
βββ meta_requirements.txt # Additional dependencies
For example, if your name is 'example' and you're building a demo_agent that searches and summarizes articles, your folder structure would look like this:
example/
βββ demo_agent/
βββ entry.py
βββ config.json
βββ meta_requirements.txt
Note: If your agent needs any libraries beyond AIOS's built-in ones, make sure to list them in meta_requirements.txt. Apart from the above three files, you can have any other files in your folder.
Your agent needs a config.json file that describes its functionality. Here's what it should include:
{
"name": "demo_agent",
"description": [
"Demo agent that can help search AIOS-related papers"
],
"tools": [
"demo_author/arxiv"
],
"meta": {
"author": "demo_author",
"version": "0.0.1",
"license": "CC0"
},
"build": {
"entry": "agent.py",
"module": "DemoAgent"
}
}
When setting up your agent, you'll need to specify which tools it will use. Below is a list of all currently available tools and how to reference them in your configuration:
Author | Name | How to call them |
---|---|---|
example | arxiv | example/arxiv |
example | bing_search | example/bing_search |
example | currency_converter | example/currency_converter |
example | wolfram_alpha | example/wolfram_alpha |
example | google_search | example/google_search |
openai | speech_to_text | openai/speech_to_text |
example | web_browser | example/web_browser |
timbrooks | image_to_image | timbrooks/image_to_image |
example | downloader | example/downloader |
example | doc_question_answering | example/doc_question_answering |
stability-ai | text_to_image | stability-ai/text_to_image |
example | text_to_speech | example/text_to_speech |
To use these tools in your agent, simply include their reference (from the "How to Use" column) in your agent's configuration file. For example, if you want your agent to be able to search academic papers and convert currencies, you would include both example/arxiv
and example/currency_converter
in your configuration.
If you would like to create your new tools, you can either integrate the tool within your agent code or you can follow the tool examples in the tool folder to develop your standalone tools. The detailed instructions are in How to develop new tools
Let's walk through creating your agent's core functionality.
First, create your agent class by inheriting from BaseAgent:
from cerebrum.agents.base import BaseAgent
from cerebrum.llm.communication import LLMQuery
import json
class DemoAgent(BaseAgent):
def __init__(self, agent_name, task_input, config_):
super().__init__(agent_name, task_input, config_)
self.plan_max_fail_times = 3
self.tool_call_max_fail_times = 3
self.start_time = None
self.end_time = None
self.request_waiting_times: list = []
self.request_turnaround_times: list = []
self.task_input = task_input
self.messages = []
self.workflow_mode = "manual" # (manual, automatic)
self.rounds = 0
AIOS provides several Query
classes for different types of interactions and use the Response
class in here to receive results from the AIOS kernel.
Query Class | Arguments | Output |
---|---|---|
LLMQuery |
messages: List , tools: List , action_type: str , message_return_type: str |
response: Response |
MemoryQuery |
TBD | response: Response |
StorageQuery |
TBD | response: Response |
ToolQuery |
tool_calls: List |
response: Response |
Here's how to import a specific query
from cerebrum.llm.communication import LLMQuery # Using LLMQuery as an example
Here's how to set up your agent's system instructions and you need to put this function inside your agent class
def build_system_instruction(self):
prefix = "".join(["".join(self.config["description"])])
plan_instruction = "".join(
[
f"You are given the available tools from the tool list: {json.dumps(self.tool_info)} to help you solve problems. ",
"Generate a plan with comprehensive yet minimal steps to fulfill the task. ",
"The plan must follow the json format as below: ",
"[",
'{"action_type": "action_type_value", "action": "action_value","tool_use": [tool_name1, tool_name2,...]}',
'{"action_type": "action_type_value", "action": "action_value", "tool_use": [tool_name1, tool_name2,...]}',
"...",
"]",
"In each step of the planned plan, identify tools to use and recognize no tool is necessary. ",
"Followings are some plan examples. ",
"[" "[",
'{"action_type": "tool_use", "action": "gather information from arxiv. ", "tool_use": ["arxiv"]},',
'{"action_type": "chat", "action": "write a summarization based on the gathered information. ", "tool_use": []}',
"];",
"[",
'{"action_type": "tool_use", "action": "gather information from arxiv. ", "tool_use": ["arxiv"]},',
'{"action_type": "chat", "action": "understand the current methods and propose ideas that can improve ", "tool_use": []}',
"]",
"]",
]
)
if self.workflow_mode == "manual":
self.messages.append({"role": "system", "content": prefix})
else:
assert self.workflow_mode == "automatic"
self.messages.append({"role": "system", "content": prefix})
self.messages.append({"role": "user", "content": plan_instruction})
You can create a workflow for the agent to execute its task and you need to put this function inside your agent class.
Manual workflow example:
def manual_workflow(self):
workflow = [
{
"action_type": "tool_use",
"action": "Search for relevant papers",
"tool_use": ["demo_author/arxiv"],
},
{
"action_type": "chat",
"action": "Provide responses based on the user's query",
"tool_use": [],
},
]
return workflow
Finally, implement the run method to execute your agent's workflow and you need to put this function inside your agent class.
def run(self):
self.build_system_instruction()
task_input = self.task_input
self.messages.append({"role": "user", "content": task_input})
workflow = None
if self.workflow_mode == "automatic":
workflow = self.automatic_workflow()
self.messages = self.messages[:1] # clear long context
else:
assert self.workflow_mode == "manual"
workflow = self.manual_workflow()
self.messages.append(
{
"role": "user",
"content": f"[Thinking]: The workflow generated for the problem is {json.dumps(workflow)}. Follow the workflow to solve the problem step by step. ",
}
)
try:
if workflow:
final_result = ""
for i, step in enumerate(workflow):
action_type = step["action_type"]
action = step["action"]
tool_use = step["tool_use"]
prompt = f"At step {i + 1}, you need to: {action}. "
self.messages.append({"role": "user", "content": prompt})
if tool_use:
selected_tools = self.pre_select_tools(tool_use)
else:
selected_tools = None
response = self.send_request(
agent_name=self.agent_name,
query=LLMQuery(
messages=self.messages,
tools=selected_tools,
action_type=action_type,
),
)["response"]
self.messages.append({"role": "assistant", "content": response.response_message})
self.rounds += 1
final_result = self.messages[-1]["content"]
return {
"agent_name": self.agent_name,
"result": final_result,
"rounds": self.rounds,
}
else:
return {
"agent_name": self.agent_name,
"result": "Failed to generate a valid workflow in the given times.",
"rounds": self.rounds,
}
except Exception as e:
return {}
To test your agent, use the run_agent command to run:
run-agent --llm_name <llm_name> --llm_backend <llm_backend> --agent_name_or_path <agent_name_or_path> --task <task_input> --aios_kernel_url <aios_kernel_url>
Replace the placeholders with your specific values:
<llm_name>
: The name of the language model you want to use<llm_backend>
: The backend service for the language model<your_agent_folder_path>
: The path to your agent's folder<task_input>
: The task you want your agent to complete<aios_kernel_url>
: The url that is connected to the aios kernel
or you can run the agent using the source code in the cerebrum/example/run_agent
python cerebrum/example/run_agent --llm_name <llm_name> --llm_backend <llm_backend> --agent_name_or_path <agent_name_or_path>> --task <task_input> --aios_kernel_url <aios_kernel_url>
Similar as developing new agents, developing tools also need to follow a simple directory structure:
demo_author/
βββ demo_tool/
βββ entry.py # Contains your tool's main logic
βββ config.json # Tool configuration and metadata
Your tool needs a configuration file that describes its properties. Here's an example of how to set it up:
{
"name": "arxiv",
"description": [
"The arxiv tool that can be used to search for papers on arxiv"
],
"meta": {
"author": "demo_author",
"version": "1.0.6",
"license": "CC0"
},
"build": {
"entry": "tool.py",
"module": "Arxiv"
}
}
In entry.py
, you'll need to implement a tool class which is identified in the config.json with two essential methods:
get_tool_call_format
: Defines how LLMs should interact with your toolrun
: Contains your tool's main functionality
Here's an example:
class Arxiv:
def get_tool_call_format(self):
tool_call_format = {
"type": "function",
"function": {
"name": "demo_author/arxiv",
"description": "Query articles or topics in arxiv",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Input query that describes what to search in arxiv"
}
},
"required": [
"query"
]
}
}
}
return tool_call_format
def run(self, params: dict):
"""
Main tool logic goes here.
Args:
params: Dictionary containing tool parameters
Returns:
Your tool's output
"""
# Your code here
result = do_something(params['param_name'])
return result
When integrating your tool for the agents you develop:
- Use absolute paths to reference your tool in agent configurations
- Example:
/path/to/your/tools/example/your_tool
instead of justauthor/tool_name
Provider π’ | Model Name π€ | Open Source π | Model String β¨οΈ | Backend βοΈ | Required API Key |
---|---|---|---|---|---|
Anthropic | Claude 3.5 Sonnet | β | claude-3-5-sonnet-20241022 | anthropic | ANTHROPIC_API_KEY |
Anthropic | Claude 3.5 Haiku | β | claude-3-5-haiku-20241022 | anthropic | ANTHROPIC_API_KEY |
Anthropic | Claude 3 Opus | β | claude-3-opus-20240229 | anthropic | ANTHROPIC_API_KEY |
Anthropic | Claude 3 Sonnet | β | claude-3-sonnet-20240229 | anthropic | ANTHROPIC_API_KEY |
Anthropic | Claude 3 Haiku | β | claude-3-haiku-20240307 | anthropic | ANTHROPIC_API_KEY |
OpenAI | GPT-4 | β | gpt-4 | openai | OPENAI_API_KEY |
OpenAI | GPT-4 Turbo | β | gpt-4-turbo | openai | OPENAI_API_KEY |
OpenAI | GPT-4o | β | gpt-4o | openai | OPENAI_API_KEY |
OpenAI | GPT-4o mini | β | gpt-4o-mini | openai | OPENAI_API_KEY |
OpenAI | GPT-3.5 Turbo | β | gpt-3.5-turbo | openai | OPENAI_API_KEY |
Gemini 1.5 Flash | β | gemini-1.5-flash | GEMINI_API_KEY | ||
Gemini 1.5 Flash-8B | β | gemini-1.5-flash-8b | GEMINI_API_KEY | ||
Gemini 1.5 Pro | β | gemini-1.5-pro | GEMINI_API_KEY | ||
Gemini 1.0 Pro | β | gemini-1.0-pro | GEMINI_API_KEY | ||
Groq | Llama 3.2 90B Vision | β | llama-3.2-90b-vision-preview | groq | GROQ_API_KEY |
Groq | Llama 3.2 11B Vision | β | llama-3.2-11b-vision-preview | groq | GROQ_API_KEY |
Groq | Llama 3.1 70B | β | llama-3.1-70b-versatile | groq | GROQ_API_KEY |
Groq | Llama Guard 3 8B | β | llama-guard-3-8b | groq | GROQ_API_KEY |
Groq | Llama 3 70B | β | llama3-70b-8192 | groq | GROQ_API_KEY |
Groq | Llama 3 8B | β | llama3-8b-8192 | groq | GROQ_API_KEY |
Groq | Mixtral 8x7B | β | mixtral-8x7b-32768 | groq | GROQ_API_KEY |
Groq | Gemma 7B | β | gemma-7b-it | groq | GROQ_API_KEY |
Groq | Gemma 2B | β | gemma2-9b-it | groq | GROQ_API_KEY |
Groq | Llama3 Groq 70B | β | llama3-groq-70b-8192-tool-use-preview | groq | GROQ_API_KEY |
Groq | Llama3 Groq 8B | β | llama3-groq-8b-8192-tool-use-preview | groq | GROQ_API_KEY |
ollama | All Models | β | model-name | ollama | - |
vLLM | All Models | β | model-name | vllm | - |
HuggingFace | All Models | β | model-name | huggingface | HF_HOME |
@article{mei2024aios,
title={AIOS: LLM Agent Operating System},
author={Mei, Kai and Li, Zelong and Xu, Shuyuan and Ye, Ruosong and Ge, Yingqiang and Zhang, Yongfeng}
journal={arXiv:2403.16971},
year={2024}
}
@article{ge2023llm,
title={LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem},
author={Ge, Yingqiang and Ren, Yujie and Hua, Wenyue and Xu, Shuyuan and Tan, Juntao and Zhang, Yongfeng},
journal={arXiv:2312.03815},
year={2023}
}
For how to contribute, see CONTRIBUTE. If you would like to contribute to the codebase, issues or pull requests are always welcome!
If you would like to join the community, ask questions, chat with fellows, learn about or propose new features, and participate in future developments, join our Discord Community!
For issues related to Cerebrum development, we encourage submitting issues, pull requests, or initiating discussions in AIOS Discord Channel. For other issues please feel free to contact the AIOS Foundation (contact@aios.foundation).