Deploy the api to modal (#267)

* prototype for modal serving * add openai secret * fix type annotation * add doc * bug fix for simulation api * add customize model, evaluator model and evaluation dimensions * Implement modal API server with Redis integration and FastAPI setup - Added a new script for the modal API server that initializes a Redis instance. - Created a persistent volume for Redis data and included a function to download initial data if not present. - Configured a Docker image with necessary dependencies including Redis Stack and FastAPI. - Implemented a web API class that sets up and cleans up the Redis connection, ensuring readiness before serving requests. - Integrated the SotopiaFastAPI application within the modal framework. --------- Co-authored-by: XuhuiZhou <zhouxuhui2018@gmail.com>
sotopia-lab · JXZhou0224 · Jan 18, 2025 · Dec 5, 2024 · Dec 8, 2024 · Dec 11, 2024
commit 1f4fb0ab38a472a3c09d752b3211be641117c608
diff --git a/docs/pages/examples/deployment.md b/docs/pages/examples/deployment.md
@@ -0,0 +1,6 @@
+# Deploy Sotopia Python API to Modal
+We offer a script to deploy Sotopia Python API to [Modal](https://modal.com/).
+To do so, simply go to the `sotopia/sotopia/ui` directory and run the following command:
+```bash
+modal deploy sotopia/ui/modal_api_server.py
+```
diff --git a/scripts/modal/modal_api_server.py b/scripts/modal/modal_api_server.py
@@ -0,0 +1,102 @@
+import modal
+import subprocess
+import time
+import os
+
+import redis
+from sotopia.ui.fastapi_server import SotopiaFastAPI
+
+# Create persistent volume for Redis data
+redis_volume = modal.Volume.from_name("sotopia-api", create_if_missing=True)
+
+
+def initialize_redis_data() -> None:
+    """Download Redis data if it doesn't exist"""
+    if not os.path.exists("/vol/redis/dump.rdb"):
+        os.makedirs("/vol/redis", exist_ok=True)
+        print("Downloading initial Redis data...")
+        subprocess.run(
+            "curl -L https://cmu.box.com/shared/static/e3vd31r7916jb70j9cgtcq9etryrxml0.rdb --output /vol/redis/dump.rdb",
+            shell=True,
+            check=True,
+        )
+        print("Redis data downloaded")
+
+
+# Create image with all necessary dependencies
+image = (
+    modal.Image.debian_slim(python_version="3.11")
+    .apt_install(
+        "git",
+        "curl",
+        "gpg",
+        "lsb-release",
+        "wget",
+        "procps",  # for ps command
+        "redis-tools",  # for redis-cli
+    )
+    .run_commands(
+        # Update and install basic dependencies
+        "apt-get update",
+        "apt-get install -y curl gpg lsb-release",
+        # Add Redis Stack repository
+        "curl -fsSL https://packages.redis.io/gpg | gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg",
+        "chmod 644 /usr/share/keyrings/redis-archive-keyring.gpg",
+        'echo "deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main" | tee /etc/apt/sources.list.d/redis.list',
+        "apt-get update",
+        "apt-get install -y redis-stack-server",
+    )
+    .pip_install(
+        "sotopia>=0.1.2",
+        "fastapi>0.100",  # TODO: remove this dependency after pypi release
+        "uvicorn",  # TODO: remove this dependency after pypi release
+    )
+)
+redis_volume = modal.Volume.from_name("sotopia-api", create_if_missing=True)
+
+# Create stub for the application
+app = modal.App("sotopia-fastapi", image=image, volumes={"/vol/redis": redis_volume})
+
+
+@app.cls(
+    image=image,
+    concurrency_limit=1,
+    allow_concurrent_inputs=5,
+    secrets=[modal.Secret.from_name("openai-secret")],
+)
+class WebAPI:
+    def __init__(self) -> None:
+        self.web_app = SotopiaFastAPI()
+
+    @modal.enter()
+    def setup(self) -> None:
+        # Start Redis server
+        subprocess.Popen(
+            ["redis-stack-server", "--dir", "/vol/redis", "--port", "6379"]
+        )
+
+        # Wait for Redis to be ready
+        max_retries = 30
+        for _ in range(max_retries):
+            try:
+                initialize_redis_data()
+                # Attempt to create Redis client and ping the server
+                temp_client = redis.Redis(host="localhost", port=6379, db=0)
+                temp_client.ping()
+                self.redis_client = temp_client
+                print("Successfully connected to Redis")
+                return
+            except (redis.exceptions.ConnectionError, redis.exceptions.ResponseError):
+                print("Waiting for Redis to be ready...")
+                time.sleep(1)
+
+        raise Exception("Could not connect to Redis after multiple attempts")
+
+    @modal.exit()
+    def cleanup(self) -> None:
+        if hasattr(self, "redis_client"):
+            self.redis_client.close()
+
+    @modal.asgi_app()
+    def serve(self) -> SotopiaFastAPI:
+        return self.web_app
diff --git a/sotopia/ui/README.md b/sotopia/ui/README.md
@@ -155,7 +155,7 @@ returns:
 **WSMessageType**
 | Type | Direction   | Description |
 |-----------|--------|-------------|
-| SERVER_MSG | Server → Client | Standard message from server (payload: `messageForRendering` [here](https://github.com/sotopia-lab/sotopia-demo/blob/main/socialstream/rendering_utils.py) ) |
+| SERVER_MSG | Server → Client | Standard message from server (payload: `EpisodeLog`) |
 | CLIENT_MSG | Client → Server | Standard message from client (payload: TBD) |
 | ERROR      | Server → Client | Error notification (payload: TBD) |
 | START_SIM  | Client → Server | Initialize simulation (payload: `SimulationEpisodeInitialization`) |
@@ -179,6 +179,13 @@ returns:
 
 **Implementation plan**: Currently only support LLM-LLM simulation based on [this function](https://github.com/sotopia-lab/sotopia/blob/19d39e068c3bca9246fc366e5759414f62284f93/sotopia/server.py#L108).
 
+**SERVER_MSG payload**
+The server message is a dictionary that has the following keys:
+- type: str, indicates the type of the message, typically it is "messages"
+- messages: Any. Typically this is the dictionary of the `EpisodeLog` for the current simulation state. (Which means the reward part could be empty)
+
+
+
 
 ## An example to run simulation with the API