Close #59

This is the start of an ongoing effort to make sotopia dependency slimmer.

Similar to tinygrad's point, having less dependencies will make more people easy to use our library with other tools that they are using (no need to resolve conflicts that don't actually exist).

I think the only 8 dependencies that we should have are:

1. pydantic: for data validation and abstraction,
2. redis-om: for data serialization, logging, and communication,
3. tqdm: for fun,
4. aiohttp: for calling RESTAPI,
5. beartype: for type guarding,
6. lxml: for handling different visibilities
7. jinja2: for prompt templates
8. and gin: for configuration, and we should remove it in the end as well.

Options:

[example]:

• pandas
• transformers
• datasets
• scipy
• torch

[litellm]:

• litellm

[chat]:

• fastapi

[testing]:

• mypy
• pytest
• pytest-asyncio

[dev]:

• ruff
• pre-commit

📑 Description

In this PR, I made progress towards this goal --

This is the current required dependency list

pandas = "^2.1.1"
lxml = "^4.9.3"
openai = "^1.11.0"
langchain = "0.1.5"
rich = "^13.6.0"
PettingZoo = "1.24.0"
redis-om = "^0.2.1"
pandas-stubs = ""
types-tqdm = ""
gin-config = "^0.5.0"
absl-py = "^2.0.0"
together = "^0.2.4"
pydantic = "1.10.12"
beartype = "^0.14.0"
langchain-openai = "^0.0.5"
litellm = "^1.23.12"

We now have 14 required dependencies (except for stubs), and removing the other 6 is hard for the following reasons.

1. pandas is only used for converting a dict into a csv. This should be a feature that we can easily implement.
2. openai and together are our "default" LLMs which we could do without, but requires more changes.
3. rich for printing, which we could conditionally use.
4. PettingZoo is our initial abstraction for environments, which we have in fact deviated from. We could do without it in a small PR.
5. litellm could be optional in a following PR

Here are our optional dependencies:

fastapi = { version = "^0.109.2", optional = true }
tabulate = { version = "^0.9.0", optional = true }
anthropic = { version = "^0.26.0", optional = true }
xmltodict = { version = "^0.13.0", optional = true }
groq = { version = "^0.4.2", optional = true }
cohere = { version = "^5.1.8", optional = true }
google-generativeai = { version = "^0.5.4", optional = true }
transformers = { version = "^4.41.0", optional = true }
datasets = { version = "^2.19.0", optional = true }
scipy = { version = "^1.13.1", optional = true }
torch = { version = "^2.3.0", optional = true }

✅ Checks

• My pull request adheres to the code style of this project
• My code requires changes to the documentation
• I have updated the documentation as required
• All the tests have passed
• Branch name follows type/descript (e.g. feature/add-llm-agents)
• Ready for code review

ℹ Additional Information