This directory contains a subset of the multi-agent environments as described in the paper FACMAC: Factored Multi-Agent Centralised Policy Gradients. The task descriptions are the same as in the implementation from Gymnasium-Robotics. These are multi-agent factorisations of MuJoCo tasks such that each agent controls a subset of the joints and only observes the local state.

Specifically, we include the following environments:

Each agent's observation vector is composed of the local state of the joints it controls, as well as the state of joints at distance 1 away in the body graph, and the state of the root body. State here refers to the position and velocity of the joint or body. All observations are continuous numbers in the range [-inf, inf].

Each agent's action space is the input torques to the joints it controls. All environments have continuous actions in the range [-1.0, 1.0], except for humanoid_9|8 where the range is [-0.4, 0.4].

Note: the two agents in humanoid_9|8 have different action space sizes. To pad the action spaces to be the same size pass "homogenisation_method":"max" to the envrionment. If using our config files, this would done as:

"ENV_NAME": "humanoid_9|8" 
"ENV_KWARGS": {"homogenisation_method":"max"}

To visualise a trajectory in a Jupyter notebook, given a list of states, you can use the following code snippet:

from IPython.display import HTML
from brax.io import html

HTML(html.render(env.sys, [s.qp for s in state_history]))

A notable difference to Gymansium-Robotics is that this JAX implementation currently fixes the observation distance to 1, whereas in the original implementation, it is a configurable parameter. This means that each agent has access to the observations of joints at distance 1 away from it in the body graph. We plan to make this a configurable parameter in a future update.