The built-in VecEnvs only support launching multiple instances of the same environment. You have to modify the Gazebo environment to include multiple robots to gather data, and then implement a VecEnv wrapper that correctly handles this data (pretends to be multiple environments, but in reality just passes on the data from a single environment with multiple agents).

Unity ML Agents repository could give some tips on how to do the former, as they "parallelize" learning in Unity environment in a similar fashion.

Thank you so much for your quick response. I will look into that.

Is this what you want?

# multiprocess environment
n_cpu = 4
env = SubprocVecEnv([lambda: gym.make('CartPole-v1') for i in range(n_cpu)])

No @ChengYen-Tang , I have a Gazebo environment, and I have step, get_obs, reset etc. functions defined for a single agent in the environment. I tried training a single agent with PPO, but the training process is too slow, I need to run multiple agents to speed up the training. Now I cannot launch multiple Gazebo instances as Gazebo needs a good amount of memory, so I want to launch multiple robot instances in a single Gazebo environment. I am not exactly sure how I can do that. Do I need to define the methods of VecEnv class? If yes it would be great if you can point me to some example code if someone has tried to do the same. I am going through Unity ML Agents but still it is not clear. Thanks for looking into the issue.

Take in consideration that, depending of the robot you're using, the simulation in Gazebo could become a bottleneck. In other words, when multiple robots are part of the simulation, the RTF of Gazebo could drop significantly since after all the gzserver is a single-process application. Simple robots are could be handled easily, instead robot with complex collisions primitives could become very expensive to compute.

Thank you @diegoferigo I will consider that.

@utsavpatel22 Hey there, have you managed to define this kind of vectorized environment with multiple robots in a single gazebo environment? If yes, how does it work? Can you share some code, or tips please? I am trying to do the same thing.
Thanks a lot in advance 🙏