git clone git@github.com:MAC-VO/MAC-VO.git --recursive

• [Nov 2024] We released the ROS-2 integration at https://github.com/MAC-VO/MAC-VO-ROS2 along with the documentation at https://mac-vo.github.io/wiki/ROS/

Docker Image See /Docker/DockerfileRoot to build the container.

Virtual Environment, Need Python 3.10+. See requirements.txt for environment requirements. Only tested on PyTorch >=2.0 and NumPy < 2.0.0. Breaking Change: must have PyPose >= 0.6.8

• The pre-trained model for MACVO, stereo TartanVO, and DPVO is in our release

• Download a demo sequence through drive and the pre-trained model for front-end model and posenet model

• Remember to change the path in the config file

• Run MAC-VO on a single sequence

  $ python MACVO.py --odom [PATH_TO_ODOM_CONFIG] --data [PATH_TO_DATA_CONFIG] --useRR

  see python MACVO.py --help for more flags and configurations.

  Example Usages (See More)

  • Run MAC-VO (Ours method):

    $ python MACVO.py --odom ./Config/Experiment/MACVO/MACVO.yaml --data ./Config/Sequence/TartanAir_abandonfac_001.yaml

  • Run MAC-VO for ablation studies

    $ python MACVO.py --odom ./Config/Experiment/MACVO/Ablation_Study/[CHOOSE_ONE_CFG].yaml --data ./Config/Sequence/TartanAir_abandonfac_001.yaml --useRR

• Run MAC-VO on Test Dataset

  $ python -m Scripts.Experiment.Experiment_MACVO --odom [PATH_TO_ODOM_CONFIG]

• Run MAC-VO Mapping Mode

  Mapping mode only reprojects pixels to 3D space and does not optimize the pose. To run the mapping mode, you need to first run a trajectory through the original mode (MAC-VO), and pass the resulting pose file to MAC-VO mapping mode by modifying the config. (Specifically, motion > args > pose_file in config file)

  $ python MACVO.py --odom ./Config/Experiment/MACVO/MACVO_MappingMode.yaml --data ./Config/Sequence/TartanAir_abandonfac_001.yaml

Every run will produce a Sandbox (or Space). A Sandbox is a storage unit that contains all the results and meta-information of an experiment. The evaluation and plotting script usually requires one or more paths of sandbox(es).

• Evaluate Sequence(s)

  Calculate the absolute translate error (ATE, m); relative translation error (RTE, m/frame); relative orientation error (ROE, deg/frame); relative pose error (per frame on se(3)).

  $ python -m Evaluation.EvalSeq --spaces SPACE_0, [SPACE, ...]

• Plot Sequence(s)

  Plot sequences, translation, translation error, rotation and rotation error.

  $ python -m Evaluation.PlotSeq --spaces SPACE_0, [SPACE, ...]

We also integrated two baseline methods (DPVO, TartanVO Stereo) into the codebase for evaluation, visualization and comparison.

• PyTorch Tensor Data - All images are stored in BxCxHxW format following the convention. Batch dimension is always the first dimension of tensor.

• Pixels on Camera Plane - All pixel coordinates are stored in uv format following the OpenCV convention, where the direction of uv are "east-down". Note that this requires us to access PyTorch tensor in data[..., v, u] indexing.

  (0, 0)----> u
    |
    |
    v
  

• World Coordinate - NED convention, +x -> North, +y -> East, +z -> Down with the first frame being world origin having identity SE3 pose.

This codebase is designed with modularization in mind so it's easy to modify, replace, and re-configure modules of MAC-VO. One can easily use or replase the provided modules like flow estimator, depth estimator, keypoint selector, etc. to create a new visual odometry.

🤗 We welcome everyone to extend and redevelop the MAC-VO. For documentation please visit the Documentation Site