This repo is the official PyTorch implementation for paper: Enhancing 3D Lane Detection and Topology Reasoning with 2D Lane Priors.

Our framework, named Topo2D, enhances both lane detection and topology reasoning capabilities by integrating 2D lane priors. The 2D and 3D lane detectors are with similar transformer-based architectures. For 3D lane detection, we use the 2D lane query features and 2D coordinates obtained by the 2D lane decoder to initialize the 3D lane queries and positional embeddings. For topology prediction, we utilize a comprehensive approach that not only involves the features from 3D lanes and traffic elements, but also integrates corresponding 2D lane features, thereby enhancing overall performance. We validate our Topo2D on the multi-view topology reasoning benchmark OpenLane-V2 and the single-view 3D lane detection benchmark OpenLane. Topo2D achieves state-of-the-art performance on both benchmarks.

• News
• Installation
• Prepare Dataset
• Train and Evaluate
• Main Results
• Citation
• Acknowledgement

• [2024/07/16] The code and models are released.
• [2024/06/05] The paper is released on arXiv.

conda create -n topo2d python=3.8
conda activate topo2d
pip install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio==0.10.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html

pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
pip install mmdet==2.14.0
pip install mmsegmentation==0.14.1

cd mmdetection3d
pip install -v -e .

git clone https://github.com/Dao-AILab/flash-attention.git
git checkout v0.2.8
python setup.py install

pip install -r requirements.txt

mkdir ckpts
cd ckpts 
wget https://download.pytorch.org/models/resnet50-19c8e357.pth

For the Openlane-V2 dataset, follow getting_started.md to download the data and install the environment, then run the following code to preprocess the data.

python tools/preprocess_openlanev2.py

The data folders are organized as follows:

data
├── OpenLane-V2
│   ├── data_dict_subset_A.json
│   ├── data_dict_subset_A_test.pkl
│   ├── data_dict_subset_A_train.pkl
│   ├── data_dict_subset_A_val.pkl
│   ├── test
│   ├── train
│   └── val

Train Topo2D with 4 GPUs:

bash tools/dist_train.sh projects/configs/topo2d/openlanev2.py 4

Evaluate Topo2D with 4 GPUs:

bash tools/dist_test.sh projects/configs/topo2d/openlanev2.py openlanev2_exp6_10_24e.pth 4 --eval openlanev2

[config] [ckpt]

If you find this repo useful for your research, please consider citing it using the following BibTeX entry.

@misc{li2024enhancing,
      title={Enhancing 3D Lane Detection and Topology Reasoning with 2D Lane Priors}, 
      author={Han Li and Zehao Huang and Zitian Wang and Wenge Rong and Naiyan Wang and Si Liu},
      year={2024},
      eprint={2406.03105},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

We thank the authors that open the following projects.

• MMDetection3D
• MapTR
• PETR
• OpenLane-v2
• TopoNet