Motion forecasting is an important component in autonomous driving systems. One of the most challenging problems in motion forecasting is interactive trajectory prediction, whose goal is to jointly forecasts the future trajectories of interacting agents. To this end, we present a large-scale interactive trajectory prediction dataset named INT2 for INTeractive trajectory prediction at INTersections. INT2 includes 612,000 scenes, each lasting 1 minute, containing up to 10,200 hours of data. The agent trajectories are auto-labeled by a high-performance offline temporal detection and fusion algorithm, whose quality is further inspected by human judges. Vectorized semantic maps and traffic light information are also included in INT2. Additionally, the dataset poses an interesting domain mismatch challenge. For each intersection, we treat rush-hour and non-rush-hour segments as different domains. We benchmark the best open-sourced interactive trajectory prediction method on INT2 and Waymo Open Motion, under in-domain and cross-domain settings.

Additionally, the dataset poses an interesting domain mismatch challenge. For each intersection, we treat rush-hour and non-rush-hour segments as different domains. Peak time (a, b, c) and trough time (d, e, f).

[coming soon]: INT2 Motion Prediciton Challenge 2023 and INT2 Interactive Motion Prediction Challenge 2023 in this challenges page.

[2023-8-9]: The INT2 Dataset, Benchmark, Visulization toolbox and Interaction filter toolbox are released in this code-base page.

[2023-8-8]: The INT2 Dataset Website are open in this website page.

• Installation
• Data structure
• Downloading

We propose an algorithm that enables us to efficiently mine our vast dataset for interactions of research value.

INT2 includes interactions between vehicles-vehicles, vechile-cyclist, and vehicle-pedestrian:

Retrieve the interaction within the scenario dataset:

python interaction_filter.py --scenario_path int2_dataset_example/scenario/0/010213250706-010213264206.pickle --output_dir
 int2_dataset_example/interaction_scenario/complete_scenario

Split the complete interactive scenario into interactive scenarios with a length of 9.1 seconds:

python split_interaction.py --interaction_scenario_path int2_dataset_example/interaction_scenario/complete_scenario/0/010213250706-010213264206.pickle --output_dir int2_dataset_example/interaction_scenario/split_scenario

The visualization of the complete interactive scenario:

python vis_interaction_scenario.py --scenario_path int2_dataset_example/interaction_scenario/complete_scenario/0/010213250706-010213264206.pickle

The results will be saved by default in the output/visualization folder, including an XML file in CommonRoad format, frame-by-frame visualization images, and a complete video.

The visualization of the interactive scenario segments split into 9.1-second lengths.

python vis_split_interaction_scenario.py --scenario_path int2_dataset_example/interaction_scenario/complete_scenario/0/010
213250706-010213264206.pickle

multiple XML files, visualization images, and videos with a length of 9.1 seconds will be saved by default in the output/visualization folder

We report collision rates so that they function as baselines for potential trajectory generation (instead of trajectory forecasting) applications. Generated trajectories should be as collision-free as possible, under the criteria mentioned above. To calculate collision:

python calculate_collision.py --scenario_path int2_dataset_example/scenario/0/010213250706-010213264206.pickle --hdmap_dir int2_dataset_example/hdmap

We rasterize both agents and road elements, where agents are represented as rectangles and the road elements are decomposed into a combination of triangles. We use the IOU criteria to detect collisions between agents by computing the overlap between their corresponding rectangles. We also detect collisions between agents and road elements by checking if the rectangles overlap with the road element triangles. The collision rate equals the number of collisions divided by the total number of agent-agent pairs or agent-boundary pairs, you can find it in the code.

We used M2I and MTR as benchmarks for our dataset.

If you want to use them, please refer to

• M2I with INT2
• MTR with INT2

Quantitative results of M2I on our INT2 dataset.

Quantitative results of MTR on our INT2 dataset.

Comming soon.

If you find this work useful in your research, please consider cite:

TODO.

• Waymo open motion dataset: https://github.com/waymo-research/waymo-open-dataset
• Commonroad: https://commonroad.in.tum.de/getting-started
• M2I: https://github.com/Tsinghua-MARS-Lab/M2I
• MTR: https://github.com/sshaoshuai/MTR