🔍 AMF Extraction: Process reference videos through pre-trained DiT to extract Attention Motion Flow (AMF)

⚙️ Motion Optimization: Guide latent denoising with AMF loss in a training-free manner to reproduce reference motion

🔄 Zero-shot Motion Injection: Optimized transformer positional embeddings can be injected in new generation for zero-shot motion transfer

📊 Evaluation: Outperforms existing methods (MOFT, SMM) across multiple metrics and human evaluation when implemented for DiTs

[10/12/2024] 🔥🔥🔥 Our paper, Video Motion Transfer with Diffusion Transformers, has been archived.