🔥🔥MLVU: Multi-task Long Video Understanding Benchmark

PyTorch code for our paper "Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grain Image Quality Assessment"

Codebase for Aria - an Open Multimodal Native MoE

[ICML'24 Spotlight] LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

[Neurips 24 Spotlight] Training in Pairs + Inference on Single Image with Anchors

[LMM + codec] A new paradigm of visual signal compression!

Accelerating the development of large multimodal models (LMMs) with lmms-eval

Open-source evaluation toolkit of large vision-language models (LVLMs), support ~100 VLMs, 40+ benchmarks

🏆 [CVPRW 2024] COVER: A Comprehensive Video Quality Evaluator. 🥇 Winner solution for Video Quality Assessment Challenge at the 1st AIS 2024 workshop @ CVPR 2024

[Neurips 24' D&B] Official Dataloader and Evaluation Scripts for LongVideoBench.

[LMM + AIGC] What do we expect from LMMs as AIGI evaluators and how do they perform?

Cornell NLVR and NLVR2 are natural language grounding datasets. Each example shows a visual input and a sentence describing it, and is annotated with the truth-value of the sentence.

An open-source implementation for training LLaVA-NeXT.

[NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"

Ongoing research training transformer models at scale

MambaOut: Do We Really Need Mamba for Vision?

Official code for Paper "Mantis: Multi-Image Instruction Tuning"

Multimodal language model benchmark, featuring challenging examples

[MM 2024 Oral] Refiner for AIGC

[CVPRW2024, Official Code] for paper "Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gap".

Tips for releasing research code in Machine Learning (with official NeurIPS 2020 recommendations)

The official Meta Llama 3 GitHub site

Evaluating text-to-image/video/3D models with VQAScore

LLM training in simple, raw C/CUDA

A list of works on evaluation of visual generation models, including evaluation metrics, models, and systems

Analysis of video quality datasets via design of minimalistic video quality models

[ACMMM 2024] AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perception