Code for paper "KOSMOS-E : Learning to Follow Instruction for Robotic Grasping" at IEEE International Conference on Intelligent Robots and System (IROS), 2024.

world modeling challenge for humanoid robots

A native PyTorch Library for large model training

A throughput-oriented high-performance serving framework for LLMs

Efficient Triton Kernels for LLM Training

Fast and memory-efficient exact attention

Official implementation of SEED-LLaMA (ICLR 2024).

Deita: Data-Efficient Instruction Tuning for Alignment [ICLR2024]

The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use th…

Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.

Mathematical Visual Instruction Tuning for Multi-modal Large Language Models

DataComp for Language Models

Autoregressive Model Beats Diffusion: 🦙 Llama for Scalable Image Generation

LOFT: A 1 Million+ Token Long-Context Benchmark

MINT-1T: A one trillion token multimodal interleaved dataset.

OpenVLA: An open-source vision-language-action model for robotic manipulation.

Official repository of the xLSTM.

Tile primitives for speedy kernels

The Prism Alignment Project

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?

Memory optimization and training recipes to extrapolate language models' context length to 1 million tokens, with minimal hardware.

Code for the curation of The Stack v2 and StarCoder2 training data

Official Implementation of ICLR'24: Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Retrieval and Retrieval-augmented LLMs

A lightweight script for processing HTML page to markdown format with support for code blocks

GLake: optimizing GPU memory management and IO transmission.

DORA (Dataflow-Oriented Robotic Architecture) is middleware designed to streamline and simplify the creation of AI-based robotic applications. It offers low latency, composable, and distributed dat…