Tools for merging pretrained Large Language Models and create Mixture of Experts (MoE) from open-source models.

Mergekit Assistant is a cutting-edge toolkit designed for the seamless merging of pre-trained language models. It supports an array of models, offers various merging methods, and optimizes for low-resource environments with both CPU and GPU compatibility.