#

pre-training

Here are 157 public repositories matching this topic...

RUCAIBox / LLMSurvey

The official GitHub page for the survey paper "A Survey of Large Language Models".

natural-language-processing pre-training pre-trained-language-models in-context-learning large-language-models llm llms chain-of-thought chatgpt rlhf instruction-tuning

Updated Aug 20, 2024
Python

modelscope / ms-swift

Use PEFT or Full-parameter to finetune 350+ LLMs or 90+ MLLMs. (LLM: Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, Gemma2, ...; MLLM: Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL, Phi3.5-Vision, ...)

Updated Sep 30, 2024
Python

dbiir / UER-py

Open Source Pre-training Model Framework in PyTorch & Pre-trained Model Zoo

Updated May 9, 2024
Python

modelscope / data-juicer

A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

Updated Sep 27, 2024
Python

ChandlerBang / awesome-self-supervised-gnn

Papers about pretraining and self-supervised learning on Graph Neural Networks (GNN).

machine-learning deep-learning graph-mining graph-neural-networks self-supervised-learning pre-training pretraining graph-self-supervised-learning

Updated Feb 2, 2024
Python

EgoAlpha / prompt-in-context-learning

Awesome resources for in-context learning and prompt engineering: Mastery of the LLMs such as ChatGPT, GPT-3, and FlanT5, with up-to-date and cutting-edge updates.

chatbot prompt language-modeling prompt-toolkit cot pre-training language-understanding prompt-learning prompt-tuning in-context-learning llm prompt-engineering chain-of-thought prompt-based-learning chatgpt chatgpt-api large-language-model prompt-design

Updated Sep 24, 2024
Jupyter Notebook

LirongWu / awesome-graph-self-supervised-learning

Code for TKDE paper "Self-supervised learning on graphs: Contrastive, generative, or predictive"

machine-learning deep-learning transfer-learning representation-learning unsupervised-learning data-augmentation graph-neural-networks self-supervised-learning pre-training pretext-task

Updated Aug 15, 2024

zjunlp / KnowLM

An Open-sourced Knowledgable Large Language Model Framework.

Updated Jun 26, 2024
Python

yzhuoning / Awesome-CLIP

Awesome list for research on CLIP (Contrastive Language-Image Pre-Training).

clip pre-training contrastive-learning

Updated Jun 28, 2024

Oscar

microsoft / Oscar

Oscar and VinVL

vqa image-captioning oscar vision-and-language pre-training image-text-search vinvl

Updated Aug 28, 2023
Python

Tencent / TencentPretrain

Tencent Pre-training framework in PyTorch & Pre-trained Model Zoo

Updated Aug 4, 2024
Python

brightmart / bert_language_understanding

Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN

nlp text-classification question-answering document-classification transfer-learning fasttext language-model textcnn attention-is-all-you-need self-attention transformer-encoder bert-model pre-training language-understanding

Updated Jan 1, 2019
Python

qingsongedu / Awesome-TimeSeries-SpatioTemporal-LM-LLM

A professional list on Large (Language) Models and Foundation Models (LLM, LM, FM) for Time Series, Spatiotemporal, and Event Data.

timeseries forecasting machinelearning deeplearning autoscaling large-models anomalydetection rca pre-training foundation-models large-language-models

Updated May 25, 2024

SalesforceAIResearch / uni2ts

[ICML2024] Unified Training of Universal Time Series Forecasting Transformers

machine-learning deep-learning time-series transformers forecasting representation-learning time-series-forecasting pre-training pre-trained-models universal-forecasting

Updated Sep 18, 2024
Jupyter Notebook

ChenRocks / UNITER

Research code for ECCV 2020 paper "UNITER: UNiversal Image-TExt Representation Learning"

transformers pytorch vision-and-language pre-training

Updated Jun 30, 2021
Python

jackroos / VL-BERT

Code for ICLR 2020 paper "VL-BERT: Pre-training of Generic Visual-Linguistic Representations".

pytorch representation-learning bert vision-and-language self-supervised-learning pre-training iclr2020 vl-bert

Updated May 22, 2023
Jupyter Notebook

Shen-Lab / GraphCL

[NeurIPS 2020] "Graph Contrastive Learning with Augmentations" by Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen

self-supervised-learning pre-training graph-neural-network contrastive-learning

Updated Jul 17, 2024
Python

princeton-nlp / LLM-Shearing

[ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

nlp efficiency pruning llama pre-training llm llama2

Updated Mar 4, 2024
Python

nancheng58 / Awesome-LLM4RS-Papers

Large Language Model-enhanced Recommender System Papers

survey awesome-list recommender-system gpt papers pre-training prompt-tuning large-language-models llm

Updated Sep 6, 2024

acbull / GPT-GNN

Code for KDD'20 "Generative Pre-Training of Graph Neural Networks"

graph-neural-networks self-supervised-learning graph-representation-learning pre-training

Updated Jul 25, 2024
Python

Improve this page

Add a description, image, and links to the pre-training topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the pre-training topic, visit your repo's landing page and select "manage topics."