grouped-query-attention

Here are 6 public repositories matching this topic...

knotgrass / attention

several types of attention modules written in PyTorch for learning purposes

transformers pytorch transformer attention attention-mechanism softmax-layer multi-head-attention multi-query-attention grouped-query-attention scale-dot-product-attention

Updated Oct 1, 2024
Python

reshalfahsi / image-captioning-mobilenet-llama3

Star

Image Captioning With MobileNet-LLaMA 3

nlp cnn pytorch transformer image-captioning image-text flickr8k-dataset mobilenetv3 pytorch-lightning kv-cache rotary-position-embedding grouped-query-attention rms-norm llama3

Updated Jun 23, 2024
Jupyter Notebook

MyDarapy / SmolLM-experiments-with-grouped-query-attention

Star

(Unofficial) building Hugging Face SmolLM-blazingly fast and remarkably powerful small language model with PyTorch implementation of grouped query attention (GQA)

transformer attention smol huggingface ml-efficiency llm grouped-query-attention smol-lm huggingface-smol-lm

Updated Oct 11, 2024
Python

lucadellalib / llama3

Star

A single-file implementation of LLaMA 3, with support for jitting, KV caching and prompting

python transformers pytorch large-language-models llm grouped-query-attention rotary-positional-embedding llama3

Updated Nov 11, 2024
Python

LukasDrews97 / DumbleLLM

Star

Decoder-only LLM trained on the Harry Potter books.

transformer byte-pair-encoding rotary-position-embedding large-language-model flash-attention grouped-query-attention

Updated Dec 20, 2024
Python

prajeshshrestha / Llama-2.0-architecture-and-inference-from-scratch-with-PyTorch

Star

pytorch pytorch-implementation kv-cache llama2 grouped-query-attention rotary-positional-embedding

Updated Aug 5, 2024
Python

Improve this page

Add a description, image, and links to the grouped-query-attention topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the grouped-query-attention topic, visit your repo's landing page and select "manage topics."

Learn more

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

grouped-query-attention

Here are 6 public repositories matching this topic...

knotgrass / attention

reshalfahsi / image-captioning-mobilenet-llama3

MyDarapy / SmolLM-experiments-with-grouped-query-attention

lucadellalib / llama3

LukasDrews97 / DumbleLLM

prajeshshrestha / Llama-2.0-architecture-and-inference-from-scratch-with-PyTorch

Improve this page

Add this topic to your repo