CUDA: reduce MMQ stream-k overhead by JohannesGaessler · Pull Request #22298 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/22298

Story Details

Last Updated
Total Articles	1

Main Topics

Nvidia GeForce RTX 3090

#nvidia-geforce-rtx-3090

high-end Ampere architecture graphics card

Handle

#handle

part of a device designed to be held

Similar to This

github.com 18h

CUDA: fuse MoE gate/up matmuls and GLU in MMQ by anujj · Pull Request #31188 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/31188

0 similar articles

github.com Oct 25

CUDA: fuse FFN gate/up matmuls and GLU in MMQ with interleaved gate/up rows by anujj · Pull Request #29948 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/29948

0 similar articles

github.com Oct 24

llama-on-legacy-gpu/README.md at main · theIvanR/llama-on-legacy-gpu: High performance llama cpp patches and fork for legacy gpus - llama-on-legacy-gpu/README ...

github.com/theIvanR/llama-on-legacy-gpu/blob/main/README.md

0 similar articles

github.com Oct 16

model : MTP support for Mellum2.1 by Xarbirus · Pull Request #30612 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/30612

0 similar articles

github.com Sep 16

qwen4exp: add hc ops by am17an · Pull Request #28901 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...

github.com/ggml-org/llama.cpp/pull/28901

0 similar articles

github.com Sep 12

ggml-cuda: hip: add missing AMD GCN MMQ config by thelittlefireman · Pull Request #27841 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/27841

0 similar articles

github.com May 29

Release b9410 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an account on GitHub.

github.com/ggml-org/llama.cpp/releases/tag/b9410

0 similar articles

github.com May 25

CUDA: add fast walsh-hadamard transform by am17an · Pull Request #23615 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/23615

0 similar articles

github.com May 25

Misc. bug: Meta backend `ggml_context` pool exhaustion with `--split-mode tensor` · Issue #22404 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/issues/22404

0 similar articles

github.com May 15

What is wrong with llama.cpp release page?

github.com/ggml-org/llama.cpp/releases

0 similar articles

github.com Apr 15

GitHub - ParmesanParty/llama.cpp: Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into ...

github.com/ParmesanParty/llama.cpp