[PoC] server: support requantizing kv cache by wadealexc · Pull Request #24134 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24134

Story Details

Last Updated
Total Articles	1

Main Topics

Cache (computing)

#cache-computing

computing component that transparently stores data so that future requests for that data can be served faster

Handle

#handle

part of a device designed to be held

Similar to This

github.com Oct 16

model : MTP support for Mellum2.1 by Xarbirus · Pull Request #30612 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/30612

0 similar articles

github.com Oct 11

Runtime support for Prism Bonsai 2 27B by bri-prism · Pull Request #29600 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/29600

0 similar articles

github.com Sep 28

GitHub - solvingSteve/Gevva0: System 0 local decision gateway powered by Gemma 4. Direct logit scoring, KV-cache sequence branching ...

github.com/solvingSteve/Gevva0

0 similar articles

github.com Aug 27

feat: import qwen4exp (Qwen3.8-Flash-Next) support from upstream PR #27742 by giveen · Pull Request #324 · TheTom/llama-cpp-turboquant

github.com/TheTom/llama-cpp-turboquant/pull/324

1 similar article

github.com Jul 26

Add MiniMax-M3 (MSA: MiniMax Sparse Attention) support by timkhronos · Pull Request #24908 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24908

1 similar article

github.com Jun 19

llama.cpp/tools/server at master · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an account on GitHub.

github.com/ggml-org/llama.cpp/tree/master/tools/server

0 similar articles

github.com Jun 18

4-bit key cache that scales each channel on its own (q4-size memory without the quality cliff plain q4_0 falls off on some models) · ggml-org/llama ...

github.com/ggml-org/llama.cpp/discussions/24518

0 similar articles

github.com Jun 08

kv-cache : avoid kv cells copies by ggerganov · Pull Request #24277 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24277

0 similar articles

github.com May 22

Asymmetric KV q8/q4 cache to preserve precision while reducing memory usage · ggml-org/llama.cpp · Discussion #23470

github.com/ggml-org/llama.cpp/discussions/23470

0 similar articles

github.com May 15

What is wrong with llama.cpp release page?

github.com/ggml-org/llama.cpp/releases

0 similar articles

github.com Apr 15

GitHub - ParmesanParty/llama.cpp: Specialized fork for (relatively) fast single-GPU inference (in CUDA) using large MoE models that don't fit fully into ...

github.com/ParmesanParty/llama.cpp

0 similar articles

github.com Apr 07

GitHub - arte-fact/llamacpp-gfx-906-turbo: Contribute to arte-fact/llamacpp-gfx-906-turbo development by creating an account on GitHub.

github.com/arte-fact/llamacpp-gfx-906-turbo

0 similar articles

github.com Apr 01

llama : rotate activations for better quantization by ggerganov · Pull Request #21038 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21038

0 similar articles