(revert) kv-cache : do not quantize SWA KV cache by ggerganov · Pull Request #21332 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21332

Story Details

Last Updated
Total Articles	1

Main Topics

Cache (computing)

#cache-computing

computing component that transparently stores data so that future requests for that data can be served faster

Llama (language model)

#llama-language-model

large language model by Meta AI

Handle

#handle

part of a device designed to be held

Similar to This

github.com Oct 15

Add Support to maion-coder arch by mailkv23 · Pull Request #29778 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/29778

0 similar articles

github.com Oct 14

common : add --spec-draft-n-ubatch by ggerganov · Pull Request #30459 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/30459

0 similar articles

github.com Sep 28

GitHub - solvingSteve/Gevva0: System 0 local decision gateway powered by Gemma 4. Direct logit scoring, KV-cache sequence branching ...

github.com/solvingSteve/Gevva0

0 similar articles

github.com Aug 15

[FR] Make GGML_CUDA_FA_ALL_QUANTS the default or add runtime warning for missing quantized FA kernels · Issue #24485 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/issues/24485

0 similar articles

github.com Jul 24

GitHub - fewtarius/CachyLLama: LLM inference in C/C++: LLM inference in C/C++. Contribute to fewtarius/CachyLLama development by creating an account on GitHub.

github.com/fewtarius/CachyLLama

0 similar articles

github.com Jun 08

kv-cache : avoid kv cells copies by ggerganov · Pull Request #24277 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24277

0 similar articles

github.com Jun 04

[PoC] server: support requantizing kv cache by wadealexc · Pull Request #24134 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24134

0 similar articles

github.com May 22

Asymmetric KV q8/q4 cache to preserve precision while reducing memory usage · ggml-org/llama.cpp · Discussion #23470

github.com/ggml-org/llama.cpp/discussions/23470

0 similar articles

github.com May 20

llama : add Gemma4 MTP by am17an · Pull Request #23398 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...

github.com/ggml-org/llama.cpp/pull/23398

0 similar articles

173

github.com May 17

GitHub - RedToasty/llama.cpp_qts: Fixing --split-mode tensor, with different KV cache quantization types.

github.com/RedToasty/llama.cpp_qts

0 similar articles

github.com May 15

What is wrong with llama.cpp release page?

github.com/ggml-org/llama.cpp/releases

0 similar articles

github.com Apr 09

ggml: backend-agnostic tensor parallelism (experimental) by JohannesGaessler · Pull Request #19378 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/19378

0 similar articles

github.com Apr 07

kv-cache : support attention rotation for heterogeneous iSWA by ggerganov · Pull Request #21513 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21513

0 similar articles

github.com Apr 07

TurboQuant - Extreme KV Cache Quantization · ggml-org/llama.cpp · Discussion #20969

github.com/ggml-org/llama.cpp/discussions/20969

0 similar articles

github.com Apr 01

llama : rotate activations for better quantization by ggerganov · Pull Request #21038 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21038

0 similar articles

github.com Mar 31

GitHub - quantumaikr/TurboQuant.cpp: turboquant.cpp: turboquant.cpp. Contribute to quantumaikr/TurboQuant.cpp development by creating an account on GitHub.

github.com/quantumaikr/TurboQuant.cpp

0 similar articles

github.com Mar 24

GitHub - cenconq25/delta-compress-llm: Exploiting temporal coherence in LLM inference-- delta encoding for KV cache compression and weight-skip prediction ...

github.com/cenconq25/delta-compress-llm

0 similar articles