kv-cache : avoid kv cells copies by ggerganov · Pull Request #24277 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24277

Story Details

Last Updated
Total Articles	1

Main Topics

Llama (language model)

#llama-language-model

large language model by Meta AI

Cache (computing)

#cache-computing

computing component that transparently stores data so that future requests for that data can be served faster

Cell (biology)

#cell-biology

basic structural and functional unit of all organisms

Similar to This

github.com Oct 15

Add Support to maion-coder arch by mailkv23 · Pull Request #29778 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/29778

0 similar articles

github.com Oct 14

common : add --spec-draft-n-ubatch by ggerganov · Pull Request #30459 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/30459

0 similar articles

github.com Sep 28

GitHub - solvingSteve/Gevva0: System 0 local decision gateway powered by Gemma 4. Direct logit scoring, KV-cache sequence branching ...

github.com/solvingSteve/Gevva0

0 similar articles

github.com Aug 15

[FR] Make GGML_CUDA_FA_ALL_QUANTS the default or add runtime warning for missing quantized FA kernels · Issue #24485 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/issues/24485

0 similar articles

github.com Jul 28

spec: add DSpark speculative decoding by wjinxu · Pull Request #25173 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/25173

0 similar articles

github.com Jun 10

Remove padding and multiple D2D copies for MTP by gaugarg-nv · Pull Request #24086 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24086

0 similar articles

github.com Jun 04

[PoC] server: support requantizing kv cache by wadealexc · Pull Request #24134 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24134

0 similar articles

github.com May 22

Asymmetric KV q8/q4 cache to preserve precision while reducing memory usage · ggml-org/llama.cpp · Discussion #23470

github.com/ggml-org/llama.cpp/discussions/23470

0 similar articles

github.com May 21

server: fix checkpoints creation by jacekpoplawski · Pull Request #22929 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/22929

0 similar articles

github.com May 20

llama : add Gemma4 MTP by am17an · Pull Request #23398 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...

github.com/ggml-org/llama.cpp/pull/23398

0 similar articles

173

github.com May 12

examples : add llama-eval by ggerganov · Pull Request #21152 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21152

0 similar articles

github.com Apr 09

ggml: backend-agnostic tensor parallelism (experimental) by JohannesGaessler · Pull Request #19378 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/19378

0 similar articles

github.com Apr 07

kv-cache : support attention rotation for heterogeneous iSWA by ggerganov · Pull Request #21513 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21513

0 similar articles

github.com Apr 07

TurboQuant - Extreme KV Cache Quantization · ggml-org/llama.cpp · Discussion #20969

github.com/ggml-org/llama.cpp/discussions/20969

0 similar articles

github.com Apr 03

(revert) kv-cache : do not quantize SWA KV cache by ggerganov · Pull Request #21332 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21332

0 similar articles

github.com Apr 01

llama : rotate activations for better quantization by ggerganov · Pull Request #21038 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21038

0 similar articles

github.com Feb 14

models : optimizing qwen3next graph by ggerganov · Pull Request #19375 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/19375

0 similar articles