common : add --spec-draft-n-ubatch by ggerganov · Pull Request #30459 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/30459

Story Details

Last Updated
Total Articles	1

Main Topics

Llama (language model)

#llama-language-model

large language model by Meta AI

Handle

#handle

part of a device designed to be held

Similar to This

github.com Oct 15

Add Support to maion-coder arch by mailkv23 · Pull Request #29778 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/29778

0 similar articles

github.com Oct 06

RPC: add `-sm tensor` by am17an · Pull Request #26610 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...

github.com/ggml-org/llama.cpp/pull/26610

0 similar articles

github.com Aug 02

DeepseekV4 MTP + DSpark by am17an · Pull Request #25784 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...

github.com/ggml-org/llama.cpp/pull/25784

0 similar articles

126

github.com Jul 28

spec: add DSpark speculative decoding by wjinxu · Pull Request #25173 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/25173

0 similar articles

github.com Jun 12

[Speculative decoding] feat: add EAGLE3 speculative decoding support by ruixiang63 · Pull Request #18039 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/18039

0 similar articles

github.com Jun 10

Remove padding and multiple D2D copies for MTP by gaugarg-nv · Pull Request #24086 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24086

0 similar articles

github.com Jun 08

kv-cache : avoid kv cells copies by ggerganov · Pull Request #24277 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/24277

0 similar articles

github.com May 20

llama : add Gemma4 MTP by am17an · Pull Request #23398 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...

github.com/ggml-org/llama.cpp/pull/23398

0 similar articles

173

github.com May 19

llama : MTP clean-up by ggerganov · Pull Request #23269 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...

github.com/ggml-org/llama.cpp/pull/23269

0 similar articles

github.com Apr 09

ggml: backend-agnostic tensor parallelism (experimental) by JohannesGaessler · Pull Request #19378 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/19378

0 similar articles

github.com Apr 07

kv-cache : support attention rotation for heterogeneous iSWA by ggerganov · Pull Request #21513 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21513

0 similar articles

github.com Apr 06

ggml: add Q1_0 1-bit quantization support (CPU) by khosravipasha · Pull Request #21273 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21273

0 similar articles

github.com Apr 03

(revert) kv-cache : do not quantize SWA KV cache by ggerganov · Pull Request #21332 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21332

0 similar articles

github.com Apr 01

llama : rotate activations for better quantization by ggerganov · Pull Request #21038 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21038

0 similar articles

github.com Mar 28

ggml: allow prefetching tensor overrides by am17an · Pull Request #21067 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/21067

0 similar articles

github.com Mar 11

metal : add GDN kernel by ggerganov · Pull Request #20361 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating ...

github.com/ggml-org/llama.cpp/pull/20361

0 similar articles

github.com Mar 08

What's the difference between batch-size and ubatch-size? · ggml-org/llama.cpp · Discussion #6328

github.com/ggml-org/llama.cpp/discussions/6328

0 similar articles

github.com Feb 14

models : optimizing qwen3next graph by ggerganov · Pull Request #19375 · ggml-org/llama.cpp

github.com/ggml-org/llama.cpp/pull/19375

0 similar articles

github.com Feb 11

spec : add ngram-mod by ggerganov · Pull Request #19164 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...

github.com/ggml-org/llama.cpp/pull/19164

0 similar articles