github.com Mar 10
server : speculative checkpointing by srogmann · Pull Request #19493 · ggml-org/llama.cpp
github.com/ggml-org/llama.cpp/pull/19493
Story Details
| Last Updated | |
| Total Articles | 1 |
Main Topics
Similar to This
Previous
I moved my local models from Ollama to llama.cpp, and the reason I stayed has nothing to do with speed
xda-developers.com/i-moved-from-ollama-to-llamacpp-and-discovered-its-better-than-most-tools/
I ran GLM 5.3, a 744-billion-parameter model, on my home lab, and the frontier suddenly feels a lot closer
xda-developers.com/you-can-run-a-744b-model-on-your-own-hardware/
I used speculative decoding to make my local LLM feel instant, and now I actually prefer it to cloud APIs
xda-developers.com/used-speculative-decoding-my-local-llm-feel-instant-prefer-cloud-apis/
github.com Oct 10
open-jet/megakernel at master · L-Forster/open-jet: Local coding agent + model runtime. Serves Qwen3.8-27B 1.5x faster than llama.cpp on an RTX 3090 ...
github.com/L-Forster/open-jet/tree/master/megakernel
DSpark: Deepseek improves GPU utilization with token rates - | Golem.de: Speculative decoding allows for significantly better GPU utilization ...
golem.de/news/dspark-deepseek-verbessert-mit-token-raten-die-gpu-auslastung-2607-210485.html
Claude in caveman mode: This developer wanted to save tokens and went viral on Reddit
t3n.de/news/claude-hoehlenmensch-entwickler-reddit-1743774/
github.com Mar 22
GitHub - ian-hailey/vllm-docker-Qwen3-5-122B-A10B-NVFP4: Docker container config for launching Qwen3.5-122B-A10B-NVFP4 with vLLM
github.com/ian-hailey/vllm-docker-Qwen3-5-122B-A10B-NVFP4
github.com Feb 11
spec : add ngram-mod by ggerganov · Pull Request #19164 · ggml-org/llama.cpp: LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an ...
github.com/ggml-org/llama.cpp/pull/19164
Next