GitHub - joongom/mlx-soloheaven:

Contribute to joongom/mlx-soloheaven development by creating an account on GitHub.

Story Details

Last Updated
Total Articles	1

Main Topics

Cache (computing)

#cache-computing

computing component that transparently stores data so that future requests for that data can be served faster

Web API

#web-api

HTTP based API used in web development

Turn 10 Studios

#turn-10-studios

American video game developer

Similar to This

VentureBeat

Credibility C

Oct 16

How flash storage can cut GPU waste from KV-cache recomputation: LLM inference can waste up to 60% of GPU compute recomputing context. WEKA explains how KV-cache ...

venturebeat.com/ai/up-to-60-of-gpu-compute-goes-to-recomputing-context-flash-storage-can-put-that-capacity-back-to-work

0 similar articles

github.com Sep 28

GitHub - solvingSteve/Gevva0: System 0 local decision gateway powered by Gemma 4. Direct logit scoring, KV-cache sequence branching ...

github.com/solvingSteve/Gevva0

0 similar articles

github.com Sep 26

Release Splash 1.1.0 · incoai/splash: A local inference engine for Apple silicon, built around the model. - Release Splash 1.1.0 · incoai/splash

github.com/incoai/splash/releases/tag/1.1.0

0 similar articles

VentureBeat

Sep 23

C2C replaces text handoffs between AI models | VentureBeat: C2C lets AI models communicate through KV caches instead of text, improving benchmark accuracy and ...

venturebeat.com/orchestration/text-handoffs-slow-ai-models-down-c2c-lets-them-communicate-through-kv-caches-instead

0 similar articles

0 0

github.com Jul 23

GitHub - firish/webfetch: Own your LLM's web search: a local search->fetch->rank pipeline that replaces hosted web-search tools ...

github.com/firish/webfetch

0 similar articles

github.com Jun 29

GitHub - GuglielmoCerri/khazad: Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets.

github.com/GuglielmoCerri/khazad

0 similar articles

github.com Apr 13

GitHub - Aryagm/dflash-mlx: Exact speculative decoding on Apple Silicon, powered by MLX.

github.com/Aryagm/dflash-mlx

0 similar articles

github.com Apr 05

GitHub - AlphaWaveSystems/tqai: TurboQuant KV cache compression for local LLM inference — 80% memory savings, near-zero quality loss on 8B+ models ...

github.com/AlphaWaveSystems/tqai

0 similar articles

github.com Apr 02

GitHub - matt-k-wong/turboquant-mlx-full: Contribute to matt-k-wong/turboquant-mlx-full development by creating an account on GitHub.

github.com/matt-k-wong/turboquant-mlx-full

0 similar articles

github.com Apr 01

GitHub - SharpAI/SwiftLM: ⚡ Native Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models ...

github.com/SharpAI/SwiftLM

0 similar articles

github.com Mar 27

GitHub - e1n00r/tinyserve: Run 20B-400B+ MoE models on 8GB consumer GPUs. 3-tier expert caching (SSD→RAM→GPU), native MXFP4/GGUF Q4_K compute ...

github.com/e1n00r/tinyserve

0 similar articles

github.com Mar 25

GitHub - Anemll/flash-moe: Contribute to Anemll/flash-moe development by creating an account on GitHub.

github.com/Anemll/flash-moe

0 similar articles

github.com Mar 02

GitHub - i3T4AN/Semantic-skill-space: Semantic Skill Space (SSS): Injecting skill embeddings into KV cache for small-model agents to recover skill behavior ...

github.com/i3T4AN/Semantic-skill-space

0 similar articles

github.com Feb 20

GitHub - fbk2111/Sentinel: Smart and cost effective LLM proxy: Smart and cost effective LLM proxy. Contribute to fbk2111/Sentinel development by creating an ...

github.com/fbk2111/Sentinel

0 similar articles