llama.cpp 1.0 Breaking Changes and How to Upgrade Safely

llama.cpp 1.0 shipped on 13 November, and ggml 1.0.0 was tagged the same day. It's the first time the engine behind a huge share of local LLM tooling has carried a 1.0 version number. It also deletes a lot of old API surface.

If you run llama-server behind a proxy, link against libllama, or split models across machines with RPC, read this before you bump the submodule. The new features are good. The breakage is real.

Key features and llama.cpp breaking changes in 1.0

The release notes read like a spring clean followed by a feature drop. Here's what matters.

Removed and renamed APIs. The notes list llama_set_warmup, 35 deprecated functions, defrag_thold and --defrag-thold as removed. They don't name the other 34 functions in that line, so your compiler is the real changelog here. llama_set_embd_state_src is now llama_set_ctx_state_src, and the NextN API moved out of llama-ext.h.

FP8 KV cache. You can now store the KV cache as FP8 with -ctk f8_e4m3 -ctv f8_e4m3, with optional static per-tensor scales. FP8 is one byte per element and f16 is two, so this halves KV-cache memory against f16. It doesn't halve your total VRAM, since weights usually dominate.

MoE expert cache. Mixture-of-experts models get an expert cache, exposed through two new fields in llama_context_params: moe_cache_size and moe_cache_cpu_share. ggml can compute a share of cache misses on the CPU and copy results back asynchronously. The notes don't mention a command-line flag, so for now this is something you wire up in code.

Speech output. The server gains an OpenAI-compatible /v1/audio/speech endpoint, plus a temperature option on /tts. Gemma 4 also gets streaming audio input, which is a different feature.

New models. Support lands for Kimi-K3.1-Pro and for Ripple-65B-Preview, a new dense 65B architecture.

Redesigned web UI. The bundled chat UI now has chat groups and a markdown editor.

The llama-server security fixes you shouldn't skip

This is the part I'd upgrade for even if I didn't care about FP8. The release fixes --cors-origins, DNS rebinding, CORS proxy SSRF and child instance access.

That's a nasty set for anyone who runs llama-server on a LAN box or a dev VM. DNS rebinding is how a malicious web page in your browser can talk to a server you thought was only reachable locally. SSRF through a CORS proxy is how that server gets tricked into fetching internal URLs for someone else.

Two new opt-in flags came with the fixes: --allow-unauthenticated-tools and --media-allow-private-urls. Their opt-in names tell you the direction of travel. If your setup leaned on tools without auth or on media from private addresses, expect to set these explicitly, and think hard before you do.

ggml 1.0 and the RPC cluster trap

ggml's own 1.0.0 release bumps the RPC protocol to v9. The notes say it brings faster loading and graceful failure on unreachable endpoints, and that the bump is there to reject incompatible peers.

Here's what that means in practice: mixed-version RPC clusters stop working. If you shard a big model across a desktop, a laptop and a Mac mini, upgrade every node together. Upgrade one and it'll refuse the others.

ggml also adds OCP FP8 support on CUDA, fused MoE routing on Metal, and a device-side MoE scheduler on SYCL. It drops the DIAG_MASK_INF and DIAG_MASK_ZERO ops, which only matters if you call ggml directly.

How much memory does the llama.cpp FP8 KV cache save?

The release notes don't put a number on this, so here's the arithmetic. KV cache size is roughly:

2 × layers × kv_heads × head_dim × context_length × bytes_per_element

Take a hypothetical model with 48 layers, 8 KV heads and a head dimension of 128, running a 32K context. At f16 that's 2 × 48 × 8 × 128 × 32,768 × 2 bytes, about 6 GiB. At FP8 it's about 3 GiB.

That 3 GiB is either a longer context or a second parallel slot on the same card. On a 24 GB card, that can decide whether a model fits once the weights are loaded. The release gives no speed or quality numbers for FP8, so measure perplexity on your own workload before you make it the default.

Real-world use cases

Long-context local chat. FP8 KV gives you twice the context for the same cache memory. Useful for repo-sized code questions on a single consumer GPU.
Self-hosted TTS behind an OpenAI client. Point any OpenAI SDK's speech call at /v1/audio/speech on your own box instead of a paid API.
Running big MoE models on mixed hardware. The new expert cache, which can push a share of cache misses to the CPU, looks aimed at setups where experts don't all fit in VRAM.
Team servers on a shared network. The security fixes close several known holes for llama-server on an office LAN, though it still shouldn't face the open internet.

Hard limitations and flaws

There's no migration guide. Neither the llama.cpp nor the ggml release includes one. You'll find out which of the 35 removed functions you used when the build fails.
1.0 isn't a stability promise. The notes say nothing about an API-stability or semver guarantee going forward. Don't read the version number as a contract.
No FP8 benchmarks. The release doesn't quantify speed or output quality at FP8. Test before you trust it for evals or anything accuracy-sensitive.
The MoE cache is code-only for now. If you only drive llama.cpp from the CLI, there's no documented switch for it yet.
Expect bindings to lag. Anything that wraps libllama has to absorb the removals and renames before you see 1.0 there.

Developer action plan for llama.cpp 1.0

Pin first. Lock your current commit or tag so a rebuild doesn't drag you into 1.0 by accident.
Grep before you build. Run grep -rnE "llama_set_warmup|llama_set_embd_state_src|defrag.thold" . across your code and launch scripts. It catches the named breakages in seconds, before a compile tells you the rest.
Compile against 1.0 on a branch. Fix the removed calls, rename llama_set_embd_state_src to llama_set_ctx_state_src, and drop any --defrag-thold from launch scripts.
Upgrade RPC nodes as a group. v9 rejects older peers, so plan one maintenance window for the whole cluster.
Re-check server flags. Audit --cors-origins, and only add --allow-unauthenticated-tools or --media-allow-private-urls if you understand the exposure.
Try FP8 KV on one workload. Run -ctk f8_e4m3 -ctv f8_e4m3, compare outputs against f16, then decide.

My verdict: upgrade llama-server this week for the security fixes alone. Treat the library upgrade as a planned migration, not a casual git pull.

llama.cpp 1.0 Breaking Changes and How to Upgrade Safely