How to Use and Deploy Kimi K3

Following Kimi K2.7’s strong run in long-horizon coding, Moonshot AI has launched another model. Kimi K3 expands the scope to coding, knowledge work, reasoning, and visual agents through a 2.8-trillion-parameter MoE with native vision and a 1-million-token context window.

Moonshot still ranks K3 behind Claude Fable 5 and GPT 5.6 Sol overall. The smaller capability gap gives enterprises a credible migration path: move repeatable, high-volume, or data-sensitive agent workloads to K3, then keep proprietary models for cases where internal evaluations show a clear advantage. Hosted access makes that test straightforward; the official weights are public, but self-hosting still needs purpose-built multi-accelerator infrastructure.

What is Kimi K3?

Kimi K3 is Moonshot AI’s 2.8-trillion-parameter flagship for long-horizon coding, reasoning, knowledge work, and visual agents. The sparse model activates 16 of 896 routed experts per token, supports native vision, and accepts up to 1 million tokens.

Moonshot attributes K3’s roughly 2.5× scaling-efficiency gain over Kimi K2 to Kimi Delta Attention, periodic Gated MLA, Attention Residuals, greater MoE sparsity, and changes to training and data. The official release still places K3 behind Fable 5 and GPT 5.6 Sol overall, but close enough to justify workload-level migration tests.

Kimi K3 launch date and availability

K3 is already available through four hosted routes. The appropriate route depends on whether a team is testing raw capability, desktop workflows, coding agents, or a production integration.

Product teams should evaluate K3 through the API. The OpenAI-compatible endpoint lets them reproduce their own prompts, tool schemas, context, permissions, and output contracts inside the application they plan to ship.

The hosted API cannot forecast a self-hosted cluster’s throughput. Moonshot reports a cache-hit rate above 90 percent for coding workloads on its tuned Mooncake stack; a new deployment begins with different traffic, prefix reuse, and cache behavior.

Where to find the Kimi K3 weights

Moonshot has published the full Kimi K3 checkpoint on Hugging Face.

The official Hugging Face checkpoint is the source for the full open-weight release. Downloading it makes sense for teams with a real multi-accelerator serving plan, not as a workstation experiment.

Moonshot’s GitHub repository now carries the model documentation, technical report, license, and release material. Use the implementation guidance for the runtime and hardware path you will actually operate.

Moonshot’s 64-plus-accelerator recommendation describes an efficient production topology, not a promise that every smaller setup will meet latency, concurrency, or reliability targets.

How to use Kimi K3 today

A migration test should reproduce the real agent loop. Send the same repository context, tools, screenshots, documents, system constraints, and output schema used in production. Measure completed work, review burden, tool failures, latency, and total session cost.

K3 launches with max reasoning effort. Moonshot says low and high modes will arrive later, so current evaluations should record output-token volume, end-to-end latency, and cost under the max setting.

K3 always enables thinking and Preserved Thinking. The API returns reasoning content separately from the final answer. Multi-turn clients must send historical reasoning content back exactly as required by the Kimi API contract.

Kimi’s own model limitations warn that missing thinking history can make generation unstable. Switching an active session from another model to K3 creates the same risk because K3 receives a conversation without the reasoning history it expects.

K3’s proactiveness requires tighter agent boundaries. Moonshot warns that ambiguous tasks can lead the model to make unexpected decisions on the user’s behalf. System prompts and AGENTS.md files should define allowed tools, filesystem scope, approval gates, external actions, and stopping conditions.

Kimi K3 usage best practices

• Start with a hosted evaluation. Use the API or Kimi Code to validate task success before reserving supernode-scale infrastructure.

• Preserve the complete thinking history. Return every required reasoning_content field during multi-turn conversations and tool loops.

• Keep one model for the full session. Avoid moving an active task from another model to K3 or replacing K3 halfway through the agent trajectory.

• Record the active reasoning mode. K3 launches with max; rerun the evaluation when low and high modes become available.

• Constrain proactive behavior. Define permissions, approval points, filesystem scope, network actions, and task completion rules in the system prompt or AGENTS.md.

• Preserve reusable prefixes. Stable system prompts, tool schemas, repository instructions, and shared documents increase the value of prefix caching.

• Measure the complete agent session. Track queueing, prefill, decode, tool turnaround, context growth, retries, and end-to-end task duration.

Kimi K3 architecture and its serving implications

K3’s scale changes the serving bottleneck. Memory still matters, but expert communication, recurrent-state caching, cross-layer residual reads, precision-specific kernels, and vision preprocessing determine whether the model runs efficiently.

KDA reduces the need to retain a conventional key and value pair for every past token. Each KDA layer instead advances a matrix-like recurrent state plus a short convolution state.

The recurrent state creates a different cache problem. A cached prefix is only useful when the runtime has the KDA state for the exact matching boundary. Storing that large state at every small token block would consume too much memory.

vLLM’s K3 work separates the physical KDA state block from the finer prefix-match unit. A request can reuse a partial prefix inside a larger state block, then copy the shared state into a private destination before extending it.

K3 still uses MLA every four layers. The serving engine must keep recurrent KDA state and paged MLA KV cache aligned to the same logical prefix. Cache transfer, offload, and disaggregated prefill and decode workers must preserve that alignment across machines.

Attention Residuals need dedicated kernels

Attention Residuals retrieve representations from earlier layer blocks instead of relying on one accumulated residual stream.

A direct implementation would add reads, writes, reductions, and normalization launches throughout K3’s 93-layer network. vLLM and NVIDIA have built fused kernels that combine the residual update, AttnRes mixing, and output RMSNorm for supported shapes.

Across 93 layers, even small penalties raise time per output token. Fused AttnRes kernels belong in the baseline serving path.

Expert parallelism makes the network part of inference

K3 routes each token to 16 of 896 experts. Sparse activation controls per-token compute, while the complete expert set still has to remain distributed across the serving system.

Expert parallelism moves token representations toward the selected experts and returns their outputs on every MoE layer. Routing quality, dispatch kernels, load balance, static shapes, accelerator links, and collective communication all affect latency.

Moonshot recommends supernodes with 64 or more accelerators because larger high-bandwidth communication domains improve inference efficiency. A cluster assembled from loosely connected nodes can have enough aggregate memory and still deliver poor token latency.

MXFP4 keeps K3 at supernode scale

K3 applies quantization-aware training from supervised fine-tuning onward. Moonshot describes MXFP4 weights with MXFP8 activations, and the released configuration uses MXFP4 weights.

The lower-precision checkpoint reduces the model-residency burden, but the theoretical four-bit weight footprint alone is roughly 1.4 TB before metadata, scales, runtime buffers, cache state, and implementation overhead.

Published implementation material covers initial K3 deployments, but each operator still needs to benchmark its own hardware, context mix, concurrency target, and failure headroom before a production rollout.

The published deployment paths cover more than one hardware stack. Hardware selection should follow the implementation you will actually run, not generic FP4 peak numbers.

Why Kimi K3 self-hosting is difficult

Model residency requires a large accelerator pool. Efficient expert execution requires a high-bandwidth communication domain. The 1M context adds recurrent KDA state, periodic KV cache, and expensive prefill work. Native vision adds preprocessing and a separate parallelism decision.

The serving stack must coordinate all of those resources under uneven agent traffic. A short request, a 500K-token research session, a visual coding loop, and a multi-hour tool-using agent consume different amounts of prefill, decode, cache, and network capacity.

The public release makes a proof of deployment possible. Production serving still requires a runtime that handles KDA state, cache transfer, expert communication, multimodal inputs, and the target hardware topology correctly.

The public checkpoint lets teams move from workload tracing to a bounded proof of deployment. Procurement should follow measured latency, concurrency, memory headroom, and failure-recovery results.

What Kimi K3 benchmarks actually establish

K3’s published results justify testing long-horizon coding, tool use, knowledge work, and visual agents. Internal task success should decide which workloads migrate. 

Moonshot’s reported K3 results use max reasoning effort, while the coding comparisons mix KimiCode, Claude Code, Codex, Terminus, and benchmark-specific harnesses. A small score difference can reflect the harness, context policy, or fallback behavior as much as the base model.

Migration tests should keep the harness, prompts, tools, permissions, retry policy, and context management fixed across models. Track successful task completion, human review, latency, and total session cost.

Graduating enterprise workflows to Kimi K3

Enterprise adoption should start with a workload, not an all-or-nothing model switch. K3 becomes a credible default when the same agent harness, context policy, tools, and review process produce comparable completed work at a lower operating cost.

K3 can replace a premium proprietary default for long-horizon coding, research, document analysis, visual QA, and tool-running workflows. Keep a closed-model fallback for novel, high-stakes, or failure-intolerant work until production evidence justifies a wider route.

Cost per completed task

Token pricing is only one input into task economics. Kimi K3 charges $0.30 per million cache-hit input tokens, $3 per million cache-miss input tokens, and $15 per million output tokens. Reasoning effort, prefix reuse, retries, tool calls, and human review determine the actual cost of a completed task.

BrowseComp can expose a meaningful score-to-cost tradeoff, but one chart does not predict another agent’s operating cost. Repeat the comparison with the same task mix, current API prices, token policy, and review process before changing a routing policy.

When self-hosting changes the cost equation

Open weights do not make self-hosting cheaper by default. A K3 deployment adds accelerator reservation, high-bandwidth networking, runtime engineering, warm capacity, monitoring, and failure coverage.

Self-hosting becomes rational when sustained traffic keeps the deployment busy, common prefixes create cache savings, or data residency and operating control outweigh API convenience. Hosted K3 remains the sensible starting point for bursty traffic and unproven workloads.

Kimi K3 production bottlenecks

• Supernode-scale residency: The MXFP4 checkpoint still spans a large accelerator domain, and production headroom must cover cache state, buffers, active sequences, and failures.

• Expert communication: Sixteen active experts per token create routing and all-to-all communication across 896 routed experts.

• Hybrid cache semantics: KDA recurrent state and MLA KV cache must advance on the same logical prefix across caching, offload, and worker transfer.

• Long-context prefill: A 1M-token window makes chunked prefill, prefix reuse, and prefill/decode disaggregation central to latency.

• Persistent thinking: Long agent sessions retain reasoning history and create highly variable decode duration, context growth, and output cost.

• Multimodal traffic: Images add preprocessing, visual tokens, and vision-tower capacity to the scheduling problem.

• Mixed workloads: Interactive requests can share infrastructure with long research or coding agents unless placement and batching policies separate them.

Loading K3 only confirms memory fit. Production readiness depends on cache reuse, communication efficiency, runtime correctness, traffic isolation, warm capacity, and recovery under partial accelerator or worker failure.

How PipeShift approaches Kimi K3 deployment

PipeShift begins with the closed-model workloads that have a credible K3 migration case.

API evaluations reproduce the real agent harness, context distribution, visual inputs, tool loops, concurrency, and approval policy. The results set targets for task success, tail latency, session cost, and review burden.

With the weights and release material now public, PipeShift can tune accelerator topology, expert-parallel placement, MXFP4 execution, KDA cache allocation, prefill and decode separation, and warm capacity around the measured workload.

Request classes remain separate so interactive work does not queue behind long research prefills or persistent coding sessions.

Frequently asked questions about Kimi K3

When did Kimi K3 launch?

Moonshot opened hosted K3 access on Jul 16, 2026. Moonshot released the full K3 checkpoint on Jul 27, 2026.

Where can I download the Kimi K3 weights?

Download the official Kimi K3 checkpoint from Moonshot’s Hugging Face repository. The model is open-weight, but the release belongs in a planned multi-accelerator serving environment, not on a conventional workstation.

Can Kimi K3 run on a single GPU or workstation?

A practical full-model deployment does not fit on a conventional workstation. The theoretical four-bit weights alone occupy roughly 1.4 TB before runtime overhead.

How many GPUs does Kimi K3 need?

Moonshot recommends supernodes with 64 or more accelerators for efficient inference. Published runtime guidance may show smaller loadable configurations, but they are not equivalent to an efficient production topology.

A smaller configuration may load the checkpoint in some runtimes. Loading the weights does not establish acceptable latency, concurrency, or failure headroom.

Does Kimi K3 support vLLM?

K3 support is now documented in the public serving ecosystem, including KDA-aware caching, multimodal handling, and reasoning and tool parsers. Test the exact runtime and hardware combination before committing production traffic.

What is the best way to test Kimi K3?

Start with the kimi-k3 API or Kimi Code. Use real tasks, preserve thinking history, keep the harness fixed, and measure completed work, latency, output tokens, tool reliability, and session cost under max reasoning effort.

Can Kimi K3 replace Claude Fable 5 or GPT 5.6 Sol?

K3 merits migration testing for coding, knowledge work, tool use, and visual agents. Moonshot still reports lower overall performance than Fable 5 and GPT 5.6 Sol.

An enterprise rollout can route qualified workloads to K3 and retain a closed-model fallback. Internal task success and operating cost should decide the boundary.