Local LLM for Code: Top Recommendations

Lisa Ernst · 06.10.2025 · Technology · 5 min

This overview examines the current local code LLMs that can be run on on-premise hardware without cloud connectivity. What matters are verifiable benchmarks, hardware requirements (VRAM/RAM), and features such as code infilling. We summarize the status and show which model fits which machine.

Quelle: Eigene Darstellung

Introduction & Fundamentals

By 'local' we mean running a model entirely on your own hardware, for example via runners such as Ollama or directly via llama.cpp/vLLM. Ollama enables easy pull/run, including with quantization. Quantisierung (e.g., GGUF Q4_K_M) significantly reduces memory usage, usually with moderate quality loss.

For practical use, the following aspects are important:

• Infilling/FIM: Targeted filling of gaps in code, supported by models such as StarCoder2 and CodeGemma.
• Context window: The ability to include longer files or projects. Qwen2.5-Coder offers here up to 128K tokens.
• Runtime budget: Rough rules of thumb for Ollama are: 7B models require at least 8 GB RAM/VRAM, 13B models 16 GB and 70B models 64 GB.

The motivation for local operation lies in privacy, reproducibility, offline work and cost control. Vendors such as BigCode/Hugging Face, Alibaba/Qwen and DeepSeek accelerate speed and transparency. Tools such as Ollama lower the entry barriers through easy pull/run and quantization (GGUF/4-Bit). Extensions such as Continue integrate local models directly into VS Code/JetBrains.

Current State & Models

Since 2024 there have been significant developments in the field of local code LLMs:

• StarCoder2 (3B/7B/15B): This model brought FIM training to The Stack v2 and a 16K context window. The 15B variant übertrifft similar-sized models across many benchmarks, such as in dieser Veröffentlichung described.
• Qwen2.5-Coder (0.5B–32B): Reports state-of-the-art (SOTA) results on open code benchmarks. The 32B-Instruct variant explicitly targets 'open-source SOTA' at EvalPlus, LiveCodeBench and BigCodeBench.
• DeepSeek-Coder-V2: Introduces an MoE design. The V2-Lite version (16B, active 2.4B) offers 128K context and is designed for local use. The larger V2 variant (236B, active 21B) leads many code benchmarks, but is not suitable for consumer hardware.
• CodeGemma (2B/7B): Focuses on efficient infilling. The 7B variant is well documented, including 4-bit setup and FIM tokens.

For fair comparisons, contamination-free benchmarks such as LiveCodeBench (rolling) and EvalPlus (HumanEval+/MBPP+). Hugging Face offers further information on this.

Quelle: nutstudio.imyfone.com

A visual representation of the best local LLMs for programming.

Practical Application & Integration

Choosing the right model strongly depends on the available hardware and the intended task:

• Laptop/8–12 GB VRAM: Qwen2.5-Coder-7B or CodeGemma-7B. These models offer strong infilling and low latency, especially in 4-bit operation.
• 16 GB VRAM: StarCoder2-15B-Instruct or DeepSeek-Coder-V2-Lite (16B aktiv 2.4B). A good balance of quality and speed.
• 24 GB+ VRAM: Qwen2.5-Coder-32B-Instruct. This model is open, powerful, and offers a large context window.
• CPU-only/small iGPU: Gemma/CodeGemma or smaller Qwen-Coder Varianten. Google explicitly demonstrates CPU operation with Ollama.

For practical use, IDE integration with Continue (VS Code/JetBrains) in conjunction with a Ollama-Server. It is advisable to actively use infilling rather than just chatting, and to perform A/B comparisons with EvalPlus - or LiveCodeBench-Problemen for your own domain.

Analysis & Evaluation

Manufacturers often emphasize 'open SOTA' (Qwen) or 'best-in-class' (StarCoder2), which is partly supported by benchmarks but also includes marketing aspects. A look at mehrere Quellen is therefore advisable. The community reports mixed experiences: while some local setups celebrate, others report variable quality on edit tasks, often due to prompting, contexts, and editor integration, as hier discussed.

Fact-check: Evidence vs. Claims

• Supported by:
  • 7B/13B/70B Rough RAM guidelines for Ollama are broadly confirmed in practice.
  • StarCoder2 offers FIM training, 16K context, and strong 15B results compared to similarly sized models (Quelle).
  • Qwen2.5-Coder 32B-Instruct claims SOTA on open code benchmarks and covers 0.5B–32B sizes, up to 128K context.
  • DeepSeek-Coder-V2-Lite: MoE with 16B (active 2.4B), 128K context. The large V2 variant shows very high code-bench scores, but is not suitable for consumer hardware.
  • CodeGemma 7B: FIM tokens are documented; 4-bit operation is possible with around 9 GB.
• Unclear/Nuanced:
  • "1 GB VRAM per billion parameters" is a community rule-of-thumb value that varies greatly with quantization, context length, and offload. Model/runner documentation is more precise here ( Qwen, Ollama).
• False/Misleading:
  • "Quantization makes models unusable." In many coding workflows, 4-bit is a good compromise. Quality losses depend on the model, task, and context length ( CodeGemma, Qwen).

Quelle: pieces.app

A diagram comparing the performance of different LLM models in the coding domain.

Conclusion & Outlook

For the search for the 'best local LLM for coding' there are real options today. Qwen2.5-Coder-32B-Instruct For 24 GB+ VRAM, it is the go-to option among open models. StarCoder2-15B-Instruct On 16 GB VRAM, it delivers very smooth infilling and stable performance. Qwen2.5-Coder-7B In the 7B segment, there are CodeGemma-7B pragmatic choices: fast, efficient, and well-documented. DeepSeek-Coder-V2-Lite scores with MoE efficiency and large context, provided it is cleanly quantized and integrated.

Utility Analysis

Weighting: Performance 60 %, local resource fit 20 %, IDE features/FIM+Context 10 %, License 10 %. Performance estimates are based on cited benchmarks/model documents.

• Qwen2.5-Coder-32B-Instruct: 8.4/10 – Highest open performance, large context window; requires more VRAM, but strong for complex tasks.
• Qwen2.5-Coder-14B-Instruct: 8.4/10 – Excellent price/performance ratio, broadly applicable, Apache-2.0 license.
• DeepSeek-Coder-V2-Lite (16B, aktiv 2.4B): 8.0/10 – Efficient MoE, 128K context; highly usable when quantized.
• StarCoder2-15B-Instruct: 7.9/10 – FIM-strong, 16K context, transparent training; robust for Edit/Completion.
• Qwen2.5-Coder-7B-Instruct: 8.0/10 – Mobile/laptop-ready, good quality with fast latency; ideal for inline edits.
• CodeGemma-7B: 7.5/10 – Lean, FIM very neat, good documentation/setup; strong for fast autocompletion.

If you want to start today, install Ollama, pulls Qwen2.5-Coder-7B or StarCoder2-15B, activate Continue in VS Code and use infilling deliberately. This way you benefit immediately, without tying yourself to a cloud provider.

Open Questions

The robustness of code quality across different programming languages and frameworks remains an open question. Rolling benchmarks address data leakage, but are not a complete guarantee ( LiveCodeBench, Hugging Face). Which metrics correlate most strongly with real productivity in the editor (Edit/Refactor/Repo context)? Aider publishes editing/refactor benchmarks, but standardization is still lacking. For local hardware, questions remain about the optimal quant/offload setup; here the runner guides and own microbenchmarks ( Qwen, Ollama).

Quelle: openxcell.com

A representation of the integration of LLMs into the development process.