Offline LLM for Texts

Lisa Ernst · 24.11.2025 · Technology · 9 min

Local writing AI on your own PC allows the creation of blog drafts, outlines, title variations, and raw versions completely offline. While initially local LLMs were primarily used for programming, they are increasingly establishing themselves as writing assistants. In contrast to cloud services like ChatGPT or Claude, local models offer full data control and no ongoing subscription costs.

Source: Own Representation

Basics of Local AI

The search for „best ai for writing“ often leads to hosted services like ChatGPT, Claude, or specialized SaaS tools. These require an internet connection and transfer texts as well as metadata to third parties. The desire for automated writing assistance is high, as shown by the popularity of 'best ai for writing' as one of the most searched 'best ai for...' queries on Google Trends, directly behind 'best ai for coding'.

Local models shift the focus: raw data remains on your own computer, latency is low, and costs depend on electricity prices, not subscriptions. Modern runtime environments like LM Studio, Ollama or AMD Gaia package these models into user-friendly interfaces. This creates a new category for writers: 'local llm for writing offline'. This means a model that is directly integrated into editors like VS Code or Obsidian and operates without an external connection.

Technical Aspects

A 'local llm for writing offline' typically consists of three layers: an open model (e.g., Llama, Qwen, Gemma, Phi, or DeepSeek-Distillate), a runtime like Ollama or LM Studio, and a client (editor, browser plugin, or desktop app).

Open Language Models

Open language models form the foundation. Meta Llama 3 und 3.1 are available as open weights, with sizes from 8 to 405 billion parameters; the 8B variants are designed for consumer hardware. Qwen3 von Alibaba encompasses dense and Mixture-of-Experts models from under 1 billion to over 200 billion parameters and has a strong focus on multilingualism. Google Gemma 3, Microsoft Phi-3 and DeepSeek-R1 complement the field with smaller, more efficient models and specialized reasoning variants.

Runtimes

Runtimes enable the operation of models. Ollama offers a unified interface and API to download and deploy various models locally with a single command. Installation and model management are handled via a desktop app and a simple CLI. LM Studio follows a similar approach but focuses on a GUI workflow for downloading, benchmarking, and launching models, and can also provide a local OpenAI-compatible server. AMD Gaia complements these tools on Windows by making local LLMs, including RAG agents, accessible via an ONNX-based backend layer and optimizing AMD Ryzen AI systems.

Integration into Writing Tools

Integration into writing tools is crucial for the workflow. Continue.dev is a popular extension for VS Code and JetBrains that integrates local models via Ollama or LM Studio as a chat and inline assistant. For Obsidian there are several community plugins like „AI LLM“ or Copilot integrations that connect directly to local Ollama or LM Studio servers.

For text generation, the hardware requirements for an LLM are less demanding than for complex coding or multi-modal reasoning. The LM-Studio-Dokumentation recommends at least 16 GB of RAM and a GPU with at least 4 GB of VRAM for Windows systems, especially with larger context windows.

A 3-4B model like Phi-3-Mini can already run usefully fast on CPU-only systems with 16 GB RAM in 4-bit quantization, provided the context length remains at 4-8K tokens. For 7-8B models like Llama 3.1 8B or Qwen3-8/14B a GPU with 8 GB of VRAM or more is recommended to enable smooth interaction with 8-32K context. A rule of thumb is that for an 8B model in 4-bit quantization, about 6-8 GB of VRAM is sufficient, with an additional 4-6 GB of VRAM for the KV cache potentially needed for 32K context. For 14-32B models, more memory or multi-GPU setups are accordingly required.

For pure writing operations, a middle ground is recommended: 16-32 GB RAM, a GPU with 8-12 GB VRAM (or Apple Silicon with Unified Memory), and a well-quantized 7-14B model. This allows for pleasant response times, consistent texts, and working with longer blog series in a single session.

Model Selection and Context

For writing, stable, natural-sounding language is more important than the latest benchmark score, especially in German. A current overview of open LLMs for German highlights Qwen3-235B-A22B, Meta-Llama-3.1-8B-Instruct and Qwen3-14B as particularly strong candidates due to their multilingualism and robust German language proficiency. Qwen3 was trained on around 36 trillion tokens and supports 119 languages and dialects, including German.

Meta Llama 3.1 8B in its 3.1 generation is explicitly described as multilingual and offers a 128K context, which is attractive for writing projects with many notes and references. Gemma-2 und Gemma-3 are characterized by efficiency; Gemma-2-2B-IT performs well in European languages. Gemma 3 increases context windows to 32K for 1B variants and 128K for larger models.

Microsofts Phi-3-Mini-4K-Instruct is kept small with 3.8B parameters but performs strongly in benchmarks and is suitable for notebooks or less powerful desktops. DeepSeek-R1 and its derived models are strong in reasoning and available as open source. Data protection concerns with online use argue for strictly locally operated variants.

For a practical writing workflow, three classes have proven effective:

• Small models (2-4B) like Phi-3-Mini or Gemma-2-2B-IT on less powerful hardware, for short texts, notes, and idea collections.
• Medium-sized models (7-8B) like Llama 3.1 8B or Qwen3-8B/14B as the 'sweet spot' for blog articles with solid coherence and good German quality.
• Larger models (14-32B) for demanding projects or long series; here, memory requirements and energy consumption increase, which is more worthwhile on workstations.

Context length determines how much text the model can process simultaneously. For a typical blog article of 1500-2500 words, 4-8K tokens are sufficient. For extensive research, previous articles, or SEO briefings, a 32K window becomes attractive. Qwen3 was trained with a default of 32K context and can be extended to 128K. Llama 3.1 also raises the context for 8B models to 128K tokens, enabling the processing of entire content collections.

The cost of long contexts is quadratic complexity: increasing the context window from 8K to 128K can theoretically increase the computational load by a factor of 256. Long contexts should only be used where they provide added value, e.g., when rewriting multiple articles or extensive research dossiers. For blog writing, a compact 8K window for quick idea generation and a 32K window for large projects have proven effective. LM-Studio-Leitfäden recommend considering context length and KV cache requirements when choosing a model, as a 32K context for 8B models can require an additional 4-6 GB of VRAM.

Integration in Editors

Integrating a local LLM directly into the editor is crucial for an efficient workflow.

VS Code: Inline Writing Assistant

Continue.dev is an open-source extension for VS Code that integrates local models via Ollama or LM Studio as an editor assistant. The configuration allows you to set the API endpoint of a local server (e.g. http://localhost:11434 with Ollama) as the source for chat and inline completions. A common practice is to install Ollama, download a model like Llama 3.1 8B or Qwen3-14B, and start it as a server. Continue is installed in VS Code and configured so that all requests go to this local endpoint. Inline suggestions then help with formulating introductions, subheadings, or transitions, while chat prompts rewrite longer sections or transfer them to different tonalities.

Those who prefer

can activate its server mode and connect it as a local AI provider through extensions like CodeGPT. A guide shows how to use the local LM Studio server in VS Code as a source for chat interaction and text suggestions.

„Build a FREE AI Coding Assistant Locally in VS Code“ – is useful for the basic configuration of Continue + Ollama, which can be transferred to writing models.

„Local AI Coding in VS Code: Installing Llama 3 with Continue“ – exemplifies how to connect Llama 3 locally with Continue; the same setup works for writing prompts.

Obsidian: Knowledge Management and Writing in One

For writers who use Obsidian as a Zettelkasten or content database, local integration is particularly exciting. The „AI LLM“-Plugin-Eintrag explicitly describes support for local LLMs via Ollama, including functions to expand and rewrite text directly within the Markdown document. A guide shows how to connect Obsidian Copilot with LM Studio by configuring the local server under an OpenAI-compatible URL.

A practical example: In LM Studio a Qwen3-7B-Instruct model runs, which is provided as a server via the 'Local LLM Service'; Obsidian Copilot or the AI LLM plugin sends selected paragraphs to this server to generate suggestions for better transitions, alternative wording, or meta descriptions – without data leaving the cloud.

Video Tips for Obsidian + Local:

„How To Run Local AI With Obsidian Copilot Plugin + LM Studio“ shows step-by-step how to set up LM Studio as a local provider for Obsidian Copilot.

„Ollama + Obsidian (LocalGPT): Writing with local models“ demonstrates how a local Ollama model supports writing through an Obsidian plugin.

Data Protection and Risks

Offline operation offers a solution for data protection concerns. However, it is important to check which models and interfaces actually operate locally only. DeepSeek has heavily influenced the open-source discourse, but has also received criticism due to potential state access in China to cloud services. The Czech NÚKIB officially warned against using DeepSeek products in critical infrastructures, as Chinese laws oblige companies to cooperate with authorities. Such concerns primarily affect hosted services, less so locally stored model weights. Nevertheless, it is important to check the tools used: Some graphical interfaces connect local models and cloud APIs by default within the same interface, and telemetry can send data even during pure localhost operation.

In regulated industries, local LLMs can bridge a gap: patient or customer data remains within their own network, while the model only accesses synthetic or anonymized information. The obligation to professionally and legally review every generated text remains; none of the current studies recommend deployment without human control.

Local writing AI has developed into a serious building block in the writing workflow. A well-configured „local llm for writing offline“ – – such as Llama 3.1 8B, Qwen3-14B or an efficient Gemma or Phi model – today delivers quality for drafts, variations, and structural suggestions for blogs, newsletters, or longform texts, without content leaving one's own infrastructure. Integration into editors like VS Code or Obsidian is crucial here. Those who have already set up a local programming assistance system can use the same infrastructure to write content. The model handles raw versions, variations, and suggestions, while structure, fact-checking, and the final polish remain with the user. Local writing AI acts as a productive co-author who never leaves the house.