Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
deepseek-v4.1-flash-ista-iq1_m
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 1.75 bits per weight GSQ-RCO
quantization (IQ1_M), the smallest size of the three (about 239 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 34 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Slightly lower quality than 2.00 bpw, and noticeably
weaker on plain English prose; ISTA-DASLab measured 98.7% of the base model on
its task average (AIME25, GPQA-Diamond, LiveCodeBench v6).
Needs about 240 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-iq2_xxs
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 2.00 bits per weight GSQ-RCO
quantization (IQ2_XXS), the middle size of the three (about 257 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 40 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 15.5% above the original
model on 4,995 positions (code 8.6%, agent work 5.3%).
Needs about 258 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-iq2_xs
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 2.25 bits per weight GSQ-RCO
quantization (IQ2_XS), the largest size of the three (about 274 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 44 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 7.3% above the original
model on 4,995 positions (code 3.2%, agent work 2.2%).
Needs about 275 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-coder-iq2_xxs
ISTA-DAS's coding version of DeepSeek-V4.1-Flash-1017: in every layer 192 of the
384 routed experts are kept and the other 192 removed, stored in the 2.00 bpw
(IQ2_XXS) layout. It is the smallest DeepSeek-V4.1 set here (about 191 GB to
download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 24 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 48% above the original
model on its test set (KL 0.40), so it is weaker than the full sets on prose and
general knowledge; it is meant for code and agent work.
Needs about 192 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-1017
# DeepSeek-V4.1-Flash-1017
Technical Report 👁️
## Introduction
**DeepSeek-V4.1-Flash-1017** is a minor update to **DeepSeek-V4.1-Flash**, delivering substantial improvements across reasoning, coding, agentic tasks, and visual benchmarks. Alongside these gains, the model reduces task execution time and output length, enabling users to complete tasks faster and at lower cost and advancing the performance–cost Pareto frontier across multiple agent harnesses.
*† denotes the text-only subset of HLE. We use the Claude Code harness for Terminal-Bench 4.0 and the Mini-SWE-V2 harness for DeepSWE v1.1.*
## Prompt Encoding
This release does not include a Jinja-format chat template. The `encoding` folder contains a self-contained Python reference implementation (`encoding.py`) with test cases for multi-turn conversations, tool calling, thinking mode, numeric reasoning effort, mid-conversation system messages, and interleaved image content.
...
Repository: localaiLicense: mit
deepseek-v4-flash-vision-exp
# DeepSeek-V4-Flash-Vision-Exp
## Introduction
We are excited to introduce **DeepSeek-V4-Flash-Vision-Exp**, our first experimental multimodal model in the DeepSeek-V4 family. It builds on the DeepSeek-V4-Flash architecture by incorporating visual modules and undergoing continued training to unlock visual understanding capabilities.
Compared to DeepSeek-V4-Flash-0731, DeepSeek-V4-Flash-Vision-Exp achieves substantial improvements on its multimodal agent capabilities, while maintaining comparable performance on text-only agent tasks.
Notes:
1. For the text agent benchmarks above, DeepSeek models are evaluated with the minimal mode of DeepSeek Harness as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.
2. † For ApexBench and Agents' Last Exam, DeepSeek-V4-Flash-0731 ignores the multimodal elements in the input.
## Repository layout
This repository contains the tokenizer, prompt encoding reference, and a
minimal PyTorch inference implementation for DeepSeek-V4 Flash Vision. The
reference inference covers the vision encoder and aligner, DFlash attention,
MoE, Hyper-Connections, and the DSpark forward path.
...
Repository: localaiLicense: mit
deepseek-v4-flash-0731
# DeepSeek-V4-Flash-0731
Technical Report👁️
## Introduction
**DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.
DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.
Notes:
1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.
2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.
## Chat Template
...
Repository: localaiLicense: mit
deepseek-v4-pro-0813
DeepSeek V4 Pro 0813 is DeepSeek's MIT-licensed flagship mixture-of-experts
model for agentic coding, reasoning, and long-horizon tool use. This entry
uses Unsloth's UD-Q4_K_XL GGUF build, split into 20 shards for llama.cpp.
Repository: localaiLicense: mit
deepseek-v4-flash
# DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Technical Report👁️
## Introduction
We present a preview version of **DeepSeek-V4** series, including two strong Mixture-of-Experts (MoE) language models — **DeepSeek-V4-Pro** with 1.6T parameters (49B activated) and **DeepSeek-V4-Flash** with 284B parameters (13B activated) — both supporting a context length of **one million tokens**.
DeepSeek-V4 series incorporate several key upgrades in architecture and optimization:
1. **Hybrid Attention Architecture:** We design a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) to dramatically improve long-context efficiency. In the 1M-token context setting, DeepSeek-V4-Pro requires only **27% of single-token inference FLOPs** and **10% of KV cache** compared with DeepSeek-V3.2.
2. **Manifold-Constrained Hyper-Connections (mHC):** We incorporate mHC to strengthen conventional residual connections, enhancing stability of signal propagation across layers while preserving model expressivity.
3. **Muon Optimizer:** We employ the Muon optimizer for faster convergence and greater training stability.
...
Repository: localaiLicense: mit
qwen3.5-9b-deepseek-v4-flash
# Qwen3.5-9B
[](https://chat.qwen.ai)
> [!Note]
> This repository contains model weights and configuration files for the post-trained model in the Hugging Face Transformers format.
>
> These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, KTransformers, etc.
Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance. Qwen3.5 represents a significant leap forward, integrating breakthroughs in multimodal learning, architectural efficiency, reinforcement learning scale, and global accessibility to empower developers and enterprises with unprecedented capability and efficiency.
## Qwen3.5 Highlights
Qwen3.5 features the following enhancement:
- **Unified Vision-Language Foundation**: Early fusion training on multimodal tokens achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.
- **Efficient Hybrid Architecture**: Gated Delta Networks combined with sparse Mixture-of-Experts deliver high-throughput inference with minimal latency and cost overhead.
...
Repository: localaiLicense: apache-2.0
deepseek-v4-flash-spark-exl3-vllm-cpp
DeepSeek V4 Flash's Spark and GB10-oriented REAP-K216 EXL3 checkpoint,
served by vllm.cpp. It needs CUDA and roughly 100 GiB for its large
rank-sliced checkpoint.
The repository is pinned to its latest recorded revision. vllm.cpp's
existing runtime evidence measured the older 22f28d32b9b29b4352eaa380ff8c2c170b2847ab
revision; this entry does not claim that the newer revision has passed the
same end-to-end gate.
Repository: localai
deepseek-v4-flash-exl3-3bpw-vllm-cpp
Experimental non-Spark DeepSeek V4 Flash EXL3 3.0bpw checkpoint served by
vllm.cpp. This is the complete, non-REAP layout and requires a large
multi-GPU CUDA system.
The publisher describes the artifact as structurally complete but has not
passed end-to-end generation. Treat this entry as an integration target,
not as a correctness- or performance-gated configuration.
Repository: localai
deepseek-v4-flash-q2
DeepSeek V4 Flash (IQ2XXS GGUF, ~81 GB) - only loadable via the ds4 backend.
Requires >=128 GB RAM. Metal (Darwin) or CUDA (Linux).
See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-flash-q2-q4
DeepSeek V4 Flash (mixed q2/q4 GGUF, ~91 GB) - only loadable via the ds4 backend.
The last 6 expert layers are kept at Q4_K (the rest IQ2XXS), trading a little
extra memory for higher quality than the pure-q2 build while still fitting in
RAM on a 128 GB machine. imatrix-tuned. Metal (Darwin) or CUDA (Linux).
See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-flash-q4-ssd
DeepSeek V4 Flash (full 4-bit experts GGUF, ~153 GB) - only loadable via the
ds4 backend, with SSD streaming enabled so it runs on a 128 GB machine even
though the weights do not fit in RAM: routed MoE experts stream from the GGUF
on SSD while the non-routed weights stay resident. SSD streaming is Metal
(Darwin) only; generation speed depends on SSD speed and the expert cache.
Tune the routed-expert cache with the 'ssd_streaming_cache_experts:NGB'
option (default: automatic budget). See https://github.com/antirez/ds4.
Repository: localai
deepseek-v4-flash-q2-mtp
DeepSeek V4 Flash (IQ2XXS GGUF, ~81 GB) paired with the optional MTP
speculative-decoding weights (~3.5 GB) for a slight speedup. Only loadable
via the ds4 backend; requires >=128 GB RAM. MTP helps only with greedy
decoding (temperature 0), so the override pins temperature to 0. Metal
(Darwin) or CUDA (Linux). See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-pro-q2-ssd
DeepSeek V4 Pro (IQ2XXS GGUF, ~433 GB, imatrix-tuned) - only loadable via the
ds4 backend, with SSD streaming so the Pro-class model can be run on a 128 GB
machine. This is experimental and slow: it needs ~433 GB of free SSD plus
enough RAM for the resident weights, KV cache, and routed-expert cache, and
is best used with thinking off for inspection or occasional work. SSD
streaming is Metal (Darwin) only. See https://github.com/antirez/ds4.
Repository: localai