Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
qwen3.8-flash-next-coder-iq1_m
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
ISTA-DASLab's Coder release (IQ1_M): half of the experts (256 of 512 per
layer) are kept, chosen on code, agent and image data, so it is smaller and
faster but weaker outside code and in languages other than English (Chinese
and other CJK text included) (about 59 GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 32 GB of RAM (23 GB of experts in RAM) with a suitable GPU, and
fast SSD storage: the 29 GB n-gram table stays on disk. Needs a GPU the strata
backend supports; NVIDIA with 12 GB of VRAM or more is the class Strata tests.
Runs only on the experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Coder GSQ-RCO quantization by ISTA-DASLab; Strata is by
Niko1221. The MTP draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-gyro-s
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
agentionai's Gyro-S quantization (trellis-coded experts), the smallest option,
for machines with little memory and lower in quality (about 59 GB to
download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 28 GB of VRAM and RAM together for the weights. Half of the file
is the n-gram table, which stays on disk and is read for every token, so keep
it on a fast local SSD. Needs a GPU the strata backend supports; NVIDIA with
12 GB of VRAM or more is the class Strata tests. Runs only on the experimental
strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF file is the Gyro quantization by agentionai; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.6-35b-a3b-ud-q4_k_xl-strata
Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with 3B active
parameters, in Unsloth's 4-bit (Unsloth Dynamic), the smallest GGUF with the
MTP layer (about 24 GB to download). It reads images through Unsloth's vision
projector, and the Strata backend parses its reasoning and tool calls.
Needs an NVIDIA or AMD GPU (the Intel Arc and Apple builds of Strata do not
run this model) and about 32 GB of RAM: the engine keeps every expert it can
in VRAM and computes the rest on the CPU. Runs only on the experimental strata
backend.
On the first load the strata backend exports the tokenizer and chat template
from the GGUF file (seconds); the engine reads the GGUF as it is. The MTP
draft layer inside the file is used for speculative decoding.
Licensed under the Apache License 2.0 (Qwen/Qwen3.6-35B-A3B). The GGUF files
are the Unsloth quantization; Strata is by Niko1221.
Repository: localaiLicense: apache-2.0
deepseek-v4.1-flash-ista-iq1_m
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 1.75 bits per weight GSQ-RCO
quantization (IQ1_M), the smallest size of the three (about 239 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 34 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Slightly lower quality than 2.00 bpw, and noticeably
weaker on plain English prose; ISTA-DASLab measured 98.7% of the base model on
its task average (AIME25, GPQA-Diamond, LiveCodeBench v6).
Needs about 240 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-coder-iq2_xxs
ISTA-DAS's coding version of DeepSeek-V4.1-Flash-1017: in every layer 192 of the
384 routed experts are kept and the other 192 removed, stored in the 2.00 bpw
(IQ2_XXS) layout. It is the smallest DeepSeek-V4.1 set here (about 191 GB to
download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 24 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 48% above the original
model on its test set (KL 0.40), so it is weaker than the full sets on prose and
general knowledge; it is meant for code and agent work.
Needs about 192 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
bge-small-en-v1.5-f16
BGE-small-en-v1.5 is a 33M English embedding model for semantic search
and retrieval. This F16 GGUF build produces 384-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
bge-small-en-v1.5-q8-0
BGE-small-en-v1.5 is a 33M English embedding model for semantic search
and retrieval. This Q8_0 GGUF build produces 384-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
bge-small-en-v1.5-q4-k-m
BGE-small-en-v1.5 is a 33M English embedding model for semantic search
and retrieval. This Q4_K_M GGUF build produces 384-dimensional embeddings
with a 512-token context. For retrieval, prefix queries with
"Represent this sentence for searching relevant passages: "; leave passages unprefixed.
Repository: localaiLicense: mit
qwen3.8-flash-next-q4
Qwen3.8-Flash-Next is Qwen's 125B-parameter, 6B-active experimental
vision-language mixture-of-experts model. It targets agentic coding,
reasoning, tool use, and long-context workloads with a native 262K-token
context window.
This default entry uses Unsloth's UD-Q4_K_XL GGUF and BF16 vision
projector. Linked variants offer Q8_0 and AtomicChat's smaller IQ4_XS
and Q4_K_M builds with a separate n-gram table shard.
Repository: localaiLicense: other
huihui-qwen3.8-27b-abliterated
Huihui Qwen3.8 27B is an abliterated vision-language model published by
huihui-ai. This BF16 GGUF build includes the shared BF16 vision projector
and enables MTP speculative decoding through llama.cpp. Q4_K, Q5_K, Q6_K,
and Q8_0 variants are available as smaller downloads.
Repository: localaiLicense: apache-2.0
ornith-1.5-35b-a3b-apex-compact
Ornith-1.5-35B-A3B in the smaller APEX Compact GGUF format, with the
shared BF16 vision projector for multimodal prompts.
Repository: localaiLicense: mit
thomson-1.0-small-q4
Thomson-1.0-Small is a 35B-parameter mixture-of-experts model with about
3B active parameters. It focuses on legal, tax, journalism, research,
reasoning, tool use, and document processing. It supports text and image
input with a native context window of 262K tokens.
This default entry uses the Q4_K_M GGUF and BF16 vision projector. A
higher-quality Q8_0 model is available as a variant.
Repository: localaiLicense: polyform-strict-1.0.0
thomson-1.0-small-q8
Thomson-1.0-Small in the higher-quality Q8_0 GGUF format, with the shared
BF16 vision projector for multimodal prompts.
Repository: localaiLicense: polyform-strict-1.0.0
qwen3.8-27b-gsq-rco-iq2-xs
Qwen3.8-27B with ISTA DASLab's smallest GSQ-RCO mixed quantization. The
2.50-bit IQ2_XS model uses 8.4 GB for the weights and retains the shared
BF16 vision projector.
Repository: localaiLicense: apache-2.0
qwen3.8-27b-gsq-rco-iq3-xxs
Qwen3.8-27B with ISTA DASLab's 3.00-bit GSQ-RCO IQ3_XXS mixed
quantization. The 10.1 GB model offers a smaller alternative to IQ3_S
and retains the shared BF16 vision projector.
Repository: localaiLicense: apache-2.0
qwen3.8-2b-q4
Qwen3.8-2B is Empero AI's smallest Qwen3.8 reasoning distillation. It uses
the Qwen3.5-2B architecture and targets mathematics, instruction
following, tool use, and edge deployment with a native 262K-token context
window. This default entry uses Q4_K_M weights; a higher-quality Q8_0
build is available as a variant.
Repository: localaiLicense: apache-2.0
nemotron-3.5-lightning-30b-a3b-q4
NVIDIA Nemotron 3.5 Lightning is a text-only hybrid Mamba-2, attention, and
mixture-of-experts model with 30B total parameters and 3B active parameters.
It targets reasoning, coding, tool use, multilingual chat, and long-context
agent workflows, with a context window of up to one million tokens.
This entry uses the official Q4_K_M GGUF. Automatic variant selection can
choose the smaller NVFP4 build or the higher-quality Q8_0 build when it fits.
Repository: localaiLicense: openmdw-1.1
nemotron-3.5-lightning-30b-a3b-nvfp4
NVIDIA Nemotron 3.5 Lightning 30B-A3B in the official NVFP4 GGUF format.
This is the smallest linked build and retains the model's reasoning, coding,
tool-use, multilingual, and long-context capabilities.
Repository: localaiLicense: openmdw-1.1
muse-glimmer-30b
Muse Glimmer is Meta Superintelligence Labs' Apache-2.0 dense 30B model for
autonomous agentic work, coding, tool use, long-horizon reasoning, and
multimodal understanding. It supports more than 100 languages, interleaved
text and image input through its 1.8B-parameter perception encoder, and a
131K-token context window.
This entry uses the publisher's higher-quality dynamic K-quant GGUF and
official quantized vision projector. Automatic variant selection can use
the smaller 17 GB quantization or a DFlash-accelerated build when it fits.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder. It preserves the model's agentic, coding, tool-use,
multilingual, and image-understanding capabilities for hosts with less
memory than the dynamic quantization requires.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb-dflash
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder and DFlash drafter. This is the lowest-memory published
build that retains image understanding and block-speculative decoding.
Flash attention is enabled for the DFlash path.
Repository: localaiLicense: apache-2.0
Page
1
of many