Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
qwen3.8-flash-next-unsloth-ud-iq4_xs
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
Unsloth's Dynamic quantization (UD-IQ4_XS, about 4-bit i-quant) (about 95 GB
to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 72 GB of RAM to keep every expert in memory; with less, the mapped
experts are read from the SSD while it answers, which works and is much
slower. Keep the files on an SSD. Needs a GPU the strata backend supports;
NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the
experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Unsloth quantization; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-unsloth-ud-q4_k_xl
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
Unsloth's Dynamic quantization (UD-Q4_K_XL, 4-bit) (about 112 GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 88 GB of RAM to keep every expert in memory; with less, the mapped
experts are read from the SSD while it answers, which works and is much
slower. Keep the files on an SSD. Needs a GPU the strata backend supports;
NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the
experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Unsloth quantization; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-unsloth-ud-q5_k_xl
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
Unsloth's Dynamic quantization (UD-Q5_K_XL, 5-bit, close to Q8_0) (about 159
GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 108 GB of RAM to keep every expert in memory; with less, the
mapped experts are read from the SSD while it answers, which works and is much
slower. Keep the files on an SSD. Needs a GPU the strata backend supports;
NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the
experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Unsloth quantization; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-unsloth-ud-q6_k_xl
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
Unsloth's Dynamic quantization (UD-Q6_K_XL, 6-bit) (about 170 GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 118 GB of RAM to keep every expert in memory; with less, the
mapped experts are read from the SSD while it answers, which works and is much
slower. Keep the files on an SSD. Needs a GPU the strata backend supports;
NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the
experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Unsloth quantization; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-unsloth-q8_0
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
Unsloth's Dynamic quantization (Q8_0, 8-bit, the reference quality) (about 189
GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 136 GB of RAM to keep every expert in memory; with less, the
mapped experts are read from the SSD while it answers, which works and is much
slower. Keep the files on an SSD. Needs a GPU the strata backend supports;
NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the
experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Unsloth quantization; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-gyro-s
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
agentionai's Gyro-S quantization (trellis-coded experts), the smallest option,
for machines with little memory and lower in quality (about 59 GB to
download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 28 GB of VRAM and RAM together for the weights. Half of the file
is the n-gram table, which stays on disk and is read for every token, so keep
it on a fast local SSD. Needs a GPU the strata backend supports; NVIDIA with
12 GB of VRAM or more is the class Strata tests. Runs only on the experimental
strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF file is the Gyro quantization by agentionai; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-gyro-m
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
agentionai's Gyro-M quantization (trellis-coded experts), about IQ3_XXS
quality with about 20% less GPU memory, 16 to 21% slower than Q2_0 (about 93
GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 35 GB of VRAM and RAM together for the weights. Half of the file
is the n-gram table, which stays on disk and is read for every token, so keep
it on a fast local SSD. Needs a GPU the strata backend supports; NVIDIA with
12 GB of VRAM or more is the class Strata tests. Runs only on the experimental
strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF file is the Gyro quantization by agentionai; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-nvfp4-gptq
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, with
every routed expert quantized to NVFP4 by GPTQ and packed for the Strata
inference engine (a Strata pack plus GGUF side files, about 129 GB to
download). It reads images through the vision projector published by
ISTA-DASLab for the original model, and the Strata backend parses its
reasoning and tool calls.
Needs an NVIDIA GPU with 12 GB of VRAM or more (RTX 20 series or newer, or a
DGX Spark) and about 80 GB of RAM (about 69 GiB of expert weights are kept in
memory), plus fast SSD storage for the 48 GiB n-gram table. Runs only on the
NVIDIA builds of the experimental strata backend.
The weights are the Strata NVFP4 conversion by Maximilian228 of
Qwen/Qwen3.8-Flash-Next, licensed under the Qwen Community License 1.0. The
MTP draft layer shipped in the repository is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.8-flash-next-nvfp4-nvidia
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
NVIDIA's NVFP4 quantization (nvidia/Qwen3.8-Flash-Next-NVFP4), converted to a
Strata pack with GGUF side files (about 129 GB to download). It reads images
through the vision projector published by ISTA-DASLab for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs an NVIDIA GPU with 12 GB of VRAM or more (RTX 20 series or newer, or a
DGX Spark) and about 80 GB of RAM (about 69 GiB of expert weights are kept in
memory), plus fast SSD storage for the 48 GiB n-gram table. Runs only on the
NVIDIA builds of the experimental strata backend.
Two licences apply: the NVIDIA Open Model License Agreement (NVIDIA's
quantization) and the Qwen Community License 1.0 (the base model). The
conversion to the Strata format is by Maximilian228. The MTP draft layer
shipped in the repository is not configured here.
Repository: localaiLicense: nvidia-open-model-license
hy4-preview
# Hy4-preview GGUF
Three GGUF builds of Hy4-Preview: https://huggingface.co/tencent/Hy4-preview
**Language / 语言:** English · 中文
**Neither file runs on stock llama.cpp.** The `hyv4` architecture is not upstream. Apply the
patches in `hy4-preview-patch/`
## English
### 1. What these are
**`Hy4-preview-Q4_K_M.gguf`** — a conventional Q4_K_M. Most tensors are Q4_K; `ffn_down_exps`
gets Q6_K on 37 layers via llama.cpp's own logic. Use this unless you are memory-constrained.
**`Hy4-preview-UD-IQ1_M.gguf`** - mixed precision with UD-IQ1_M strategy at ~2.44 bpw, roughly **half the size** for the
same model. The routed-expert `gate`/`up` projections run at 1.75 bpw (IQ1_M) and 2.0625 bpw (IQ2_XXS).
**`Hy4-preview-STQ1_0.gguf`** — mixed precision with MIX-STQ1_0 strategy at ~2.38 bpw, roughly **half the size** for the
same model. The routed-expert `gate`/`up` projections run at 1.3125 bpw (STQ1_0) on 29 layers and
2.0625 bpw (IQ2_XXS) on the other 48. See section 3.
### 2. Running them
Build a patched llama.cpp
```bash
git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git checkout 0cea36222
...
Repository: localai
s1-mini-f16
S1-mini by Superwhisper in the publisher's 1.4 GB F16 GGUF format. This
variant preserves full model fidelity for hosts with enough memory.
Repository: localaiLicense: s1-mini-license
llm-jp-4-33b-thinking-bf16
LLM-jp-4-33B-thinking in the official 66.4 GB BF16 GGUF format. This
variant preserves the original model precision for hosts with enough
memory.
Repository: localaiLicense: apache-2.0
qwen3.8-flash-next-q8
Qwen3.8-Flash-Next in the higher-quality Q8_0 GGUF format, with the shared
BF16 vision projector. This build preserves more model quality but needs
more memory than the default Q4 variant.
Repository: localaiLicense: other
qwen3.8-flash-next-atomic-iq4
Qwen3.8 Flash Next in AtomicChat's AD-3.84bpw IQ4_XS M64 GGUF
build, with the F16 vision projector. The n-gram table occupies a
separate shard. This entry enables memory mapping and disables llama.cpp
automatic parameter fitting as required by the publisher.
Repository: localaiLicense: other
qwen3.8-flash-next-atomic-q4
Qwen3.8 Flash Next in AtomicChat's AD-4.27bpw Q4_K_M M64 GGUF
build, with the F16 vision projector. The n-gram table occupies a
separate shard. This entry enables memory mapping and disables llama.cpp
automatic parameter fitting as required by the publisher.
Repository: localaiLicense: other
ling-3.0-tiny-q8
Ling-3.0-tiny in the higher-quality Q8_0 GGUF format. This variant
preserves more model fidelity for hosts with enough memory.
Repository: localaiLicense: mit
hy-mt2-1.8b-q8
Hy-MT2-1.8B in the higher-quality 1.9 GB Q8_0 GGUF format. This variant
preserves more model fidelity for hosts with enough memory.
Repository: localaiLicense: apache-2.0
ling-3.0-flash-iq2
Ling-3.0-flash in the higher-quality 44.7 GB AD-IQ2_XS GGUF format. This
variant preserves more model fidelity for hosts with enough memory.
Repository: localaiLicense: mit
qwen3.8-27b-q8
Qwen3.8-27B in the official Q8_0 GGUF format. This variant provides higher
model fidelity for hosts with enough memory.
Repository: localaiLicense: apache-2.0
underdog-saluki-27b-1.0
Underdog Saluki 27B 1.0 is a mixed IQ2 quantization of Qwen3.8-27B
for chat, reasoning, and tool use. This llama.cpp build includes the
F16 vision projector and uses the embedded chat template.
The weights and projector total approximately 8.83 GB before runtime memory.
Repository: localaiLicense: apache-2.0
qwen3.8-9b-q8
Qwen3.8-9B in the higher-quality Q8_0 GGUF format. This variant preserves
more model fidelity for hosts with enough memory.
Repository: localaiLicense: apache-2.0
Page
1
of many