Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
qwen3.8-flash-next-unsloth-ud-q5_k_xl
Qwen3.8-Flash-Next, a 125B-parameter hybrid mixture-of-experts model, in
Unsloth's Dynamic quantization (UD-Q5_K_XL, 5-bit, close to Q8_0) (about 159
GB to download).
It reads images through the vision projector published for the original model,
and the Strata backend parses its reasoning and tool calls.
Needs about 108 GB of RAM to keep every expert in memory; with less, the
mapped experts are read from the SSD while it answers, which works and is much
slower. Keep the files on an SSD. Needs a GPU the strata backend supports;
NVIDIA with 12 GB of VRAM or more is the class Strata tests. Runs only on the
experimental strata backend.
On the first load the strata backend builds the Strata pack from the GGUF
files (a few minutes, a few GB of extra disk); a finished pack is never
rebuilt. The experts are read from the GGUF files in place (mapped); add
--resident-experts to engine_args to copy them into RAM when there is room.
Licensed under the Qwen Community License 1.0 (inherited from
Qwen/Qwen3.8-Flash-Next): free use, but a Model as a Service or AI Work
Assistant business needs a licence from Qwen. Read the LICENSE file in the
repository first.
The GGUF files are the Unsloth quantization; the vision projector is
ISTA-DASLab's file for the original model; Strata is by Niko1221. The MTP
draft layer is not configured here.
Repository: localaiLicense: qwen-community-1.0
qwen3.5-0.8b-artemis-q5
TurinTech Artemis Q5_K_M quantization of Qwen3.5 0.8B for text chat,
reasoning, and tool use. This text-only build omits the vision encoder
and MTP layer. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-2b-artemis-q5-k-m
TurinTech Artemis Q5_K_M quantization of Qwen3.5 2B for local tool calls,
maths, and instruction following. Accepts image input through the
bundled F16 vision projector. Uses the embedded chat template and an
8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-4b-artemis-q5-k-m
TurinTech Artemis Q5_K_M quantization of Qwen3.5 4B for local chat and
tool calls. Accepts image input through the bundled F16 vision
projector. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-4b-artemis-q5-k-s
TurinTech Artemis Q5_K_S quantization of Qwen3.5 4B for local chat and
tool calls. Accepts image input through the bundled F16 vision
projector. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
qwen3.5-9b-artemis-q5-k-s
TurinTech Artemis Q5_K_S quantization of Qwen3.5 9B for local chat and
tool calls. Accepts image input through the bundled F16 vision
projector. Uses the embedded chat template and an 8192-token context.
Repository: localaiLicense: apache-2.0
dirk-qwen3.8-27b-q5
Dirk in the higher-quality Q5_K_XL GGUF format, with MTP speculative
decoding and the shared F16 vision projector for multimodal prompts.
Repository: localaiLicense: apache-2.0
huihui-qwen3.8-27b-abliterated-q5
Huihui Qwen3.8 27B in Q5_K GGUF format, with the shared BF16 vision
projector and MTP speculative decoding through llama.cpp.
Repository: localaiLicense: apache-2.0
ornith-1.5-9b-q5
Ornith-1.5-9B in the Q5_K_M GGUF format, with the shared BF16
vision projector for multimodal prompts.
Repository: localaiLicense: mit
ornith-1.5-35b-a3b-q5
Ornith-1.5-35B-A3B in the Q5_K_M GGUF format, with the shared
BF16 vision projector for multimodal prompts.
Repository: localaiLicense: mit
nex-n2.5-mini-q5
Nex-N2.5-mini is a 35B-parameter Qwen3.5 MoE model for coding, tool use,
and computer and browser tasks with image input. This Q5_K_M GGUF build
includes the F16 vision projector and uses the embedded chat template.
Repository: localaiLicense: apache-2.0
tiel-coder-35b-a3b-q5-mtp
Tiel-Coder-35B-A3B in Q5_K_XL format with MTP speculative decoding and a
BF16 vision projector.
Repository: localaiLicense: mit
sharp-spark-x2.5-4b-q5
Sharp-Spark is an imatrix quantization of XHToken's Spark-X2.5-4B text model
with an adjusted chat template for coding. This Q5_K_XL build uses the
embedded Sharp-Spark template and a 32K-token default context.
Repository: localaiLicense: apache-2.0
qwen3.8-35b-a3b-distill-q5
Empero's Qwen3.8 distillation into Qwen3.6-35B-A3B has 35B total
parameters and about 3B active per token. This Q5_K_M GGUF build
uses the embedded reasoning template and includes the F16 vision
projector. Vision is inherited from the base and was not evaluated
by the publisher.
Repository: localaiLicense: apache-2.0
gemma-4-12b-it-q5
Gemma 4 12B IT is Google's 12B instruction-tuned model for chat,
reasoning, tool use, and image understanding. This Unsloth Q5_K_M
GGUF build includes the F16 vision projector and uses the embedded
chat template. It defaults to a 32,768-token context.
Repository: localaiLicense: apache-2.0
gemma-4-e4b-hauhaucs-aggressive-q5
HauhauCS Aggressive is an uncensored derivative of Google's Gemma 4 E4B
instruction model. This Q5_K_M GGUF supports text chat and image input
through the publisher's bundled F16 multimodal projector.
Uses the embedded chat template and the publisher's sampling settings.
The default context is 8192 tokens. The publisher declares the Gemma license.
Repository: localaiLicense: gemma
lythri-7b-a4b-q5
Lythri 7B A4B is a Gemma 4 E4B fine-tune for conversational companionship,
with 7.46B total parameters and 4.5B active parameters.
This Q5_K_M GGUF build runs text chat through llama.cpp with the embedded
chat template and a 32,768-token context.
Repository: localaiLicense: apache-2.0
zero-gemma4-e4b-openzero-q5-k-m
Zero Gemma4-E4B OpenZero is a Gemma 4 E4B instruction fine-tune for
local coding, research, chat, and agentic workflows. It was trained on
2,033 curated OpenZero examples and is distributed as one merged Q5_K_M
GGUF file.
License: OpenZero Community Source v1; see the model repository for terms.
Repository: localaiLicense: other
lfm2.5-24b-a2b-q5
LiquidAI LFM2.5-24B-A2B is a text-only reasoning model with 24B total
parameters and 2.45B active parameters for multilingual chat and tool use.
This entry uses the official Q5_K_M GGUF quantization.
Repository: localaiLicense: other
neohorse-1-4b-q5
NeoHorse-1-4B is TokenRhythm's text-only Qwen3.5-4B fine-tune for coding,
reasoning, and agentic tasks. This official Q5_K_M GGUF build uses the
embedded Jinja chat template and a 32K-token default context.
Repository: localaiLicense: apache-2.0
minimax-h3-fl2va-pruned-q5
MiniMax-H3 served by vllm.cpp, LocalAI's own C++ port of vLLM. It generates
video AND audio jointly, so a clip comes back as an MP4 with a soundtrack.
Ask for speech in the prompt and the model lip-syncs it.
Runtime: vllm.cpp (backend vllm-cpp). Partition: fl2va. DiT layout:
pruned. Format: Q5_0.
This entry is the FL2VA partition, which serves text-to-video (t2va) and first/last-frame conditioning (fl2va). It refuses reference conditioning: for that use the matching ref2va entry. The DiT is the pruned DiT (AdaLN curve table in place of the per-block modulation projection; about 20B instead of 33B parameters) (~13.9 GB).
Render status: loads, not rendered. The vllm.cpp model page says the pruned Q5_0 file loads through the same path as Q8_0; only Q8_0 has been rendered.
The vllm.cpp page records its renders with the DiT expanded to bf16. This
entry does not expand it and keeps the quantised weights resident, so a
render in that mode is not covered by the page.
The weights are one DiT, the Qwen3-VL-32B encoder (~14.6 GB), the video VAE
(~10.4 GB), the audio VAE and the tokenizer, plus the two VAE configs that
carry the latent statistics. The default canvas is 1344x768 at 124 frames
and 24 fps, about 5.2 seconds. Generation is slow: roughly 176 s per denoise
step at that canvas on a 20-SM device. Muxing the finished frames needs
ffmpeg on the host.
Repository: localaiLicense: other
Page
1
of many