Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
Attention: Trust Remote Code is required for this model
wemm-embedding-4b
WeMM-Embedding-4B is Tencent's mid-sized Apache-2.0 multilingual embedding
model built on Qwen3.5. This entry serves the original bfloat16 safetensors
with LocalAI's Transformers backend and produces 2,560-dimensional
normalized embeddings for text retrieval, semantic search, and RAG.
The upstream model can also embed images and videos. LocalAI currently
exposes text input through its embeddings API for this backend.
Repository: localaiLicense: apache-2.0
qwen3.8-4b-q4
Qwen3.8-4B is Empero AI's full-parameter distillation of Qwen3.8 2.4T
A95B into the Qwen3.5-4B architecture. It targets mathematics, reasoning,
instruction following, and tool use with a native 262K-token context
window. This default entry uses Q4_K_M weights; a higher-quality Q8_0
build is available as a variant.
Repository: localaiLicense: apache-2.0
qwen3.8-4b-q8
Qwen3.8-4B in the higher-quality Q8_0 GGUF format. This variant preserves
more model fidelity for hosts with enough memory.
Repository: localaiLicense: apache-2.0
qwen3.8-4b-distill-q4
Qwen3.8 4B Distill is an Apache-2.0, text-only Qwen3.5 4B fine-tune
distilled from Qwen3.8 2.4T A95B reasoning traces. It targets reasoning,
coding, instruction following, and function calling with a 262K native
context window. This entry uses the balanced Q4_K_M GGUF quantization; the
Q8_0 variant offers higher fidelity.
Repository: localaiLicense: apache-2.0
qwen3.8-4b-distill-q8
Qwen3.8 4B Distill in the higher-fidelity Q8_0 GGUF format. This text-only
Qwen3.5 4B fine-tune targets reasoning, coding, instruction following, and
function calling with a 262K native context window.
Repository: localaiLicense: apache-2.0
parable-qwen3-4b-claude-fable-5
# Parable-Qwen3-4B-Claude-Fable-5
Qwen3 4B fine-tuned on genuine Claude Fable 5 agent traces. Thinking-mode
reasoning, agent/terminal task flavor, tool-call formatting.
Repository: localaiLicense: apache-2.0
qwen3.6-14b-a3b-fablevibes
Qwen3.6-14B-A3B-FableVibes is an Apache-2.0 mixture-of-experts reasoning
model distilled from Fable 5 and Claude Opus traces, with additional tool
calling and coding data. It retains Qwen 3.6 vision support while pruning
the 35B-A3B base to a 14B consumer-oriented footprint. This default entry
uses the recommended Q4_K_M GGUF quantization and its Q8_0 multimodal
projector.
Repository: localaiLicense: apache-2.0
qwen3.6-14b-a3b-fablevibes-q8
Qwen3.6-14B-A3B-FableVibes is an Apache-2.0 mixture-of-experts reasoning
model distilled from Fable 5 and Claude Opus traces, with additional tool
calling and coding data. This entry uses the near-lossless Q8_0 GGUF
quantization and its matching Q8_0 multimodal projector.
Repository: localaiLicense: apache-2.0
agents-a1-4b
Agents-A1-4B is InternScience's Apache-2.0 dense 4B agentic model, based on
Qwen3.5. It is trained for long-horizon search, engineering and scientific
research, instruction following, tool use, and multimodal tasks. This entry
uses the official Q4_K_M GGUF quantization and vision projector.
Repository: localaiLicense: apache-2.0
agents-a1-4b-q8
Agents-A1-4B is InternScience's Apache-2.0 dense 4B agentic model, based on
Qwen3.5. It is trained for long-horizon search, engineering and scientific
research, instruction following, tool use, and multimodal tasks. This entry
uses the official Q8_0 GGUF quantization and vision projector.
Repository: localaiLicense: apache-2.0
qwen3-4b-dflash
Qwen3-4B paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend. This is the canonical DFlash pairing documented upstream (`z-lab/Qwen3-4B-DFlash` + `Qwen/Qwen3-4B`).
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3-4B target; the ~0.5 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0 (Qwen3-4B target) / MIT (z-lab DFlash drafter).
Repository: localaiLicense: apache-2.0
qwen3.5-4b-dflash
Qwen3.5-4B paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3.5-4B target; the ~0.6 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0 (Qwen3.5-4B target) / MIT (z-lab DFlash drafter).
Repository: localaiLicense: apache-2.0
serenity-26b-a4b
.mc-wrap{background:#0d1117;color:#c9d1d9;font-family:'Inter',sans-serif;max-width:920px;margin:0 auto;padding:24px;border-radius:16px;box-sizing:border-box}
.mc-wrap *{box-sizing:border-box}
.mc-wrap h1,.mc-wrap h2,.mc-wrap h3,.mc-wrap h4{color:#e6edf3;border:none}
.mc-wrap p{color:#c9d1d9}
.mc-wrap strong{color:#7ee8d0}
.mc-wrap a{color:#7ee8d0;text-decoration:none}
.mc-wrap ul{list-style:none;padding-left:0;margin:0}
.mc-wrap li{color:#c9d1d9;margin-bottom:8px;padding-left:4px}
.mc-wrap code{background:#161b22;color:#7ee8d0;padding:2px 8px;border-radius:4px;font-family:'JetBrains Mono',monospace;font-size:.88em;border:1px solid rgba(126,232,208,.15)}
.mc-hdr{text-align:center;padding:40px 32px;background:#0d1117;border:1px solid #21262d;border-radius:24px;margin-bottom:20px;position:relative;overflow:hidden}
.mc-hdr::before{content:'';position:absolute;top:0;left:0;right:0;height:3px;background:linear-gradient(135deg,#7ee8d0,#a78bfa,#c4b5fd)}
.mc-name{font-family:'Space Grotesk',sans-serif;font-size:2.8em;font-weight:800;margin:0;letter-spacing:-.02em;background:linear-gradient(135deg,#7ee8d0,#a78bfa,#c4b5fd);-webkit-background-clip:text;-webkit-text-fill-color:transparent;backg
...
Repository: localaiLicense: apache-2.0
melody1437-26b-a4b-v2.0
@import url('https://fonts.googleapis.com/css2?family=Poppins:wght@400;600&family=Playfair+Display:ital,wght@0,400;0,700&family=Roboto+Mono:wght@400;500&display=swap');
body {
font-family: 'Poppins', sans-serif;
background: #1a1a2e;
background-image:
radial-gradient(circle at 50% 50%, rgba(76, 201, 240, 0.05) 0%, transparent 70%),
url('https://www.transparenttextures.com/patterns/cubes.png');
color: #e0e0e0;
margin: 0;
padding: 20px;
line-height: 1.6;
}
.container {
max-width: 900px;
margin: 0 auto;
background: rgba(26, 32, 44, 0.95);
border-radius: 8px;
padding: 40px;
box-shadow: 0 4px 30px rgba(0, 0, 0, 0.5), 0 0 0 1px #2a3b55;
border: 1px solid #2a3b55;
position: relative;
overflow: hidden;
backdrop-filter: blur(5px);
}
.header {
text-align: center;
margin-bottom: 30px;
position: relative;
z-index: 1;
border-bottom: 1px solid #2a3b55;
padding-bottom: 15px;
}
...
Repository: localaiLicense: apache-2.0
dark-scarlett-v0.3-26b-a4b
Hugging Face |
GitHub |
Launch Blog |
Documentation
License: Apache 2.0 | Authors: Google DeepMind
Gemma is a family of open models built by Google DeepMind. Gemma 4 models are multimodal, handling text and image input (with audio supported on small models) and generating text output. This release includes open-weights models in both pre-trained and instruction-tuned variants. Gemma 4 features a context window of up to 256K tokens and maintains multilingual support in over 140 languages.
Featuring both Dense and Mixture-of-Experts (MoE) architectures, Gemma 4 is well-suited for tasks like text generation, coding, and reasoning. The models are available in four distinct sizes: **E2B**, **E4B**, **26B A4B**, and **31B**. Their diverse sizes make them deployable in environments ranging from high-end phones to laptops and servers, democratizing access to state-of-the-art AI.
Gemma 4 introduces key **capability and architectural advancements**:
* **Reasoning** – All models in the family are designed as highly capable reasoners, with configurable thinking modes.
...
Repository: localaiLicense: apache-2.0
gemma-4-26b-a4b-it-qat
Hugging Face |
GitHub |
Launch Blog |
Documentation
License: Apache 2.0 | Authors: Google DeepMind
> [!Note]
> This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model.
> Four versions of the QAT checkpoints are available:
> * **Unquantized QAT checkpoints** (Q4_0): Half-precision weights extracted from the QAT pipeline, ideal for custom downstream compilation and research. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B, and their drafter models.
> * **GGUF** (Q4_0): Ready-to-deploy formats for broad ecosystem compatibility. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B.
> * **Mobile-optimized** (wNa8o8): A custom schema engineered explicitly for mobile hardware efficiency. It features targeted 2-bit decoding layers, optimized KV caches, and static activations to maximize VRAM savings. Available for Gemma 4 E2B and E4B.
> * **Compressed Tensors** (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B
...
Repository: localaiLicense: apache-2.0
gemma-4-e4b-it-qat-q4_0
Gemma 4 E4B is a multimodal (text + image) instruction-tuned model from Google DeepMind, optimized with Quantization-Aware Training (QAT) to preserve bfloat16-level quality at a fraction of the memory. E4B is a MatFormer "effective 4B" elastic variant, balancing quality and footprint for on-device and edge deployments. This is the official Google Q4_0 GGUF, shipped with its multimodal projector.
License: Apache 2.0 | Authors: Google DeepMind
Repository: localaiLicense: apache-2.0
zero-gemma4-e4b-openzero-q5-k-m
Zero Gemma4-E4B OpenZero is a Gemma 4 E4B instruction fine-tune for
local coding, research, chat, and agentic workflows. It was trained on
2,033 curated OpenZero examples and is distributed as one merged Q5_K_M
GGUF file.
License: OpenZero Community Source v1; see the model repository for terms.
Repository: localaiLicense: other
gemma-4-26b-a4b-it-qat-q4_0
Gemma 4 26B-A4B is a multimodal (text + image) instruction-tuned Mixture-of-Experts model from Google DeepMind, optimized with Quantization-Aware Training (QAT) to preserve bfloat16-level quality at a fraction of the memory. With 26B total parameters and ~4B active per token, it delivers large-model quality at a much lower inference cost. This is the official Google Q4_0 GGUF, shipped with its multimodal projector.
License: Apache 2.0 | Authors: Google DeepMind
Repository: localaiLicense: apache-2.0
gemma-4-e4b-it-qat-mtp
Gemma 4 E4B IT QAT (Google DeepMind) paired with its Multi-Token Prediction (MTP) drafter head for speculative decoding on the llama.cpp backend. The Q4_K_XL target carries the full multimodal (text + image) model; the small `mtp-gemma-4-E4B-it` head predicts several tokens ahead which the target verifies in parallel, accelerating generation with no change to output quality. E4B is a MatFormer "effective 4B" elastic variant, balancing quality and footprint for on-device and edge deployments.
The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here. It uses the upstream `gemma4-assistant` architecture registered by llama.cpp PR #23398, so it loads on stock llama.cpp without any patch.
License: Apache 2.0 | Authors: Google DeepMind (target/drafter checkpoints), Unsloth (GGUF conversion)
Repository: localaiLicense: apache-2.0
gemma-4-26b-a4b-it-apex
AI model: gemma-4-26b-a4b-it-apex
Repository: localaiLicense: gemma
Page
1
of many