Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
qwen3.8-27b-turbo-fable-cold-fusion-735-882-heretic-uncensored-neo-coder-max-mtp
RELEASE #1 GGUFS [including detailed notes, how to use, benches and much more]:
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NEO-CODER-MAX-MTP-GGUF
Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU (release #1, others pending...)
( repo has 10+ other versions (and 3 branches) noted below that EXCEED the performance of all QWEN 27B models, including fine tunes. )
First, special thanks to Nightmedia for working on the first three stages prior to heretic'ing/post staging and benching everything (3 sections below).
A number of my finetunes - both released and non-released - were used here as well as some third parties. Full details will be disclosed upon final release as the project shores up.
THREE example generations [snippets] from STAGE1-PART2, STAGE1b-PART2 and STAGE2-rplus2 at the bottom of the page.
Release(s) will be GGUFS first (linked here directly) then source code shortly thereafter [now released/open].
Some additional work and/ spawning of new branches from branch(es) below is still going on.
NEW: Branch 3 added, see below.
COMPLETED AND PENDING RELEASES:
...
Repository: localaiLicense: apache-2.0
nemotron-3-embed-1b-q4
Nemotron-3-Embed-1B is NVIDIA's multilingual text embedding model for
retrieval, semantic search, and RAG. This compact Q4_K_M GGUF produces
2,048-dimensional normalized embeddings and supports 36 languages. Prefix
retrieval queries with `query: ` and documents with `passage: `.
Repository: localaiLicense: openmdw-1.1
minicpm5-1b-claude-opus-fable5-v2-thinking
# MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking
GGUF quantizations for local deployment: **MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-GGUF**
中文说明
**MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking** is a compact 1B **Thinking** language model built on openbmb/MiniCPM5-1B. Compared with V1, this V2 release is further fine-tuned on **Fable 5** data with a stronger focus on **tool calling / function calling**, while also improving **coding** and **instruction-following**. It keeps MiniCPM5's native Thinking chat template and XML tool-call format.
Previous version: **MiniCPM5-1B-Claude-Opus-Fable5-Thinking** (V1)
For llama.cpp / Ollama / LM Studio deployment, see the **GGUF repository**.
## Overview
## Capabilities
- **Tool calling (enhanced in V2)** — more reliable XML / function-calling style tool use on top of MiniCPM5's native format
- **Coding** — code generation, debugging, and software-engineering-style tasks
- **Instruction following** — more reliable adherence to user prompts and structured constraints
- **Thinking mode** — chain-of-thought reasoning via the MiniCPM5 chat template
- **Long context** — up to **128K tokens** (131,072 tokens per `config.json`)
...
Repository: localaiLicense: apache-2.0
bonsai-8b-1bit
Bonsai 8B (PrismML) is an end-to-end 1-bit language model built on the Qwen3-8B dense
architecture (GQA, SwiGLU, RoPE, RMSNorm, 36 layers, 65,536 context). Every weight is a
single sign bit (`-scale` / `+scale`) with one FP16 scale per group of 128 weights, for
an effective 1.125 bits/weight and a ~1.15 GB footprint (14.2x smaller than FP16) while
matching full-precision 8B instruct models at ~70.5 average across 6 benchmark
categories.
The Q1_0 quantization is only decodable by the PrismML llama.cpp fork, so this entry
runs on LocalAI's `bonsai` backend (that fork), not the stock `llama-cpp` backend.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
bonsai-27b-1bit
Bonsai 27B (PrismML) is a full 27B-class reasoning model in end-to-end 1-bit weights,
derived from the Qwen3.6-27B hybrid-attention backbone (~75% linear attention, 262K
context). At a true 1.125 bits/weight it deploys in ~3.9 GB (~14.2x smaller than FP16)
while retaining 89.5% of FP16 intelligence across 15 thinking-mode benchmarks (math
91.66, coding 81.88). Ships an optional 4-bit vision tower (mmproj) for image input,
included here.
The Q1_0_g128 weights and hybrid-attention kernels are only in the PrismML llama.cpp
fork, so this runs on LocalAI's `bonsai` backend. A GPU is recommended. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
minicpm5-1b-claude-opus-fable5-thinking
# MiniCPM5-1B-Claude-Opus-Fable5-Thinking
GGUF quantizations for local deployment: **MiniCPM5-1B-Claude-Opus-Fable5-Thinking-GGUF**
中文说明
**MiniCPM5-1B-Claude-Opus-Fable5-Thinking** is a compact 1B **Thinking** language model built on openbmb/MiniCPM5-1B. It is further fine-tuned on **Fable 5** data to improve **coding** and **instruction-following** while keeping MiniCPM5's native Thinking chat template and tool-call format.
For llama.cpp / Ollama / LM Studio deployment, see the **GGUF repository**.
## Overview
## Capabilities
- **Coding** — code generation, debugging, and software-engineering-style tasks
- **Instruction following** — more reliable adherence to user prompts and structured constraints
- **Thinking mode** — chain-of-thought reasoning via the MiniCPM5 chat template
- **Tool calling** — inherits MiniCPM5's XML tool-call format
- **Long context** — up to **128K tokens** (131,072 tokens per `config.json`)
## Quick start
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "GnLOLot/MiniCPM5-1B-Claude-Opus-Fable5-Thinking"
...
Repository: localaiLicense: apache-2.0
serenity-26b-a4b
.mc-wrap{background:#0d1117;color:#c9d1d9;font-family:'Inter',sans-serif;max-width:920px;margin:0 auto;padding:24px;border-radius:16px;box-sizing:border-box}
.mc-wrap *{box-sizing:border-box}
.mc-wrap h1,.mc-wrap h2,.mc-wrap h3,.mc-wrap h4{color:#e6edf3;border:none}
.mc-wrap p{color:#c9d1d9}
.mc-wrap strong{color:#7ee8d0}
.mc-wrap a{color:#7ee8d0;text-decoration:none}
.mc-wrap ul{list-style:none;padding-left:0;margin:0}
.mc-wrap li{color:#c9d1d9;margin-bottom:8px;padding-left:4px}
.mc-wrap code{background:#161b22;color:#7ee8d0;padding:2px 8px;border-radius:4px;font-family:'JetBrains Mono',monospace;font-size:.88em;border:1px solid rgba(126,232,208,.15)}
.mc-hdr{text-align:center;padding:40px 32px;background:#0d1117;border:1px solid #21262d;border-radius:24px;margin-bottom:20px;position:relative;overflow:hidden}
.mc-hdr::before{content:'';position:absolute;top:0;left:0;right:0;height:3px;background:linear-gradient(135deg,#7ee8d0,#a78bfa,#c4b5fd)}
.mc-name{font-family:'Space Grotesk',sans-serif;font-size:2.8em;font-weight:800;margin:0;letter-spacing:-.02em;background:linear-gradient(135deg,#7ee8d0,#a78bfa,#c4b5fd);-webkit-background-clip:text;-webkit-text-fill-color:transparent;backg
...
Repository: localaiLicense: apache-2.0
gemma-4-31b-scotoma-2-q4
Gemma 4 31B Scotoma 2 is a multimodal Gemma 4 31B instruction-tuned model from ReadyArt. It applies a bounded refusal edit and preference training intended to reduce repetitive prose patterns while retaining the base model's text and image capabilities. This entry uses the 18.7 GB Q4_K_M GGUF and the matching Q8_0 vision projector.
License: Apache 2.0 | Base model: Google Gemma 4 31B IT
Repository: localaiLicense: apache-2.0
gemma-4-31b-scotoma-2-q8
Gemma 4 31B Scotoma 2 is a multimodal Gemma 4 31B instruction-tuned model from ReadyArt. It applies a bounded refusal edit and preference training intended to reduce repetitive prose patterns while retaining the base model's text and image capabilities. This higher-fidelity entry uses the 32.6 GB Q8_0 GGUF and the matching Q8_0 vision projector.
License: Apache 2.0 | Base model: Google Gemma 4 31B IT
Repository: localaiLicense: apache-2.0
gemma-4-31b-it-qat-q4_0
Gemma 4 31B is the largest dense multimodal (text + image) instruction-tuned model in the Gemma 4 family from Google DeepMind, optimized with Quantization-Aware Training (QAT) to preserve bfloat16-level quality while dramatically reducing the memory required to load the model. This is the official Google Q4_0 GGUF, shipped with its multimodal projector.
License: Apache 2.0 | Authors: Google DeepMind
Repository: localaiLicense: apache-2.0
gemma-4-31b-it-qat-mtp
Gemma 4 31B IT QAT (Google DeepMind), the largest dense model in the family, paired with its Multi-Token Prediction (MTP) drafter head for speculative decoding on the llama.cpp backend. The Q4_K_XL target carries the full multimodal (text + image) model; the small `mtp-gemma-4-31B-it` head predicts several tokens ahead which the target verifies in parallel, accelerating generation with no change to output quality. Dense models like 31B are the sizes that benefit most from MTP.
The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here. It uses the upstream `gemma4-assistant` architecture registered by llama.cpp PR #23398, so it loads on stock llama.cpp without any patch.
License: Apache 2.0 | Authors: Google DeepMind (target/drafter checkpoints), Unsloth (GGUF conversion)
Repository: localaiLicense: apache-2.0
lfm2.5-8b-a1b
Try LFM •
Docs •
LEAP •
Discord
# LFM2.5-8B-A1B
LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.
- **On-device personal assistant**: Designed to power real-life applications, chaining tool calls, and following complex instructions on all devices.
- **Compressed performance**: Competitive with much larger dense and MoE models on instruction following and agentic tasks.
- **Unmatched throughput**: Fastest in its size class on both CPU and GPU inference, with day-one support for llama.cpp, MLX, vLLM, and SGLang.
Find more information about LFM2.5-8B-A1B in our blog post.
**AA-Omniscience Index (higher is better) rewards correct answers and penalizes hallucinations. Scores range from -100 to 100. See more results on Artificial Analysis.*
## 🗒️ Model Details
LFM2.5-8B-A1B is a general-purpose text-only model with the following features:
...
Repository: localaiLicense: other
lfm2.5-8b-a1b-dspark
LFM2.5-8B-A1B with LiquidAI's DSpark speculative drafter. This build pairs
the Q4_K_M target with the compact Q4_K_M draft sidecar for lower-memory
hosts. DSpark proposes blocks of tokens that the target model verifies,
which preserves the target model's output while accelerating generation.
Repository: localaiLicense: other
lfm2.5-8b-a1b-q8-dspark
LFM2.5-8B-A1B with LiquidAI's DSpark speculative drafter. This build pairs
the higher-quality Q8_0 target with the recommended F16 draft sidecar for
the best acceptance length. DSpark proposes blocks of tokens that the
target model verifies, which preserves the target model's output while
accelerating generation.
Repository: localaiLicense: other
gemma-4-31b-it
Google Gemma 4 31B-IT is the largest dense model in the Gemma 4 family with 31B parameters. It handles text and image input, generating text output, with a 256K context window and support for 140+ languages. Provides the highest quality outputs in the Gemma 4 lineup, well-suited for complex reasoning, summarization, and image understanding tasks.
Repository: localaiLicense: apache-2.0
lightonocr-2-1b
LightOnOCR-2-1B is a compact Apache-2.0 vision-language model for optical
character recognition and document understanding. It extracts text,
tables, forms, and structured content from images and PDFs in multiple
languages. This default entry uses the Q8_0 model and vision projector.
Repository: localaiLicense: apache-2.0
lightonocr-2-1b-f16
LightOnOCR-2-1B F16 is the full-precision GGUF build for optical character
recognition and multilingual document understanding. It pairs the F16
language model with the matching F16 vision projector.
Repository: localaiLicense: apache-2.0
antares-1b
Antares-1B is an Apache-2.0 Granite 4.0 model specialized for vulnerability
localization in real-world codebases. It operates as a terminal agent that
navigates repositories, inspects source files, and identifies vulnerable
file paths. This entry uses the Q4_K_M GGUF quantization.
Repository: localaiLicense: apache-2.0
antares-1b-q8
Antares-1B is an Apache-2.0 Granite 4.0 model specialized for vulnerability
localization in real-world codebases. It operates as a terminal agent that
navigates repositories, inspects source files, and identifies vulnerable
file paths. This entry uses the higher-quality Q8_0 GGUF quantization.
Repository: localaiLicense: apache-2.0
baidu_ernie-4.5-21b-a3b-thinking
Over the past three months, we have continued to scale the thinking capability of ERNIE-4.5-21B-A3B, improving both the quality and depth of reasoning, thereby advancing the competitiveness of ERNIE lightweight models in complex reasoning tasks. We are pleased to introduce ERNIE-4.5-21B-A3B-Thinking, featuring the following key enhancements:
Significantly improved performance on reasoning tasks, including logical reasoning, mathematics, science, coding, text generation, and academic benchmarks that typically require human expertise.
Efficient tool usage capabilities.
Enhanced 128K long-context understanding capabilities.
Note: This version has an increased thinking length. We strongly recommend its use in highly complex reasoning tasks. ERNIE-4.5-21B-A3B-Thinking is a text MoE post-trained model, with 21B total parameters and 3B activated parameters for each token.
Repository: localaiLicense: apache-2.0
liquidai_lfm2-8b-a1b
LFM2 is a new generation of hybrid models developed by Liquid AI, specifically designed for edge AI and on-device deployment. It sets a new standard in terms of quality, speed, and memory efficiency.
We're releasing the weights of our first MoE based on LFM2, with 8.3B total parameters and 1.5B active parameters.
LFM2-8B-A1B is the best on-device MoE in terms of both quality (comparable to 3-4B dense models) and speed (faster than Qwen3-1.7B).
Code and knowledge capabilities are significantly improved compared to LFM2-2.6B.
Quantized variants fit comfortably on high-end phones, tablets, and laptops.
Repository: localaiLicense: lfm1.0
Page
1
of many