Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
qwen3.6-35b-a3b-ud-q4_k_xl-strata
Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with 3B active
parameters, in Unsloth's 4-bit (Unsloth Dynamic), the smallest GGUF with the
MTP layer (about 24 GB to download). It reads images through Unsloth's vision
projector, and the Strata backend parses its reasoning and tool calls.
Needs an NVIDIA or AMD GPU (the Intel Arc and Apple builds of Strata do not
run this model) and about 32 GB of RAM: the engine keeps every expert it can
in VRAM and computes the rest on the CPU. Runs only on the experimental strata
backend.
On the first load the strata backend exports the tokenizer and chat template
from the GGUF file (seconds); the engine reads the GGUF as it is. The MTP
draft layer inside the file is used for speculative decoding.
Licensed under the Apache License 2.0 (Qwen/Qwen3.6-35B-A3B). The GGUF files
are the Unsloth quantization; Strata is by Niko1221.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-ud-q6_k_xl-strata
Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with 3B active
parameters, in Unsloth's 6-bit (Unsloth Dynamic), closer to the full model
GGUF with the MTP layer (about 34 GB to download). It reads images through
Unsloth's vision projector, and the Strata backend parses its reasoning and
tool calls.
Needs an NVIDIA or AMD GPU (the Intel Arc and Apple builds of Strata do not
run this model) and about 44 GB of RAM: the engine keeps every expert it can
in VRAM and computes the rest on the CPU. Runs only on the experimental strata
backend.
On the first load the strata backend exports the tokenizer and chat template
from the GGUF file (seconds); the engine reads the GGUF as it is. The MTP
draft layer inside the file is used for speculative decoding.
Licensed under the Apache License 2.0 (Qwen/Qwen3.6-35B-A3B). The GGUF files
are the Unsloth quantization; Strata is by Niko1221.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-q8_0-strata
Qwen3.6-35B-A3B, a 35B-parameter mixture-of-experts model with 3B active
parameters, in Unsloth's 8-bit, the closest to the full model GGUF with the
MTP layer (about 39 GB to download). It reads images through Unsloth's vision
projector, and the Strata backend parses its reasoning and tool calls.
Needs an NVIDIA or AMD GPU (the Intel Arc and Apple builds of Strata do not
run this model) and about 52 GB of RAM: the engine keeps every expert it can
in VRAM and computes the rest on the CPU. Runs only on the experimental strata
backend.
On the first load the strata backend exports the tokenizer and chat template
from the GGUF file (seconds); the engine reads the GGUF as it is. The MTP
draft layer inside the file is used for speculative decoding.
Licensed under the Apache License 2.0 (Qwen/Qwen3.6-35B-A3B). The GGUF files
are the Unsloth quantization; Strata is by Niko1221.
Repository: localaiLicense: apache-2.0
nl2sh-1.5b-q4
nl2sh-1.5b is a 1.5B Qwen2.5-Coder fine-tune that converts plain-English
requests into single POSIX or Bash commands. This Q4_K_M GGUF is 941 MB
and is designed for fast CPU inference.
Use the system prompt from the model card and review every generated
command before execution. The model can produce destructive commands and
cannot inspect the local filesystem.
Repository: localaiLicense: apache-2.0
lfm2.5-1.2b-instruct
Try LFM • Docs • LEAP • Discord
# LFM2.5-1.2B-Instruct
LFM2.5 is a new family of hybrid models designed for **on-device deployment**. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.
- **Best-in-class performance**: A 1.2B model rivaling much larger models, bringing high-quality AI to your pocket.
- **Fast edge inference**: 239 tok/s decode on AMD CPU, 82 tok/s on mobile NPU. Runs under 1GB of memory with day-one support for llama.cpp, MLX, and vLLM.
- **Scaled training**: Extended pre-training from 10T to 28T tokens and large-scale multi-stage reinforcement learning.
Find more information about LFM2.5 in our blog post.
## 🗒️ Model Details
LFM2.5-1.2B-Instruct is a general-purpose text-only model with the following features:
...
Repository: localaiLicense: other
lfm2.5-8b-a1b
Try LFM •
Docs •
LEAP •
Discord
# LFM2.5-8B-A1B
LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.
- **On-device personal assistant**: Designed to power real-life applications, chaining tool calls, and following complex instructions on all devices.
- **Compressed performance**: Competitive with much larger dense and MoE models on instruction following and agentic tasks.
- **Unmatched throughput**: Fastest in its size class on both CPU and GPU inference, with day-one support for llama.cpp, MLX, vLLM, and SGLang.
Find more information about LFM2.5-8B-A1B in our blog post.
**AA-Omniscience Index (higher is better) rewards correct answers and penalizes hallucinations. Scores range from -100 to 100. See more results on Artificial Analysis.*
## 🗒️ Model Details
LFM2.5-8B-A1B is a general-purpose text-only model with the following features:
...
Repository: localaiLicense: other
maple-preview-tq1-0-head-q4-k
Maple-Preview is DeepGrove's 20B mixture-of-experts reasoning model
with about 1B active parameters. This build uses TQ1_0
ternary GGUF weights with a Q4_K output head.
It uses the publisher's CPU configuration and embedded Jinja chat
template with an 8K default context. The native context is 128K tokens.
Repository: localaiLicense: mit
maple-preview-tq1-0-head-f16
Maple-Preview is DeepGrove's 20B mixture-of-experts reasoning model
with about 1B active parameters. This build uses TQ1_0
ternary GGUF weights with a F16 output head.
It uses the publisher's CPU configuration and embedded Jinja chat
template with an 8K default context. The native context is 128K tokens.
Repository: localaiLicense: mit
maple-preview-tq2-0-head-q4-k
Maple-Preview is DeepGrove's 20B mixture-of-experts reasoning model
with about 1B active parameters. This build uses TQ2_0
ternary GGUF weights with a Q4_K output head.
It uses the publisher's CPU configuration and embedded Jinja chat
template with an 8K default context. The native context is 128K tokens.
Repository: localaiLicense: mit
maple-preview-tq2-0-head-f16
Maple-Preview is DeepGrove's 20B mixture-of-experts reasoning model
with about 1B active parameters. This build uses TQ2_0
ternary GGUF weights with a F16 output head.
It uses the publisher's CPU configuration and embedded Jinja chat
template with an 8K default context. The native context is 128K tokens.
Repository: localaiLicense: mit
ced-base-f16
CED (Consistent Ensemble Distillation, Xiaomi) is a sound-event classifier that tags everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark, ...) into the 527-class AudioSet ontology. This is the f16 GGUF for the ced backend (a standalone C++/ggml port). Recommended default: fastest on CPU and near-lossless. Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-tiny-f16
CED-tiny (5.5M params, Pi-class / edge) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-mini-f16
CED-mini (9.6M params, low-power) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-small-f16
CED-small (22M params, balanced size/accuracy) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
longcat-video-avatar-1.5
LongCat-Video-Avatar-1.5 served by LocalAI's dedicated CUDA backend. Turns
speech plus a prompt into an avatar video, optionally conditioning on a
portrait, and continues across multiple segments for longer audio.
Avatar generation also loads tokenizer, text encoder, and VAE components
from LongCat-Video. Plan for very large downloads and substantial NVIDIA
GPU or unified memory; CPU and macOS execution are unsupported.
Repository: localaiLicense: mit
qwen3-coder-30b-a3b-vllm-cpp
Qwen3-Coder-30B-A3B on vllm.cpp: a coding and agentic-tool-use model, 30B
total parameters with about 3B active per token, gated token-exact against
vLLM on this engine.
The tool-call parser is named explicitly rather than auto-detected, and that
matters here. Qwen3-Coder's tool dialect is byte-identical on the wire to
another family's, so template sniffing cannot separate the two and would fall
back to the wrong parser. With qwen3_coder named, tool calls arrive as real
tool_calls on the OpenAI response.
This is the bf16 checkpoint, roughly 57 GB of weights, which is what the
engine was gated on. Being bf16 rather than NVFP4 it does not need Blackwell
on its own account, but LocalAI's CUDA images for this backend are currently
built for Blackwell-family GPUs only, so on an older card use the CPU build.
Repository: localaiLicense: apache-2.0
qwen3-4b-vllm-cpp
Qwen3-4B on vllm.cpp, in bf16. The small end of the engine's gated dense
family, which reaches parity with vLLM on every axis at concurrency 1.
bf16 rather than NVFP4 on purpose: this is the entry that runs where the
flagship NVFP4 checkpoints cannot, including Apple Silicon via Metal, Vulkan
and plain CPU. Roughly 8 GB of weights, plus about 4.5 GB of KV cache at the
context configured here.
Tool calling and the thinking split are parsed inside the engine.
Repository: localaiLicense: apache-2.0
qwen3-0.6b-vllm-cpp
Qwen3-0.6B on vllm.cpp, in bf16. Roughly 1.4 GB of weights, which makes it
the cheapest way to confirm a vllm-cpp install actually serves before
committing disk and memory to one of the large checkpoints.
It runs anywhere the backend does, CPU included, and it is a real chat model
rather than a stub, so tool calling and the thinking split can be exercised
on it too.
Repository: localaiLicense: apache-2.0
qwen3-streaming-assistant
English voice assistant preset: Silero VAD, Parakeet TDT 0.6B v3, Qwen3
0.6B chat, and Qwen3-TTS 0.6B CustomVoice with Ryan's voice. Streams LLM
text into one persistent TTS session while audio is emitted.
This installs only the pipeline configuration. First install the separate
gallery models silero-vad-ggml, parakeet-cpp-tdt-0.6b-v3, qwen3-0.6b, and
qwen3-tts-cpp-customvoice. Pin its variant to qwen3-tts-cpp-customvoice
for Q8. WebRTC also needs the opus backend.
Requires a LocalAI build with incremental Qwen3-TTS input support.
CPU synthesis may be slower than playback; GPU and browser performance
are not established by this preset.
Repository: localaiLicense: mit
moss-tts-cpp-v1_5-q8_0
MOSS-TTS-Local v1.5 (C++/ggml, moss-tts.cpp), Q8_0 (~6 GB), near-lossless. Native
C++ text-to-speech for the OpenMOSS MOSS-TTS-Local Transformer v1.5 (a GPT-J local
transformer decoded through the MOSS-Audio-Tokenizer-v2 neural codec), 48kHz stereo
output with reference-audio voice cloning (set `voice` to a reference .wav). Verified
per component against the reference PyTorch and about 2x faster per frame on CPU.
Repository: localaiLicense: apache-2.0
allenai_olmo-3.1-32b-think
The **Olmo-3.1-32B-Think** model is a large language model (LLM) optimized for efficient inference using quantized versions. It is a quantized version of the original **allenai/Olmo-3.1-32B-Think** model, developed by **bartowski** using the **imatrix** quantization method.
### Key Features:
- **Base Model**: `allenai/Olmo-3.1-32B-Think` (unquantized version).
- **Quantized Versions**: Available in multiple formats (e.g., `Q6_K_L`, `Q4_1`, `bf16`) with varying precision (e.g., Q8_0, Q6_K_L, Q5_K_M). These are derived from the original model using the **imatrix calibration dataset**.
- **Performance**: Optimized for low-memory usage and efficient inference on GPUs/CPUs. Recommended quantization types include `Q6_K_L` (near-perfect quality) or `Q4_K_M` (default, balanced performance).
- **Downloads**: Available via Hugging Face CLI. Split into multiple files if needed for large models.
- **License**: Apache-2.0.
### Recommended Quantization:
- Use `Q6_K_L` for highest quality (near-perfect performance).
- Use `Q4_K_M` for balanced performance and size.
- Avoid lower-quality options (e.g., `Q3_K_S`) unless specific hardware constraints apply.
This model is ideal for deploying on GPUs/CPUs with limited memory, leveraging efficient quantization for practical use cases.
Repository: localaiLicense: apache-2.0
Page
1
of many