Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
s1-mini-f16
S1-mini by Superwhisper in the publisher's 1.4 GB F16 GGUF format. This
variant preserves full model fidelity for hosts with enough memory.
Repository: localaiLicense: s1-mini-license
llm-jp-4-33b-thinking-bf16
LLM-jp-4-33B-thinking in the official 66.4 GB BF16 GGUF format. This
variant preserves the original model precision for hosts with enough
memory.
Repository: localaiLicense: apache-2.0
mxbai-embed-large-v1-f16
Mixedbread's mxbai-embed-large-v1 in the official full-precision F16
GGUF format. This 335M-parameter English BERT model produces
1,024-dimensional embeddings for retrieval, semantic search, and RAG.
Repository: localaiLicense: apache-2.0
ced-base-f16
CED (Consistent Ensemble Distillation, Xiaomi) is a sound-event classifier that tags everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark, ...) into the 527-class AudioSet ontology. This is the f16 GGUF for the ced backend (a standalone C++/ggml port). Recommended default: fastest on CPU and near-lossless. Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-tiny-f16
CED-tiny (5.5M params, Pi-class / edge) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-mini-f16
CED-mini (9.6M params, low-power) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-small-f16
CED-small (22M params, balanced size/accuracy) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
qwen3-coder-30b-a3b-vllm-cpp
Qwen3-Coder-30B-A3B on vllm.cpp: a coding and agentic-tool-use model, 30B
total parameters with about 3B active per token, gated token-exact against
vLLM on this engine.
The tool-call parser is named explicitly rather than auto-detected, and that
matters here. Qwen3-Coder's tool dialect is byte-identical on the wire to
another family's, so template sniffing cannot separate the two and would fall
back to the wrong parser. With qwen3_coder named, tool calls arrive as real
tool_calls on the OpenAI response.
This is the bf16 checkpoint, roughly 57 GB of weights, which is what the
engine was gated on. Being bf16 rather than NVFP4 it does not need Blackwell
on its own account, but LocalAI's CUDA images for this backend are currently
built for Blackwell-family GPUs only, so on an older card use the CPU build.
Repository: localaiLicense: apache-2.0
qwen3-4b-vllm-cpp
Qwen3-4B on vllm.cpp, in bf16. The small end of the engine's gated dense
family, which reaches parity with vLLM on every axis at concurrency 1.
bf16 rather than NVFP4 on purpose: this is the entry that runs where the
flagship NVFP4 checkpoints cannot, including Apple Silicon via Metal, Vulkan
and plain CPU. Roughly 8 GB of weights, plus about 4.5 GB of KV cache at the
context configured here.
Tool calling and the thinking split are parsed inside the engine.
Repository: localaiLicense: apache-2.0
qwen3-0.6b-vllm-cpp
Qwen3-0.6B on vllm.cpp, in bf16. Roughly 1.4 GB of weights, which makes it
the cheapest way to confirm a vllm-cpp install actually serves before
committing disk and memory to one of the large checkpoints.
It runs anywhere the backend does, CPU included, and it is a real chat model
rather than a stub, so tool calling and the thinking split can be exercised
on it too.
Repository: localaiLicense: apache-2.0
moss-tts-cpp-v1_5-f16
MOSS-TTS-Local v1.5 (C++/ggml, moss-tts.cpp), F16 (~9.4 GB), code-exact versus the
reference. 48kHz stereo text-to-speech with reference-audio voice cloning.
Repository: localaiLicense: apache-2.0
magpie-tts-cpp-357m-f16
Magpie TTS Multilingual 357M (C++/ggml, magpie-tts.cpp), F16 (~784 MB). 22.05kHz mono
text-to-speech with 5 baked voices and 9+ languages.
Repository: localaiLicense: other
allenai_olmo-3.1-32b-think
The **Olmo-3.1-32B-Think** model is a large language model (LLM) optimized for efficient inference using quantized versions. It is a quantized version of the original **allenai/Olmo-3.1-32B-Think** model, developed by **bartowski** using the **imatrix** quantization method.
### Key Features:
- **Base Model**: `allenai/Olmo-3.1-32B-Think` (unquantized version).
- **Quantized Versions**: Available in multiple formats (e.g., `Q6_K_L`, `Q4_1`, `bf16`) with varying precision (e.g., Q8_0, Q6_K_L, Q5_K_M). These are derived from the original model using the **imatrix calibration dataset**.
- **Performance**: Optimized for low-memory usage and efficient inference on GPUs/CPUs. Recommended quantization types include `Q6_K_L` (near-perfect quality) or `Q4_K_M` (default, balanced performance).
- **Downloads**: Available via Hugging Face CLI. Split into multiple files if needed for large models.
- **License**: Apache-2.0.
### Recommended Quantization:
- Use `Q6_K_L` for highest quality (near-perfect performance).
- Use `Q4_K_M` for balanced performance and size.
- Avoid lower-quality options (e.g., `Q3_K_S`) unless specific hardware constraints apply.
This model is ideal for deploying on GPUs/CPUs with limited memory, leveraging efficient quantization for practical use cases.
Repository: localaiLicense: apache-2.0
lightonocr-2-1b-f16
LightOnOCR-2-1B F16 is the full-precision GGUF build for optical character
recognition and multilingual document understanding. It pairs the F16
language model with the matching F16 vision projector.
Repository: localaiLicense: apache-2.0
hunyuan-ocr-bf16
HunyuanOCR in BF16 GGUF format for maximum model and vision-projector fidelity. It runs on llama.cpp and supports document parsing, text spotting, information extraction, and text-image translation.
Repository: localaiLicense: tencent-hunyuan-community
huihui-ai_huihui-gpt-oss-20b-bf16-abliterated
This is an uncensored version of unsloth/gpt-oss-20b-BF16 created with abliteration (see remove-refusals-with-transformers to know more about it).
Repository: localaiLicense: apache-2.0
depth-anything-3-base-f16
Depth Anything 3 (base), f16 — half precision (~233 MB), no measurable
accuracy loss vs f32. Depth + camera pose.
Repository: localaiLicense: apache-2.0
depth-anything-2-base-f16
Depth Anything V2 (base / ViT-B), f16 — half precision, no measurable
accuracy loss vs f32. Relative monocular depth only (no pose). Use
GenerateImage (src -> depth PNG) or the Depth endpoint.
Repository: localaiLicense: apache-2.0
qwen3-reranker-0.6b
The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
**Exceptional Versatility**: The embedding model has achieved state-of-the-art performance across a wide range of downstream application evaluations. The 8B size embedding model ranks No.1 in the MTEB multilingual leaderboard (as of June 5, 2025, score 70.58), while the reranking model excels in various text retrieval scenarios.
**Comprehensive Flexibility**: The Qwen3 Embedding series offers a full spectrum of sizes (from 0.6B to 8B) for both embedding and reranking models, catering to diverse use cases that prioritize efficiency and effectiveness. Developers can seamlessly combine these two modules. Additionally, the embedding model allows for flexible vector definitions across all dimensions, and both embedding and reranking models support user-defined instructions to enhance performance for specific tasks, languages, or scenarios.
**Multilingual Capability**: The Qwen3 Embedding series offer support for over 100 languages, thanks to the multilingual capabilites of Qwen3 models. This includes various programming languages, and provides robust multilingual, cross-lingual, and code retrieval capabilities.
**Qwen3-Reranker-0.6B** has the following features:
- Model Type: Text Reranking
- Supported Languages: 100+ Languages
- Number of Paramaters: 0.6B
- Context Length: 32k
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0, f16
Repository: localaiLicense: apache-2.0
qwen3-embedding-4b
The Qwen3 Embedding model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
**Exceptional Versatility**: The embedding model has achieved state-of-the-art performance across a wide range of downstream application evaluations. The 8B size embedding model ranks **No.1** in the MTEB multilingual leaderboard (as of June 5, 2025, score **70.58**), while the reranking model excels in various text retrieval scenarios.
**Comprehensive Flexibility**: The Qwen3 Embedding series offers a full spectrum of sizes (from 0.6B to 8B) for both embedding and reranking models, catering to diverse use cases that prioritize efficiency and effectiveness. Developers can seamlessly combine these two modules. Additionally, the embedding model allows for flexible vector definitions across all dimensions, and both embedding and reranking models support user-defined instructions to enhance performance for specific tasks, languages, or scenarios.
**Multilingual Capability**: The Qwen3 Embedding series offer support for over 100 languages, thanks to the multilingual capabilites of Qwen3 models. This includes various programming languages, and provides robust multilingual, cross-lingual, and code retrieval capabilities.
**Qwen3-Embedding-4B-GGUF** has the following features:
- Model Type: Text Embedding
- Supported Languages: 100+ Languages
- Number of Paramaters: 4B
- Context Length: 32k
- Embedding Dimension: Up to 2560, supports user-defined output dimensions ranging from 32 to 2560
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0, f16
Repository: localaiLicense: apache-2.0
qwen3-embedding-8b
The Qwen3 Embedding series model series is the latest proprietary model of the Qwen family, specifically designed for text embedding and ranking tasks. Building upon the dense foundational models of the Qwen3 series, it provides a comprehensive range of text embeddings and reranking models in various sizes (0.6B, 4B, and 8B). This series inherits the exceptional multilingual capabilities, long-text understanding, and reasoning skills of its foundational model. The Qwen3 Embedding series represents significant advancements in multiple text embedding and ranking tasks, including text retrieval, code retrieval, text classification, text clustering, and bitext mining.
**Exceptional Versatility**: The embedding model has achieved state-of-the-art performance across a wide range of downstream application evaluations. The 8B size embedding model ranks **No.1** in the MTEB multilingual leaderboard (as of June 5, 2025, score **70.58**), while the reranking model excels in various text retrieval scenarios.
**Comprehensive Flexibility**: The Qwen3 Embedding series offers a full spectrum of sizes (from 0.6B to 8B) for both embedding and reranking models, catering to diverse use cases that prioritize efficiency and effectiveness. Developers can seamlessly combine these two modules. Additionally, the embedding model allows for flexible vector definitions across all dimensions, and both embedding and reranking models support user-defined instructions to enhance performance for specific tasks, languages, or scenarios.
**Multilingual Capability**: The Qwen3 Embedding series offer support for over 100 languages, thanks to the multilingual capabilites of Qwen3 models. This includes various programming languages, and provides robust multilingual, cross-lingual, and code retrieval capabilities.
**Qwen3-Embedding-8B-GGUF** has the following features:
- Model Type: Text Embedding
- Supported Languages: 100+ Languages
- Number of Paramaters: 8B
- Context Length: 32k
- Embedding Dimension: Up to 4096, supports user-defined output dimensions ranging from 32 to 4096
- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0, f16
Repository: localaiLicense: apache-2.0
Page
1
of many