Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
longcat-video
LongCat-Video served by LocalAI's dedicated CUDA backend. Generates video
from a text prompt or a start image. The SDPA attention path works without
FlashAttention and is suitable for CUDA 13 ARM64 systems such as DGX Spark.
This is a very large checkpoint (roughly 83 GB in Hugging Face storage) and
requires Linux with an NVIDIA CUDA GPU plus substantial memory and disk.
Repository: localaiLicense: mit
longcat-video-avatar-1.5
LongCat-Video-Avatar-1.5 served by LocalAI's dedicated CUDA backend. Turns
speech plus a prompt into an avatar video, optionally conditioning on a
portrait, and continues across multiple segments for longer audio.
Avatar generation also loads tokenizer, text encoder, and VAE components
from LongCat-Video. Plan for very large downloads and substantial NVIDIA
GPU or unified memory; CPU and macOS execution are unsupported.
Repository: localaiLicense: mit
qwen3.8-27b-exl3-vllm-cpp
Qwen3.8-27B EXL3 3.5bpw served by vllm.cpp, LocalAI's C++ vLLM-style
runtime. The published checkpoint generates on CUDA and measured 16.7
tokens/s on GB10 with the pinned revision and the limits configured here.
This is the target-only setup. Use the DFlash2 variant for the measured
speculative-decoding configuration. The entry downloads the complete
revision-pinned repository, including its configuration, tokenizer, index,
and safetensors shards.
Repository: localai
deepseek-v4-flash-spark-exl3-vllm-cpp
DeepSeek V4 Flash's Spark and GB10-oriented REAP-K216 EXL3 checkpoint,
served by vllm.cpp. It needs CUDA and roughly 100 GiB for its large
rank-sliced checkpoint.
The repository is pinned to its latest recorded revision. vllm.cpp's
existing runtime evidence measured the older 22f28d32b9b29b4352eaa380ff8c2c170b2847ab
revision; this entry does not claim that the newer revision has passed the
same end-to-end gate.
Repository: localai
deepseek-v4-flash-exl3-3bpw-vllm-cpp
Experimental non-Spark DeepSeek V4 Flash EXL3 3.0bpw checkpoint served by
vllm.cpp. This is the complete, non-REAP layout and requires a large
multi-GPU CUDA system.
The publisher describes the artifact as structurally complete but has not
passed end-to-end generation. Treat this entry as an integration target,
not as a correctness- or performance-gated configuration.
Repository: localai
qwen3-coder-30b-a3b-vllm-cpp
Qwen3-Coder-30B-A3B on vllm.cpp: a coding and agentic-tool-use model, 30B
total parameters with about 3B active per token, gated token-exact against
vLLM on this engine.
The tool-call parser is named explicitly rather than auto-detected, and that
matters here. Qwen3-Coder's tool dialect is byte-identical on the wire to
another family's, so template sniffing cannot separate the two and would fall
back to the wrong parser. With qwen3_coder named, tool calls arrive as real
tool_calls on the OpenAI response.
This is the bf16 checkpoint, roughly 57 GB of weights, which is what the
engine was gated on. Being bf16 rather than NVFP4 it does not need Blackwell
on its own account, but LocalAI's CUDA images for this backend are currently
built for Blackwell-family GPUs only, so on an older card use the CPU build.
Repository: localaiLicense: apache-2.0
qwen3-tts-llamacpp
Qwen3-TTS 1.7B Base served by the llama.cpp backend, using upstream's own
GGUF conversion. Runs on the full llama-cpp accelerator matrix (CUDA, ROCm,
SYCL, Vulkan, Metal). Streaming output and zero-shot voice cloning: set
`voice` to a reference clip or a saved Voice Library profile, which is
required since the Base checkpoint has no built-in speaker. 24kHz mono,
10 languages. Q8_0 backbone (~1.8 GB) plus a Q8_0 projector.
Repository: localaiLicense: apache-2.0
llm-compiler-13b-imat
LLM Compiler is a state-of-the-art LLM that builds upon Code Llama with improved performance for code optimization and compiler reasoning.
LLM Compiler is free for both research and commercial use.
LLM Compiler is available in two flavors:
LLM Compiler, the foundational models, pretrained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimizations;
and LLM Compiler FTD, which is further fine-tuned to predict the best optimizations for code in LLVM assembly to reduce code size, and to disassemble assembly code to LLVM-IR.
Repository: localaiLicense: other
llm-compiler-13b-ftd
LLM Compiler is a state-of-the-art LLM that builds upon Code Llama with improved performance for code optimization and compiler reasoning.
LLM Compiler is free for both research and commercial use.
LLM Compiler is available in two flavors:
LLM Compiler, the foundational models, pretrained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimizations;
and LLM Compiler FTD, which is further fine-tuned to predict the best optimizations for code in LLVM assembly to reduce code size, and to disassemble assembly code to LLVM-IR.
Repository: localaiLicense: other
llm-compiler-7b-imat-GGUF
LLM Compiler is a state-of-the-art LLM that builds upon Code Llama with improved performance for code optimization and compiler reasoning.
LLM Compiler is free for both research and commercial use.
LLM Compiler is available in two flavors:
LLM Compiler, the foundational models, pretrained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimizations;
and LLM Compiler FTD, which is further fine-tuned to predict the best optimizations for code in LLVM assembly to reduce code size, and to disassemble assembly code to LLVM-IR.
Repository: localaiLicense: other
llm-compiler-7b-ftd-imat
LLM Compiler is a state-of-the-art LLM that builds upon Code Llama with improved performance for code optimization and compiler reasoning.
LLM Compiler is free for both research and commercial use.
LLM Compiler is available in two flavors:
LLM Compiler, the foundational models, pretrained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimizations;
and LLM Compiler FTD, which is further fine-tuned to predict the best optimizations for code in LLVM assembly to reduce code size, and to disassemble assembly code to LLVM-IR.
Repository: localaiLicense: other
deepseek-v4-flash-q2
DeepSeek V4 Flash (IQ2XXS GGUF, ~81 GB) - only loadable via the ds4 backend.
Requires >=128 GB RAM. Metal (Darwin) or CUDA (Linux).
See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-flash-q2-q4
DeepSeek V4 Flash (mixed q2/q4 GGUF, ~91 GB) - only loadable via the ds4 backend.
The last 6 expert layers are kept at Q4_K (the rest IQ2XXS), trading a little
extra memory for higher quality than the pure-q2 build while still fitting in
RAM on a 128 GB machine. imatrix-tuned. Metal (Darwin) or CUDA (Linux).
See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-flash-q2-mtp
DeepSeek V4 Flash (IQ2XXS GGUF, ~81 GB) paired with the optional MTP
speculative-decoding weights (~3.5 GB) for a slight speedup. Only loadable
via the ds4 backend; requires >=128 GB RAM. MTP helps only with greedy
decoding (temperature 0), so the override pins temperature to 0. Metal
(Darwin) or CUDA (Linux). See https://github.com/antirez/ds4 for details.
Repository: localai