Model Gallery
Discover and install AI models from our curated collection
Find Your Perfect Model
Filter by Model Type
Browse by Tags
deepseek-v4.1-flash-ista-iq1_m
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 1.75 bits per weight GSQ-RCO
quantization (IQ1_M), the smallest size of the three (about 239 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 34 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Slightly lower quality than 2.00 bpw, and noticeably
weaker on plain English prose; ISTA-DASLab measured 98.7% of the base model on
its task average (AIME25, GPQA-Diamond, LiveCodeBench v6).
Needs about 240 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-iq2_xxs
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 2.00 bits per weight GSQ-RCO
quantization (IQ2_XXS), the middle size of the three (about 257 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 40 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 15.5% above the original
model on 4,995 positions (code 8.6%, agent work 5.3%).
Needs about 258 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-iq2_xs
DeepSeek-V4.1-Flash-1017, a 763B-parameter mixture-of-experts model (384 experts
per layer, 6 used per token), in ISTA-DAS's 2.25 bits per weight GSQ-RCO
quantization (IQ2_XS), the largest size of the three (about 274 GB to download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 44 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 7.3% above the original
model on 4,995 positions (code 3.2%, agent work 2.2%).
Needs about 275 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
deepseek-v4.1-flash-ista-coder-iq2_xxs
ISTA-DAS's coding version of DeepSeek-V4.1-Flash-1017: in every layer 192 of the
384 routed experts are kept and the other 192 removed, stored in the 2.00 bpw
(IQ2_XXS) layout. It is the smallest DeepSeek-V4.1 set here (about 191 GB to
download).
It writes a reasoning trace before the answer, and the Strata backend splits it
from the reply and parses the model's DSML tool calls. Text only: images are not
supported by this backend.
Needs a GPU with 12 GB of VRAM or more and about 24 GB of RAM at the least
(Strata's figure; its measurements use 46 to 64 GB: an RTX 5070 with 64 GB
answered at about 5 tokens per second, an RTX 3060 with 46 GB at 8 to 10). Less
RAM still runs, slowly. Strata measured a perplexity 48% above the original
model on its test set (KL 0.40), so it is weaker than the full sets on prose and
general knowledge; it is meant for code and agent work.
Needs about 192 GB of free disk, on a fast NVMe drive: the 111 GB n-gram table
(Engram) stays on disk and is read for every token. The Engram table and the 8
GB DSpark draft model are shared by all DeepSeek-V4.1 entries and downloaded
once into strata/models/deepseek-v4.1-flash-shared.
Runs only on the experimental strata backend, on NVIDIA CUDA 13 (x86_64 and
arm64), AMD Radeon RX 9070 and R9700 (ROCm) and Apple Silicon (Metal). Strata
ships no DeepSeek engine for CUDA 12, Intel or other AMD cards.
On the first load the strata backend builds the Strata pack from the GGUF files
(under a minute, a few MB of extra disk); loading the model afterwards takes a
few minutes. The default context is 8192 tokens, the size Strata checked; raise
context_size if the GPU has room.
Licensed under the MIT License (DeepSeek-V4.1-Flash-1017 and the quantized
files). The GGUF files are the GSQ-RCO quantization by ISTA-DAS (ISTA-DASLab);
the DSpark draft model is Unsloth's; Strata is by Niko1221.
Repository: localaiLicense: mit
qwythos-9b-v2
Empero AI
# Qwythos-9B-v2 — the new and improved Qwythos
The next iteration of Qwythos: **all the reasoning of Qwythos-9B, with the looping behavior fixed.** v2 keeps the deep chain-of-thought, the uncensored research posture, and the 1M-token context of its predecessor, and cleans up the rough edges that showed up in real use.
- 🔁 **Looping behavior eliminated** — repetition/degeneration under greedy or low-temperature decoding dropped from **6.7% → 0%**. You can serve it *without* leaning on `repetition_penalty` as a band-aid.
- 🧠 **Reasoning fully preserved** — MMLU, GSM8K, GPQA, ARC and HumanEval are all held at (or above) the v1 level. This is a *hygiene* upgrade, not a capability regression.
- 🧩 **MTP head restored** — the native multi-token-prediction module (dropped in the previous export) is back, so config and weights agree and speculative-decoding setups work.
- 🪪 **Cleaner identity** — the model no longer prefaces unrelated answers with its identity; it introduces itself only when you actually ask.
- 🔓 **Still intentionally uncensored** for research, cybersecurity, red-teaming, biology, chemistry, pharmacology and clinical work.
- 📜 **St
...
Repository: localaiLicense: apache-2.0
qwen3.6-27b-fable-fusion-711-uncensored-heretic-nm-dau-neo-max-mtp
Important: This is the first fine tune to exceed 700 "arc-c" (The OpenAI, Claude and Gemini "zone of intelligence") in both 8 bit and 4 bit. This repo contains both "regular" and "MTP" Neo MAX Imatrix quants.
Qwen3.6-27B-Fable-Fusion-711-Uncensored-Heretic-NM-DAU-NEO-MAX-MTP-GGUF
The strongest, smartest open source multi-stage model fine tune for consumer hardware ever and BUILT on consumer hardware via Unsloth.
The first model of this size/type to breach "700" ARC-C in both 8 bit and 4 bit; hench the "711" in the name.
This model (both 4 bit and 8 bit) exceeds the base Qwen 3.6 27B in 6 out of 7 benchmarks, and matches it on the 7th AND exceeds all 7 benchmarks for Qwen3.6-35B-A3B.
The 700 "intelligence club" is reserved for OpenAI, Claude and Gemini closed source models.
This is the one they fear.
This is a multi-stage fine tune, multi-fine tune, and multi-stage merge.
A Colab between myself (multiple fine tunes, including multi-stage), Nightmedia (merge/benching), TeichAI (Polaris Dataset), armand0e (Light fable 5 traces) and trohrbaugh (heretic'ing the model).
...
Repository: localaiLicense: apache-2.0
kimi-k2.6
🤗 huggingchat
|
📰 Tech Blog
## 1. Model Introduction
Kimi K2.6 is an open-source, native multimodal agentic model that advances practical capabilities in long-horizon coding, coding-driven design, proactive autonomous execution, and swarm-based task orchestration.
### Key Features
- **Long-Horizon Coding**: K2.6 achieves significant improvements on complex, end-to-end coding tasks, generalizing robustly across programming languages (Rust, Go, Python) and domains spanning front-end, DevOps, and performance optimization.
- **Coding-Driven Design**: K2.6 is capable of transforming simple prompts and visual inputs into production-ready interfaces and lightweight full-stack workflows, generating structured layouts, interactive elements, and rich animations with deliberate aesthetic precision.
- **Elevated Agent Swarm**: Scaling horizontally to 300 sub-agents executing 4,000 coordinated steps, K2.6 can dynamically decompose tasks into parallel, domain-specialized subtasks, delivering end-to-end outputs from documents to websites to spreadsheets in a single autonomous run.
- **Proactive & Open Orchestration**: For autonomous tasks, K2.6 demonstra
...
Repository: localaiLicense: modified-mit
ced-base-f16
CED (Consistent Ensemble Distillation, Xiaomi) is a sound-event classifier that tags everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark, ...) into the 527-class AudioSet ontology. This is the f16 GGUF for the ced backend (a standalone C++/ggml port). Recommended default: fastest on CPU and near-lossless. Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-base-q8
CED (Consistent Ensemble Distillation, Xiaomi) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). This is the q8_0 GGUF for the ced backend: smallest footprint (~88 MB, ~6.5x less memory than the PyTorch reference) and near-lossless (identical top-5 tags). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-tiny-f16
CED-tiny (5.5M params, Pi-class / edge) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-tiny-q8
CED-tiny (5.5M params, Pi-class / edge) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). q8_0 GGUF for the ced backend (smallest footprint, near-lossless). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-mini-f16
CED-mini (9.6M params, low-power) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-mini-q8
CED-mini (9.6M params, low-power) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). q8_0 GGUF for the ced backend (smallest footprint, near-lossless). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-small-f16
CED-small (22M params, balanced size/accuracy) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-small-q8
CED-small (22M params, balanced size/accuracy) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). q8_0 GGUF for the ced backend (smallest footprint, near-lossless). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
gpt-oss-20b
Welcome to the gpt-oss series, OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases.
We’re releasing two flavors of the open models:
gpt-oss-120b — for production, general purpose, high reasoning use cases that fits into a single H100 GPU (117B parameters with 5.1B active parameters)
gpt-oss-20b — for lower latency, and local or specialized use cases (21B parameters with 3.6B active parameters)
Both models were trained on our harmony response format and should only be used with the harmony format as it will not work correctly otherwise.
This model card is dedicated to the smaller gpt-oss-20b model. Check out gpt-oss-120b for the larger model.
Highlights
Permissive Apache 2.0 license: Build freely without copyleft restrictions or patent risk—ideal for experimentation, customization, and commercial deployment.
Configurable reasoning effort: Easily adjust the reasoning effort (low, medium, high) based on your specific use case and latency needs.
Full chain-of-thought: Gain complete access to the model’s reasoning process, facilitating easier debugging and increased trust in outputs. It’s not intended to be shown to end users.
Fine-tunable: Fully customize models to your specific use case through parameter fine-tuning.
Agentic capabilities: Use the models’ native capabilities for function calling, web browsing, Python code execution, and Structured Outputs.
Native MXFP4 quantization: The models are trained with native MXFP4 precision for the MoE layer, making gpt-oss-120b run on a single H100 GPU and the gpt-oss-20b model run within 16GB of memory.
Repository: localaiLicense: apache-2.0
gpt-oss-120b
Welcome to the gpt-oss series, OpenAI’s open-weight models designed for powerful reasoning, agentic tasks, and versatile developer use cases.
We’re releasing two flavors of the open models:
gpt-oss-120b — for production, general purpose, high reasoning use cases that fits into a single H100 GPU (117B parameters with 5.1B active parameters)
gpt-oss-20b — for lower latency, and local or specialized use cases (21B parameters with 3.6B active parameters)
Both models were trained on our harmony response format and should only be used with the harmony format as it will not work correctly otherwise.
This model card is dedicated to the smaller gpt-oss-20b model. Check out gpt-oss-120b for the larger model.
Highlights
Permissive Apache 2.0 license: Build freely without copyleft restrictions or patent risk—ideal for experimentation, customization, and commercial deployment.
Configurable reasoning effort: Easily adjust the reasoning effort (low, medium, high) based on your specific use case and latency needs.
Full chain-of-thought: Gain complete access to the model’s reasoning process, facilitating easier debugging and increased trust in outputs. It’s not intended to be shown to end users.
Fine-tunable: Fully customize models to your specific use case through parameter fine-tuning.
Agentic capabilities: Use the models’ native capabilities for function calling, web browsing, Python code execution, and Structured Outputs.
Native MXFP4 quantization: The models are trained with native MXFP4 precision for the MoE layer, making gpt-oss-120b run on a single H100 GPU and the gpt-oss-20b model run within 16GB of memory.
Repository: localaiLicense: apache-2.0
qwen3-8b-jailbroken
This jailbroken LLM is released strictly for academic research purposes in AI safety and model alignment studies. The author bears no responsibility for any misuse or harm resulting from the deployment of this model. Users must comply with all applicable laws and ethical guidelines when conducting research.
A jailbroken Qwen3-8B model using weight orthogonalization[1].
Implementation script: https://gist.github.com/cooperleong00/14d9304ba0a4b8dba91b60a873752d25
[1]: Arditi, Andy, et al. "Refusal in language models is mediated by a single direction." arXiv preprint arXiv:2406.11717 (2024).
Repository: localaiLicense: apache-2.0
minicpm-v-4_6
MiniCPM-V 4.6 is the most edge-deployment-friendly model in the MiniCPM-V series, with a total of 1.3B parameters. Built on Qwen3.5-0.8B and SigLIP2-400M, it features ultra-efficient architecture with mixed 4x/16x visual token compression for on-device deployment on iOS, Android, and HarmonyOS.
Repository: localaiLicense: apache-2.0
dans-personalityengine-v1.0.0-8b
This model is intended to be multifarious in its capabilities and should be quite capable at both co-writing and roleplay as well as find itself quite at home performing sentiment analysis or summarization as part of a pipeline. It has been trained on a wide array of one shot instructions, multi turn instructions, role playing scenarios, text adventure games, co-writing, and much more. The full dataset is publicly available and can be found in the datasets section of the model page.
There has not been any form of harmfulness alignment done on this model, please take the appropriate precautions when using it in a production environment.
Repository: localaiLicense: apache-2.0
llama-3.1-8b-arliai-formax-v1.0-iq-arm-imatrix
Quants for ArliAI/Llama-3.1-8B-ArliAI-Formax-v1.0.
"Formax is a model that specializes in following response format instructions. Tell it the format of it's response and it will follow it perfectly. Great for data processing and dataset creation tasks."
"It is also a highly uncensored model that will follow your instructions very well."
Repository: localaiLicense: unlicense
Page
1
of many