Conway Research Shrinks Qwen 27B Model to 8 GB

Conway Research has launched Underdog Saluki 27B 1.0, a 2-bit IQ2 GGUF quantization of the Qwen3.8-27B model. By compressing the original 54 GB footprint down to just 7.89 GB, the free, Apache 2.0-licensed release brings a large model within consumer hardware memory limits. It runs out of the box on stock llama.cpp, Ollama, LM Studio, and vLLM, and can be paired with an optional 629 to 928 MB mmproj vision add-on to accept image inputs. The release has already logged over 15,000 downloads on Hugging Face.

Despite the severe compression, Conway’s evaluations show Saluki surpassing the full-size baseline on function execution. Tested on Underdog Bench, a 120-task subset of the BFCL v4 leaderboard, Saluki passed 88 tasks compared to 84 for the original 54 GB model. It completed 42 parallel tool calls versus 35 by the full baseline and retained a 96% average score across nine benchmarks. By comparison, the competing 5.95 GB Bonsai 2 model completed 70 of the 120 tasks; Saluki uses roughly 1.94 GB more storage to achieve 18 additional passes.

The aggressive quantization process relies on specialized calibration data to preserve precise tool-calling formatting, but it introduces trade-offs in other reasoning areas. The model experiences a 15% to 18% performance drop in competition math, along with declines in code generation and letter-level puzzles, plus occasional formatting slips. However, instruction-following capabilities remain close to the uncompressed baseline.

For developers building local autonomous agents, Saluki demonstrates that ultra-low bit quantization can maintain or even refine complex structured outputs without requiring massive enterprise GPUs. While users requiring heavy mathematical precision may need full-precision models, developers focused on workflow automation and tool integration now have a lightweight option that runs efficiently on personal devices.