Open-Source Reasoning Models in Production: Comparing DeepSeek-R1 Distillations, QwQ-32B, Sky-T1, and Marco-o1

The release of frontier reasoning models demonstrated that allocating test-time compute to explicit chain-of-thought generation significantly elevates problem-solving accuracy in mathematics, software engineering, and scientific reasoning. While proprietary systems conceal their training pipelines and reasoning traces, the open-source ecosystem has produced diverse approaches to replicating and deploying reasoning capabilities.

Engineering teams evaluating open-source reasoning models face fundamental architectural divergences. Some systems rely on pure supervised fine-tuning (SFT) over large-scale synthetic trajectories distilled from frontier models, others employ direct reinforcement learning (RL) with rule-based reward functions, and others integrate inference-time tree search algorithms such as Monte Carlo Tree Search (MCTS).

This analysis examines the architectural foundations, training pipelines, benchmark performance, and production serving trade-offs across four prominent open-weight reasoning families: DeepSeek-R1 Distillations, Alibaba's QwQ-32B, UC Berkeley NovaSky's Sky-T1, and Alibaba International's Marco-o1.

Architectural Paradigms and Training Methodologies

The mechanisms used to instill step-by-step reasoning determine not only baseline model capability, but also token efficiency, generalization bounds, and serving latency.

1. DeepSeek-R1 Distillations (1.5B to 70B)

DeepSeek AI demonstrated in the DeepSeek-R1 technical report that reasoning capabilities discovered through pure reinforcement learning via Group Relative Policy Optimization (GRPO) can be transferred effectively to smaller dense architectures through supervised distillation alone.

• Base Models: Dense architectures across two main foundations: Qwen2.5 (1.5B, 7B, 14B, 32B) and Llama 3 (8B, 70B).
• Data Curation Pipeline: 800,000 curated synthetic samples, comprising approximately 600,000 reasoning-oriented trajectories (math, logic, coding) and 200,000 non-reasoning instruction samples (creative writing, factual QA, translation).
• Training Objective: Direct SFT using cross-entropy loss over reasoning tokens formatted within explicit <think>...</think> tags. The smaller models do not undergo direct RL fine-tuning; instead, they learn to reproduce the exploration, backtracking, and self-correction patterns exhibited by DeepSeek-R1-671B.

2. Alibaba QwQ-32B

Unlike pure SFT distillations, Alibaba's QwQ-32B is built by applying direct reinforcement learning to the dense Qwen2.5-32B foundation.

• Training Pipeline: Multi-stage post-training combining an initial cold-start reasoning dataset with large-scale reinforcement learning. Rewards are computed using verifiable outcomes in code execution and exact mathematical proofs, alongside general instruction-following rubrics.
• Agentic Integration: Unlike reasoning models strictly confined to pure text CoT, QwQ-32B is explicitly optimized to integrate tool usage, environment interaction, and function calling into its deliberation traces.
• Inference Pattern: Produces detailed reasoning trajectories characterized by iterative hypothesis generation, counterexample testing, and dynamic strategy pivots during complex multi-step problems.

3. NovaSky Sky-T1-32B

Developed by UC Berkeley's Sky Computing Lab, Sky-T1 was created to demonstrate data-efficient reasoning distillation on a constrained compute budget (under $450).

• Data Strategy: The researchers curated the SkyThought dataset containing 17,000 high-quality reasoning trajectories filtered from larger models using rejection sampling and programmatic verifiers.
• Training Recipe: Fine-tuned on Qwen2.5-32B-Instruct using LoRA and full-parameter regimes.
• Token Efficiency Variants: Addressing the phenomenon of reasoning token bloat, the team developed Sky-T1-32B-Flash, which applies length penalties and condensed CoT supervision to reduce output token volume without compromising accuracy on standard math benchmarks.

4. Marco-o1

Introduced by Alibaba International Digital Commerce, Marco-o1 takes a hybrid path combining supervised fine-tuning with inference-time Monte Carlo Tree Search (MCTS) and reflection mechanisms.

• Base Architecture: Fine-tuned on Qwen2-7B-Instruct using the Open-O1 CoT dataset.
• Search Mechanics: Implements MCTS during inference, utilizing token-level and step-level confidence scores as value heuristics to guide node expansion and path selection.
• Target Domain: Specifically designed for open-ended reasoning tasks, translation, and non-verifiable domains where rule-based ground-truth evaluators cannot easily compute scalar rewards.

Technical Specifications and Benchmark Comparison

The following table summarizes the architectural profiles and published benchmark results across these model families:

Benchmark Analysis

1. Frontier Math and Logic: QwQ-32B and DeepSeek-R1-Distill-32B achieve near parity on MATH-500 (94.1% vs. 94.3%) and AIME 2024 (74.0% vs. 72.6%), outperforming proprietary models such as OpenAI o1-preview and o1-mini.
2. Coding and Function Execution: QwQ-32B demonstrates superior coding synthesis (61.4% on LiveCodeBench) and tool-calling execution (66.4% on Berkeley Function Calling Leaderboard), attributable to its reinforcement learning environment incorporating tool-invocation rewards.
3. Data Efficiency: Sky-T1 confirms that 17,000 highly filtered trajectories suffice to elevate a 32B base model from 55% to 88.2% on MATH-500, though larger 800k distillation sets (DeepSeek-R1) or direct RL (QwQ) are necessary for competition-grade olympiad mathematics.

Production Serving Considerations and Latency Economics

Deploying reasoning models in production architectures introduces unique throughput and cost profiles that differ substantially from traditional instruction-tuned LLMs.

1. Token Expansion and Time-to-First-Answer (TTFA)

Standard instruction models generate concise answers with typical output lengths between 150 and 800 tokens. In contrast, reasoning models generate an extensive internal monologue prior to producing the final user-facing response:

• Token Multiplier: Reasoning traces frequently range from 1,500 to 12,000 tokens for complex mathematical or multi-file coding prompts.
• Latency Impact: Inter-token latency (ITL) remains constant based on hardware batch size, but perceived user latency (Time-to-First-Answer) scales linearly with the length of the <think> block.
• Cost Scaling: API and self-hosted GPU compute costs scale directly with total generated tokens. While input prompt caching amortizes prefix costs, autoregressive decoding over 5,000 thinking tokens increases per-query serving expense by 5x to 10x.

2. Mitigating Overthinking on Deterministic Tasks

A critical pitfall in production deployment is reasoning overthinking: simple factual retrieval or extraction tasks (such as "What is the capital of France?" or "Extract JSON key from text") can trigger hundreds of thinking tokens where the model evaluates edge cases unnecessarily.

Production mitigation techniques include:

• Dual-Model Routing: Classify query complexity at an ingress gateway (e.g., LiteLLM or RouteLLM), directing low-complexity extraction and classification queries to lightweight non-reasoning models (e.g., Qwen2.5-7B-Instruct) while reserving reasoning models for high-complexity math, code generation, and logical deduction.
• Thinking Token Caps: Set strict maximum generation boundaries on thinking blocks or apply system prompts instructing the model to constrain deliberation depth for specific task categories.
• Streaming Parser Delimiters: In user-facing interfaces, use streaming server-sent event (SSE) parsers to separate the <think> block from the response payload, allowing users to collapse thinking logs while reviewing answers as they generate.

3. VRAM and Quantization Topologies

For 32B parameter reasoning models (the primary workhorse tier for open-source reasoning):

• FP16 / BF16: Requires approximately 68 GB to 72 GB of VRAM for weights alone, necessitating an 80 GB A100/H100 or dual 48 GB RTX 6000 Ada / A6000 GPUs.
• AWQ / GPTQ (INT4): Compresses model weights to approximately 18 GB to 20 GB, enabling deployment on a single 24 GB GPU (NVIDIA RTX 4090 or A10G) with adequate remaining VRAM for standard KV cache allocations under moderate batch sizes.
• FP8 Serving: Native FP8 weights with FP8 KV cache in vLLM or SGLang require approximately 36 GB VRAM, fitting comfortably on a single 48 GB GPU or dual 24 GB GPUs while retaining full mathematical reasoning precision.

Implementation Guide: Serving Reasoning Models with vLLM

When serving reasoning models such as DeepSeek-R1-Distill-Qwen-32B or QwQ-32B with vLLM, configure chunked prefill, long context allowances, and structured reasoning tags.

Example Launch Configuration

vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-32B \
  --tensor-parallel-size 1 \
  --gpu-memory-utilization 0.92 \
  --max-model-len 32768 \
  --enable-chunked-prefill \
  --kv-cache-dtype fp8 \
  --port 8000

Parsing Streaming Responses in Application Logic

import openai

client = openai.OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="token-vllm"
)

response = client.chat.completions.create(
    model="deepseek-ai/DeepSeek-R1-Distill-Qwen-32B",
    messages=[
        {"role": "user", "content": "Solve the algebraic equation: 3x^2 - 12x + 9 = 0"}
    ],
    temperature=0.6,
    max_tokens=4096,
    stream=True
)

in_thinking_block = False
thinking_buffer = []
final_answer_buffer = []

for chunk in response:
    delta = chunk.choices[0].delta.content or ""
    if "<think>" in delta:
        in_thinking_block = True
        delta = delta.replace("<think>", "")
    if "</think>" in delta:
        in_thinking_block = False
        parts = delta.split("</think>")
        thinking_buffer.append(parts[0])
        final_answer_buffer.append(parts[1])
        continue

    if in_thinking_block:
        thinking_buffer.append(delta)
    else:
        final_answer_buffer.append(delta)

print("Reasoning trace length (chars):", len("".join(thinking_buffer)))
print("Final Answer:\n", "".join(final_answer_buffer).strip())

Architectural Selection Matrix

Sources

• DeepSeek-AI. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (arXiv:2501.12948, 2025).
• Qwen Team, Alibaba Cloud. QwQ-32B Technical Announcement and Leaderboard Performance (2025).
• NovaSky AI, UC Berkeley Sky Computing Lab. SkyThought: Sky-T1 Open Reasoning Models and Datasets (2025).
• Alibaba International Digital Commerce. Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions (arXiv:2411.14405, 2024).
• DeepLearning.AI. QwQ-32B Challenges DeepSeek-R1 and Other Larger Reasoning Models (2025).