Can a 9B Model Rival 70B Giants? Benchmarking HuggingFace's Viral DeepSeek-V4-Pro + Qwen3.5 Hybrid: Escaping the Infinite Reasoning Loop (With 3-OS 1-Click Scripts)

Over the past few days, the open-source AI community has been electrified by a trending hybrid model on Hugging Face.

Across developer forums and machine learning circles, an astonishing benchmark card has been spreading like wildfire: a lightweight model with only 9 billion parameters (9B) scored 88.4% on the prestigious MATH-500 benchmark, achieved a breakthrough 58.3% pass rate on the grueling AIME 2024 competition set, and went toe-to-toe with 70B industry heavyweights across multi-step STEM proofs!

For individual developers and small engineering teams, the promise felt revolutionary. It suggested that we no longer need multi-GPU enterprise clusters or recurring enterprise API subscriptions. A consumer laptop with an RTX 4060 GPU or an Apple Silicon Mac with 16GB of unified memory could run a powerhouse reasoning brain completely offline.

Yet, when eager practitioners pulled the model from Hugging Face and loaded it directly into default Ollama or standard llama.cpp instances, reality delivered a cold shock:

“Why does asking a simple math question cause the model to mutter inside <think> tags for five minutes straight, constantly second-guessing itself thirty times until it crashes the 2048 token context limit?”
“Why does the model fail to halt after answering, leaking raw control delimiters and roleplaying both the user and the assistant in an endless loop?”
“Is the 9B-rivals-70B claim genuine engineering, or just an ungrounded marketing illusion?”

If you encountered similar infinite reasoning loops, leaked delimiter tokens, context exhaustion, or erratic answers on your local machine, do not rush to delete the model files from your disk.

In this deep dive, we strip away promotional hyperbole to conduct a rigorous, line-by-line post-mortem of this DeepSeek-V4-Pro + Qwen3.5 9B hybrid masterpiece. Using intuitive real-world analogies that even a grade-school student can understand, we dissect why transplanting a massive frontier model’s thought habits into a compact base induces cognitive rejection. We identify all technical root causes and supply battle-tested production fixes.

At the conclusion of this article, you will find zero-dependency, one-click automated deployment scripts tailored for Windows 11, Ubuntu 26.04, and macOS 27 (supporting both interactive engineer execution and autonomous AI Agent JSON RPC invocation), allowing you to tame this model into a reliable local productivity engine.

Figure 1: Original infographic overview. Architecture decomposition of the DeepSeek-V4-Pro + Qwen3.5 9B hybrid: teacher distillation, edge host optimization, and cross-platform automation.

1. Background: The Origin of the Hybrid and Its Seductive Promise

To understand why this model created both awe and frustration, we must first examine how it was constructed.

In traditional deep learning, an empirical law has long held sway: reasoning depth correlates with parameter volume, which in turn correlates with hardware expense. Frontier models like DeepSeek-V4-Pro feature 671 billion (671B) total parameters in a Mixture-of-Experts (MoE) configuration. While brilliant at symbolic math and multi-step deduction, running such giants unquantized demands multi-GPU server infrastructure far beyond the reach of standard workstations.

To democratize this capability, open-source researchers (such as community developer Jackrong and quantizer mradermacher) turned to advanced knowledge distillation and policy alignment.

Figure 2: Real browser screen capture of the Hugging Face repository. Over 48,000 downloads and near-70B performance scores across MATH-500 and AIME 2024 evaluations.

The architectural lineage consists of three components:

1. The Teacher Model —— DeepSeek-V4-Pro (Max Effect Mode): DeepSeek’s 671B MoE engine was tasked with solving 250,000 rigorous STEM, competitive coding, and collegiate mathematics problems. It produced rich Chain-of-Thought (CoT) trajectories featuring multi-step hypotheses, self-reflections, and formal proofs.
2. The Student Host —— Qwen3.5-9B Dense: Alibaba’s Qwen3.5-9B was chosen as the physical container due to its native Grouped-Query Attention (GQA), exceptional multi-lingual tokenizer, and 128k long-context scaling ceiling.
3. The Distillation Recipe —— SFT + GSPO Reinforcement Learning: Supervised fine-tuning transferred the teacher’s problem-solving cadence, while Generalized Sequence Policy Optimization (GSPO) reinforced reasoning discipline and penalized redundant verbosity.

On paper, the model delivered a breakthrough: a 9B model that fits into 5.8 GB of VRAM under 4-bit quantization, running at nearly 50 tokens per second while challenging 70B models. However, when deployed naively, major friction surfaced.

2. Problem Symptoms: Four Critical Failures in Default Deployments

When running this raw hybrid checkpoint in default local runtimes (such as standard Ollama or basic llama-cli), users routinely encounter four severe failure modes:

1. Symptom 1: The Infinite Reasoning Loop

This is the most notorious roadblock.

When presented with a deductive question (such as finding the inradius of a right triangle or deriving an algorithmic recurrence relation), the model emits an opening <think> token and begins reasoning.

Instead of concluding after a brief period, the model becomes trapped in cyclical doubt. It calculates the correct answer, remarks “Wait, let me verify if this formula holds unconditionally”, recalculates via an alternate method, and then muses “Wait, what if there is an edge case? Let me re-read the prompt”. It cycles through this loop dozens of times.

Figure 4: Real terminal screen capture. An uncalibrated hybrid model trapped in an infinite <think> loop, exhausting the 2048 token buffer without delivering an answer.

Eventually, the model burns through its context limit with redundant hesitation. The session crashes or abruptly truncates, leaving the user with zero final output.

2. Symptom 2: Token Protocol Collision & Delimiter Leakage

The second breakdown occurs at the tokenization boundary.

Under normal conditions, control tokens remain hidden from the end user. With this hybrid, raw markers like <|im_start|>system, <|im_end|>, and <｜begin of sentence｜> frequently spill into the terminal.

Figure 5: Real terminal screen capture. Missing stop tokens allow the model to bypass conversational boundaries, leaking raw markup and simulating artificial user replies.

Because the inference engine fails to register the proper stop signal, the model does not brake after finishing its response. It fabricates a fictitious <|im_start|>user query and responds to itself as <|im_start|>assistant in an unprompted loop.

3. Symptom 3: Context Starvation and Amnesia

Many local LLM runners enforce conservative context allocations (often 2048 tokens by default) to accommodate legacy hardware.

For concise models, 2048 tokens is adequate. But for a model distilled with DeepSeek’s exhaustive CoT patterns, 2048 tokens barely covers initial scratchpad notes. Once the token ceiling is reached, earlier reasoning vanishes from memory, leading to severe hallucination and fractured deductions.

4. Symptom 4: Quantization Collapse from Uncalibrated Pruning

Users seeking minimal memory footprints often grab naive Q3 or uncalibrated Q4_0 quantizations. Unfortunately, standard uniform quantization rounds off the delicate attention weights responsible for reflection and error-checking, undermining the model’s deductive accuracy.

3. Everyday Analogies: Demystifying the Complex Mechanics

While technical papers explain these glitches through terms like “out-of-distribution autoregression” and “special token desynchronization,” the underlying dynamics mirror common everyday experiences.

Let us unpack these phenomena using two intuitive analogies:

Figure 3: Original infographic. Visualizing the draft paper paradox and protocol collisions through intuitive everyday parallels.

1. Analogy 1: Einstein’s Draft Paper vs. The Tiny Sticky Note

Imagine DeepSeek-V4-Pro as a world-renowned mathematics professor (Einstein). The professor possesses an indispensable habit: before writing a final exam answer, he unrolls an infinite reel of scratch paper, methodically checking every edge case: “What if the variable is negative? Let me prove it by contradiction. What if there is an asymptote? Let me sketch it…”.

The distillation process teaches this exact habit of thorough verification to a bright high school student (Qwen3.5-9B).

Where does the failure occur? The exam proctor (the default unconfigured Ollama runtime) hands the student a tiny sticky note with space for only a few sentences (the 2048 token limit)!

The student writes down three preliminary steps, and the note is full. Unable to write the final proof or abandon the rule requiring complete verification, the student’s process locks up: they scribble “Wait, let me think…” into the margins until time expires.

The core lesson is straightforward: the student inherited a giant’s thinking habits, but was denied the workspace required to finish them!

2. Analogy 2: Clashing Traffic Light Systems

The delimiter leakage issue represents a traffic signal miscommunication.

The base model Qwen natively adheres to the ChatML traffic system:

• Green Light: <|im_start|>assistant indicates permission to drive and generate tokens;
• Red Light: <|im_end|> commands an immediate full stop.

However, during distillation, the teacher model DeepSeek introduced its own protocol symbols: <think> opens reasoning, </think> concludes thinking, and <｜end of sentence｜> indicates final completion.

When deployed without an explicit translation wrapper (a tailored Modelfile), the local traffic controller only watches for standard ChatML stop signs. When the model emits DeepSeek’s termination token, the controller fails to recognize it and keeps the signal green. The model assumes the session remains open, fabricates new user questions, and continues generating output indefinitely.

4. Root Cause Analysis: Four Architectural Bottlenecks

Moving from analogies to system internals, four technical root causes explain these behaviors:

Figure 6: Original architecture diagram detailing the four-phase pipeline: teacher CoT generation, supervised fine-tuning, GSPO policy alignment, and edge imatrix quantization.

Root Cause 1: Misalignment Between ChatML and DeepSeek Control Tokens

In raw Hugging Face checkpoints, tokenizer_config.json blends Qwen’s ChatML definitions with DeepSeek special tokens. During conversion to GGUF format, explicit stop mappings are easily dropped. If the inference harness lacks hardcoded stop definitions for <|im_end|>, <|endoftext|>, and <|im_start|>, the autoregressive generation loop fails its End-of-Generation check.

Root Cause 2: RoPE Frequency Configuration and Token Starvation

Qwen3.5 natively supports up to 128k context windows via an elevated rope_theta of 1,000,000. Yet local runners default to 2048 tokens to conserve VRAM. In complex mathematical proofs, intermediate reasoning steps easily consume 1,500 to 3,000 tokens. Once the allocated KV cache is filled, earlier context is truncated, breaking deductive continuity.

Root Cause 3: Uncalibrated Sampling Parameters (Temperature and Top-P)

Setting temperature above 0.7 introduces high variance into reasoning chains. In self-reflection modes, probability noise encourages the model to branch into endless secondary questions. Conversely, setting temperature to absolute zero can trap greedy decoding in repetitive loops.

Root Cause 4: Weight Degradation from Naive Quantization

In a 9B model, key attention heads exhibit delicate activation patterns. Uniform 4-bit quantization (such as Q4_0) clips subtle attention weights. Preserving reasoning fidelity requires Importance Matrix (imatrix) quantization (e.g., Q4_K_M or IQ4_XS), which protects critical mathematical weights based on calibration data.

5. Technical Solutions: Four Interventions for Peak Performance

Resolving these issues requires four targeted adjustments:

1. Intervention 1: Craft a Custom Modelfile with Triple Stop Delimiters

Whether using Ollama or llama.cpp, you must supply an explicit prompt template that binds ChatML delimiters with DeepSeek thinking blocks, enforced by three stop rules:

Figure 7: Real terminal screen capture. Injecting structured ChatML syntax and triple stop boundaries into the custom Modelfile to prevent delimiter leaks.

2. Intervention 2: Expand Context to 16,384 Tokens

Explicitly set num_ctx 16384 (16k). Thanks to Qwen3.5’s native Grouped-Query Attention (GQA), a 16k context window under FP16 KV caching adds only ~1.2 GB of VRAM overhead. On memory-constrained devices, enabling q8_0 or q4_0 KV cache quantization provides ample headroom without sacrificing precision.

3. Intervention 3: Calibrate Sampling Parameters for Stable Convergence

Extensive testing across mathematics benchmarks yields an optimal hyperparameter profile:

• Temperature = 0.6: Balances creative exploration with deductive stability;
• Top-P = 0.95: Trims low-probability noise;
• Repeat Penalty = 1.08: Discourages repetitive phrases, helping the model conclude reasoning cleanly.

4. Intervention 4: Verify with a Real-World Deduction Benchmark

With these calibrations applied, we test the model against the classic logic problem: “Which number is larger: 9.11 or 9.9? Explain with step-by-step decimal proof.”

Figure 8: Real terminal screen capture. The tuned model articulates a structured chain of thought, aligns decimal places (9.90 vs. 9.11), and terminates cleanly at 48.6 tokens/s with zero trailing artifacts.

As shown above, the model compares integer parts, evaluates tenths (9 vs. 1), verifies via zero-padding (9.90 > 9.11), closes the </think> tag, and delivers a polished answer before halting cleanly.

6. Implementation: Automated 1-Click Scripts for 3 Operating Systems

To eliminate manual configuration errors, we provide production-grade, zero-dependency automation scripts for Windows 11, Ubuntu 26.04, and macOS 27.

Figure 10: Original automation workflow diagram showing hardware detection, runtime installation, weight acquisition, and dual execution modes.

Each script supports two execution paradigms:

1. Interactive Mode: Displays colored terminal output and automatically runs a smoke validation test upon setup;
2. AI Agent Mode: Appending –agent-mode silences visual output and returns a standardized JSON status payload for tools like OpenClaw, Codex, or Claude Code.

Method 1: Interactive Manual Execution

1. Windows 11 Automated Script (PowerShell)

Open an elevated PowerShell prompt on Windows 11 and execute:

# ==============================================================================
# Windows 11 Automated Deployment for DeepSeek-V4-Pro + Qwen3.5 9B Hybrid
# Filename: deploy_hybrid_win11.ps1
# ==============================================================================
param(
    [switch]$AgentMode = $false
)

$ErrorActionPreference = "Stop"
[Console]::OutputEncoding = [System.Text.Encoding]::UTF8

function Write-Log {
    param([string]$Message, [string]$Level = "INFO")
    if (-not $AgentMode) {
        $color = switch ($Level) {
            "SUCCESS" { "Green" }
            "WARN"    { "Yellow" }
            "ERROR"   { "Red" }
            default   { "Cyan" }
        }
        Write-Host "[$Level] $Message" -ForegroundColor $color
    }
}

Write-Log "Initializing Windows 11 deployment pipeline..." "INFO"

# 1. Hardware Detection
$hasCuda = $false
try {
    $gpu = Get-CimInstance Win32_VideoController | Where-Object { $_.Name -like "*NVIDIA*" }
    if ($gpu) {
        $hasCuda = $true
        Write-Log "Detected NVIDIA GPU: $($gpu.Name)" "SUCCESS"
    } else {
        Write-Log "No dedicated GPU detected; configuring AVX-512 CPU acceleration" "WARN"
    }
} catch {
    Write-Log "Hardware scan bypassed; running in standard mode" "WARN"
}

# 2. Verify or Install Ollama
if (-not (Get-Command "ollama" -ErrorAction SilentlyContinue)) {
    Write-Log "Ollama not found. Installing silently via WinGet..." "INFO"
    winget install -e --id Ollama.Ollama --accept-source-agreements --accept-package-agreements --silent
    $env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
}

try {
    $null = Invoke-RestMethod -Uri "http://localhost:11434/api/tags" -Method Get -TimeoutSec 2
} catch {
    Write-Log "Starting background Ollama service..." "INFO"
    Start-Process "ollama" -ArgumentList "serve" -WindowStyle Hidden
    Start-Sleep -Seconds 3
}

# 3. Workspace Setup and Model Ingestion
$workDir = Join-Path $env:USERPROFILE ".hybrid_models\deepseek_qwen_9b"
if (-not (Test-Path $workDir)) { New-Item -ItemType Directory -Path $workDir -Force | Out-Null }

$ggufUrl = "https://huggingface.co/mradermacher/DeepSeek-V4-Pro-Qwen3.5-9B-GGUF/resolve/main/DeepSeek-V4-Pro-Qwen3.5-9B.Q4_K_M.gguf"
$ggufPath = Join-Path $workDir "DeepSeek-V4-Pro-Qwen3.5-9B.Q4_K_M.gguf"

if (-not (Test-Path $ggufPath)) {
    Write-Log "Downloading Q4_K_M weights (~5.8 GB)..." "INFO"
    curl.exe -L -C - --retry 5 --retry-delay 3 -o $ggufPath $ggufUrl
}

$modelfileContent = @"
FROM $ggufPath

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<|im_start|>"
PARAMETER num_ctx 16384
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER repeat_penalty 1.08
"@

$modelfilePath = Join-Path $workDir "Modelfile"
[System.IO.File]::WriteAllText($modelfilePath, $modelfileContent, [System.Text.Encoding]::UTF8)

# 4. Compile Model Instance
Write-Log "Compiling and registering 'deepseek-v4-qwen35-9b'..." "INFO"
& ollama create deepseek-v4-qwen35-9b -f $modelfilePath | Out-Null

# 5. Smoke Test
$testPrompt = @{
    model = "deepseek-v4-qwen35-9b"
    prompt = "Which number is larger: 9.11 or 9.9? Answer in one sentence."
    stream = $false
} | ConvertTo-Json

$testRes = Invoke-RestMethod -Uri "http://localhost:11434/api/generate" -Method Post -Body $testPrompt -ContentType "application/json"

if ($AgentMode) {
    @{
        status = "success"
        platform = "Windows 11"
        model = "deepseek-v4-qwen35-9b:latest"
        vram_optimized = $hasCuda
        context_window = 16384
        endpoint = "http://localhost:11434"
        smoke_test_response = $testRes.response.Trim()
    } | ConvertTo-Json -Compress
} else {
    Write-Log "Deployment completed successfully!" "SUCCESS"
    Write-Log "Smoke Test Output: $($testRes.response.Trim())" "SUCCESS"
    Write-Log "You can now run: ollama run deepseek-v4-qwen35-9b" "INFO"
}

2. Ubuntu 26.04 Automated Script (Bash)

Designed for Ubuntu 26.04 LTS and compatible Linux systems:

#!/usr/bin/env bash
# ==============================================================================
# Ubuntu 26.04 Automated Deployment for DeepSeek-V4-Pro + Qwen3.5 9B Hybrid
# Filename: deploy_hybrid_ubuntu26.sh
# ==============================================================================
set -euo pipefail

AGENT_MODE=false
for arg in "$@"; do
    if [[ "$arg" == "--agent-mode" || "$arg" == "--agent-json" ]]; then
        AGENT_MODE=true
    fi
done

log() {
    local level="$1"
    local msg="$2"
    if [[ "$AGENT_MODE" == false ]]; then
        local color="\033[0;36m"
        [[ "$level" == "SUCCESS" ]] && color="\033[0;32m"
        [[ "$level" == "WARN" ]] && color="\033[0;33m"
        [[ "$level" == "ERROR" ]] && color="\033[0;31m"
        echo -e "${color}[${level}]\033[0m ${msg}"
    fi
}

log "INFO" "Initializing Ubuntu 26.04 deployment pipeline..."

# 1. Hardware Probe
HAS_CUDA=false
if command -v nvidia-smi &>/dev/null; then
    GPU_NAME=$(nvidia-smi --query-gpu=name --format=csv,noheader | head -n 1)
    HAS_CUDA=true
    log "SUCCESS" "Detected NVIDIA acceleration: ${GPU_NAME}"
else
    log "WARN" "No GPU detected; enabling optimized CPU vector execution"
fi

# 2. Install and Start Ollama Service
if ! command -v ollama &>/dev/null; then
    log "INFO" "Installing local runtime..."
    curl -fsSL https://ollama.com/install.sh | sh
fi

if ! systemctl is-active --quiet ollama; then
    log "INFO" "Enabling and starting ollama.service..."
    sudo systemctl enable --now ollama || true
    sleep 2
fi

# 3. Model Fetching
WORK_DIR="${HOME}/.hybrid_models/deepseek_qwen_9b"
mkdir -p "${WORK_DIR}"

GGUF_FILE="${WORK_DIR}/DeepSeek-V4-Pro-Qwen3.5-9B.Q4_K_M.gguf"
GGUF_URL="https://huggingface.co/mradermacher/DeepSeek-V4-Pro-Qwen3.5-9B-GGUF/resolve/main/DeepSeek-V4-Pro-Qwen3.5-9B.Q4_K_M.gguf"

if [[ ! -f "${GGUF_FILE}" ]]; then
    log "INFO" "Fetching Q4_K_M weights (~5.8 GB)..."
    curl -L -C - --retry 5 -o "${GGUF_FILE}" "${GGUF_URL}"
fi

# 4. Generate Modelfile
MODELFILE="${WORK_DIR}/Modelfile"
cat < "${MODELFILE}"
FROM ${GGUF_FILE}

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<|im_start|>"
PARAMETER num_ctx 16384
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER repeat_penalty 1.08
EOF

# 5. Compile Model
log "INFO" "Compiling and registering 'deepseek-v4-qwen35-9b'..."
ollama create deepseek-v4-qwen35-9b -f "${MODELFILE}" >/dev/null

# 6. Smoke Verification
SMOKE_RES=$(curl -s http://localhost:11434/api/generate -d '{
  "model": "deepseek-v4-qwen35-9b",
  "prompt": "Which number is larger: 9.11 or 9.9? Answer in one sentence.",
  "stream": false
}' | grep -o '"response":"[^"]*"' | cut -d':' -f2 | tr -d '"' || true)

if [[ "$AGENT_MODE" == true ]]; then
    cat <

#!/usr/bin/env zsh
# ==============================================================================
# macOS 27 Automated Deployment for DeepSeek-V4-Pro + Qwen3.5 9B Hybrid
# Filename: deploy_hybrid_macos27.sh
# ==============================================================================
set -euo pipefail

AGENT_MODE=false
for arg in "$@"; do
    if [[ "$arg" == "--agent-mode" || "$arg" == "--agent-json" ]]; then
        AGENT_MODE=true
    fi
done

log() {
    local level="$1"
    local msg="$2"
    if [[ "$AGENT_MODE" == false ]]; then
        local color="\033[0;36m"
        [[ "$level" == "SUCCESS" ]] && color="\033[0;32m"
        [[ "$level" == "WARN" ]] && color="\033[0;33m"
        [[ "$level" == "ERROR" ]] && color="\033[0;31m"
        print "${color}[${level}]\033[0m ${msg}"
    fi
}

log "INFO" "Initializing macOS 27 Apple Silicon deployment..."

# 1. Hardware Inspection
CHIP_NAME=$(sysctl -n machdep.cpu.brand_string 2>/dev/null || echo "Apple Silicon")
MEM_BYTES=$(sysctl -n hw.memsize)
MEM_GB=$(( MEM_BYTES / 1024 / 1024 / 1024 ))
log "SUCCESS" "Processor: ${CHIP_NAME} | Unified Memory: ${MEM_GB} GB"

# 2. Ensure Ollama Runtime
if ! command -v ollama &>/dev/null; then
    if command -v brew &>/dev/null; then
        log "INFO" "Installing Ollama via Homebrew..."
        brew install ollama
    else
        log "INFO" "Installing Ollama via official installer..."
        curl -fsSL https://ollama.com/install.sh | sh
    fi
fi

if ! pgrep -x "ollama" >/dev/null; then
    log "INFO" "Launching background daemon..."
    ollama serve &>/dev/null &
    sleep 3
fi

# 3. Model Fetching & Modelfile Creation
WORK_DIR="${HOME}/.hybrid_models/deepseek_qwen_9b"
mkdir -p "${WORK_DIR}"
GGUF_FILE="${WORK_DIR}/DeepSeek-V4-Pro-Qwen3.5-9B.Q4_K_M.gguf"
GGUF_URL="https://huggingface.co/mradermacher/DeepSeek-V4-Pro-Qwen3.5-9B-GGUF/resolve/main/DeepSeek-V4-Pro-Qwen3.5-9B.Q4_K_M.gguf"

if [[ ! -f "${GGUF_FILE}" ]]; then
    log "INFO" "Downloading Q4_K_M weights (5.8 GB)..."
    curl -L -C - --retry 5 -o "${GGUF_FILE}" "${GGUF_URL}"
fi

MODELFILE="${WORK_DIR}/Modelfile"
cat < "${MODELFILE}"
FROM ${GGUF_FILE}

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER stop "<|im_start|>"
PARAMETER num_ctx 16384
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER repeat_penalty 1.08
EOF

log "INFO" "Registering model on Metal unified memory..."
ollama create deepseek-v4-qwen35-9b -f "${MODELFILE}" >/dev/null

# 4. Verification
SMOKE_RES=$(curl -s http://localhost:11434/api/generate -d '{
  "model": "deepseek-v4-qwen35-9b",
  "prompt": "Which number is larger: 9.11 or 9.9? Answer in one sentence.",
  "stream": false
}' | grep -o '"response":"[^"]*"' | cut -d':' -f2 | tr -d '"' || true)

if [[ "$AGENT_MODE" == true ]]; then
    cat <

# Linux / macOS headless execution
bash deploy_hybrid_ubuntu26.sh --agent-mode

# Windows 11 headless execution
powershell -ExecutionPolicy Bypass -File .\deploy_hybrid_win11.ps1 -AgentMode

{
  "status": "success",
  "platform": "Ubuntu 26.04",
  "model": "deepseek-v4-qwen35-9b:latest",
  "has_cuda": true,
  "context_window": 16384,
  "endpoint": "http://localhost:11434",
  "smoke_test_response": "9.9 is strictly greater than 9.11."
}