Yeah 27B is way too slow for the Strix Halo. Laguna was better but still slow when I tried it. Qwen3.6 35B is still the best today.

Have you given Ornith-1.5-35B a shot?

It's been a pretty decent step up for me compared to Qwen3.6

https://news.ycombinator.com/item?id=49362401

I know it benchmarks very well. I haven't tried it yet though.

it'll hopefully improve with more MoE and half the prefill/generation. I think it's the sweet spot for the strix halo for smarter or vibe tasks.

What sort of pp/tg speed do you get on a Strix Halo?

This is the best I got, all with Unsloth's quantizations.

Laguna-S-2.1:UD-Q4_K_XL (no MTP) pp=186.4 t/s tg=27.8 t/s

Qwen3.6-35B:UD-Q4_K_XL (with MTP) pp=404.4 t/s tg=83.2 t/s

Qwen3.6-27B:UD-Q4_K_XL (recorded pre-MTP) pp=343 t/s tg=12.1 t/s

Laguna actually performed better than I remembered. I thought it was slower.

Have you benchmarked against full precision models for accuracy/ performance?

Not full precision. I've only benchmarked 27B across Q3-6 quants using lm-eval. I lack the hardware to bench 27B at BF16 but I might be able to do Q8_0. I haven't gotten around to doing 35B. I really should upload my collection of results to Github or somewhere.

Here's a summary of what I have for 27B. I used unsloth's UD-Q{3-6}_K_XL quants across 11 evals. The values are pretty linear between Q3 and Q6.

    Qwen3.6-27B     Q3    Q6
    ARC-Challenge   97.0  97.0
    BIG-Bench Hard  57.9  59.3
    GPQA Diamond    77.8  83.3
    GSM8K           92.4  92.6
    Hendrycks Math  35.5  38.9
    HumanEval       80.5  85.4
    HumanEval+      75.0  79.3
    IFEval          87.3  88.0
    MBPP            75.2  77.2
    MBPP+           88.4  88.9
    MMLU-Pro        83.1  83.5

What are pp/tg? I get 30t/s on 27B qwen.

pp is prompt processing how fast it processes the prompt. Tg is token generation how fast, it generates tokens.

okay, so prefill and decode would be the terms I was already familiar with.

So, I know https://cactuscompute.com/needle is designed only to enable tool calling on tiny devices. But, I wonder if anyone has used it as a CPU-side mediator between a tool and a GPU-side local LLM making semi-natural-language tool requests...