Neuetricks · On-prem frontier LLM · 11 May 2026

I / VI · Verdict

The verdict

On-prem buys locality and sovereignty.
Not capability.

A 10-person Australian SME asked whether to build an on-prem frontier LLM. The hardware bar is achievable; the capability gap is not closable with hardware. Three named conditions flip the answer. Most SMEs satisfy none of them.

The default verdict, and the three flips

Default

Do not build. An 8× H200 stack meets the throughput bar, but the best open-weight model trails the closed frontier by three to fifteen points on the benchmarks that decide multi-step agentic work.

AUD 1.59M / 3 yrs5–15 pt gap, frontier vs open

01 Workload

Bounded by retrieval and summarisation.

No novel reasoning at the edge, no high-fidelity agentic multi-step tool use. The open-weight gap collapses to under five points and the stack is fit for purpose.

Read the capability gap

02 Compliance

Air-gap, not sovereignty.

If the operative need is Australian-resident weights — not a physical air-gap — AWS Sydney or Azure Australia East delivers identical capability at 30–50% of the three-year TCO.

Read the cost comparison

03 Connectors

Notion, Gmail, Calendar, Word, Excel — not core.

Every named connector is OAuth-gated SaaS that routes data through US-domiciled cloud. If they drive the workflow, the on-prem story is already punctured.

Read the OAuth puncture

IIThe capability gap

open-weight trails the closed frontier by three to six months and five to fifteen points on agentic benchmarks

gap.html

IIIThe bandwidth floor

only the 8× H200 SXM HGX node meets the bar; DGX Spark and Mac Studio miss by one to two orders of magnitude

silicon.html

IVThree-year TCO, on-prem vs cloud-sovereign

AUD 1.59M midpoint on-prem against AUD 400–700k for the cloud-sovereign alternative that also closes the capability gap

costs.html

VThe OAuth puncture

five named connectors, five US-domiciled clouds, five OAuth boundaries that exit the building regardless of where the model lives

puncture.html

VIWhere the verdict flips

the three conditions written as a threshold ledger, with the default probability each holds for an SME annotated

flips.html

SourceOn-prem frontier LLM briefing, 11 May 2026 — §Verdict and §Conditions under which the recommendation flips. AUD throughout; USD/AUD planning rate 1.50, RBA spot ≈ 1.38.