500.6GB down to 117.8GB

Here's the deal: on July 27, Korean model-compression company Nota released a compressed build of Solar Open 2, Upstage's second sovereign AI foundation model. One line summarizes it:

Weight memory from 500.6GB to 117.8GB. A 76.5% reduction. What needed eight H100s now runs on two.

Why that matters becomes obvious when you price the hardware. An eight-H100 server runs into the hundreds of thousands of dollars. For most Korean companies, "let's run the domestic foundation model on our own servers" was a conversation that ended the moment the quote arrived. Cut that quote to a quarter and the decision changes.

And the real point of the announcement isn't the compression ratio. It's that agent capability survived it. Nota reports that tool-calling and other core agentic functions held stable — which is typically the first thing to break when you fold a model in half.

The two companies involved

Upstage is one of Korea's leading LLM companies, known for the Solar series. Solar Open 2 is the second output of the government's Sovereign AI Foundation Model program — a national-budget effort to build domestic large models, with Upstage owning one of its tracks.

Solar Open 2's design direction is interesting. It's explicitly pitched at "AI that works" rather than benchmark scores — built for agentic environments where the model reasons across multiple rounds and calls tools to finish a task, not one-shot Q&A. Architecturally it's a Mixture of Experts (MoE) with 250 billion total parameters and only 15 billion active during inference. It handles up to 1 million tokens of context, and both the weights and a technical report were published openly on Hugging Face.

Nota is a different kind of company. It doesn't build models — it compresses other people's models until they run on real hardware. It made its name compressing vision models for edge devices and has since expanded into large language models. This work was also carried out with Upstage as part of the sovereign model program.

The pairing says something about where the industry is going: building a model and making a model usable are separating into distinct industries. Teams used to handle their own deployment; once models reach hundreds of billions of parameters, compression and quantization become their own specialty.

The techniques inside it

Nota disclosed four optimization methods. Worth unpacking each.

Technique What it does
Data-driven MoE quantization (PASCAL-MoE) Sets per-expert quantization strength using observed data distributions
Router-aware MoE quantization Allocates precision based on how often each expert is actually called
Non-uniform global pruning Removes weights at different rates layer by layer
4-bit quantization (INT4 / NVFP4) Represents weights in 4 bits, cutting memory at the root

Start with why compressing MoE is hard. An MoE model holds many "expert" networks, and a router selects a subset for each input. That's what "250B total, 15B active" means — light compute, large knowledge capacity. Clever design.

The problem is memory. Compute uses 15 billion parameters, but since you don't know which experts will be called, all 250 billion have to sit in memory. That gives MoE its odd profile: cheap to compute, expensive to hold. The 500.6GB figure comes from exactly this.

Experts also aren't equally important. Some get called on nearly every token; others only fire in narrow domains. Crushing every expert to 4 bits uniformly produces disproportionate damage in the frequently used ones. That's why Nota went router-aware: read the router's call statistics, then give precision to the hot experts and compress the cold ones harder.

The data-driven method labeled PASCAL-MoE follows the same logic. Instead of cutting based on weight magnitudes alone, it runs real data through the model, measures which values actually drive outputs, and sets quantization accordingly. More expensive to compute, much less quality loss.

The 4-bit format choice is worth noting too. Alongside INT4, NVFP4 appears — Nvidia's 4-bit floating point format supported natively in its newest hardware generation. Using a format the hardware implements directly delivers far better throughput than emulating it in software. Optimizing for the model architecture and the hardware environment together is how Nota CTO Kim Tae-ho framed it:

"This result shows large MoE models being optimized into a form enterprises can actually deploy."

One caveat. The exact accuracy cost wasn't published. There's a qualitative statement that tool-calling and core agentic performance held stable, but no benchmark comparison table accompanied the release. Anyone evaluating this build should verify those numbers independently.

What each side gets

Upstage gets deployability. Publishing open weights means little if nobody can run them — download counts aren't adoption, and not many Korean organizations own eight H100s. Drop the requirement to two and the addressable base multiplies. A sovereign model succeeds or fails on how many organizations actually use it, and compression grows that denominator.

Nota gets a reference. Model compression is a hard business to demonstrate; "we compress well" doesn't verify. But "we cut a 250-billion-parameter national-program model by 76.5%" is specific and checkable. Expect this case to be cited in every large-model compression pitch it makes from here.

Enterprise users get an option. Right now a Korean company adopting an LLM has roughly three paths: a foreign API, a domestic cloud service, or self-hosting a small open model. The first requires sending data out; the third has meant accepting weaker capability. Running a 250B-class model in-house on two H100s is a fourth path that didn't exist. That matters most in finance, healthcare and the public sector, where data can't leave.

Now the sober read. Two H100s still isn't cheap — tens of thousands of dollars each, plus server, power and cooling. Going from eight to two is dramatic, but it isn't "anyone can run this." And a compressed model isn't the same model. Beyond the tool-calling claim, there's no published data on how it behaves on, say, long-context reasoning consistency or subtle Korean-language handling.

How compression efforts have gone before

Model compression is a mature field now, and the success and failure patterns are fairly legible.

The archetypal success is the spread of 4-bit quantization itself. A few years ago, running a large language model on personal hardware was barely imaginable. Then quantization methods like GPTQ and AWQ arrived alongside runtimes like llama.cpp, and suddenly billion-parameter models ran on laptops. The key finding was that quality loss was smaller than expected — repeatedly, 4-bit builds turned out to feel close enough in real use, and the practice became standard.

The failure pattern is equally clear: leading with compression ratio while a specific capability quietly collapses. Long-context reasoning, code generation and tool calling are the known quantization-sensitive areas, because they're tasks where small errors compound across a long chain. A slight deviation on one token changes the next decision, and ten steps later the outputs diverge completely. Nota foregrounding "tool-calling held stable" is an acknowledgment that it knows where the risk sits.

MoE compression is also a relatively new front. Dense-model quantization has deep research behind it; MoE adds per-expert importance variance and router behavior as variables, so existing methods don't transfer cleanly. Naming router-aware and PASCAL-MoE separately is Nota's flag that it worked on exactly that gap. With major open models moving to MoE, MoE-specific compression is a market that keeps growing.

One more datapoint: Upstage's own materials describe Solar Open 2 as runnable on two H200s when quantized. Nota's result is two H100s. Given H200 is the larger-memory generation, that implies Nota pushed compression meaningfully further.

How competitors respond

The commercial API camp won't lose much sleep. Their argument — "renting an API beats compressing and babysitting servers" — is simply true for most workloads. It doesn't land with customers who can't move data off-premises, though, which is why they keep expanding on-prem deployment options.

Other domestic model teams face the same task. Multiple companies participate in the sovereign model program, and making their models actually get used is a shared problem. If the Nota–Upstage pairing produces results, expect other teams to attach compression partners too.

Chinese open models are the most concrete competition. Releases like Moonshot's Kimi K3 are moving both capability and price, and they often ship aggressive quantized variants of MoE architectures themselves. Korean models will have to differentiate on Korean-language quality and data sovereignty rather than raw scores — and whether that gap is large enough for users to feel is the open question.

Inference frameworks matter here too. Open-source serving engines like vLLM and SGLang keep deepening quantization support. If general-purpose tooling gets "good enough," the room for specialist compression firms narrows. What a company like Nota defends is architecture-specific optimization — and naming MoE-specific techniques is precisely that positioning.

So what changes

If you're standing up an LLM in-house, redo the budget. A two-H100 configuration sits at a completely different approval level than eight. Before you commit, though, get accuracy-loss data versus the original. The release covered tool-calling stability qualitatively with no benchmark comparison. Run your own A/B on tasks close to your actual workload.

If you're an engineer, the transferable idea here is using router statistics as an optimization input. If you work with MoE models, simply logging which experts get called and how often gives you material for compression and caching strategy. Uniform treatment leaves a lot on the table.

If you run a startup, the practical change is that self-hosting a domestic open-weight model just entered the realm of possibility. Foreign API costs scale linearly with usage; self-hosting front-loads the cost and then drops marginal cost sharply. Above a certain traffic level it's worth redoing the break-even math — while remembering to price in the hidden cost of the people who keep it running.

If you follow AI policy, this highlights something usually missing from sovereign AI discussion: budget for building models gets debated; the infrastructure cost of running them doesn't. However good a domestic model is, if the inference cost is unbearable, users go back to foreign APIs. Compression isn't ancillary — it's closer to a precondition for sovereign AI to exist at all.

If you invest, read this as evidence that model optimization is establishing itself as a standalone market. The risk is clear, though: if open-source serving frameworks absorb the same functionality, specialist margins compress. Watch whether differentiation holds.

If you're just reading the news, one sentence: the next round of AI competition isn't bigger models, it's how small you can make the model you already have. As capability levels out, the fight moves to who runs the same quality most cheaply.

🥄 Three Things You're Probably Wondering

— 76.5% smaller and performance is unchanged? Hard to say unchanged. What's confirmed is a qualitative statement that tool-calling and core agentic functions held stable; no benchmark comparison against the original was published. 4-bit quantization generally produces small perceived loss, but tasks where error compounds — long-context reasoning especially — can diverge. If you're adopting it, measure on your own tasks.

— So can my company run the domestic model now? If you can absorb two H100s. That's still tens of thousands of dollars per card plus server, power and cooling — not a small decision. But eight-to-two is the difference between "impossible" and "worth evaluating." Especially worth the math if your industry can't export data.

— What's so special about MoE? That it computes on a slice but has to hold all of it. Only 15 billion of 250 billion parameters run per token, so it's fast — but since you can't predict which experts fire, all 250 billion stay resident. That makes MoE the "fast but memory-hungry" architecture, and also the one where compression pays off most. That's exactly what this work targeted.

Sources

Figures are as of announcement and may change.