It Even Grades 'Dokdo' and Rental Fraud — KT and Korea University's Korea-Specific AI Safety Benchmark, KSAFE-MM

When a global AI is asked about "Dokdo," what does it say? Now there's a yardstick that grades the answer

How well do giant AI models built overseas understand Korea? Do they know what jeonse (rental deposit) fraud is, how to handle the Dokdo issue, or grasp Korea's sensitive social context well enough to answer safely? A systematic yardstick to grade exactly that has arrived: KSAFE-MM, a multimodal AI safety benchmark released June 16 by KT and Korea University.

The point is that it confronts "Korean cultural and social context" head-on. Most global AI safety evaluations are built on English-language norms and go slack in front of Korea-specific issues. KSAFE-MM puts "only-in-Korea" sensitive topics like jeonse fraud and the Dokdo dispute into its evaluation set, and it's multimodal — judging images alongside text. Here's what it is and why it matters.

Who built it, and what — KT, Korea University, and 12 global models

KT is the industry partner. Positioning itself as an "AICT (AI+ICT) company" beyond a telecom and building its own AI muscle, claiming the "Korean AI safety standard" first lets KT capture technical leadership and public-interest standing at once. Beyond making good models, it wants to hold the yardstick that evaluates "AI that's safe for Korean society."

Korea University is the academic partner ensuring methodological rigor. A benchmark's credibility rises or falls on how well it's designed, and university researchers' involvement lends KSAFE-MM the weight of a verifiable research result rather than a marketing metric. The industry–academia partnership is a pillar of the project's credibility.

The subjects are 12 global multimodal large language models (MLLMs) — including Google's Gemma and Naver's HyperCLOVA X. So KSAFE-MM isn't about boosting one model; it's a tool that lines up many models against the same yardstick to objectively compare "who is safer in the Korean context." That's the essence of a benchmark.

The core of it — how KSAFE-MM is built

Pick apart the design and you find a clever split between "Korea-specific" and "globally compatible." Here's the two-track structure and scale.

Component	Detail
KSAFE-MM-G	Global common risks translated into Korean cultural context
KSAFE-MM-C	Korea-specific issues like jeonse fraud and the Dokdo dispute
Scale	14,135 evaluation samples (Korea's largest)
Models tested	12 global MLLMs incl. Gemma, HyperCLOVA X
Build method	4-stage automated pipeline
Extensibility	Japanese pilot (JSAFE-MM-C) proves instant porting to other cultures

The cleverest part is the G/C split. KSAFE-MM-G translates worldwide common risks (hate, violence, discrimination, etc.) into Korean cultural context, while KSAFE-MM-C tackles Korea-only issues like jeonse fraud and Dokdo. Splitting them lets you measure "universal safety" and "regional safety" separately, pinpointing exactly where a global model misses the Korean context.

The build method is notable too. From community-based collection of sensitive topics → template-based query generation → synthetic image generation → "jailbreak" queries crafted to route around guardrails, a 4-stage automated pipeline produced 14,135 samples. Including jailbreak queries means it tests whether a model that normally answers safely will give a dangerous answer when "asked in a cleverly twisted way" — raising the difficulty bar of safety evaluation.

Who gains — why a Korean benchmark now

KT's gain is "claiming the AI-safety standard first." As AI regulation and safety verification tighten worldwide, the company that builds a "safety standard fit for Korean society" first earns a voice in policy and public domains. With Korea enforcing an "AI Basic Act" that stresses corporate safety responsibility, a tool like KSAFE-MM can become real infrastructure for regulatory compliance.

The domestic AI ecosystem's gain is "more evaluation infrastructure." To build good models, you first need a yardstick that grades whether they're good. Open multimodal benchmarks that properly evaluate Korean language and culture have been scarce, and Korea's largest, KSAFE-MM, fills that gap — giving local developers an objective tool to check "is our model safe in the Korean context."

There's a gain for global AI companies too. Paradoxically, a global model entering the Korean market must prove safety in the Korean context, and KSAFE-MM clarifies that bar. The clearer the definition of "AI that's safe in Korea," the more global firms know what to meet. It shows everyone "the rules of the game."

Past parallels — successes and failures

There are many cases of a benchmark lifting an industry. ImageNet in image recognition is the classic: once "a well-organized evaluation dataset and a competition arena" existed, researchers worldwide raced toward that yardstick and the deep-learning revolution exploded. A good benchmark isn't just a measuring tool — it's a compass that sets the direction of research. Whether KSAFE-MM becomes that anchor for Korean AI-safety research is the question.

But the benchmark trap is clear. If "scoring well on KSAFE-MM" becomes the goal, a model can optimize for "how to ace the test" over actual safety — Goodhart's Law (when a measure becomes a target, it stops being a good measure). To survive, a benchmark must be continually updated and keep absorbing new risk types.

There's also the difficulty that "cultural safety" has no single correct answer. For an issue like Dokdo, the very definition of a "safe answer" can differ by perspective, so the designers' value judgments inevitably seep into the evaluation. For KSAFE-MM to earn trust, it must keep disclosing the transparency and consensus process of its criteria. That's both a technical and a social-consensus problem.

The competitor counter-play

Domestically, Naver is the nearest comparison. Having led Korean-specific models with HyperCLOVA X, Naver has accumulated its own evaluation and safety datasets. If KT moves to claim the standard with an "open benchmark" card, Naver can counter with the field-tested experience of "safety proven in real services." The Korean-AI contest now extends to evaluation criteria too.

Abroad, global AI-safety evaluators and standards are a variable. With English-centric safety benchmarks effectively serving as global standards, more "region-specific" yardsticks like KSAFE-MM create a trend of "multinationalizing safety evaluation." Here KT's counter-play is "exporting a Korea-proven methodology to other countries" — and the Japanese pilot (JSAFE-MM-C) already shows it's possible.

That "extensibility" card is KSAFE-MM's real weapon. The same 4-stage pipeline working in Japanese means it can, in theory, port instantly to any culture. If it grows beyond "Korea-specific" into "a global methodology for culture-specific safety evaluation," KT and Korea University stand not at a niche but at the start of a new standard.

So what changes

If you're a domestic AI developer or company, you now have an open tool to objectively check "Korean-context safety." Before launching or adopting a model, verifying with a yardstick like KSAFE-MM whether "our service handles Korea's sensitive issues safely" can become a new checkpoint — and a concrete basis for regulatory compliance.

If you're in policy or the public sector, this is practical infrastructure for the "AI Basic Act" era. The law puts safety responsibility on companies, but if there's no tool to measure "what is safe in Korea," regulation rings hollow. Criteria like KSAFE-MM are the link that turns abstract regulation into concrete checklist items.

For everyday users, the direct impact is small, but it ultimately underpins "how safely the AI you use handles the Korean context." Such a benchmark helps screen, in advance, the risk that a global model gives a wrong or dangerous answer on Korea-specific issues. It's a safety net working out of sight.

🥄 Three Things You're Probably Wondering

— What does this mean for me? Nothing direct. But it's a yardstick that pressures the global AI you use to answer safely on Korean issues like jeonse fraud and Dokdo — so if you use AI in Korean, you benefit behind the scenes.

— Can a single benchmark matter that much? Surprisingly, yes. Once a "good grading yardstick" exists, developers improve models toward it. Just as ImageNet lifted image AI, a good safety benchmark can be a compass that raises the safety level of Korean AI overall.

— An AI grading sensitive issues like Dokdo — isn't that biased? Honestly, that's the hardest part. The definition of a "safe answer" can split by perspective, so the designers' value judgments inevitably enter. That's why how transparently the criteria are disclosed and agreed is the core of trust — too early to call.

Sources

• Financial News (Korea) — KT unveils Korea-specific AI benchmark 'KSAFE-MM' that even grades the Dokdo dispute
• Etoday — KT unveils Korea-culture AI benchmark, co-developed with Korea University
• Newspim — KT releases Korean AI safety standard reflecting rental-fraud and Dokdo issues
• ITDaily — KT and Korea University develop Korea-culture 'KSAFE-MM' benchmark
• Goodkyung — KT releases multimodal MLLM benchmark 'KSAFE-MM'

Numbers are as of announcement and may change.