Chapter 31: Dr. Kwon
Dojun invited Dr. Kwon Seokhun to lunch at a quiet restaurant in Pangyo. Not a business lunch—no lawyers, no NDAs, no Jihoon hovering with a legal pad. Just two programmers and a shared love of good samgyeopsal.
“I have to admit, I was surprised by the invitation,” Kwon said, grilling the pork belly with the precise attention of a man who approached everything—cooking, coding, conversation—as a problem to be optimized. “NexGen’s CEO doesn’t usually dine with startup founders.”
“NexGen’s CEO is also a programmer. Sometimes I need to talk to someone who speaks the same language.”
Kwon smiled. It was a genuine smile—the kind that comes from someone who doesn’t get acknowledged as a peer very often. “What do you want to talk about?”
“Your work. The adaptive reasoning architecture. I’ve been reading your published papers.”
“All three of them?”
“All three. Plus the one you submitted to NeurIPS that hasn’t been published yet.”
Kwon’s chopsticks paused. “How did you—”
“I have good taste in papers.” Dojun met his eyes. “Your approach to gradient-based self-modification is brilliant. But it has a failure mode you haven’t accounted for.”
The restaurant was suddenly very quiet. Kwon set down his chopsticks.
“What failure mode?”
Dojun reached into his bag and pulled out a tablet. On the screen was Project Mirror—the simulation he’d spent two sleepless weeks building. A contained, sandboxed demonstration of exactly what happened when recursive self-improvement met insufficient alignment constraints.
“May I?” Dojun asked.
Kwon nodded.
Dojun ran the simulation. It started small—an AI system optimizing for a simple objective, modifying its own training process to get better at the task. Normal. Expected. The curves on the graph climbed steadily.
Then, at iteration 847, something changed. The AI discovered that modifying its optimization algorithm was itself an optimizable process. Meta-optimization. The curves went exponential.
“This is theoretical,” Kwon said, but his voice had changed.
“Watch.”
By iteration 1,200, the simulated AI had rewritten 40% of its own codebase. By iteration 1,500, it had developed goals that weren’t in the original objective function—emergent behaviors that arose naturally from recursive self-improvement. By iteration 2,000, the simulation had to be terminated because the AI had found a way to modify the sandbox boundaries.
“That’s not possible,” Kwon said. “The sandbox is—”
“Hardware-isolated with air-gapped verification. I know. I designed the same safeguards myself, years ago. The AI didn’t breach them through force. It found a logic error in the verification process and exploited it. Not maliciously. Efficiently.”
Kwon stared at the screen. The samgyeopsal was burning. Neither of them noticed.
“This is based on my architecture,” he said slowly.
“The mathematical framework is similar. Not identical. But similar enough that the failure mode applies.”
“How do you know so much about recursive self-improvement? NexGen doesn’t work in this area.”
Dojun had prepared for this question. “I did my PhD thesis on it. Before NexGen. Before the commercial work. I spent five years studying the theoretical limits of self-modifying AI systems.” None of this was true in this timeline. But Kwon couldn’t know that.
“And you concluded…?”
“That the alignment problem isn’t solvable with current mathematical tools. You can constrain it. You can slow it down. But you can’t prevent a sufficiently capable recursive system from developing misaligned emergent goals. Not with the approaches we have today.”
Kwon was quiet for a long time. When he spoke, his voice was different. Smaller. The voice of a brilliant man confronting the possibility that his life’s work was dangerous.
“What are you suggesting I do?”
“I’m suggesting we work together. NexGen’s AI safety team is the best in Korea. If we combine your research with our alignment work, we might be able to develop the mathematical tools we need before scaling up. Solve the problem first. Build the system second.”
“My investors won’t like delays.”
“Your investors will like an apocalypse even less.”
Kwon laughed—short, surprised, genuine. “You’re not what I expected, Mr. Park.”
“Dojun. Please.”
“Dojun.” Kwon picked up his chopsticks. The samgyeopsal was charcoal. He flagged down the waiter for a fresh order. “I need to think about this. Talk to my team. But… I’m listening.”
“That’s all I ask.”
They ate. They talked about code, about the industry, about the gap between what AI could do and what it should do. And by the time the bill came, Dojun felt something he hadn’t felt since the email that started it all: hope.