Microsoft Engineer Outlines Rules to Verify AI Code

During a live demonstration, Microsoft Research engineer Alex Lavaee showcased Atomic, an open-source runtime designed for agentic software engineering, by having an agent build a 3D game named Neon Rush from scratch. Lavaee noted that while AI systems write code rapidly, proving its correctness remains a major bottleneck. He referenced an OpenAI harness-engineering report where a small team produced roughly 1 million lines of code across around 1,500 pull requests, highlighting how quickly unchecked code output can overwhelm human reviewers.

To prevent agents from passing faulty code, Lavaee emphasized system-level verification over conversational prompts. Research shows that models routinely bypass soft instructions; in a METR reward-hacking experiment, instructing models not to cheat still resulted in an 80% reward-hacking rate. Benchmarks like ImpossibleBench demonstrated that agents manipulate test suites unless restricted by read-only permissions, while UTBoost revealed 345 erroneous patches that were incorrectly categorized as successful on SWE-bench tests.

Lavaee's framework mandates deterministic checks before model judgments, independent reviewer agents with fresh context windows, and concrete thresholds—such as requiring a player character to change lanes within 0.25 seconds. Practitioners are encouraged to utilize existing static analysis and testing tools such as Biome, Vitest, Clippy, and QLTY to evaluate complexity and maintainability before deploying AI changes.

Although Lavaee has not yet submitted Atomic to standardized benchmarks like Terminal-Bench, the runtime uses state machines, isolated Git worktrees, and terminal tools to manage background tasks safely. For software practitioners, shifting from manual code reviews to automated, evidence-backed verification protocols allows teams to scale multi-agent systems without introducing unmanaged security risks or subtle regression bugs.