MENTIS Framework Adds Human Beliefs to AI World Models

Current AI world models like Sora, Genie 3, JEPA, and Marble only track physical attributes like motion and occlusion, failing to anticipate human actions driven by hidden mental states. To bridge this gap, researchers introduced the Mental World Modeling (MWM) framework. Its training-free reference implementation, MENTIS, splits actions into physical carriers and mental payloads, simulating how choices affect both physical and mental states. The pipeline evaluates options based on physical plausibility, mental consistency, and social appropriateness.

To evaluate the framework, the researchers created Menti-Bench, a dataset of 448 decision scenes consisting of 320 text descriptions, 100 picture stories, and 28 sound-video clips, with 78 percent involving multiple characters. They tested eight language models, including OpenAI's GPT-5.6-Sol and GPT-4.1, alongside Anthropic's Claude Fable 5, Claude Opus 4.8, and Claude Haiku 4.5. Using MWM, the average F1 accuracy score reached 87.9, compared to 63.3 for direct answers and 77.9 for self-consistency. Humans scored 98.5 under the same protocol. Notably, the weakest model using MWM (GPT-4.1 at 84.9) outperformed the strongest model using direct self-consistency (GPT-5.6-Sol at 83.6).

The study highlighted the necessity of both channels. Removing the mental channel dropped scores by an average of 12.1 points, while removing the physical channel caused a 16.5-point decline. Predicting transitions independently instead of coupling them lost 6.4 points. In interpersonal scenes, the F1 score improved by 26.4 points, whereas object-focused scenes saw a 14.0-point gain. The performance gap between MWM and direct answers was 28 points for GPT-4.1 and 21 points for GPT-5.6-Sol.

For AI practitioners building collaborative agents or service robots, this framework suggests that scaling up compute or retrying direct answers cannot substitute for explicit mental modeling. The researchers identified next-state simulation as the primary bottleneck. Replacing simulated transitions with human reference solutions yielded a 3.5-point boost, while perfect initial states added 2.8 points and perfect observations added 1.7 points. Fully optimizing these intermediate transition simulations will be crucial for developers aiming to close the remaining gap to human-level prediction.