MatrAIx

MatrAIx is a population-scale simulated-user evaluation framework for testing AI systems and digital products with diverse persona agents.

Author: Koutian Wu; GitHub: ktwu01

Every product team wants to know how real users will react before launch. Will they leave when the price goes up? Will they keep using the app after the AI assistant gives a wrong answer? How long will they wait for a page to load? The honest way to find out is a human study, and human studies are slow, expensive, and small. You get dozens of people when you need thousands, and you get the people who answer surveys, who are not everyone.

MatrAIx tries to replace part of that with simulated users. I am one of the 93 authors of the paper, “MatrAIx: Simulating the World with 8.3 Billion Persona Agents,” posted to arXiv on August 4, 2026.

Three parts

Persona 8B. A set of 8.3 billion persona records, each described along 1,290 categorical dimensions. Some are sampled from a dependency graph, so that correlated traits stay consistent: a retired farmer in a rural county should not also be a 22-year-old crypto trader in Manhattan. Others are derived from human-written profiles. The team released a quality-filtered coreset of about one million personas, 599,847 grounded in real human profiles and 400,000 synthetic.

Playground. Four environments where persona agents use a product: Survey, AI Chatbot, Web, and App. A persona can fill out a questionnaire, chat with an assistant, click through a website, or tap through a mobile app.

Tasks. 1,010 application tasks across more than 25 domains, including commerce, software, finance, and healthcare.

Does a persona behave like its description?

This is the question that decides whether the whole idea works. If you tell an agent it is price-sensitive and impatient, does it act price-sensitive and impatient, or does it act like a generic helpful assistant?

The paper tests this directly. In a controlled study of 400 trials covering ten behavioral attributes across all four environments, the declared behavior was expressed, or correctly held back, in 366 trials, which is 91.5 percent.

Beyond that check, the team ran 18,189 evaluation trials on eight representative tasks, with persona agents powered by Claude Opus 4.8, GPT 5.5, and Claude Haiku 4.5. The feedback varied by persona in the ways you would hope to measure: hesitation after a price increase, willingness to keep using a product after an AI failure, tolerance for latency.

What I am skeptical about

91.5 percent trait fidelity is a strong result for the question it answers. It is a different question from whether simulated users predict what real users do. A persona that faithfully acts “impatient” still acts the way a language model imagines an impatient person acts. If the model’s picture of impatience is off, every persona inherits the same error, and a million agents agree with each other for the wrong reason.

So I read MatrAIx as a fast, cheap first filter. Run the simulated panel to find the obvious failures and the surprising splits between user groups. Then spend the human study budget on the questions the simulation flagged. That workflow is already a large saving, and it does not require trusting the simulation more than the evidence supports.

The work has been picked up widely, with coverage in WIRED Czech, Numerama, Forbes Türkiye, NZZ am Sonntag, and Hugging Face Daily Papers. The code and persona data are on GitHub, and there is a demo video.

A simulated crowd is useful the moment it tells you where to look, long before it tells you the answer.

Explore MatrAIx.