Digital Users and Simulated Environments for
AI Training and Evaluation

About The Workshop

Many applications depend on understanding human behavior, from A/B testing and recommender systems to usability research and interactive LLM evaluation. Recruiting participants is slow, costly, and hard to scale, while live experiments can disrupt users or exclude rare populations. LLM-based digital users offer a scalable proxy: they generate human-like responses, navigate interfaces, express preferences, and pursue goals. Yet evaluating them in live websites, apps, and tools is difficult because content, layouts, locations, and bot-detection policies change constantly. Simulated environments provide stable testbeds, but they also have drawbacks: they may lack real-world diversity, miss important interface dynamics, encode unrealistic assumptions, and fail to generalize to real use.

So far, research on digital users and research on simulated environments have largely progressed as separate fields, leaving shared questions unresolved. We argue these are interrelated challenges: realistic digital users need faithful environments, and useful environments need realistic, diverse, fair behavior grounded in human data. This workshop brings the two communities together to develop methods for user modeling, benchmarks, behavioral calibration, environment construction, responsible simulation, and deployment. The goal is to assess when simulation is valid, where it fails, and how it can support scalable, reproducible research without treating synthetic proxies as human replacements.