Why embodied cognition models are critical to the future of design AI

In design work, we help to create technology that best serves people’s needs. At the moment, we increasingly see AI make progress toward amazing performance in select design tasks, but clearly not all of them. Is there something wrong specifically with how current design AI is founded, and could there be a different way to push the development further?

I explore the concept of embodied cognition, the idea that thinking is shaped not only by the brain, but also by the body and its interaction with the physical environment. The current AI movement largely ignores this, which is why I want to highlight complementary work that may prove critical to the next breakthroughs in design AI.

Where limitations of current design AI solutions come from

Most current AI solutions for digital product design rely on frontier large language models (LLMs). This means these models have learned all they know from the plethora of written and audio-video sources, digital and physical. Although one can be continuously surprised by how much the models know, they still exhibit an uncanny lack of understanding in many very human matters, known as the jagged frontier.

One reason for this is that the models have not gained their information through interaction with the physical world (resulting in so-called embodied knowledge), but only through chunks of digital information. That is why some researchers, not only in robotics, have long sought to build other types of AI models that are based on the dynamics of the physical world. These now provide complementary perspectives for LLMs, and they are not only so-called world models (see, e.g., Genie by Google). What should we think about their relevance in the LLM-dominated AI debate?

Professor Antti Oulasvirta from Aalto University is the most celebrated human-computer interaction (HCI) researcher in Finland. His HCI career spans nearly two decades, and among many achievements, he was the first Finnish member elected to the American ACM SIGCHI Academy in 2025.
Currently, he runs the Computational Behavior Lab at Aalto, employing over 20 researchers at various career levels, from students to more than a handful of postdocs, who usually make the most scientific progress. His lab is one of the first institutes in the European ELLIS network focused on AI and machine learning research. Altogether, if you want to hear what serious researchers think of future design AI, Antti is clearly the person to talk to.

From embodied cognition to algorithmic representation of humans

“The focus of our research is how algorithms can represent humans realistically,” Antti explains to me around a cup of coffee one early September morning at Aalto University. Before he sat down, he showed me around a small lab where they were recently recording athletes’ movements during endurance sports for a very specific modeling task. Now we’re sitting in a new office building where Antti leads his groundbreaking work, and I’m trying hard to follow his dense tale of what his group is currently working on.

I’ve known Antti for over 25 years, and his evolving research interests have never stopped to amaze me. Back then, as today, his interest has been understanding humans through robust theories that can predict our behavior and preferences. The way that’s done has just radically changed over the years. When Antti was first appointed at Aalto as a professor, his lab was called User Interfaces; now it’s dubbed Computational Behavior. The new name describes his aims marvelously.

His group models that predict human behavior and thus help design the technology we interact with.

The algorithmic representation of human behavior and intent sounds like fancy basic science, but it does come down to practical applications. These include topics such as autonomous driving and amplified reading. The special ingredient of his current methodology is using a large combination of existing and custom-made models. For example, their present approach might include modeling of hundreds of muscles as well as cognitive mechanisms such as working memory and attention. These become crucial in applications where physical and digital must meet.

Waiting for the robots – where European research lags

“We must start to investigate the human-robot interaction. My Chinese colleague has seven humanoid robots at this lab, and that is not exceptional by local standards. Here in Europe, we are clearly behind in that research area,” Antti says in a concerned tone.

In China, humanoid robots have indeed gotten a head start. China deployed over 10,000 humanoid robots in 2025. However, the applications still seem to concentrate on traditional manufacturing as an extension of existing robotic automation, as well as education. For example, UBTECH’s Walker 1 humanoid robots have been employed in various factory environments, including car manufacturing. They are claimed to assemble complete cars without human help.

Robotics is an important research area in HCI, as we expect various forms of robots to become increasingly important in the future. This connects to the opening argument that one inherent limitation of LLM-style AIs comes from their teaching corpora (learning material) that are limited in ways that become apparent in robotics. It’s both how the physical world works as well as how humans behave in that world.

To develop effective robots for home and work, Antti sees it essential that the robots can understand (or model, to remain neutral to AI consciousness debate) what the humans around do, and intend to do.

This requires the robots to have comprehensive models of people as the current models are overly simplistic according to Antti. Earlier approaches of making robots more understandable to humans, such as the Baxter by Rethink Robotics (2012), are inadequate. Robot makers are striving for performance exceeding human capacity and the scenario of peaceful co-existence, human safety must be guaranteed. And naturally, the Chinese are into this as well, developing embodied foundation models. In China, embodied intelligence is mentioned in the 2025 Government Work Report as a national strategic target. There’s some catching up to do on the western front.

What the future AI-assisted design may look like

However, most of us are not waiting to design robots, but more traditional interfaces with advanced tools, so I poked Antti about that topic as well. I knew their group had repeatedly ventured into creating experimental AI design tools, so I was keen to know what they are currently looking into.

“We are actively developing systems that combine LLMs and Bayesian models. I believe AI systems will, in the near future, be capable of solving most UI design challenges and getting the right design. But it remains a job for humans to steer the AI to solve the right challenges, point it to the right problem space.”

The lure of Bayesian optimization comes from solving design problems optimally. According to Antti, inclusion of Bayesian principles allows to develop design models that boost the traditional double diamond design process by learning from past progress even if the problem is reframed as often happens in design. In their recent study, Bayesian model represents the design problem, and the LLM creates solutions under the supervision of Bayesian model solve. So, the endgame for human designers seems to be framing problems for AI, then getting getting solutions for review and refinement. On a high level, this sounds like much what we’re already doing, but with increasingly sophisticated models.

The other direction is the use of embodied cognition models that bring in more comprehensive understanding of how humans think. They call it “simulation intelligence” that embeds intelligence about the person with model of the world in which they dwell. This could be applied across contexts, not only for robotics.

“Ten years ago, most of design efforts were spent in interface creation, in future it will be defining these tasks and refining the AI outputs.”