The synthesis of human motion has traditionally been addressed through task-dependent models that focus on specific challenges, such as predicting future motions or filling in intermediate poses conditioned on known key-poses. In this paper, we present a novel task-independent model called MASK-M, which can effectively address these challenges using a unified architecture. Our model obtains comparable or better performance than the state-of-the-art in each field. Inspired by Vision Transformers (ViTs), our MASK-M model decomposes a human pose into body parts to leverage the spatio-temporal relationships existing in human motion. Moreover, we reformulate various pose-conditioned motion synthesis tasks as a reconstruction problem with different masking patterns given as input. By explicitly informing our model about the masked joints, our MASK-M becomes more robust to occlusions. Experimental results show that our model successfully forecasts human motion on the Human3.6M dataset. Moreover, it achieves state-of-the-art results in motion inbetweening on the LaFAN1 dataset, particularly in long transition periods.

Esteve Valls Mascaro, Hyemin Ahn and Dongheui Lee, A Unified Masked Autoencoder with Patchified Skeletons for Motion Synthesis, AAAI Conference on Artificial Intelligence (2024). (ArXiV, opens an external URL in a new window, Webpage, opens an external URL in a new window)

© Esteve Valls Mascaro

AAAI Overall Motivation

AAAI Overall Motivation

AAAI Overall Motivation

Programming by demonstration is reaching industrial applications, which allows non-experts to teach new tasks without manual code writing. However, a certain level of complexity, such as online decision making or the definition of recovery behaviors, still requires experts that use conventional programming methods. Even though, experts cannot foresee all possible faults in a robotic application. To encounter this, we present a framework where user and robot collaboratively program a task that involves online decision making and recovery behaviors. Hereby, a task-graph is created that represents a production task and possible alternative behaviors. Nodes represent start, end or decision states and links define actions for execution. This graph can be incrementally extended by autonomous anomaly detection, which requests the user to add knowledge for a specific recovery action. Besides our proposed approach, we introduce two alternative approaches that manage recovery behavior programming and compare all approaches extensively in a user study involving 21 subjects. This study revealed the strength of our framework and analyzed how users act to add knowledge to the robot. Our findings proclaim to use a framework with a task-graph based knowledge representation and autonomous anomaly detection not only for initiating recovery actions but particularly to transfer those to a robot.

Thomas Eiband, Christoph Willibald, Isabel Tannert, Bernhard Weber, and Dongheui Lee, Collaborative Programming of Robotic Task Decisions and Recovery Behaviors. Autonomous Robots (AURO), 47(2), 229-247, 2023. DOI:10.1007/s10514-022-10062-9. Paper, opens an external URL in a new window

© Thomas Eiband

Collaborative programming of robotic task decisions and recovery behaviors

Collaborative programming of robotic task decisions and recovery

behaviors

Collaborative programming of robotic task decisions and recovery behaviors

Smart manufacturing requires easily reconfigurable robotic systems to increase the flexibility in presence of market uncertainties by reducing the set-up times for new tasks. One enabler of fast reconfigurability is given by intuitive robot programming methods. On the one hand, offline skill-based programming (OSP) allows the definition of new tasks by sequencing pre-defined, parameterizable building blocks termed as skills in a graphical user interface. On the other hand, programming by demonstration (PbD) is a well known technique that uses kinesthetic teaching for intuitive robot programming, where this work presents an approach to automatically recognize skills from the human demonstration and parameterize them using the recorded data. The approach further unifies both programming modes of OSP and PbD with the help of an ontological knowledge base and empowers the end user to choose the preferred mode for each phase of the task. In the experiments, we evaluate two scenarios with different sequences of programming modes being selected by the user to define a task. In each scenario, skills are recognized by a data-driven classifier and automatically parameterized from the recorded data. The fully defined tasks consist of both manually added and automatically recognized skills and are executed in the context of a realistic industrial assembly environment.

Thomas Eiband, Florian Lay, Korbinian Nottensteiner, Dongheui Lee, Automatic Skill Recognition in a Knowledge-driven Robot Programming Framework, 5th International Conference on Industry 4.0 and Smart Manufacturing, 2023. (Paper, opens an external URL in a new window)

Automatic Skill Recognition in a Knowledge-driven Robot Programming Framework

Automatic Skill Recognition in a Knowledge-driven Robot Programming Framework

Automatic Skill Recognition in a Knowledge-driven Robot Programming Framework