Project Page

Open-Set Ego-Noise Separation for Legged-Robot Audition via Annotation-Free Adaptation and Pretrained-Model Transfer

Koki Shoda, Jun Younes Louhi Kasahara, Aoba Koyanagi, Qi An, and Atsushi Yamashita

The University of Tokyo

Examples

Example of ego-noise recorded by a humanoid robot, Unitree G1.
Example of ego-noise recorded by a quadruped robot, Unitree Go1.

Real-World Video Demo

Important setup: The audio in these videos was recorded by microphones mounted on the robot. Environmental sounds were played from an external speaker during robot operation.

Demo

Each example shows references first, followed by separated target estimates from each method.

CLAP Score is the CLAP audio embedding cosine similarity between the target and the separated estimate. SAJ Score is the overall separation judgment score. Higher is better.

Spectrogram color scale