publications

2024

1. 

   Cloth-Splatting:3D Cloth State Estimation from RGB Supervision

   Alberta Longhini, Marcel Büsching, Bardienus Pieter Duisterhof, and 4 more authors

   In 8th Annual Conference on Robot Learning (CoRL), 2024

   Abs PDF Website

   We introduce Cloth-Splatting, a method for estimating 3D states of cloth from RGB images through a prediction-update framework. Cloth-Splatting leverages an action-conditioned dynamics model for predicting future states and uses 3D Gaussian Splatting to update the predicted states. Our key insight is that coupling a 3D mesh-based representation with Gaussian Splatting allows us to define a differentiable map between the cloth’s state space and the image space. This enables the use of gradient-based optimization techniques to refine inaccurate state estimates using only RGB supervision. Our experiments demonstrate that Cloth-Splatting not only improves state estimation accuracy over current baselines but also reduces convergence time by ∼ 85 %.

2. 

   Unfolding the Literature: A Review of Robotic Cloth Manipulation

   Alberta Longhini, Yufei Wang, Irene Garcia-Camacho, and 8 more authors

   Annual Review of Control, Robotics, and Autonomous Systems, 2024

   Abs PDF

   The realm of textiles spans clothing, households, healthcare, sports, and industrial applications. The deformable nature of these objects poses unique challenges that prior work on rigid objects cannot fully address. The increasing interest within the community in textile perception and manipulation has led to new methods that aim to address challenges in modeling, perception, and control, resulting in significant progress. However, this progress is often tailored to one specific textile or a subcategory of these textiles. To understand what restricts these methods and hinders current approaches from generalizing to a broader range of real-world textiles, this review provides an overview of the field, focusing specifically on how and to what extent textile variations are addressed in modeling, perception, benchmarking, and manipulation of textiles. We finally conclude by identifying key open problems and outlining grand challenges that will drive future advancements in the field.

3. 

   AdaFold: Adapting Folding Trajectories of Cloths via Feedback-loop Manipulation

   Alberta Longhini, Michael C Welle, Zackory Erickson, and 1 more author

   Robotics and Automation Letters (RA-L), 2024

   Abs PDF Website

   We present AdaFold, a model-based feedback-loop framework for optimizing folding trajectories. AdaFold extracts a particle-based representation of cloth from RGB-D images and feeds back the representation to a model predictive control to replan folding trajectory at every time-step. A key component of AdaFold that enables feedback-loop manipulation is the use of semantic descriptors extracted from geometric features. These descriptors enhance the particle representation of the cloth to distinguish between ambiguous point clouds of differently folded cloths. Our experiments demonstrate AdaFold’s ability to adapt folding trajectories of cloths with varying physical properties and generalize from simulated training to realworld execution.

4. 

   Standardization of cloth objects and its relevance in robotic manipulation

   Irene Garcia-Camacho, Alberta Longhini, Michael Welle, and 3 more authors

   2024 IEEE International Conference on Robotics and Automation (ICRA), 2024

   Abs PDF Website

   The field of robotics faces inherent challenges in manipulating deformable objects, particularly in understanding and standardising fabric properties like elasticity, stiffness, and friction. While the significance of these properties is evident in the realm of cloth manipulation, accurately categorising and comprehending them in real-world applications remains elusive. This study sets out to address two primary objectives: (1) to provide a framework suitable for robotics applications to characterise cloth objects, and (2) to study how these properties influence robotic manipulation tasks. Our preliminary results validate the framework’s ability to characterise cloth properties and compare cloth sets, and reveal the influence that different properties have on the outcome of five manipulation primitives. We believe that, in general, results on the manipulation of clothes should be reported along with a better description of the garments used in the evaluation. This paper proposes a set of these measures.

5. 

   Distilling Semantic Features for 3D Cloth Representations from Vision Foundation Models

   Alberta Longhini, Marcel Büsching, Bardienus Pieter Duisterhof, and 3 more authors

   In ICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024

   Abs PDF

   This study explores the use of vision foundation models to enhance 3D representations of cloth-like deformable objects. By focusing on the distillation of semantic information from RGB images, we examine the potential of pre-trained Visual-Language Models in capturing complex folded configurations of cloth. Our investigation reveals the challenges and preliminary successes in leveraging semantic information to improve the understanding and tracking of deformable object states.

2023

1. 

   Edo-net: Learning elastic properties of deformable objects from graph dynamics

   Alberta Longhini, Marco Moletta, Alfredo Reichlin, and 4 more authors

   In 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023

   Abs PDF

   We study the problem of learning graph dynamics of deformable objects that generalizes to unknown physical properties. Our key insight is to leverage a latent representation of elastic physical properties of cloth-like deformable objects that can be extracted, for example, from a pulling interaction. In this paper we propose EDO-Net (Elastic Deformable Object - Net), a model of graph dynamics trained on a large variety of samples with different elastic properties that does not rely on ground-truth labels of the properties. EDO-Net jointly learns an adaptation module, and a forward-dynamics module. The former is responsible for extracting a latent representation of the physical properties of the object, while the latter leverages the latent representation to predict future states of cloth-like objects represented as graphs. We evaluate EDO-Net both in simulation and real world, assessing its capabilities of: 1) generalizing to unknown physical properties, 2) transferring the learned representation to new downstream tasks.

2. 

   Elastic Context: Encoding Elasticity for Data-driven Models of Textiles Elastic Context: Encoding Elasticity for Data-driven Models of Textiles

   Alberta Longhini, Marco Moletta, Alfredo Reichlin, and 6 more authors

   In 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023

   Abs PDF

   Physical interaction with textiles, such as assistive dressing or household tasks, requires advanced dexterous skills. The complexity of textile behavior during stretching and pulling is influenced by the material properties of the yarn and by the textile’s construction technique, which are often unknown in real-world settings. Moreover, identification of physical properties of textiles through sensing commonly available on robotic platforms remains an open problem. To address this, we introduce Elastic Context (EC), a method to encode the elasticity of textiles using stress-strain curves adapted from textile engineering for robotic applications. We employ EC to learn generalized elastic behaviors of textiles and examine the effect of EC dimension on accurate force modeling of real-world non-linear elastic behaviors.

2021

1. 

   Textile taxonomy and classification using pulling and twisting

   Alberta Longhini, Michael C Welle, Ioanna Mitsioni, and 1 more author

   In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021

   Abs PDF

   Identification of textile properties is an important milestone toward advanced robotic manipulation tasks that consider interaction with clothing items such as assisted dressing, laundry folding, automated sewing, textile recycling and reusing. Despite the abundance of work considering this class of deformable objects, many open problems remain. These relate to the choice and modelling of the sensory feedback as well as the control and planning of the interaction and manipulation strategies. Most importantly, there is no structured approach for studying and assessing different approaches that may bridge the gap between the robotics community and textile production industry. To this end, we outline a textile taxonomy considering fiber types and production methods, commonly used in textile industry. We devise datasets according to the taxonomy, and study how robotic actions, such as pulling and twisting of the textile samples, can be used for the classification. We also provide important insights from the perspective of visualization and interpretability of the gathered data.

2. 

   Learning the tuned liquid damper dynamics by means of a robust EKF

   Alberta Longhini, Michele Perbellini, Stefano Gottardi, and 3 more authors

   In 2021 American Control Conference (ACC), 2021

   Abs PDF

   The tuned liquid dampers (TLD) technology is a feasible and cost-effective seismic design. In order to improve its efficiency it is fundamental to find accurate models describing their dynamic. A TLD system can be modeled through the Housner model and its parameters can be estimated by solving a nonlinear state estimation problem. We propose a robust extended Kalman filter which alleviates the model discretization and the fact that the noise process is not known. We test the effectiveness of the proposed approach by using some experimental data corresponding to two classical seismic waves, namely the El Centro wave and the Hachinohe wave.

3. 

   DLO@Scale - A Large-Scale Meta Dataset for Learning Non-Rigid Object Pushing Dynamics

   Robert Gieselmann, Alberta Longhini, Alfredo Reichlin, and 2 more authors

   In NeurIPS 2021 - Workshop on Physical Reasoning and Inductive Biases for the Real World, 2021

   Abs PDF Website

   The ability to quickly understand our physical environment and make predictions about interacting objects is fundamental to us humans. To equip artificial agents with similar reasoning capabilities, machine learning can be used to approximate the underlying state dynamics of a system. In this regard, deep learning has gained much popularity but is relying on the availability of large-enough datasets. In this work, we present DLO@Scale, a new dataset for studying future state prediction in the context of multi-body deformable linear object pushing. It contains a large collection of 100 million simulated interactions enabling thorough statistical analysis and algorithmic benchmarks. Our data is generated using a high-fidelity physics engine which simulates complex mechanical phenomena such as elasticity, plastic deformation and friction. An important aspect is the large variation of the physical parameters making it suitable for testing meta learning algorithms. We describe DLO@Scale and present a first empirical evaluation using neural network baselines