publications

2024

1. 

   Distilling Semantic Features for 3D Cloth Representations from Vision Foundation Models

   Alberta Longhini, Marcel Büsching, Bardienus Pieter Duisterhof, and 3 more authors

   ICRA 2024 Workshop on 3D Visual Representations for Robot Manipulation, 2024

   Abs HTML

   This study explores the use of vision foundation models to enhance 3D representations of cloth-like deformable objects. By focusing on the distillation of semantic information from RGB images, we examine the potential of pre-trained Visual-Language Models in capturing complex folded configurations of cloth. Our investigation reveals the challenges and preliminary successes in leveraging semantic information to improve the understanding and tracking of deformable object states.

2023

1. 

   FlowIBR: Leveraging Pre-Training for Efficient Neural Image-Based Rendering of Dynamic Scenes

   Marcel Büsching, Josef Bengtson, David Nilsson, and 1 more author

   arXiv preprint, arXiv.2309.05418, Sep 2023

   Abs HTML

   We introduce a novel approach for monocular novel view synthesis of dynamic scenes. Existing techniques already show impressive rendering quality but tend to focus on optimization within a single scene without leveraging prior knowledge. This limitation has been primarily attributed to the lack of datasets of dynamic scenes available for training and the diversity of scene dynamics. Our method FlowIBR circumvents these issues by integrating a neural image-based rendering method, pre-trained on a large corpus of widely available static scenes, with a per-scene optimized scene flow field. Utilizing this flow field, we bend the camera rays to counteract the scene dynamics, thereby presenting the dynamic scene as if it were static to the rendering network. The proposed method reduces per-scene optimization time by an order of magnitude, achieving comparable results to existing methods - all on a single consumer-grade GPU.