Prof. Hendrik Lensch:
Deep Learning for Videos
and Unstructured Point Clouds

9. 11. 2018 od 13:30, Malá Strana – posluchárna S5

Videos and unstructured point sets pose specific problems to deep learning approaches. In videos the shear amount of data, missing labels and the temporal continuity have to be considered. We present a scalable and robust method for computing a non-linear temporal video alignment across videos of different weather conditions. The approach autonomously manages its training data for learning a meaningful representation in an iterative procedure each time increasing its own knowledge. It leverages on the temporal nature of the videos themselves to remove the need for manually created labels. A second approach addresses video motion deblurring to correct for camera shake or moving objects in the scene. Based on a large, synthetically enhanced training data set we introduce a novel recurrent network architecture to deblur frames taking temporal information into account by aggregating scene information across frames. The framework successfully performs blind multi-frame deconvolution and even removes spatially varying blur.

The third part addresses deep learning on unstructured data. Traditional convolution layers are specifically designed to exploit the natural data representation of images – a fixed and regular grid. However, unstructured data like 3D point clouds containing irregular neighborhoods. We introduce a natural generalization flex-convolution of the conventional convolution layer along with an efficient GPU implementation, which allows for transfering best-practices and design choices from 2D-image learning methods directly to point cloud processing. We demonstrate competitive performance on benchmark sets using fewer parameters and lower memory consumption and obtain significant improvements on a million-scale real-world dataset. Ours is the first which allows to efficiently process 7 million points concurrently.

© 2013–2018 Univerzita Karlova, Matematicko-fyzikální fakulta. Design noBrother.
Za obsah odpovídá OPMK.