4D ANALYSIS BEYOND THE VISIBLE SPECTRUM IN REAL-LIFE ENGINEERING APPLICATIONS

The research work was supported by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under the “First Call for H.F.R.I. Research Projects to support Faculty members and Researchers and the procurement of high-cost research equipment grant” (Project Number: FRI-FM17-2972).

 

 

Our World is 3D: The world we live is three-dimensional (3D). 3D geometry and time are probably the two most critical parameters of our lives! We perceive information from this world by several senses but it is doubtful that 3D vision is the most critical one. The 3D geometry plus the time in fact depicts the whole of our world, while the conventional 2D representation (using images or videos) is just a subset of the whole. To attribute human-like properties in machines we need to develop new 3D vision algorithms suitable for different types of computer systems, that is, 3D computer vision tools.

Our World is Temporally Dynamic: Time is an essential parameter of our world. We, the humans, the animated beings and the objects of this world are positioning in the 3D space and moving within it to make our world an “active/vivid” environment. Thus, only capturing and digitalizing objects’ 3D geometry is not adequate to attain intelligent capabilities to human-made machines and endow them with senses as we, the humans, use to explore, understand and interact with our world. Thus, we need to adopt a time-varying 3D modelling process that enables the reconstruction of the spatial and temporal diversity of the objects yielding a 4D (3D geometry plus the time) modelling/analysis.

As dimension increases the complexity exponentially increases as well. Thus, we need to research on new algorithms and tools that can make 4D computer vision applicable to real-life engineering applications in which time performance is critical. To reduce the cost, we exploit spatio-temporal dependencies on the 3D processed data including joint reconstruction, information predictive models and shape repairing algorithms to dramatically reduce the time required for a 4D modelling. In addition, the models should be enriched with semantics (describing historical, cultural, functional and constructional values of the object) to make them retrievable and searchable. Furthermore, semantics accelerates reconstruction since only the most salient information is taken into account.

Another important aspect is the exploitation of visual information other than of the visible Red Green Blue (RGB) spectrum. Currently, almost all the research on 3D analysis is for RGB data while other bands, such as thermal or hyper-spectral sensing, are left behind. These bands can provide useful information on object material type or can indicate material defects and decay phenomena.

4DBeyond researches on 4D analysis and computer vision for three real-life engineering applications, supporting one post doc and three PhD candidacies.

(i) 4D modeling in tangible/intangible cultural heritage assets (CH),

(ii) visible, hyper-spectral and thermal data analysis for identifying complex humans’ behaviors and

(ii) monitoring of tunnel infrastructures to identify cracks or other defects.   

Our Publications

Automatic 3D Modeling and Reconstruction of Cultural Heritage Sites from Twitter Images

This paper presents an approach for leveraging the abundance of images posted on social media like Twitter for large scale 3D reconstruction of cultural heritage landmarks. Twitter allows users to post short messages, including photos, describing a plethora of...

ADAPTABLE AUTOREGRESSIVE MOVING AVERAGE FILTER TRIGGERINGCONVOLUTIONAL NEURAL NETWORKS FOR CHOREOGRAPHIC MODELING

Choreographic modeling, that is identification of key choreographic primitives, is a significant element for Intangible CulturalHeritage (ICH) performing art modeling. Recently, deep learning architectures, such as LSTM and CNN, have been utilized forchoreographic...

Unsupervised 3D Motion Summarization Using Stacked Auto-Encoders

In this paper, a deep stacked auto-encoder (SAE) scheme followed by a hierarchical Sparse Modeling for Representative Selection (SMRS) algorithm is proposed to summarize dance video sequences, recorded using the VICON Motion capturing system. SAE’s main task is to...

Follow us

8 + 2 =