4D BEYOND

NiDS 2021-Best paper Award

4dbeyond — Mon, 11 Oct 2021 08:16:44 +0000

Automatic 3D Modeling and Reconstruction of Cultural Heritage Sites from Twitter Images

4dbeyond — Wed, 26 May 2021 06:04:13 +0000

This paper presents an approach for leveraging the abundance of images posted on social media like Twitter for large scale 3D reconstruction of cultural heritage landmarks. Twitter allows users to post short messages, including photos, describing a plethora of activities or events, e.g., tweets are used by travelers on vacation, capturing images from various cultural heritage assets. As such, a great number of images are available online, able to drive a successful 3D reconstruction process. However, reconstruction of any asset, based on images mined from Twitter, presents several challenges. There are three main steps that have to be considered: (i) tweets’ content identification, (ii) image retrieval and filtering, and (iii) 3D reconstruction. The proposed approach first extracts key events from unstructured tweet messages and then identifies cultural activities and landmarks. The second stage is the application of a content-based filtering method so that only a small but representative portion of cultural images are selected to support fast 3D reconstruction. The proposed methods are experimentally evaluated using real-world data and comparisons verify the effectiveness of the proposed scheme. https://www.mdpi.com/2071-1050/12/10/4223/htm

ADAPTABLE AUTOREGRESSIVE MOVING AVERAGE FILTER TRIGGERINGCONVOLUTIONAL NEURAL NETWORKS FOR CHOREOGRAPHIC MODELING

4dbeyond — Tue, 25 May 2021 14:20:12 +0000

Choreographic modeling, that is identification of key choreographic primitives, is a significant element for Intangible CulturalHeritage (ICH) performing art modeling. Recently, deep learning architectures, such as LSTM and CNN, have been utilized forchoreographic identification and modeling. However, such approaches present sensitivity to capturing errors and fail to model thedynamic characteristics of a dance, since they assume a stationarity between the input-output data. To address these limitations,in this paper, we introduce an AutoRegressive Moving Average (ARMA) filter into a conventional CNN model; this means thatthe classification output feeds back to the input layer, improving overall classification accuracy. In addition, an adaptive imple-mentation algorithm is introduced, exploiting a first-order Taylor series expansion, to update network response in order to fit dancedynamic characteristics. This way, the network parameters (e.g., weights) are dynamically modified improving overall classificationaccuracy. Experimental results on real-life dance sequences indicate the out-performance of the proposed approach with respect toconventional deep learning mechanisms. https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/V-2-2020/467/2020/isprs-annals-V-2-2020-467-2020.pdf

Unsupervised 3D Motion Summarization Using Stacked Auto-Encoders

4dbeyond — Tue, 25 May 2021 13:51:03 +0000

In this paper, a deep stacked auto-encoder (SAE) scheme followed by a hierarchical Sparse Modeling for Representative Selection (SMRS) algorithm is proposed to summarize dance video sequences, recorded using the VICON Motion capturing system. SAE’s main task is to reduce the redundant information embedding in the raw data and, thus, to improve summarization performance. This becomes apparent when two dancers are performing simultaneously and severe errors are encountered in the humans’ point joints, due to dancers’ occlusions in the 3D space. Four summarization algorithms are applied to extract the key frames; density based, Kennard Stone, conventional SMRS and its hierarchical scheme called H-SMRS. Experimental results have been carried out on real-life dance sequences of Greek traditional dances while the results have been compared against ground truth data selected by dance experts. The results indicate that H-SMRS being applied after the SAE information reduction module extracts key frames which are deviated in time less than 0.3 s to the ones selected by the experts and with a standard deviation of 0.18 s. Thus, the proposed scheme can effectively represent the content of the dance sequence. https://www.mdpi.com/2076-3417/10/22/8226