Semantic Image Interpretation - Integration of Numerical Data and Logical Knowledge for Cognitive Vision

Donadello, Ivan (2018) Semantic Image Interpretation - Integration of Numerical Data and Logical Knowledge for Cognitive Vision. PhD thesis, University of Trento.

[img]
Preview
PDF - Doctoral Thesis
4Mb
[img]PDF - Disclaimer
Restricted to Repository staff only until 9999.

907Kb

Abstract

Semantic Image Interpretation (SII) is the process of generating a structured description of the content of an input image. This description is encoded as a labelled direct graph where nodes correspond to objects in the image and edges to semantic relations between objects. Such a detailed structure allows a more accurate searching and retrieval of images. In this thesis, we propose two well-founded methods for SII. Both methods exploit background knowledge, in the form of logical constraints of a knowledge base, about the domain of the images. The first method formalizes the SII as the extraction of a partial model of a knowledge base. Partial models are built with a clustering and reasoning algorithm that considers both low-level and semantic features of images. The second method uses the framework Logic Tensor Networks to build the labelled direct graph of an image. This framework is able to learn from data in presence of the logical constraints of the knowledge base. Therefore, the graph construction is performed by predicting the labels of the nodes and the relations according to the logical constraints and the features of the objects in the image. These methods improve the state-of-the-art by introducing two well-founded methodologies that integrate low-level and semantic features of images with logical knowledge. Indeed, other methods, do not deal with low-level features or use only statistical knowledge coming from training sets or corpora. Moreover, the second method overcomes the performance of the state-of-the-art on the standard task of visual relationship detection.

Item Type:Doctoral Thesis (PhD)
Doctoral School:Information and Communication Technology
PhD Cycle:29
Subjects:Area 09 - Ingegneria industriale e dell'informazione > ING-INF/03 TELECOMUNICAZIONI
Area 01 - Scienze matematiche e informatiche > INF/01 INFORMATICA
Area 01 - Scienze matematiche e informatiche > MAT/02 ALGEBRA
Area 09 - Ingegneria industriale e dell'informazione > ING-INF/05 SISTEMI DI ELABORAZIONE DELLE INFORMAZIONI
Area 01 - Scienze matematiche e informatiche > MAT/01 LOGICA MATEMATICA
Funders:Fondazione Bruno Kessler
Repository Staff approval on:23 Apr 2018 11:10

Repository Staff Only: item control page