Content uploaded by Gennaro Vessio
Author content
All content in this area was uploaded by Gennaro Vessio on Jan 24, 2022
Content may be subject to copyright.
Understanding Art with AI: Our Research Experience
Giovanna Castellano, Gennaro Vessio
Department of Computer Science, University of Bari Aldo Moro, Italy
Abstract
Articial Intelligence solutions are empowering many elds of knowledge, including art. Indeed, the
growing availability of large collections of digitized artworks, coupled with recent advances in Pattern
Recognition and Computer Vision, oer new opportunities for researchers in these elds to help the
art community with automatic and intelligent support tools. In this discussion paper, we outline
some research directions that we are exploring to contribute to the challenge of understanding art
with AI. Specically, our current research is primarily concerned with visual link retrieval, artwork
clustering, integrating new features based on contextual information encoded in a knowledge graph, and
implementing these methods on social robots to provide new engaging user interfaces. The application
of Information Technology to ne arts has countless applications, the most important of which concerns
the preservation and fruition of our cultural heritage, which has been severely penalized, along with
other sectors, by the ongoing COVID pandemic. On the other hand, the artistic domain poses entirely
new challenges to the traditional ones, which, if addressed, can push the limits of current methods to
achieve better semantic scene understanding.
Keywords
Digital Humanities, Visual arts, Articial Intelligence, Deep Learning
1. Introduction
Articial Intelligence is revolutionizing numerous elds of knowledge and has established
itself as a key enabling technology. Among the various domains that have been powered by
AI-based solutions there is also the artistic one. In fact, in recent years, a large-scale digitization
eort has been made, which has led to the increasing availability of huge digitized artwork
collections. And this availability, combined with the recent advances in Pattern Recognition
and Computer Vision, has opened up new opportunities for researchers in these elds to assist
domain experts, particularly art historians, in the study and analysis of visual arts. Among other
benets, a deeper understanding of visual arts can favor their use by an ever wider audience,
thus promoting the spread of culture. Visual arts, and more generally our cultural heritage, play
a role of primary importance for the economic and cultural growth of our society [1,2].
The ability to recognize characteristics, similarities and, more generally, patterns within and
between digitized artworks, in order to favor a deeper study, inherently falls within the domain
of human aesthetic perception [
3
]. Since this perception is highly subjective, and inuenced
by various factors, not least the emotion the artwork evokes in the observer, it is extremely
dicult to conceptualize. However, representation learning techniques, such as those on which
AIxIA 2021 Discussion Papers
giovanna.castellano@uniba.it (G. Castellano); gennaro.vessio@uniba.it (G. Vessio)
0000-0002-6489-8628 (G. Castellano); 0000-0002-0883-2691 (G. Vessio)
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
2. Visual Link Retrieval
visual link retrieval
historical knowledge discovery
Figure 1: Query examples and corresponding visually linked paintings.
3. Artwork Clustering
Figure 2: Sample images from the clusters found among Picasso’s artworks.
4. Computer Vision & Knowledge Graphs
5. Social Robotics
social robots
6. Conclusion
Digital Humanities
Acknowledgments
References
2
