Multimodal Transformers for Cultural Heritage Protection: Identifying Genuine Artefacts in Online Discussions.

Enlarge

As part of the Digital Heritage Research Colloquium, Dr. Sara Ferro from the Italian Institute of Technologies -Centre for Cultural Heritage Technology will present a lecture titled “Multimodal Transformers for Cultural Heritage Protection: Identifying Genuine Artefacts in Online Discussions.”

Abstract:

The illicit trade in cultural artefacts presents an enduring global challenge, with genuine objects often circulating online among replicas and falsified items. This study proposes a multimodal transformer-based framework designed to assist in identifying potentially authentic artefacts within online discussions, thereby supporting efforts to monitor and counteract illicit trafficking. The approach integrates Vision Transformers (ViTs) and Text Transformers to analyse both the visual characteristics of artefacts and the accompanying discourse drawn from Reddit communities. ViTs extract detailed stylistic and material cues from images, while Text Transformers interpret linguistic signals, sentiment, and consensus patterns within user comments relating to authenticity. Through a multimodal fusion mechanism, the model learns to correlate visual and textual evidence, distinguishing between likely replicas and artefacts that merit further expert examination. Rather than offering a definitive authenticity judgement, the system prioritises items with strong indicators of genuineness for targeted human review. This methodology demonstrates the potential of transformer architectures to enhance digital monitoring workflows, offering a scalable and data-driven tool for cultural heritage protection and anti-trafficking initiatives.

The lecture takes place on Thursday, 18 December 2025, at 18:00 c.t., in Lecture Hall A119, LMU Main Building, Geschwister-Scholl-Platz 1.

Flyer Download: Dr Ferro Public Lecture (JPG, 6,032 KB)