Intermediality and Computational Humanities Hackathon 14-15 November 2024

REGISTRATION CLOSED ON 6 OCTOBER

Thank you all for the great interest in this event.

Analysing intersections and interconnections of different media in digitised sources has recently gained attention in Arts, Humanities, and Digital Humanities (DH). In a paper published last year, Smits and Wevers make their case for a multimodal turn in DH. New technologies like large language models provide automated support for recognising specific contents, structures, and relations within pictorial and textual contents of cultural heritage data. But how are research communities approaching the analysis of text and image sources? Which challenges do researchers encounter when dealing with multimodal sources?

The Digital Humanities Team at the University of Vienna and the Ottoman Nature in Travelogues (ONiT) project are pleased to announce a hackathon dedicated to exploring current challenges and solutions related to the analysis of texts, images, and multimodal sources. The hackathon is targeting scholars with backgrounds in Arts and Humanities as well as Computer Science with an interest in Computational Humanities methods. The aim of the hackathon is to bring together tool providers and art and history researchers to collaboratively focus on the analysis of digitised texts, images, or an inter-medial approach.

The hackathon will be a two-day event starting on Thursday, November 14 at 9:00 CET and ending on Friday, November 15 at 15:00 CET. On the first day of the event, participants will be invited to propose topics and challenges related to the theme of the hackathon that they wish to work on. Working groups will then be formed from the participants to investigate these challenges. The working groups will be flexible, and participants will be welcome to work on multiple topics throughout the course of the event. With this in mind, participants are invited to suggest for potential topics and lines of enquiry in the registration form below.

The hackathon will include two keynote lectures and a social dinner events for the participants sponsored by the University of Vienna and the ONiT project. Small bursaries will be available as necessary for the covering of travel costs.

Keynote Day 1: Thomas Smits and Melvin Wevers (University of Amsterdam)

Are we at a Multimodal Turn? Navigating AI Developments for Historical Research

In the dynamic field of Digital Humanities (DH), research methodologies have traditionally focused on textual data. The introduction of deep learning, however, has expanded this focus by facilitating the automated analysis and labeling of visual materials. Although powerful, these initial methods required large training datasets. The field experienced another shift with the development of multimodal deep learning architectures like Contrastive Language Image Pre-training (CLIP). More recent advancements have integrated GPT-inspired interfaces for visual analysis, significantly enhancing the scope of multimodal research. As a result, humanists are now on the brink of fully embracing computational visual analysis.

This keynote aims to spotlight these advancements and probe deeper into their alignment with multimodal theory. By doing so, it strives to understand their ramifications on the humanistic engagement with visual mediums. As we journey through this alignment, we find ourselves at a crossroads, grappling with pressing dilemmas of practicality, adaptability, and choice. Can the humanistic community keep pace with these swift technological evolutions? And, more fundamentally, is there an imperative to stay abreast, or should we gravitate towards more established techniques, offering greater control and explainability?

Keynote Day 2: Matteo Valleriani (Max Planck Institute for the History of Science, TU Berlin, Tel Aviv University)

Networks and AI: General trend analysis of multimodal historical data

This presentation explores the integration of network analysis and artificial intelligence (AI) in the study of multimodal historical data. By examining various data types, including visual elements, texts, tables, and metadata, we demonstrate how AI and network theories can uncover patterns and trends in historical research. We will highlight the role of explainable AI (XAI) in enhancing the interpretability of these complex analyses. The session aims to provide insights into how the convergence of these technologies can transform history writing, offering new perspectives on the relationship between macro and micro histories.

If you have any questions, feel free to send us an email at hackathon.geschichte@univie.ac.at.

The event is co-sponsored by the thematic platform MLA2S (Machine Learning at the Austrian Academy of Sciences) of the OeAW.