Fantastic Futures Canberra 2024

Fantastic Futures (FF24), the international conference on AI for Libraries, Archives, and Museums, was held at the National Film and Sound Archive of Australia (NFSA) in Canberra, Australia, on 15 to 18 October 2024.

The conference included a combination of international and Australian keynote presentations, workshops, demonstrations, academic papers and creative commissions. It was enfolded by a rich on-site experience comprising two days of bespoke pre-conference workshops, a networking dinner, Canberra tours and tailored events, including unique access to the NFSA collection and a performance by award-winning Australian audiovisual artist and composer Robin Fox.

Under the 2024 conference theme of Artificial Intelligence in the Future of Work in GLAMs (Galleries, Libraries, Archives and Museums), the four-day event facilitated exploration of the current state and potential futures of artificial intelligence and generative AI within the GLAM sector, through the lenses of history, language and culture in relation to place, particularly in an Australasian context. 

About AI4LAM

AI4LAM (Artificial Intelligence for Libraries, Archives, Museums) is a collaborative framework for libraries, archives and museums to organise, share and elevate their knowledge about and use of artificial intelligence. Individually we are slow and isolated; collectively we can go faster and farther. The Australia and Aotearoa New Zealand Regional Chapter of AI4LAM was initiated in 2020.

Session 1: Language & culture

In Conversation: Language, culture and the machines

KEY SPEAKERS:
KATHY REID
PETER-LUCAS JONES

HOSTED BY:
KEIR WINESMITH

From voice-activated devices to speech-to-text tools, voice recognition technology has grown in the last 15 years to become an indispensable part of modern living. As we tread alongside the path of its development, we pause to ask: who gets left behind? Kathy Reid (ANU), Peter-Lucas Jones (Te Hiku Media) and Keir Winesmith (NFSA) discuss bias in voice data, weaknesses in certain demographics and leading the movement to take ownership of language technologies away from American giants.

FF24 Day 1 - 1 - Peter-Lucas Jones, Kathy Reid, Keir Winesmith

Grant Heinrich

The first 140 days: How we're teaching an American transcription engine to speak Australian

With machine learning, AI and transcription being largely influenced by US companies, the NFSA set itself a challenge: turning an American transcription engine into one that can understand Australian accents and vernacular speech. Grant Heinrich discusses issues about influencing large data models with small collections of training data, a novel approach to accuracy, crowdsourcing corrections via Bowerbird UX, managing collection entities, and security around sensitive material.

We are looking at how we can transcribe our entire collection so we can find everything that's in it. It's a first step towards what we call a 'conversational archive'.

Grant Heinrich

Honiana Love

Join us on a journey to the future

Ngā Taonga Sound & Vision is Aotearoa New Zealand’s audiovisual archive with a collection spanning more than 100 years of rich history. Honiana Love offers an overview of the innovations they've adopted to transform an at-risk and largely inaccessible physical collection to a digitally preserved archive, whilst retaining their commitment to traditional Māori knowledge.

Javier de la Rosa

The Mímir Project: Evaluating the impact of copyrighted materials on generative large language models for Norwegian languages

Javier de la Rosa takes us through the research, evaluations and conclusions resulting from the Mímir Project: an initiative by the Norwegian government that aims to assess the significance and influence of copyrighted materials in the development and performance of generative large language models, specifically tailored for Norwegian languages.

Session 2: Conversational archive

Peter Leonard, Lindsay King

Beyond ChatGPT: Transformers models for collections

Transformers-based models have shown that many complicated problems in culture beyond digital text – a human voice, a handwritten word, even a scene from a motion picture – are now tractable to computation. Added to this is the simultaneous development of 'conversational interfaces' as a way of interacting with cultural material. Peter Leonard and Lindsay King draw on their experience working with Stanford's collections to explore several non-commercial multi-modal large language models and what conversational modalities can offer GLAM patrons.

Mia Ridge

Closing the loop: Integrating enriched metadata into collections platforms

Ingesting enhanced data appropriately and responsibly into collections management and discovery platforms is a key factor in operationalising AI to improve collections documentation and discovery, but is often difficult in practice. Mia Ridge presents original research into the extent to which projects can integrate enriched data into collections platforms, with success and failure results gathered from over 60 projects.

Joshua Ng

Archives in the cloud: Exploring machine learning to transform Archives New Zealand’s digital services for agencies

In 2022, Archives New Zealand spearheaded an innovative proof-of-concept project to explore the potential of machine learning and hyperscale cloud computing in auto-classifying digital public records and surfacing information of interest to Māori. Joshua Ng provides an overview of the process, the solutions that were developed and what to consider in the design and usage of these tools.

Peter Broadwell (Presented by Peter Leonard)

Applying advances in person detection and action recognition AI to enhance indexing and discovery of video cultural heritage collections

The ability to recognise and digitally encode human movements and actions, both in individual and interpersonal contexts, holds great potential for elevating the accessibility and understanding of digitised audiovisual materials. As part of the Machine Intelligence for Motion Exegesis (MIME) project, collaborators worked to apply new transformer AI-based video analysis models to archival recordings of theatre performances, political speeches and other events, exemplifying the methods’ potential relevance to any video collection containing human movement.

Session 3: Research & community

In Conversation: Research, collaboration and community

KEY SPEAKERS:
KIRSTEN THORPE
SYDNEY SHEP

HOSTED BY:
ROXANNE MISSINGHAM

Memory institutions across the globe are re-examining collecting practices and their role, historically, as reinforcers of the colonial project. Progress has been made in bringing Indigenous voices to the forefront, but with the proliferation of AI, archives are faced with an additional layer of responsibility as custodians of data. Kirsten Thorpe (University of Technology Sydney), Sydney Shep (Victoria University of Wellington) and Roxanne Missingham (ANU) discuss community-first approaches to archiving, data sovereignty and converting principles into action.

FF24 Day 1 - 9 - Kirsten Thorpe, Sydney Shep, Roxanne Missingham

Terhi Nurmikko-Fuller (Presented by Janet McDougall)

Mapping Indigeneity in institutional repositories

In the age of AI and large datasets, smarter data that has been represented in ways that capture various perspectives and nuances is needed. This has the power to simultaneously reduce historical and social bias and maximises the potential of bringing mapped data and powerful algorithms together. Janet McDougall reports on a linked data project in motion at the ANU, identifying all content relating to Indigenous research across its myriad repositories.

Benjamin Lee, Kath Bode, Andrew Dean

Reanimating and reinterpreting the archive with AI: Unifying scholarship and practice

From the application of AI-generated poetry to understand one of the earliest examples of generative language from the 1960s to using AI in the investigation of Irishness in Australian newspaper fiction, Benjamin Lee, Andrew Dean and Kath Bode present case studies on how academic AI projects can further scholarly inquiry and reinterpret digital collections.

James Smithies, Karaitiana Taiuru

Large language models and transnational research: Introducing the AI as infrastructure (AIINFRA) project

Despite the growing use of large language models as expert chatbots and research assistants, there have been no obvious or accepted ways to evaluate their quality as research tools. The AI as Infrastructure (AIINFRA) project aims to develop resources to resolve that situation, including a test framework and prototype AI research tool, with a focus on transnational collaboration, Slow AI, and Indigenous design principles.

Beth Shulman, Charlotte Bradley, Ripley Rubens

Imaginative Restoration

The National Institute of Dramatic Arts, ANU School of Cybernetics and the NFSA embarked on a collaborative project to demystify AI and conceptualise ways emerging technology could be used on archival media. The end result is Imaginative Restoration: an interactive installation that uses damaged media that has been restored with AI and breathes new meaning into it through drawings added by a user.

Session 4: Creativity & audiovisual

In Conversation: Creativity, artists and generative AI

KEY SPEAKERS:
KARTINI LUDWIG
ERYK SALVAGGIO

HOSTED BY:
MEAGAN LOADER

Generative AI remains a divisive topic among artists. On one hand, the nature of the machine presents us with unique transformations and speed in execution. On the other, the mechanisms that enable these feats spark concern about copyright and the role of the artist in the 21st century. Join Meagan Loader (NFSA) in conversation with artists Kartini Ludwig (Kopi Su) and Eryk Salvaggio (metaLAB (at) Harvard University) as they talk creative critical engagement, datafication, ethics and the possibilities when art and sound-making meets AI.

FF24 Day 2 - 1 - Kartini Ludwig, Eryk Salvaggio, Meagan Loader

Emmanuelle Bermès

Computer vision in the museum: Perspectives at the MAD Paris

The Musée des Arts Décoratifs in Paris (MAD) partnered with Ecole des chartes (ENC) and the national library of France (BnF) in order to bring together expertise with artificial intelligence and to prototype the automated annotation of large paper collections. Emmanuelle Bermès shares their experiments using the YOLO (You Only Look Once) model on the Royère collection of architecture and furniture drawings to perform object detection using a specific ontology.

Jeff Williams

AI ambassadors: How to demystify AI and encourage experimentation

In January 2024, ACMI launched an AI Ambassadors program with the goal of enhancing staff understanding through practical experimentation. Led by ACMI's Technology Team and primarily involving non-technical staff, the program focuses on demystifying machine learning, exploring generative AI fundamentals, debating ethics and usage, and providing assignments to expand on these topics through action. Jeff Williams discusses practical insights on how to approach knowledge-sharing and foster an environment of experimentation, enabling the integration of AI more effectively into organisations and removing the barriers to adoption.

Jon Dunn

Whisper applied to digitised historical audiovisual materials

Beginning in 2015, Indiana University (IU) undertook a massive project to digitise rare and unique audio, video and film assets from across IU’s collections which resulted in over 350,000 digitised items. With accessibility needs in mind, IU Libraries is exploring the feasibility of using the open source Whisper automated speech recognition tool in conjunction with IU’s HPE Cray EX supercomputer to generate captions or transcripts for this collection. Jon Dunn shares successes and failures, experiences processing files using a shared research-oriented high-performance computing environment and future plans for using the output to support accessibility and discovery.

Emily Pugh

AI-generated metadata and the culture of image search

PhotoTech is a four-year project headed by the Getty Research Institute, which focused on using AI for both art-historical research and metadata generation in relation to digital image collections. Based on the outcomes of this project, Emily Pugh provides an overview of the current approaches to image searching as they exist in the field of cultural heritage, showing the disconnects between these approaches and those required by an environment of AI-enhanced search.

Session 5: Culture & interpretation

Kirsten Thorpe, Lauren Booker, Robin Wright

FF24 Day 2 - 6 - Lauren Booker, Robin Wright, Kirsten Thorpe

Responsibility for the care and protection of Indigenous Knowledges in AI

Join Associate Professor Kirsten Thorpe (UTS), Dr Lauren Booker (UTS) and Robin Wright (Digital Preservation Coalition) as they introduce the 2024 University of Glasgow-led project iREAL: Inclusive Requirements Elicitation for AI in Libraries to support respectful management of Indigenous knowledges. iREAL aims to develop a model for responsible AI systems development in libraries seeking to include knowledge from Indigenous communities, specifically drawing on experiences of Aboriginal and Torres Strait Islander communities in Australia. The panel discusses ethical considerations needed to address bias in data held in libraries, archives and museums; the need for Indigenous participation in data governance; and the importance of wide sector dialogue in shaping the deployment of Indigenous data within AI systems.

Key is also the notion of consent... And in the context that we're working with historical collections, how do we embody and think about notions of free, prior, informed consent when people didn't have the ability to participate in those processes previously?

Kirsten Thorpe

Morgan Strong

Converging AI to access digital content in art museums

The Queensland Art Gallery | Gallery of Modern Art (QAGOMA) embarked on an ambitious project to leverage AI to make their digital content easily accessible on personal devices. By pointing a mobile device at an artwork – 2D or 3D – the artwork is identified rapidly to return related digital content, such as descriptions, colour and shape analyses, interactive components and interactive questions. Morgan Strong explores the selection process, challenges, onboarding visitors, model training and design considerations for the app.

Miguel Escobar Varela

AI-assisted analysis of Malay-language periodicals in Singapore

Miguel Escobar Varela offers an overview of the early stages of a collaborative project that brings historians, digital humanists and computer scientists together to explore the interconnectedness of Singapore's history with the broader Malay-speaking world during the late colonial period. The study takes a look at Malay-language periodicals published in Singapore, utilising a unique methodology that combines optical character recognition (OCR) and large language models to convert newspapers and magazines written in Jawi to Rumi, enabling the processing and analysis of a vast corpus of historical periodicals at an unprecedented scale.

Katrina Grant, Marni Williams

From small data to big knowledge networks: Preparing research practices and publishing infrastructure for the AI era

Humanities research data often remain hidden behind traditional forms of publishing or paywalls, creating a disconnect from this rich data being easily accessed, reused, aggregated and linked. To address this, a group of publishers, art historians and digital humanists at the Power Institute for Art and Visual Culture at the University of Sydney are developing new approaches to research that allow scholars, artists, community knowledge holders and museum partners to produce rich data sets, annotated objects and data visualisations, and to communicate them beyond the codex as part of an open publishing infrastructure.

Svetlana Koroteeva

FF24 Day 2 - 10 - Music Classification Pipeline Chart

Exploring possibilities to enhance bibliographical records for music collections using machine learning

Traditional music cataloguing in libraries involves a lot of manual work, where librarians manually categorise each piece of music by genre. This process is time-consuming and sometimes lacks consistency. Svetlana Koroteva presents their study on developing a tool that can correctly identify genres from records and audio features: beginning from selecting a model and preparing the data to training the model.

Session 6: Institutional adoption

Digitisation was only the start

Neil Fitzgerald

For around the last 30 years the British Library has digitised its collections. Historically, this work was undertaken not from core grant-in-aid funding but via external sources including research, philanthropic and commercial. Neil Fitzgerald explores potential approaches available to overcome barriers to applying state-of-the-art research, including collaboration via open frameworks and mutuality – an alternative path to reliance on purely commercial options that enables everyone to contribute to and benefit from the value chain.

Just one more access point: LLM assistance with authority control

Laura McGuiness

The National Security Research Center is a library within the Los Alamos National Laboratory (LANL) – a US laboratory responsible for solving national security challenges. In a field where reliable author attribution and access to research is critical, Laura McGuiness talks about the challenges of name disambiguation stemming from inconsistent sources, their investigation into utilising machine learning to create valid name authority records and the complexities of implementing a robust system.

19th century handwriting and model-making with Transkribus

Kate Follington (Presented by Asa Letourneau)

Government archives from the 19th century have traditionally been difficult to transcribe due to the variety of cursive handwriting found within records, with the added challenge of the scale of records held within state collections. Public Record Office Victoria shares their journey of experimentation using the tool Transkribus to build different machine learning models alongside public models to work out which transcription approach will yield the best return on investment in time and outcome.

FF24 Day 2 - 14 - Search system, LLMs and Vector Databases Chart

Enhanced stewardship and data sovereignty through the implementation of an ontology-enhanced large language model (LLM)

Lizabeth Johnson

The National Security Research Center within LANL manages an analogue and digital collection of lab reports from in-house scientists and partner facilities, in addition to gifted archival collections. Lizabeth Johnson outlines the library's plan to enhance searchability across multiple repositories by implementing a vector database and a tailored ontology, relayed to a large language model.

Evaluation of techniques that improve findability of historic images in a large and diverse corpus using AI vision models and embeddings

Francis Crimmins

The vast majority of the images held by the National Library of Australia do not have any detailed textual description, and keyword searches rely on matches against the title and author fields. Francis Crimmins showcases their experiments into implementing a system that uses an AI vision model to generate a detailed textual description of images from the catalogue, which are then given vector embeddings to be used in search and the exploration of similar images.

Adopting Whisper: creating a front end optimized for processing needs

Rachel Senese Myers

Georgia State University Library has a rich collection of audiovisual material within its Special Collections and Archives department, with over 3,000 audiovisual assets available to patrons. In order to facilitate better accessibility to current and future assets, the Library has started a three-phased project to identify, create and expand tools for fast AI-generated transcriptions while balancing responsible implementation.

Gender in a (supposedly) non-gendered historical space: Generative AI, sentiment analysis and graph databases in the Rockefeller Foundation Collections of the 1930s

Barbara Shubinski

The Rockefeller Archive Center initiated a research project aimed at the digitisation and subsequent analysis of analogue records from the Rockefeller Foundation collection. This resulted in the construction of a Neo4j knowledge graph database, encompassing over 80,000 nodes representing various entities and creating a visualisation of over one million relationships. Barbara Shubinksi demonstrates the potential in research and data analysis by showcasing an aspect the project uncovered: gendered assumptions within a corpus that ostensibly ignored gender, biases and institutional hierarchies.

AI literacy for librarians in low-resource, multicultural communities

Kara Kennedy

Emerging AI tools have seen use in recent years as a way to shortcut learning, potentially enabling students and teachers to customise lessons and overcome socio-economic and language barriers by providing on-demand access. Kara Kennedy offers a vision of what AI literacy looks like for librarians and their colleagues working in a low-resource educational environment servicing a high number of low-socioeconomic, multicultural customers.

Applying AI to accelerate data transcription from digital objects

Karen Thompson

Herbaria provide data that document the diversity, distribution and evolutionary history of life on earth, but these collections face the same challenges many museums currently contend with – a need for digitisation and findability. Karen Thompson introduces Hespi: a tool developed by the University of Melbourne that effectively identifies the location and orientation of text within an image then applies text recognition to provide a transcription.

Fantastic Futures 2025

Fantastic Futures 2025 will be held at the British Library (London) from 3 to 5 December 2025.

Visit the AI4LAM website for more information.

Fantastic Futures 2025 at the British Library on 3-5 December 2025