Fantastic Futures 2024 - Day 2 - Session 13

This transcript was generated by NFSA Bowerbird and may contain errors.

Anyway, first of all, I just want to say a big thank you to everyone for giving my first AI existential crisis this morning at 2am, which I think is a good thing because basically it means I care a lot now about what I'm doing. Okay, so this is really two stories in one in the next five minutes. First story is of myself going from an absolute noob when it came to AI and ML, knowing nothing in July last year, to knowing a little bit more and just enough to help provide a service for researchers now. Where's my clicker? Okay, so... It's also the story of the agency I work for, Public Record Office Victoria, which is a very small government agency. It's the State Archives of Victoria, Melbourne. And dipping our toes for the very first time in AI, machine learning, we're like a little baby just learning to crawl. Very, very cute. And I'd like to thank also not just everyone for giving me that existential crisis but Neil Fitzgerald for also giving the first five minutes of my talk on transcribers. That's fantastic, helps me. Oh there's this green, great. So 2023 July last year, I knew nothing, it was a black box, absolutely nothing about AIML. Spent looking at the free service of transcribers. had a go, was incredibly upset and gutted because what came out of transcribus was absolute garbage the first time I used it. Probably my fault, not theirs, no defamation cases things. Then from September to December, I got a bit more confident, I started creating a private model, went from about 14% character error recognition rate to about 5%, which was okay, not great, but something, so 95% correct. And then I used that for my first one-hand ML build, tested that against the Ground Truth records. That was okay, but still not incredibly happy with that. tried the public model which was about 2.75% accurate, so about 97 say, that was the character error recognition rate, 2.75. That's using the Text Titan public model for transcribers. Much, much better results. It's amazing how they improved. from July 23 to say January 24. Absolutely amazing, exponential improvements. So I got really, really confident, started using their public model, made leaps and bounds, and got to a point where I was so confident, I then tried that across a if I can get to the next slide, a whole series of, well a collection actually of about five different series of records, numbering about 6,712 pages. And I used the public model with that character error recognition rate of 2.75%, took about 46 hours to transcribe those 6,000 plus pages which was unbelievably good for a very small organisation such as us and really made me think, okay, there is definitely legs with this thing, I can keep going with this and learning, learning, learning as much because I was learning and testing and learning and testing at the same time concurrently. And then we came to our fantastic future where we were able then look at tabular records for the first time using the beta site and beta service from transcribers. Because remember they're learning as well as I'm learning, transcribers are improving all the time. They come out with this tabular records model which I used for the first time. My private model actually is much better than their public model for some weird reason. So I used my private model that I trained for tabular records. and got to a stage where I thought, you know what, we can actually start doing some really good work with our volunteers. So I got a small set of volunteers and this is coming up now where we can actually use that small set of volunteers and internally we can start working out a workflow for the volunteers to look at the transcriber's past transcriptions, make corrections where necessary, and we'll feed that back into improving the model if we can. And we're at a stage now where if that happens, we're going to have the ability, hopefully, through using AAAF Viewer and possibly a collaboration with Glycerin to provide a service, a crowdsourcing service, where people will come to our viewer They'll see the image, they'll see the transcription, they can make corrections in real time and will feed that service back into improving our metadata and transcriptions along the way. So that's where we're at now from someone who knew nothing in July 23 to hopefully providing some kind of crowdsourcing service with a AAA viewer, digitised records and the transcription from the machine learning model. We're currently now on a scholar account where we can process up to about 36,000 pages a year. That's costing us about 5K, not too bad really. And as I said, we're a small government agency and we're just dipping our toes. And I'm learning at the same time, just enough to teach other people in the organisation. So, I think we're pretty happy where we're going. I mean, my manager, Kate Follington, who's somewhere here in the audience, she's doing a photographic image description tagging AI project at the same time concurrently as my transcription project. So I'm hoping together we can start this culture of ML and I've had my existential crisis so I'm over that and hopefully now I can bring a bit more confidence in terms of data sovereignty and governance to the whole culture. And that's where we're at. Thanks for your time.