As of yesterday I have 99% completed my first ever AI commission, a video essay on the work of Michel Gondry, the writer/director of films including ‘Eternal Sunshine of the Spotless Mind’ and ‘The Science of Sleep’. I share it here with you as secret Xmas celebration gift! [It still needs a few tweaks here and there but for now this is a 720p preview version].
The commission arrives via Dr. Jack Sargeant, who for some time now has been producing the extras for Blu Ray editions released by Melbourne’s Umbrella imprint. The video essay accompanies ‘Science of Sleep.’
Jack previously asked me to contribute feature length commentaries for Shane Carruth’s ‘Primer’ and ‘Upstream Colour’. I also did some shorter audio commentaries for the found footage ‘Hell House’ series reissues. This was my first video essay commission and I was excited thinking about the possibilities of the form.
After ‘Mysteries of the Inky Rocks’ I decided to go big on the project and conceptualise it as taking place in the same world as that previous video, and thus, for the eagle eyed, you’ll notice a few references to Limanora scattered here and there. Indeed, the opening is set at the Kuleshov Institute for Creative Geography which, as everyone knows, is located in Nivaria, just down the road from the Oceanic Society.
The video is both direct and abstract, as befitting the subject, and recreates in a loose way some of Gondry’s signature visual and thematic motifs. As I was making the video, I took a break and Rachel and I went to visit my old friend Julie Ann Shea and her partner Paul at their place up north near Kempsey. Julie asked me whether I translated my dreams directly into the essay, and while some fragments of real dreams are included, what I discovered while I was snoozing under the gum trees, was that my subconscious was busy composing the video as I slept, and solved a few problems for me.
For those interested in the how, here’s a quick break down of production.
For ‘Dream Spaces’ I was working within a budget that roughly equated to a shooting ratio of 3:1 [three generations to every generation used]. Planning is key to maximising Runway as a video generation software, and for the most part I was able to pull off useable shots closer to a 1:1 ratio. When I forgot about my plan, things went awry and the ratio shot up. Meanwhile, the US$ rose against the Australian dollar, and the budget became redundant anyway, as the $ to credits ratio changed.
Images were generated in MidJourney and animated in Runway. I noticed a huge upswing in usability in the video software as the image to video + text prompts were capable of some incredible subtlety. Take a look at some of the facial expressions – these weren’t an accident – they were part of the prompt. Six months ago it was a miracle to get a figure to move, now you can type in ‘quizzical’ or ‘sadness’ and Runway can translate that into a facial expression.
The punk band sequence at the Petites Merdes club was a combination of image to video generations, but I also used Runway’s video to video technique for the lead singer. I shot a video of my lead actor Teddie N. Frost standing against a wall in our bedroom with Rachel holding a flashlight to the side for fill, then fed that video into Runway with a prompt saying something like ‘black and white film of punk singer on stage with lighting behind’ – and voila, the end result was incredible. Then there was a lot of editing matching up body movements and gestures to the lyrics in the song.
Also, a big shout out to Ross Wood who did some post production trickery for me on the shot of the theatre, animating the lights going down.
Speaking of which, ‘Dream Spaces’ is notable as the first time I’ve used Udio’s ai music generation software to create songs and incidental music. Unless you know it’s ai, I think it’s impossible to tell.
The whole thing was edited and sound mixed in iMovie, which turns out to be a completely capable semi pro piece of software. Plans are afoot to upgrade to Premier Pro in the near future, but for now, humble iMovie is fine.
sciencefictional.net 