Art[of]Me - Prototype
It is the integration of generative audio-visual AI tools as collective co-creators of dreams and memories using real-time data in live-action film sequences.
Our test movie was
"MetaMorphosis":
It is the story of two entities fighting against their inner boundaries.
Basisvideo without AI content:
Art[of]Me - Prototype
It is a smart controller for cinematic events, to bring back a sense of performance and dreamlike unpredictability to what is essentially a static medium by using AI sequences which are generated while watching.
Idea
How do memories feel like?
When we remember, real experiences become blurred with stories we have heard, pictures we have seen, and things we have forgotten. We mix our own memories with the "data" our society produces. And AI collects these common memories. AI is not producing, it is reproducing. So we decided to use Generative AI to extend a shortfilm to put in common memories.
But to show these fleeting consistence, we chose not to preproduce the AI content, we wanted to do it in real-time, while watching the movie. So it is always same but diferent, like memories.
Parameter // "Influencer"
How do you feel, that this is really happening in this moment?
A big discussion was to find the right parameters to influence our prompts in ComfyUI for the MEMORY-PARTs.
For the prototype we decided to go for eye-catching "influencers":
LOCATION - the place where we are right now
PICTURE - taken while audience is arriving
HEAD LINE - loaded from "APA"-Website
SPECIAL PLACE - audience wrote on post-its while waiting for the show.
But there are more ideas to come like infrared picture.
Hard- and Software
So CorebreakerLabs put the team together: Juliana Neuhuber, director and mixed media artist
Felix Knoche, VFX specialist and AI artist
Andreas Heydecke and Chris Tatzer, Cyber Security specialists and Hard- and Software developer
Christian Stiegler, technology and ethics.
We’re working with two different AI layers so that we can achieve the required real-time-action speed. Even so, our setup “just” needs two consumer PCs with better graphics cards.
While the film is playing, the AI-generated frames are created; this takes about 50 seconds, and has to be considered in the editing.
Performance
So the 1st PC playes the shortmovie and creates parallel the AI parts based on our "influencers" in ComfyUI and combines it with the video.
This movie is now sent to the 2nd PC, where we add the 2nd layer of AI.
With a macropad now we can morph the original picture in real-time to AI generated pictures.
Prompts define the characters transformation form like "yellow cyber octopus" or "red spidery dragon".
In full transformation we see which trainingsdata the tool is useing. An interesting inside.
So, every performance is unique and fleeting, just like a moment in the theater.
What our audience during
the festival say:
pictures by Thomas Weilguny:
https://www.picdrop.com/tomweilguny/7QAVsnot8V
16th of October, 2026
“I cried; it really moved me (wow)”
– Flo
“It was a truly fascinating experience on several levels—from highlighting the use of AI in movies to playing with perception and the question of what is real and what is AI-generated, as well as the question of whether and how AI “thinks” [...] 😌 Truly a special experience that got me thinking—in a positive way!”
– Tomas
“Especially the part with the explanations and the live changes—that was the most exciting part for me in the show!”
– Michael