A generated stop motion video
Inspired by: Paul Trlllo
Description of the work:
It’s easy to generate single images with AI, but creating a consistent sequence, where one image flows naturally into the next, is much more challenging. Generative video tools can currently produce clips of up to one minute, but how do you build a longer film made of connected scenes?When filmmaker Paul Trillo created the music video The Hardest Part for the band Washed Out, OpenAI’s video tool Sora only allowed for clips of 20 seconds at most. The video tells the story of a young couple’s life in a single, uninterrupted camera movement, from their sweet first encounter at school to adulthood. To create this effect, Trillo generated nearly 700 separate clips, often using prompts that were several paragraphs long. Each shot was between 15 and 20 seconds, amounting to over 230 minutes of raw footage. These were then edited into the final 4-minute video. In 2024, Trillo won the first-ever Golden Nica for AI in Art with this video clip.
One of the prompt Paul used:
“one continuous zoom through the scene is a tunnel of time, following behind a young couple walking through the bedroom into another room, an infinite apartment in one point perspective, the couple are slow dancing in the hallway appearing infinite. the scene is moving rapidly, showing a straight view of infinite hallway with sheets of fabric floating in the air. we zoom through the foggy hallway. lit from above. depth of field slight out of focus background with sheets of fabric floating in the background. the shot is moody and cinematic, with a slight vignette and a warm, vintage tone from the era. the shot is captured on 35mm film, fuji film stock with an anamorphic 24mm lens halation and chromatic aberration. soft color palette subdued tones nostalgic scene from retro 80s cinema film grain aesthetic”
Exercise:
Reflect on the process: Where kept the image generation tool things consistent? Where did it introduce unexpected variations? How did these shape your final narrative?
Come up with an easy storyline for your stop motion animation. Choose something that transforms in a visible way, like a flower blooming, a character’s journey, a developing Polaroid photograph, or a day-to-night transition.
Create your first image using an image generation tool by describing your opening scene in detail. Include information about style (like cartoon, photorealistic or claymation), the color palette, and specific elements you want to appear.
After receiving your first image, prompt for the next frame by describing a subtle change to the scene. For example: “The same scene, but the flower’s petals are now 10% more open” or “The same character walking, but now their right foot is forward instead of their left.”
Continue this process, creating 25-30 frames that show what happens next in your story. With each prompt, keep it consistent by looking at the previous image and only changing what needs to change.
If the AI adds something unexpected or the frames don’t match perfectly, see it as a ‘happy accident’ that adds to your story. It can be nice to let the machine have a voice in the video too
Once you have all your frames, download them and use a simple video editing tool (even Photoshop or PowerPoint can work) to compile them into a stop motion animation. Experiment with different frame rates. Aim for about 0.2 seconds per frame to create a video that’s around 5 seconds long.
You can add sound or music to make your animation more fun.