Inspired by
Paul Trillo
Create an AI generated stop motion video
It’s easy to generate single images with AI, but creating a consistent sequence, where one image flows naturally into the next, is much more challenging. Generative video tools can currently only produce short clips. How do you build a longer film made of connected scenes? When filmmaker Paul Trillo created the music video The Hardest Part for the band Washed Out, OpenAI’s video tool Sora only allowed for clips of 20 seconds at most. The video tells the story of a young couple’s life in a single, uninterrupted camera movement, from their sweet first encounter at school to adulthood. To create this effect, Trillo generated nearly 700 separate clips, often using prompts that were several paragraphs long. Each shot was between 15 and 20 seconds, amounting to over 230 minutes of raw footage. These were then edited into the final 4-minute video. In this exercise you’ll play with creating continuity by generating your own stop motion video.
Here’s an example of one of the prompts Paul used:
“one continuous zoom through the scene is a tunnel of time, following behind a young couple walking through the bedroom into another room, an infinite apartment in one point perspective, the couple are slow dancing in the hallway appearing infinite. the scene is moving rapidly, showing a straight view of infinite hallway with sheets of fabric floating in the air. we zoom through the foggy hallway. lit from above. depth of field slight out of focus background with sheets of fabric floating in the background. the shot is moody and cinematic, with a slight vignette and a warm, vintage tone from the era. the shot is captured on 35mm film, fuji film stock with an anamorphic 24mm lens halation and chromatic aberration. soft color palette subdued tones nostalgic scene from retro 80s cinema film grain aesthetic”
About the artist
Paul Trillo is a multi-disciplinary artist, writer, and director who challenges both his own curiosity and illusion with his experimental, conceptual, and highly technical films.
Link:
- Come up with an easy storyline for your stop motion animation. Choose something that transforms in a visible way, like a flower blooming, a character’s journey, or a day-to-night transition.
- Pick an image generation tool. If you don’t have access to DALL·E or Midjourney the free tool craiyon.com might work well. Create your first image by describing your opening scene in detail. Include information about style (is it a cartoon, photorealistic, or claymation?), the color palette, and specific elements you want to appear.
- After receiving your first image, prompt for the next frame by describing a subtle change to the scene. For example: “The same scene, but the flower’s petals are now 10% more open” or “The same character walking, but now their right foot is forward instead of their left.”
- Continue this process, creating 25-30 frames that show what happens next in your story. With each prompt, keep it consistent by looking at the previous image and only changing what needs to change.
- If the AI adds something unexpected or the frames don’t match perfectly, see it as a ‘happy accident’ that adds to your story. It can be nice to let the machine have a voice in the video too.
- Once you have all your frames, download them and use an online stop motion tool (such as Eagle Animation) or Photoshop to compile them into a stop motion animation. Experiment with different frame rates. Aim for about 0.2 seconds per frame to create a video that’s around 5 seconds long.
- You can add sound or music to make your animation more fun.
- Reflect on the process: At which points did the image generation tool keep things consistent? Where did it introduce unexpected variations? How did these shape your final narrative?