[TUTO SD1.5] Basic guide to create animation with Stable Diffusion

[Attention] This was the workflow I was using with in 2023, with SD1.5 in a1111 (see the corresponding tutorials)
They are not related to the new method i'm experimenting with, as in end of 2025)

So! You wanna create anim with stable diffusion? Here is a quick guide that goes through the basics The first thing you'll need before eve


So! You wanna create anim with stable diffusion? Here is a quick guide that goes through the basics

The first thing you'll need before even opening Stable Diffusion, is an image sequence with the video you want to process.
I think it's waaaay better if you have an image sequence with the depth informations. It looks like this :

I'm personally using Maya to do quick animation, then i export those depth sequences, as well as other ones that can really help. But for this tutorial we'll keep it the simpliest possible.

In stable diffusion, you'll also need to have the Controlnet extension. This one is mandatory to create animations based on image sequences.

In most cases, you'll also need to have the temporalnet model installed

================ 1st step : create reference picture ========
Use the first frame of your depth pass, go in txt2img, plug it in controlnet like this :

Then, generate a first picture. It's gonna be the base of your animation. You can take your time for this one, just have fun. I usually set a high batch count, then I pick my favorite. This is the result :

===================== Step 2 : generate animation ==================

Now, go to img2img using the same positive prompt, negative prompt, seed, checkpoint, ...
Basically, use the same everything, like this :

Then scroll down to Controlnet. There are infinite ways to use it, depending on your animation you'll need to mix the CNET units in different ways to have good results.
Again, for this tutos, I'll just pick the simpliest way : 2 cells. One depth cell for the animation, and one  temporalnet cell, for consistency. Here are how I set them up :

Then, click on generate, wait for your frames to be done and voilà! You have your animation!

But that's it for this intro tutorial! You should have the basics now to create your animations!
Have fun!

If you want more, I shared with my patreons the resources I'm using to create all my animations Which includes :

- An in-depth, step-by-step guide on my worflow, with all the subtle details and tips I'm using through the full process.
- all my image sequences, like depth, normal, segmentation, opacity, beauty, ...
- python script I developped to process easily images (create videos, process pictures, batch rename, ...)
- script to setup an environment on a rented GPU, so you can have the fatest machine ever in few minutes
- and of course, specific animations, requests, tips, and more