Diffusion Models for Video Generation
<p><a href="https://lilianweng.github.io/posts/2021-07-11-diffusion-models/">Diffusion models</a> have demonstrated strong results on image synthesis in past years. Now the research community has started working on a harder task—using it for video generation. The task itself is a superset of the image case, since an image is a video of 1 frame, and it is much more challenging because:</p> <ol> <li>It has extra requirements on temporal consistency across frames in time, which naturally demands more world knowledge to be encoded into the model.</li> <li>In comparison to text or images, it is more difficult to collect large amounts of high-quality, high-dimensional video data, let along text-video pairs.</li> </ol> <blockquote> <p><br/><b> 🥑 Required Pre-read: Please make sure you have read the previous blog on <a href="https://lilianweng.github.io/posts/2021-07-11-diffusion-models/">“What are Diffusion Models?”</a> for image generation before continue here. </b><br/><br/></p>