Topview Logo
  • Create viral videos with
    GPT-4o + Ads library
    Use GPT-4o to edit video empowered by Youtube & Tiktok & Facebook ads library. Turns your links or media assets into viral videos in one click.
    Try it free
    gpt video

    You Need This Hack To Get Consistent AI Video Using Stable Diffusion Controlnet and EBsynth | Part 1

    blog thumbnail

    You Need This Hack To Get Consistent AI Video Using Stable Diffusion Controlnet and EBsynth | Part 1

    In this article, we will explore a fascinating method to create consistent AI videos using stable diffusion, control net, and EBsynth. Inspired by the incredible videos made by Tokyo Japan, a creative creator who has mastered the art of creating high-resolution, realistic AI videos, we will delve into the two hacks he uses to outsmart the AI and achieve impressive results.

    Tokyo Japan shares his knowledge in a Reddit post that garnered over 500,000 views. Unfortunately, due to a well-known protest, popular subreddits have temporarily gone dark. However, Tokyo Japan generously shared all the details through email, allowing us to learn and replicate his technique. Before we dive into the hacks, let's take a moment to appreciate some of the amazing videos Tokyo Japan has created.

    Stunning AI Videos

    Tokyo Japan's AI videos are truly remarkable. They exhibit consistent high resolution and realistic visuals, setting them apart from other programs like stable warp fusion and Runway ml's gen 1, which are limited to digital graphics animation and anime-style videos. With Tokyo Japan's temporal consistency method, we can create realistic-looking videos without any flickering issues.

    In the video, we see various examples of Tokyo Japan's impressive work. From transforming a colored woman into a white woman to creating a Sylvester Stallone version of the author, the videos are sharp, clear, and lifelike.

    The Two Hacks

    Tokyo Japan employs two clever hacks to achieve consistent AI videos. The first hack involves using a free website called Sprite Sheet Packer. By creating a grid from multiple images, stable diffusion can generate consistent and flicker-free videos. While some users might suggest using the Temporal Kit extension from Chiara Rouse, Tokyo Japan prefers to load the grid image into the text-to-image tab and control it with stable diffusion's control net. By doing so, he can override the underlying video with greater consistency and fewer flickering issues, resulting in impressive and lifelike videos.

    To apply this hack, the video frames need to be exported in a 512 by 512 square format. Software like DaVinci Resolve can be used to export the frames, or the process can be done directly on a free website like AsGift.com. Once the frames are exported, they can be arranged into a grid using Sprite Sheet Packer, and the resulting grid image can be loaded into stable diffusion's text-to-image tab.

    The second hack involves using specific models and prompts in stable diffusion to refine the videos further. Tokyo Japan uses three models - Art and Arrows, Realistic Vision, and Cine Division - to create a realistic look. These models need to be downloaded from the Civid.ai website and installed in the stable diffusion models folder. Additionally, a variational autoencoder (VAE) filter is required, and all the models used by Tokyo Japan share the same VAE.

    With all the settings prepared, stable diffusion can generate consistent and flicker-free images. The prompts provided by Tokyo Japan help ensure consistent colors and lighting across the frames. By following the steps outlined in the article, you can replicate this process and create stunning AI videos.

    Keywords

    • AI videos
    • Stable diffusion
    • Control net
    • EBsynth
    • Consistency
    • Realistic visuals
    • Flickering issues
    • Sprite Sheet Packer
    • Grid method
    • Temporal Kit extension
    • Text-to-image
    • Models and prompts
    • Variational autoencoder (VAE)
    • Colors and lighting

    FAQ

    Q: Can I use other programs instead of stable diffusion for creating consistent AI videos? A: While stable diffusion is the main program used by Tokyo Japan, there are alternatives like Runway ml's gen 1 and stable warp fusion. However, these programs are currently limited to digital graphics animation and anime-style videos.

    Q: Are there any limitations to this method? A: The method described in this article provides impressive results, but it may not be suitable for all types of videos. Experimentation and fine-tuning may be required to achieve desired outcomes in different scenarios.

    Q: Can I use EBsynth without stable diffusion? A: EBsynth is a powerful tool for generating smooth transitions between images. While stable diffusion enhances consistency and reduces flickering, EBsynth can still be used independently to create visually pleasing videos.

    Q: How can I overcome flickering issues in AI videos? A: Tokyo Japan's technique of using a grid and controlling the images with stable diffusion's control net helps overcome flickering issues. Additionally, experimenting with models, prompts, and sampling steps can contribute to reducing flickering and achieving smoother results.

    Q: Can this method be applied to longer videos? A: Yes, in the second part of this tutorial, Tokyo Japan will explore techniques for creating longer videos by incorporating more keyframes and using larger grids. This method can be adapted to create videos ideal for platforms like YouTube shorts, Instagram, and TikTok.

    By following Tokyo Japan's hacks and techniques, you can create consistent and impressive AI videos using stable diffusion, control net, and EBsynth. Stay tuned for the second part of this tutorial, where we will delve deeper into the breakthroughs and knowledge gained by Tokyo Japan over the last three months.

    Disclaimer: The content shared in this article is based on the video presented and the techniques used by Tokyo Japan. The author has made efforts to provide accurate information but encourages readers to refer to the original video for a comprehensive understanding of the process.

    One more thing

    In addition to the incredible tools mentioned above, for those looking to elevate their video creation process even further, Topview.ai stands out as a revolutionary online AI video editor.

    TopView.ai provides two powerful tools to help you make ads video in one click.

    Materials to Video: you can upload your raw footage or pictures, TopView.ai will edit video based on media you uploaded for you.

    Link to Video: you can paste an E-Commerce product link, TopView.ai will generate a video for you.

    You may also like