My Journey to AI Video Generation

Intro

We live in an era dominated by AI. Whether it's Siri answering our questions or algorithms suggesting our next movie, artificial intelligence has ingrained itself into our daily routines. Being an enthusiast, I decided to take on the challenge of creating an AI animated video.

Experiment 1: AI Image Generators to Video Creation

First, I used an AI image generator and leonardo.ai or midjourney.ai to generate the image and make it a video using runway ML or Pika Labs. But it did not give me much success as the face got distorted in those. Even if I successfully generate an image, Runway ml or Pika Labs gives you only a few seconds of videos; more is needed to create an AI video with enough length to make it very user-attractive.

Experiment 2: Transforming an Existing Video

Not to be deterred, I thought, "Why not morph an existing Creative Commons licensed video into an animation?" The Apple App Store, brimming with AI-based apps like Videoleap, Loopsie, Glam and Lisa AI, and AI mirror, seemed like the next logical playground.

So, I converted an existing Creative Commons licensed video to an animated one. I tried to use different AI apps from the Apple App Store, like Videoleap, Loopsie, Glam and Lisa AI, and AI Mirror. Videoleap gives me distorted scenes, and AI mirror gives me the best result. But still, AI Mirror or similar apps are suitable for face detection. When a face is recognized correctly, it can create better results. Otherwise, the generated frames in the video become distorted. When a face is recognized correctly, it can create better results. Otherwise, the generated frames in the video become distorted.

Experiment 3: Video to Anime Generator Tools

Then I stumbled upon the idea of turning videos into anime. This venture led me to a slew of tools, both software and web applications. Some prominent ones were:

imageupscaler.com 
Frommedia.io 
media.io/video-cartoonizer.html
ailab.wondershare.com/tools/video-cartoonizer.html

as mentioned on blogs such as https://yourlifestylebusiness.com/ai-video-cartoonizer/  and https://typecast.ai/learn/ai-video-cartoonizer-top-5-2023/.

Furthermore, I took a deep dive into the blogosphere and  My research also led me to VToonify repo and its hosted version on Hugging Face.

Much to my chagrin, these tools faced a common issue: they relied heavily on human face detection. If there was no face detected or if the same face appeared sporadically, the animation lacked consistency.

The Final Frontier: PowerDirect to the Rescue

After much trial and error, the savior emerged in the form of PowerDirect from the App Store. Unlike other tools, PowerDirect doesn’t solely depend on completely on AI, but instead augments it with traditional conversion techniques. 

So, in a nutshell, They use their own conversion technique with a boost of AI rather than depend entirely on AI like other tools mentioned. The result? A seamless video-to-animation conversion!

So, with the help of this app and some video editing skills, I was able to generate a video meshing up a Creative Commons music video and a Creative Commons love song like this:

Conclusion

The journey to creating an AI animated video was not an easy one. It was rife with challenges, disappointments, and moments of enlightenment. While pure AI tools have their merits, sometimes a blend of traditional techniques and AI is what brings out the best results. After all, isn't mixing old and new the essence of creativity?