New AI Video Models & Big Updates from Hailuo, Runway, Adobe & Krea!

I’m diving into some of the most exciting breakthroughs in AI video generation. First up is a mind-blowing technique called Diffusion as Shader (DAS), which merges 3D tracking with diffusion so you can seamlessly control subjects, locations, and even camera movements. Then, we explore two new game-changers from Adobe—one of them is open-source!—including TransPixar for text-to-video with built-in transparency and FaceLift for single-image 3D head reconstruction using Gaussian splatting.

I’ll also give you a rapid rundown of cool new updates from MiniMax, Runway, and Krea, like subject reference tools, 4K upscaling, and built-in audio generation. You’ll see how these tools push boundaries, from full-body animations to real-time 4D facial capture.

LINKS:
Diffusion As Shader: https://igl-hkust.github.io/das/
TransPixar: https://wileewang.github.io/TransPixar/
Facelift: https://www.wlyu.me/FaceLift/

Chapters
00:00 – Intro
00:30 - A Breakthrough in AI Video (DAS)
00:51 - A Brief Primer on Diffusion and Shaders
01:59 – How DAS Solves 2D Animation Issues
03:08 – Benefits of 3d for AI Video
03:44 - Using DAS with FLUX Depth
04:22 – Video-to-Video Magic & Mesh Animations
05:26 – Adobe’s Open-Source TransPixar for Transparency
07:22 – FaceLift for 3D Heads & 4D Novel View Synthesis
08:44 –MiniMax Subject Reference Now Available and What's Next
09:15 - Gen-3 4K Upscale
09:46 – Krea's Video and Sound Generation