Google Unleashes ‘Lumiere’ – The Ultimate AI Text-to-Video

Revolutionizing Storytelling

Google’s newest idea, Lumiere, is a time-space diffusion model that can be edited on demand and uses text and images to create realistic AI-generated films. A new time-and-space diffusion model, Lumiere, is in the works at Google. It can take an image or piece of text and transform it into a realistic artificial intelligence-generated video, complete with on-demand editing capabilities.

Read: Google’s Snap & Transform: Unlocking Surprising Image Magic with MobileDiffusion

Create Dynamic Videos

Lumiere uses its “Space-Time U-Net architecture” to simulate “realistic, diverse and coherent motion.” All of the video’s running time can be generated in an instant with just one model pass.What this means is that users can create dynamic videos by either uploading still images with prompts or by entering textual descriptions of what they wish to see in the film.
As the article states, users have been drawing comparisons between Lumiere and ChatGPT, with the latter serving as a platform for text and image-to-video generating, stylization, editing, animation, and more.

Read: Outsmarting the Bots: US Regulator Drops the Hammer on AI-voice Robocalls

Even though there are other AI video generators like Pika and Runway, the researchers claim their method is unique since it uses a single pass to generate videos involving the temporal data dimension. Words like “state-of-the-art” and “an incredible breakthrough” have been flying about X as users describe this new discovery, with some going so far as to predict that video generation would “gonna get crazy” in the coming year.

Features

Lumiere can create 80 frames at 16 frames per second; it was trained on a dataset consisting of 30 million videos and text descriptions. Nevertheless, the data used to train the model remains u**********, which is a contentious issue in the fields of artificial intelligence and copyright law. Numerous lawsuits alleging copyright infringement have been brought against developers of generative AI models since their public availability surged, with the main grounds being the alleged exploitation of content during training. The New York Times was one of the most notable media outlets to publish a lawsuit accusing Microsoft and ChatGPT’s developer, OpenAI, of “illegally” sourcing their work for educational reasons.