Hengyu from the Qubit of Concave Fei Temple | Official account QbitAI

Hey duck, duck duck fell!

Duck Duck is real, wrestling is real, but the coffee and the hand that serves the coffee are given by the AI.

△Fortunately, the coffee in my hand only spilled 100 million points

From an open-source project, VideoDoodles, the discussion on the Internet has skyrocketed in the past two days.

The beauty of this project is that it can add hand-drawn graffiti to the video, which can be cute and lovely, or weird.

It's just like drawing it frame by frame.

This is still relatively difficult for most 2D video editing software on the market today. If you want to achieve the same effect, you generally have to draw it frame by frame · by the creator.

Blame the effort.

VideoDoodles, on the other hand, can create this effect by reconstructing a 3D scene in a video and then placing a flat canvas in it.

Many people said that they can use it to decorate videos, add cartoon effects or flowers, and then further process them into memes/memes.

Funny videos are also arranged, and it's on the way!

You can add whatever hand-drawn effect you want

VideoDoodles is what Aunt Sauce claims to be effectual:

Insert hand-drawn animations into your videos easily and arbitrarily, or blend seamlessly with your video content.

Are you sure? Let's take a look.

Just like adding arms and hands to a duck, VideoDoodles can add limbs to any species.

Flamingos can leisurely soak in the water with their own rocking red wine glasses.

The small train came from a distance, waving his hands, and drove past the camera.

Of course, it's okay to add a able-bodied animal to the video clip.

Be like this little squirrel on the shoulder:

It can be seen that no matter how much the lady walks forward, the little squirrel stays honestly on her shoulder.

In addition to body movements, you can also use VideoDoodles to add post-variety show characters to video clips.

Static text:

Moving text:

All have good spawn effects.

And, in the aftermath of a recently concluded Olympic Games, VideoDoodles can add hand-drawn drawings to a wide variety of large movements.

Or the arrow points to:

Or Superman cartoon effects:

How?

To add hand-drawn animation to video content, it is now common practice to use the video itself as the bottom layer, and then use manual painting to draw 2D animation effects frame by frame.

This is relatively inefficient, and requires a hand-drawn animation that tracks movement, resolves perspective distortions, and adapts occluders.

It can be done, but it takes time and time.

So, how does VideoDoodles handle the tracking of dynamic objects and the synchronization of animations?

It provides a simple interface to help you complete video doodles - "simple" means that both veterans and novices can easily draw.

Behind the project, 3D scene reconstruction technology is used to realize the natural integration of graffiti into the video scene, including complex visual effects such as tracking, perspective distortion and occlusion.

Specifically, VideoDoodles recreates the video content in 3D, then places flat canvases in the video that are tracked in the 3D scene, and then adds hand-drawn animation frame by frame.

Through a custom calculation and tracking algorithm, VideoDoodles locks the objects in the video scene corresponding to the canvas, and completes the canvas to move and rotate with the movement of the objects.

In the user interface of VideoDoodles, users can fine-tune the canvas in the UI of the 2D image space through keyframe position and orientation.

The system carefully inserts keyframes by tracking the movement of moving objects in the video.

In short, VideoDoodles uses its own algorithm to solve occlusion and perspective transformation based on the user's intent expressed by keyframes, renders the scene-aware canvas at any frame of the video, and makes the canvas closely follow the moving object.

And keyframes can have any number of positions and orientations.

In the image below, VideoDoodles renders the corresponding canvas coherently throughout the video by using the direction & position of the red part of the first frame and the position of the green part of the last frame.

In practice, the process is roughly as follows:

Finally, let's introduce:

The VideoDoodles study was conducted by researchers from Adobe Research, the University of the Côte d'Azur, and the Delft University of Technology.

The head of the R&D team said:

Our goal is to simplify the creative process of "video doodles".

This emerging mixed-media art form combines video content and hand-drawn animation that can be used for entertainment or educational purposes. With VideoDoodles, I hope that more people will be able to easily "inject imagination into the real world".

This article is from Xinzhi self-media and does not represent the views and positions of Business Xinzhi.If there is any suspicion of infringement, please contact the administrator of the Business News Platform.Contact: system@shangyexinzhi.com