Hi, there. So I'm Eric Nicolas Smit. I'm a visual designer. I come from Luxembourg
in Europe and it's great to be here to talk to you about my work
and also about my tool, Cinema 4D. So this presentation will focus on motion tracking
and visual effects or how you can integrate 3D elements in film-it
sequences. Now, this is a very difficult task, because even if you have a very nice
3D model, putting it in front of your video won't make it. You need to match the
camera setting, the camera angle, the camera motion. You also need to match the
illumination, the co-stakes [SP], the reflection. It's a very long work, but
luckily, with Cinema 4D, we can do it a bit faster. So what I would like to do now
is to work with you on two visual effect sequences. We will be integrating a car
in a film-it sequence. But before starting, I
wanted to show you a bit of my work, and especially my work in
the visual effects. So this is an example of a 3D model integrated into a film-it
sequence. It's from my short movie, ControFigura, and I'll just play it back,
and we have other examples in this video. So here, I integrated the statue in this
place that was filmed and here, you can see the statue alone. I sculpted it in
Cinema 4D. Here is the location with the model, the textured model,
and then with atmospheric effects. And here is another shot from the same short
Here, it's a static camera. And another one where I actually [inaudible] a
statue and even replaced the vegetation. And the last one from this short movie.
And these shots comes from a serie of spots advertising that I did for this
company. I integrated the truck in many different location. The idea was to
express the expansion of the company by making it travel all over the world.
So it goes to London. Here, you see the shot without the truck. And here you see
Paris, in front of the Arc de Triomphe and the truck is right here. And again,
Paris on the Boulevard with the truck. And the final shot in Boga-Boga Island.
Okay, so now here's the scene we will be working on. I'll try to do it from
beginning to end. So we will do every steps. The first one will be 2D Tracking,
then we will be doing 3D Solving, then integration, and then rendering,
and post production. This is a view of the Golden Gate Bridge.
The camera is moving with the traffic and the camera is on a truck.
And we travel from the gates to the exits. So here, I have already loaded my Motion
Tracker. This is the tool that I will use to get the camera movement,
the camera angle, and also the scale, and the orientation of the scene.
So this tracker will help me to get a 3D camera that matches the 2D camera in the
footage. So this is my sequence and here, I have my Motion Tracker.
I've already loaded the footage here. It's a stock footage. And then here,
in the 2D Tracking, I have many option. I can use Auto Track, I can use Manual
Tracking, but I will try Auto Track now. I think it's always better to try it with a
Auto Process and then fix later on if you see problems.
I'll be lazy and I'm just click here, Auto Track. And, oh, when I click Auto Track,
Cinema 4D will scan the image and try to find features that stay in view
and that can be traced for a very long period. So it can get an ID of the
movement in 2D and then extract this data and transform it into a 3D motion.
So I'll run the Auto Track here. And it's now processing the frames and the 2D Track
is finished. It was quite fast. So now we see that we have points
here that appears on my footage and these points are attached to the structure
of the footage. We see the points on the bridge here and on the fence,
on the concrete blocks, and they are all pretty glued to these structures.
There is, however, a few tracks acting weird. If you look here, in this area
here, right here, one is just moving a bit erratically, chaotically.
This is an error and we need to remove it. So I'll just grab it here and just
delete it. And then we have also tracks here in front of the car,
right here. We have sliding tracks. These are errors. These are issues that we need
to get rid of.
Also, you'll notice that we have tracks here on the car and we don't want that.
We don't want to trace the movement of the car. We just want to trace the movement of
the camera. So I'll just have to remove them as well. I'll just click Delete.
This one as well. This cleaning is necessary. You cannot avoid it.
Oh, sorry, up, Delete. And that should be it. We have also here,
sliding tracks. I'll just get rid of them. And here, again, a track acting weird.
And instead of doing this manually, I could also use here, a filter that is
called Smart Acceleration. And Smart Acceleration, it will analyze groups of
tracks and see if they all behave coherently, if there is not one that is
acting really weird compared to the other. So if I reduce here the Smart
Acceleration, you'll see that many tracks will disappear and these are the bad ones.
These are the one that are just noisy and with an incoherent behavior.
So Cinema 4D gets rids of them. I'll just remove here, an extra track
here in the sky, because it's useless to have a track on the sky,
because there's absolutely nothing to trace.
So now it seems we have a decent 2D Tracking. I'll remove just this one and
then I'll stop. So we can now do the 3D Solving. And 3D Solving,
Cinema 4D will just take these 2D Tracks, and try to find their position in a 3D
space and also try to find what was the camera movement and the focal lengths.
So here we go in 3D Solving. And the Solve Mode, I'll leave it to Full 3D
Reconstruction. We have also Planar Reconstruction and Nodal Pan
and these are for a static camera or camera moving just like this.
But here, we want to analyze it with a full movement of the camera.
And we have here, the focal length and I said it's known and constant.
Constant because I doubt that the focal length was changed in this sequence
and known because it's always good to indicate a focal length.
I will talk about this later in the second part, but to get this 24 millimeters,
I actually rent a camera calibrator. But this will be the second part of my
presentation, so just keep it in mind and I'll talk about it later.
So here, I have all the settings to run the 3D Solver. I'll just click on it
and we will see what happens. So it's running the 3D Solver
and the solving is finished. And now we have what is called Features.
And you see all these big green circles, and if I move my camera, they are glued to
every element in the footage: the road, here we have the lamp,
the fence. They stick to it. And if I move away from my source camera
here in a 3D view, you can see my points here, my features. It's a representation
of the 3D space. And then my camera is moving. I go back here to my source
And we see here, all the points in the 3D space. Now, if we start adding geometries
here, for example, a plane...just move it here. Let's move it on the z-axis.
The issue here is that its position and its orientation are not very useful.
We don't know where this plane should be in the footage. It's just in the air
and we have absolutely no notion of its position compared to the film-it sequence.
So what we need to do now is to calibrate the cluster of points
here, the group of features, to get a scene that is correctly orientated at the
correct position, at the correct scale. So then we will be able to integrate our car
or any other geometries. So to transform this cluster of point that is just not
organized, I need to use a special tag. These are the Motion Tracking Tags.
And we have here, in the motion…sorry, Motion Tracking Tags, we have many
different tags. We have Planner, Position, Vector, and these are all
little helpers that will help us to transform this unorganized cluster of
point into a usable 3D scene.
So let me switch here to the Motion Tracker Layout here. I'll go back to my
camera. And here, I just had these constrained. Now, let me talk about them
a little bit more. We have here...the first we will be using is the Position
Constraint here. And the Position Constraint, what it will do is that it
will move the whole group of point to a position in the 3D space.
And for me, I want this group of point to have its center on the road.
I want to grab one point and make it the center of my 3D universe.
So for this, I will grab, here, my Anchorman in Cinema 4D.
It's the Create Position Constraint. And now, I have to click on a point
here, on one of the track, and this will be the center of my world.
So I'll try to find one that could act as this. And I will grab this point
here, right here, and I'll say that this is the center of my world.
I click on it and now I have a new tag. It's the Position Constraint.
And it says, "Feature to Position." It means that the whole group is now moved
here, linked to this point, and this will be my zero origin, the center of my world.
Now that we have done this, we need to set the orientation of the scene
and I want my scene to be orientated exactly in the same direction as the road.
So I want my z-axis to be on the road. So for this, I need to use a Vector
Constraint and for the Vector Constraint, I cannot use a single point.
I'll have to use two points and say to Cinema 4D that these two points are
So now, I need to find two points that are correctly aligned
here in this direction. So we could use, for example, these two points because they
are on the z-axis. We could also use here, the marking on the road as a line.
So I need to check, to search for two points that could be perfect for this.
So let me check. Here, I found two here just on the markings,
this one here. So I'll click here, the Vector Constraint command.
I'll just click on the first here and on the second and I will say
here that the axis should be the z-axis. So now this line is the z-axis
of my scene and I will say that the length is known. So this line is
probably… Let me check by just putting a car next to it. I would say it's
maybe, 6 meter, 8 meter. I'll just try with 8 meter. Okay. So now my scene is
correctly scaled. You can see that the point are now much smaller.
And now I can just put geometry cars and they will be at the right position,
at the right angle, and maybe at the right size as well.
So let's start by placing here, a plane. So I'll just put a plane
here. And you see it's directly here at the center that I indicated with my
Position Constraint. It's exactly where I had my track. So I'll just make it
larger. This could be my road, like so. And just longer. Let's try this.
Like so. And I'll switch here to a Line Shading, so we can see the perspective
lines. Okay. So now you see that my plane here is exactly aligned to the marking on
the floor. So that means that my perspective are correct. It's also
aligned here to the border. Let me move it a little bit here
and maybe reduce its size.
We only have a little issue here on this side. We see it's not exactly aligned.
I think it is caused by this road that is not perfectly flat. There is
maybe a curve to it on this side or this side, so we need to fix it.
You always need to fix a Motion Tracking because it's never perfectly correct.
But this was already pretty good, but I'll just fix it here, by changing the
coordinates. I'll try 0.5 here. No, that's worse. I'll just use minus.
Okay, that's already better and maybe here. I'll just Shift-click
here. No, that's too much. Zero, it's at 0.2. Okay. Yeah, that's correct.
I think now that we have something that is correctly aligned, more or less… And this
now can be my road.
So I have a road and this is already a big part of the job because the next step
will only be to put a car on top of it. So this road will not be visible.
I only want it to be an object that catch shadows and occlusion.
So I will make it invisible. I'll now switch to my standard layout,
because we don't need the tracking anymore, by dropping the footage in it.
So this is my road. I drop the footage, and I'll switch here,
the Mapping to Camera Mapping, and I'll drop here, my camera.
It means that it will get… The footage will be projected on it exactly as it is
on the view we see behind it. And since I don't want it to be visible,
I'll just put on it a Compositing Tag. So we have here, a Compositing Tag
and I'll say, "Compositing Background." It means when I render it,
it won't have any illumination, any shading. It will be just flat,
just like the background. And I'll just place another tag here,
a Display Tag, and switch here to Constant Shading, so that it doesn't get
in the way and we can work without noticing it.
So now my setup is ready. I have my road. I'll make it a bit larger.
I can bring in my car and I would like to have a car here in the center line,
right here, going from here to here. So I'll just open up my car.
It's right here. This is my car, my sport car, the one that we will be integrating
in the shot. It's already animated. I just animated the wheels
and also included a little bit of vibration to simulate irregularities on
the road. It's just a little Vibration Tag here. It's just a random motion,
a bit like to simulate the bumps and to make the movement a bit more interesting.
So now I can grab my car. I'll just copy it and I'll just load it in my scene,
paste. Switch the display here to Quick Shading. And let me [inaudible] like this.
Here, it's right here. So we have the car now in the scene and in a static position.
You can see that it's already correctly integrated. It doesn't move.
It's fixed to the road. I'll just have to adjust its size a little bit here.
It seems a bit small, so I'll make it a bit larger. Okay. It seems okay.
Now I'll make it the trial of my road, to just change its orientation so that it's
perfectly aligned to the road. Okay. And now, I'll just animate it,
the car, and I will animate it just by using two keyframes. One for the start,
one for the end. So this is my beginning. I'll just check if I'm really
on the road, yeah. And I'll add another one at the end. Switch
here to this view, like so. So, here again. And I'll check again.
I have to make my road a bit longer. Oh, that's smaller. Okay.
I'm just checking if it's correctly positioned on the road. Then I have
here, the second keyframe. Okay, so my car is now animated in the scene.
I will switch the Curve here to a Linear Curve, because I want a constant
speed. So I'll go here, open the Curve Editor, and just switch
here to a Linear curve. So now the car has a constant speed,
in my view. And in terms of movements, position, perspective,
alignment, it's more or less correct. I could work on it a little bit further.
But now if I hit Render, the issue is that there is no interaction between the
car and the road. We have here, just a black car. There's no shadow,
no occlusion, and it doesn't look like it belongs to the same environment.
So what we need to do now is to work on the interaction between the car and its
environment and this is done by using shadows, reflection, occlusion,
and all kinds of effects that can create a link between the world in the footage
and my 3D object.
So the first thing I will do is to add a sunlight, a direct light.
And to find its orientation, it's easy in this shot because we have very strong
shadows. And we see that we have shadows right here, below the car.
That indicates that the sun is coming from the top, that it's the middle of the day.
And if I look at the highlights here, I can see that the sun is coming a little
bit from the side. We see it also here, in the concrete block.
It's not totally vertical, so I will just create here, a sun.
I will use an infinite lights. Like so. And what I will do to set its orientation,
I will use here, a command that is called Set Active Object as Camera.
This allows me to see the scene from the light as if I was holding a projector.
So I'll click on it. And I can now just move around. And I have my car.
So we have a God-sun view. So we see my car. And I've just put my light right on
the top here, a bit from the side. And I'll just activate here, Shadows.
I'll use Area Shadows. And now, if I move back to my view, hit Render,
we have shadows here, that match more or less the orientation of the other one.
Now what is missing is a diffuse lights. We have only one direct lights.
So I'll re-create a diffuse lights. I will do a quick diffuse light using an
Evironment Object. An Environment Object creates a totally flat
illumination. And I will include, as well, Ambient Occlusion
here in the effects. Ambient Occlusion. Ambient Occlusion plus Environment
Light creates a very fast and cheap… I mean, you don't need any kind of
complicated setup. It creates a nice global illumination, an omnidirectional
global illumination. Now, if I hit Render, we have now…the car has now more
shadowing and the contact between the car and the road, we see has a bit of
occlusion, here. We don't notice that much, but there is a bit of occlusion.
Now what is missing is the reflection. We have the car and it's now totally black
because it doesn't reflect anything. This car has a metallic shader,
so it should reflect something. There should be something around it that
creates, generates reflection. But since the footage is behind the car,
it's totally black. Our universe is completely black. So what I need to do is
to create something that will create the reflection. So for this,
I will be using an Environment Map on a sky. So I create a sky
here and I create a new material with an Environment Map.
I will a use High-Dynamic-Range Environment Map here. It's an image
that is more or less similar to my bridge. It's a structure with, yeah, the same
kind of color tone. So I drop it here, on the sky. And now we have a sky,
but I don't want it to be visible. I want to see the footage, but not this image.
So I will make it invisible to the camera, but visible only to the reflection.
So I use...here, for this, I use the Compositing Tag. Compositing Tag are
really useful for visual effects. And I'll just turn off everything excluding the
scene by reflection.
So now this sky will only be seen by the reflection. So now if I hit Render, we
have a car with a reflection on it. So the reflection are a bit too strong,
so I will reduce here, the intensity of my High-Dynamic-Range Image,
like so, and maybe change its color a little bit to match the sky in the
footage. And now, if I hit Render, we have something much better. That's maybe too
pinky. Okay. And now to set the orientation of the sky, I will use
here, just a sphere here in my scene, make it larger, and drop
here, the same material and just rotate it, so that the light is coming from the
same direction, from the front and a bit from the top. And then I'll use the
rotation value of this sphere. I'll just copy it to my sky. So I'll just go into
Coordinates, copy this value, copy and paste it in my sky.
And it's done. Just delete my sphere. And now, we should have a more accurate
reflection, something that is more [inaudible] to the scene.
And so here we have my car driving on the Golden Gate Bridge.
So what is missing now is that we don't have any kind of motion blur.
And this is essential when you're doing compositing shots. You need to match the
motion blur, otherwise, it will look totally fake. Now, since the camera is
moving with the traffic, we don't have that much blur. The relative position
between the car and the camera is minimal. On the other side, we have a very strong
motion blur, but here, not that much. If we check this car here, the car here on
the right, it has a strong motion blur, but not as intense as the other one.
And we need to match this. So for this, we will activate Cinema 4D's motion blur,
it's in the Physical Settings here. I have the Motion Blur.
And now, the only thing that I have to do is to adjust the shutter speed of my
physical camera. It's now set to 130 and we'll just try with 130 to see how it
looks. Since motion blur is calculated at the end, we have to see it in the
Picture Viewer. So I'll open the Picture Viewer and make a test render.
So now we have the car with the motion blur. If we compare it to the other car,
it looks already more integrated. I'll just move it a bit. Let's bring it further
here. And I'll hit... I have Render again.
Okay, now we see there is a stronger motion blur when it reach the lower part
of the image and it's looking already really good. Now, the next step for us
would be to...of course, we could work further on it. We could work on the
shaders, improve the illumination, improve the pass [SP] of the car.
But we don't have much time, so I will move to the next step which is rendering
and post production. So now what I do is that I render this scene in multi-passes.
And in multi-passes, what I need is a mask for the car, so I can filter it
independently, and then I'll render also shadows pass, occlusion pass.
I'll show you how they look here. So here, I have my rendered.
I rendered it a few days ago. [inaudible] here. No, this is not it.
So here's the render. Okay. And here, I have my car mask, an object buffer,
and here, I have my ambient occlusion pass. And here, I have the shadow pass.
And now what I do is that I blend them in my compositing software.
It can be After Effect on any other. And then I filter each passes and add
a grain, I add a color mapping, I add all sort of highlights, glows, to mimic
better the style of the footage. Because we can see here, in my footage, that we
have very strong highlights here. Some sort of glows, shine.
So that I need to reproduce not in Cinema 4D, but in a post production package.
So I will show you now the finish-it sequence and this is the final sequence
with post production. I'll just play it back. Okay. So that was tracking and
compositing for an animated camera.
But sometimes, you have to work with a static shot. And if you have a static
camera shot, then the Motion Tracker is useless because you cannot trace something
that is static. There is nothing to trace, nothing to solve. So for this, we're using
a different set of tools and I'll switch now to a different scene
here. It must be… It's loading now. It takes a little while. Okay.
Now, this is a shot here in Paris. You saw it earlier in the video,
on the Boulevard, and the camera is static. So how can we get now the camera
angle, the camera position, and all the information we need to make a correct
integration if the camera is not moving? So if it's not moving, we only need the
focal length and the camera orientation. And this, we can find it using a tag that
is called the Camera Calibrator. And the Camera Calibrator is a tool that extracts
the focal lengths, the orientation, and the position of the camera from the
footage. And to make it work, we need to trace lines in the footage.
And these lines will be my axis. We need to use parallel line for each axis, so
Cinema 4D can calculate the vanishing point.
So what we do here, is we click on the tag here and we go in Calibrate.
And we add a line. So I add a line here and this line, I will place it right here
to here and I will say that this will be my z-axis. Again, I want the z-axis to be
in the traffic here, along the road. So I Shift-click on it and make it blue.
In Cinema 4D, the z-axis is always blue. Now, I need to add a second one so that we
have parallel lines for the vanishing points. So I click another line here
and I just place it here and here. And again, Shift-click on it to make it blue.
So now we see that the z-vanishing point
is already solved and we need to add another access to get the final
result, so I click Add Line. And now, I will work on the y-axis,
the vertical axis, and I will place it here, on the edge of this building.
And Shift-click on it to make it green. Green is always y. And then I add another
line here and put it here and here. It could be in other place,
it doesn't matter. And again, I Shift-click to make it green.
Okay, now we see that the x-vanishing point is indirectly solved.
The y-vanishing point is solved, z is solved as well, the camera focal length is
solved. Everything is solved. So Cinema 4D knows exactly what is the camera focal
length, but what we miss is the camera position. And this is done using a pin.
So we add a pin here and the pin appears here. It's a bit small.
And we'll just put it right here, at this position. And this will be the center of
my world. Now, if I click on this access, I can also say that this line has a known
length. Here, Z, I can say "Yes," I activate it, and I will say that this line
is maybe, let's say, 10 meter. Okay. So now everything is solved.
We have a camera. If I move away, here from the shot, I'll just hide it.
[inaudible]. And click. And I'll just move here,
here. Just hide the background. And, okay, I have my camera here,
and Cinema 4D evaluated it to be a 46 millimeter. And we have its position and
orientation. And now, again, if I want to place my road, I just have to add a plane.
Move it here. I'll switch to Display to Line. And I can just adjust its scale.
And we have here, a road that is correctly aligned. You know,
the perspective are totally accurate. So now I have my road. And again,
I can add the footage as a texture, the composting tag, bring in my car,
and do exactly the same that we did before but I won't do it. I'll just move to the
finish-it scene because it's exactly the same procedure. So let me find my scene
Oh, I need to open from here. So here, we have the complete scene with the car and
the traffic. And again, I have a sky, and environment illumination,
and a sun, and my plane as the road. So you can see it here, my car on the road.
Okay, so that was my integration, little examples, my two example.
One was an animated camera, one was a static camera. But now I would like to
talk to you about a brand-new feature in Release19. It's 3D reconstruction.
And what we have now is that if I switch now back to my Golden Gate Bridge
sequence… Let me load it. Sorry. Here, my Golden Gate Bridge sequence.
What we had from the 3D Solving was only a grid of points, a cluster of features,
these little green dots. But now with Release19, Cinema 4D offers you more.
And I will show you in a different scene where the effect is really striking.
Here, I'll switch to the...here, my building. So we have here… Let me charge
it. We have here, fly over buildings. And this shot was already tracked and solved.
And we can see here, the points that we have, the cluster of point.
If I move out of this view, we have here, this cluster of points.
But now we have an extra tab here in the Motion Tracker. It's Dereconstruction.
And here, Cinema 4D will try to re-create the scene in 3D. So it will map the
footage on a 3D cloud point.
So I'll show you how it looks. I could run it now, the run 3D
here, the reconstruction, the Run Scene Reconstruction, but since it'll take a few
minutes, it was 10 minutes, I'll just skip to the result. So here, we have the result
here. And I'll just here, hide everything. This is what we have now from
the 3D reconstruction. So what Cinema 4D did, it's tried to re-create the image
but in a 3D space. If I move away from my camera here, we can see the building,
the structure, the color, and this is extremely useful if you're doing
compositing. Because here, you can really understand the 3D shape of the scene,
you get a sense of distance, you see the color. You can even see
here, plants in the windows. So this is a very nice indication of where is what in
your 3D scene. It's much better than if I switch off the point cloud to have just
here, a cluster of point. This is way more useful.
And if I open here, my previous scene, the one on the Golden Gate Bridge,
here, I did the reconstruction as well. It took here, maybe 20 minutes.
And here, I can really see the bridge, and see where my car is traveling in the
3D space. I see also the large concrete blocks and the bridge structure.
And this is really useful if you need to reconstruct the scene,
if you need to replace elements. You have really a fantastic visualization of the
Okay, so this was my overview of motion tracking, camera calibration,
and integration of 3D elements in 3D scene. I hope it will stimulate you to try
it by yourself. I will be here on the computer behind, so if you want to talk to
me or you want to try it with me, we can play with it together
here. So that was my overview. I'm Eric Nicolas Smit. Thank you
and I hope you enjoyed it.