Canyon Scene Reconstruction: Camera Tracking for Scene Reconstruction

Photo of Rick Barrett

Instructor Rick Barrett

Share this video
  • Duration: 08:17
  • Views: 3596
  • Made with Release: 19
  • Works with Release: 19 and greater

Step one in Scene Reconstruction is tracking the 3D Camera. See the basic process and special considerations.

In order to generate a Scene Reconstruction in Cinema 4D, you first have to use the Motion Tracker to determine the location of the 3D camera based on the video footage. In this video, we’ll perform a basic camera track and discuss some special considerations when tracking the camera for scene reconstruction.



With scene reconstruction, we're generating 3D geometry from 2D footage. And the first step within Cinema 4D, is to track the 3D camera. And you do that using Cinema 4D's Motion Tracker. So create a new Motion Tracker object, and we'll need to load the footage into this object. So down here in the Attribute Manager, we're going to click on this button with the three dots and load the footage to track. Once again, Artbeats was kind enough to provide this footage at 720p resolution so that you could follow along with the tutorial. So once you load the footage, by default it's going to down sample the footage in order to speed up the tracking performance. Now it's important to keep in mind that this resampling is completely independent from the reconstruction itself. So, you can... Go ahead and stick with a lower resampling value if you'd like. I often like to just jump all the way up and see my footage at full resolution. Now in order to complete the track itself, we're going to jump into the 2D Tracking tab. And, the default number of tracks of 300 will work okay for this footage. I like to start my track from the middle of the footage somewhere, and we'll just go ahead and click the Auto Track button. And this will take a minute while it preloads the footage and processes all the frames. I'm going to come back to you as soon as this process is complete. So as soon as the 2D track has finished processing, you'll get these pink track points within the view, overlaid on the footage. And you'll see a little line showing you the vector of how each point is moving through space. The important thing here is that you want to make sure that you have points that are evenly distributed, and that you've got points on the object itself that you want to reconstruct. You'll notice here that I don't have any points on the water because there's just not enough texture detail there. But the good news is that I don't need to track or reconstruct the water because that's simply a flat plane that I can put underneath the entire geometry after it's generated. So, this tracking looks pretty good, and I know from previous tests that it's going to work out okay. What we need to do now is take this 2D track and generate a 3D camera from the tracking information. And you do that using the 3D Solve tab. So, you don't really need to adjust any of the options here. You want to make sure that you are doing a full 3D reconstruction. Again, this won't work with a Nodal Pan or a Planner Track. You need a full 3D reconstruction in order to reconstruct the 3D geometry from this footage. So we'll just hit the Run 3D Solver button. And again, this is going to take a minute while it processes all those tracks and determines the actual camera position based on the 2D track points. So once again, I'll be back with you as soon as that's done. When the 3D Solve is complete, you'll see and notice in the bottom-left corner that says the "Deferred Solve Finished." And if we twirl open the Motion Tracker here, you'll see that you now have a solved camera. When I select that, you'll see the features that have actually been solved. These are the 3D positions of each of those 2D tracking points, and they're colored green to red based on the confidence that the tracker had in the position of that point. And again, the things that influence this are the things that I covered in the previous tutorial about making sure that you have even lighting, good texture detail, minimal blur, limited shadows, and avoiding reflections. The important thing here is that in order to get a good reconstruction, you need a good even distribution of the green tracking points on the areas that you want to reconstruct. So we kind of lose the reconstruction as we get back here to the end, but I'm not worried about that too much because I'm mostly focusing in this animation on the area here from frame zero to about 160 or 180, and we're going to cut off the render there. If we jump out of the Perspective view here, you can see what this looks like in 3D space. So here from the Top view, you can see that our camera is moving through this series of points, and that actually looks pretty on the ball. And here in a Front view, you can see it as well, as well as the Right view. Now what you'll notice here on the Right view is that it looks like our track points are moving upwards and the camera is more or less staying flat. But what we know is that the water level should basically be flat, and the camera should be tilting down more. And the way that we will fix that is by adding constraints. So we'll go in here to the Motion Tracker, right-click and choose from the Motion Tracker tags, we want to choose a Planner Constraint. And this is going to give us a triangle here in the view. And what we need to do is choose three track points that we know should lie in the same plane. And in this case, we know that the water level should be consistent here. So we can choose three track points that fit right along the water. So just drag each of the points of that triangle on to three of the track points that are right at the water line. And then over here in the Attribute Manager, we need to tell Cinema 4D that the axis for that plane is the y axis. Before I do that, I'm going to jump back into the Four-way view so that you can see what it does. And when I hit that, you can see that it flattens out the entire canyon, so that now our camera is pointing down and the canyon itself, the water line is flat. Now the next thing that we want to do is give this a little bit of scale. That's going to be important especially because we're going to do a dynamic simulation later. And if we don't have some sense of the proper scale for this, then we're not going to get…the dynamics aren't going to look believable. So, we're going to go ahead and add another Motion Tracker tag. And this time, we're going to use a Vector Constraint. And again, you're going to get this pink line here in the view, and you want to drag the points on either end of that line. Let's use this point here and this point here. Now I've never been to this canyon, so I can only guess at the scale of it. We actually don't even really have a good reference for scale in the scene, we only see the canyon itself. But I'm going to guess that that's about 30 feet. So we'll go in here to the length and say, Known, and type in 30 feet. And you'll notice that I'm working in centimeters because that's Cinema 4D's default unit. But I'm American, and I still haven't grasped the metric system. So I just type in 30 feet, and Cinema does all the math for me, which is great. I'm not going to set a known axis for this because I really am not concerned with any axis other than the y axis for this simulation, and I've already set that up with my plane. So putting more axes into the mix will just confuse things. Finally, I'm going to add a Position Constraint, and that's going to set the world origin with respect to all of my tracking points. So again, we'll go down to Motion Tracker tags and choose Position Constraint. And we've got this orange outline dot, it's a little bit hard to see, but we can drag it down here and choose a spot. Let's choose that spot right there, and we're going to say that, that is 000. So if I jump back into the 3D view, you can see that my scale has gotten a lot bigger because I set the scale there for my scene. And you can see that the water line here is right along the zero y coordinate, and we basically have our canyon all lined up and ready to go. So with this, our Motion track is complete. Our camera is all set up. And the next step is to actually create the scene reconstruction. And we're going to do that in the next tutorial.
Resume Auto-Scroll?