Canyon Scene Reconstruction: Generating a Point Cloud from Video Footage

Photo of Rick Barrett

Instructor Rick Barrett

Share this video
  • Duration: 09:28
  • Views: 4110
  • Made with Release: 19
  • Works with Release: 19 and greater

Learn the C4D Motion Tracker basic parameters affecting scene reconstruction.

In this video, you’ll learn how to use Cinema 4D’s Scene Reconstruction to generate a point cloud from video footage. We’ll explore various attributes that affect the reconstruction, including the iterations, subsampling, point density, patch size and minimum texture detail.



In this tutorial, we'll generate a point cloud based on the track points that we generated in the previous tutorial. Once again, when you're doing scene reconstruction, it all starts with a good solid camera track. And you need to have these green track-points that indicate that we've got good solid tracking information, and you need them evenly distributed on the objects that you want to reconstruct. Now, all of the reconstruction is generated from the standpoint of those green tracking points. So, if you don't have enough of those, you're not going to get enough of the object reconstructed. But you don't need to necessarily flood your footage with thousands of tracking points. That's what I thought originally, but I've come to discover that, really, you just need them to be evenly distributed. And to actually generate the reconstruction itself, we, again, jump into the motion-tracker object itself. And we're going to jump into the Reconstruction tab. And in here, there's a number of different preset options for the various parameters that you can set when reconstructing your geometry. So, a good place to start is just the Preview option. What this does is it simply generates a quick point cloud to make sure that you can do any sort of reconstruction. So, after choosing the preview preset, we'll hit the Scene Reconstruction button and give it just a second to generate that point cloud. And you can see that that was pretty quick because it is just the preview. And here, we can see the point cloud that was generated. And it doesn't look so bad. It looks like it's going to work. We obviously need a lot more points. And for that, you go in here and you increase the point density. So, right now, that point density is five and this goes all the way up to eight, which would be one point for every pixel. But this is also influenced by the footage subsampling here. The footage subsampling, basically allows you to resample the footage much as you do on the footage tab. But again, the resampling for the scene reconstruction is completely independent of the resampling that you do for the camera reconstruction. So, a footage subsampling of zero is equivalent to 100%. So, in that case, if you have zero and a point density of eight, you'll get a point for every pixel. In most cases, Maxon recommends that you use a subsampling of one or two because you'll actually get a better reconstruction if it doesn't need to worry about every single pixel in the reconstruction. So, we're going to go with a footage sampling of one. And let's increase this point there to be just seven and see what we get. So I'm going to hit Run Scene Reconstruction again. And this time, you'll notice that it takes a little bit longer because, again, it's dealing with twice as much footage. It's gone from, basically, 25% of the original footage size to 50% of the original footage size. And we're generating more points. I'm going to give this a second and I'll be right back. So that took a little over a minute and you can see that now we have a much denser point cloud. This is a much better representation of the geometry itself. In fact, I'm going to jump into the Solo mode here so that we can easily switch between the point cloud that we got originally and the new point cloud that was generated as a result of the increased sampling. Now, you see that we've got some points that are showing up up here in this area, and that is possibly problematic, but we're going to keep going here and see if we can get those to filter out. And the way we can improve this point cloud further is with these other parameters in the reconstruction tab. The next option here is Patch Size, and this determines the size of the patches that are generated out from each of the tracking points. And the larger the patch size, you'll get a better reconstruction but it's going to take longer. Let's go ahead and increase that to five. The next setting here is the minimum texture detail, and this allows you to define the minimum amount of detail that it has to see in order to reconstruct that area. Increasing this is great for eliminating areas of blue sky or areas that shouldn't be reconstructed because there just isn't enough detail. Remember I said that it's incredibly important for scene reconstruction that you have that texture detail. So that's why we're minimizing the blur and the shadows as much as possible. If you're not getting enough reconstructed geometry, you can reduce this value a little bit. You just want to make sure that you don't reduce it so much that you start reconstructing areas that shouldn't be reconstructed. In this case, let's go ahead and drop it down to 0.5 and see what happens. The next option here is to filter the small groups. And what this does is it filters the areas of points that aren't really connected to larger areas of points. And this is usually a good idea because it's going to eliminate sort of the errant pixels or the errant points that are reconstructed in the wrong place. So, we'll go and activate that option. And finally, the Minimum Angle. The default value here of 10 degrees is usually going to work great. Basically, this is the minimum angle of parallax in the camera or a movement in the camera that's necessary in order to set a new point to generate additional reconstruction. Basically, if you set this too high, you won't get any scene reconstruction because there won't be enough parallax in the camera to meet that minimum angle. But if your camera is moving very slowly, you might want to reduce this setting just a little bit. I'm going to leave it at the default of 10 degrees. And let's go ahead and run the scene reconstruction again. Once again, this is going to take even a little bit longer because we've increased the calculations that are necessary. So, I'll be with you in a minute. Now that completed, and the point cloud was generated based on those new settings. You can see that we're still getting some trash up here in the upper area of the canyon, but it is cleaning up. And the next thing that we can do to clean that up even further is to allow the tracker to calculate multiple times. Each time it calculates, it will basically add more points and refine the points that are already there. And in order to do that, we go into the motion tracker in the reconstruction tab and we increase these iterations. Now, every iteration that you add is basically going to add that much more time to the reconstruction itself. So going from one iteration to two is going to double the time, from two to three, is going to double the time again, but those iterations are really necessary in order to clean up the reconstruction. So, I'm going to go ahead and set this up to three. This is probably going to take five minutes or so so it's a good time to go grab a cup of coffee. Of course, I'll be back with you in just a second. Now that that reconstruction is done, you can see that we've got a much more dense or a solid point cloud. Those extra iterations really help to close up some of the gaps that were being left from the reconstruction that only used one iteration. We do, however, still have these errant points in the upper part of the frame and those will cause a bit of an issue when we do the mesh reconstruction. I did a little bit of experimentation and I discovered that, actually, if we lower the point density a little bit here, we can get rid of those points and get a much higher quality reconstruction. So, you do have to play with these settings a little bit. And I've been adjusting them individually so that I could explain a little bit of what they do. You can use the preset options as a good starting point. The high quality is basically the last simulation that we ran. The medium quality reduces the iterations to two and the point density to six. And so, we'll run again with these settings, but I'm going to activate the Point Cloud Only option because we're going to worry about mesh reconstruction in the next tutorial. Now you can see that with the settings from the medium quality preset, we've got a good point cloud here that has removed all of those errant points that were in the center of the top of the canyon here. And we don't have quite the level of point density because we've lowered the point density as well as the iterations, but this is going to be perfectly sufficient for generating a mesh. The point cloud itself is just an array of points that have color applied to them through the vertex color tag. And it's important to note that this Drop Points Always option ensures that the points will always display even when you're not in points mode with that object selected. We'll look at a technique later in this tutorial series for rendering the points, but basically, as they are now, they serve as a great reference for placing objects in the scene. In the next tutorial, we're going to look at how to actually generate a 3D mesh from the scene reconstruction.
Resume Auto-Scroll?