The Perception Guide to FUI: Setting Up a Wrist Mounted Interface, Part 1

Photo of Perception

Instructor Perception

Share this video
  • Duration: 11:34
  • Views: 2544
  • Made with Release: 18
  • Works with Release: 18 and greater

Tracking the Camera and an Object in Cinema 4D for a VFX FUI shot

Justin Molush (Senior Designer at Perception) breaks down analyzing and tracking complex shots with C4Ds Camera and Object tracker. You'll learn how to generate and refine an automatic track for the Camera, as well as add manual tracking markers to track a specific object.



- The primary focus of this tutorial will be exploring the scene and object tracking capabilities inside of Cinema. I'll be breaking into two parts here, the first being the technical aspect of tracking the shot, looking at it, identifying points that conflict with a clean track, and finally solving the camera, then moving on to tracking an object inside of that scene, and using appropriate geometry to get the spatial transform correct that works for our purposes. In the second part, we will be a little more open to exploratory and interpretation on your end, but here I'll be making a futuristic hexial-like wrist UI, that the subtle hand gestures are navigating through. This part's a bit less technical and is ripe for motion exploration, and once you have a real solid track from the previous part, it should allow you to iterate very quickly and really explore. You'll be seeing everything in real time in your scene, and without a lot of round-tripping to other applications will help really get some motion and the look dialed in the first time. All right, so immediately we're just going to create a motion tracker and promptly point it to our footage. I'm going to turn the Resampling all the way up to 100% because I really want the tracker to look at all aspects of this shot. And we're going to turn the pattern size up right away since we're trying to track the background in particular and the shot is out of focus. So setting a relatively large area to search is pretty important, given that out of focus points will tend to slip if they're not relatively large. In addition, we're going to feed it as much camera information as we can. You can leave this Auto. Cinema usually solves it fairly well. And at this point, we're going to just go right into the automatic tracking and take a look at how the tracker's seeing it. You can see it's scattering dots pretty much all over the place, including on the hand. So we're just going to go ahead and autotrack it, and see what kind of solve it's going to give us. This is going to require a little bit of refinement. And then we're going to use some of the Autocull tools to cut down some points, manually select some, and just clean up the track in general. So this is what we get back from running the Autosolve. You'll see there are some points on the hand that are really not favorable since we're trying to solve for the back, general camera wiggle. And you'll see on the edge of the hand, there's quite a few points that really don't lend itself well to a proper 3D solve. So we're going to go ahead and just increase the minimum length of the tracking point that we want it to be in the scene. Since the camera moves relatively quickly, that should cut off a lot of the points from the hand. And it does, but we're going to go in and manually pick out some of these points, since it should be a fairly quick cleanup. Actually, just get rid of all of those, and I think there's some on the bottom there, yeah, that slip a little bit, and get rid of the top and the bottom there. And it looks like for the most part, minus some of the edges that are receiving some extra motion from the hand, it looks like it's isolated the background fairly successfully. At this point, we're ready to kind of take a look at what the tracker's going to give us. We're going to go ahead and, we're not going to manual track anything at this time, but we're going to go under reconstruction and run the solver, and let's see how our track is doing. We can always go back and refine this after the fact. Cinema's very good at keeping this data live until you lock it. So, let's scrub on through. You can immediately see the ground plane is moving pretty frantically to match some of the hand camera shake, and let me look at it from wide, yep, the camera's got a lot of the hand shake and wiggle from the original shot. So, at this point, we're going to lock the data because we're going to go ahead and create an object track. And here's where things are going to get a little more interesting. We're going to do some manually assigned tracks. If you go back into the original motion tracker, since we have lock this, we're going to be able to go back in here and manually track individual tracking markers on the hand. So we're just going to go ahead, right click, and create a point. These function the same as 2D trackers, so we're going to expand the size of it, and just go over and hit manual track, and kind of look at the track. You can see at the end here, it breaks a little bit, but if we move it once, you'll see the track will autocorrect and go through to the end. So the goal here at this point is just to very quickly go through and create points for each one of these tracking markers. Hit Manual Track, give it a quick scrub on the timeline, and especially look towards the beginning and see which ones break, and just patch the tracks manually, and make sure that they're accurate, and they follow it all the way to the end. And especially the beginning, since that's where we have the largest area of transform. That's going to be the one that is most suspect to losing these tracking markers. So, the manual track is very fast and it will autocomplete throughout the entire timeline as you hit track, but when you adjust it like that, you'll see that, yep, it's going to come through to completion. You might need more than one extra keyframe in there on these user tracks to get it to follow if there's some complex transforms going on like this. And you can see it'll go red and break, and to notify you if there's a major issue. But, yep, you can see there are some missing frames in here. So we're just going to go through and manually patch the frames so the tracker has the gaps. So if we scrub through the rest of it, it looks like the rest is pretty successful. So just keep going through, manually creating tracking markers for all the points, and just double checking, making sure they're actually on it. We're just going to nudge and make sure, since we're kind of lost the entire view of that point, we're just going to give it a little nudge and make sure it's still on point. Manual track again, and scrub. And this one drew us pretty hard, so let's move it to there. You'll see it auto updates. So this might take a little bit of fixing. Sometimes the trackers get a little off when you do the initial adjustments. So you always want to just scrub the length just to validate that all these points are on target and doing exactly what you want them. Because this is going to influence how the object we're going to track, which is the hand here, transforms in space. And we really want to get some natural movement. So if it gets too complicated, we can always just create a new tracking point and...yep, it's still losing it a little bit. But we're going to go one frame, update it, and yep. You'll see there the movement has updated pretty successfully, and that one is fairly latched. Now this one's off a little bit. So, it is fairly quick, just go in create a new point, hit Manual Track, let it track, scrub back and forth, double check to see where it breaks. You're going to go in the last frame here. Just put it back on the point and see how it updates. Little update to the track. It's kind of off here. Undo that and let's just see how, if we really need to patch that. Overall, it looks pretty good. We're going to do it right in the middle there, see if that's going to influence it negatively on both ends. Looks good. And, it goes off a little bit there again. And this is the part of the process where you're going to really want to put in a little bit of time to make sure that the track is reacting as you expect. Because this is going influence the overall quality of the track. We are moving relatively quickly here and trying to get a lot of ground covered at this time, but this is where a good chunk of time can be spent in making sure the track is of high quality. So now that you've spent some time getting all your tracking points in a good place, it's time to bring in our hand geometry and actually get the transform information onto the object and see where we're at. So first things first, is you need to select all the tracking points and go into the object tracker and hit Add Points. And that's going to pull all the information from the motion tracker in the object tracker and luckily, I have a hand that is ready for us to use that is mildly posed in a semi-correct orientation. So now what we're going to do is go in here and just reset the anchor point to the hand to make it a little more manageable because this was part of a larger model that I extracted for this, and spend some time in getting it in position. So, a lot of this is having an eye, just in general for how an object should be related to the camera. In general, what is the scale of the scene when we look at it from large here. Like, how far is this object? Is it a correct scale and what in general is the size of the scene that you've solved? So the more accurately you can place this, the more accurately all these trackers are going to apply on the object and we're going to get a more accurate solve. Before we finish this, we're going to go in there on this last point here, and just add tracking data for that as well, since we can always subtract it if we feel it's not providing an adequate transform. So we're going to just follow the exact same rules. Fix it at the end there, should auto update, and now we've got our last bit of information in there. We're going to go back into the object tracker, going to assign the piece of geometry and we're going to assign the tracking point to it. And we're going to drop it under there. We've scrubbed a few frames, so now we're just going to re-position the hand one last time. We're going to find a nice intermediate frame, that has a relatively neutral pose close to it and really try to get... I'm looking specifically at the curvature of the wrist since that's where the majority of points are. So we drop into a wire frame. We're going to specifically look at how the curvature of the wrist on the model and the shot match, and try to line up this lower boundary, since that fall-off is going to be relatively important in order for us to get an adequate transform on this and making sure it's accurate. So, you'll be rewarded if you spent a lot of time earlier on getting these tracking points fairly accurate. So we're going to double-check. The orientation looks fairly good. So I think it's time to go into the object tracker, apply these tracks and see what it does. So under Reconstruction, we're not going worry about those check boxes yet, but we're going to run the solve. And when you click on it, scrub through, you will see that it has now inherited the transforms of the points we have tracked throughout this tutorial. You'll see it is not perfect. The goal of this was not to get the fingers relatively accurate, but to localize a transform on this area of the wrist to use as an accurate basis for this wrist-mounted UI that we're now going to do. So, we have a camera tracked in the shot, we have an object tracked in the scene, and we have all this transformed data that we can now use to mount our wrist-mounted UI.
Resume Auto-Scroll?