Implementing Cineware/C4D with Mettle Skybox
Posted: 17 March 2017 06:14 AM   [ Ignore ]  
Total Posts:  2
Joined  2014-12-20

Would love to see how everyone is working with a C4D file to then get placed / composited into 360 footage using AE and Mettle’s Skybox v2! Or even just the workflow of mimicking the Skybox camera in C4d to work back and forth. I know there is a VR plugin for C4D, but that actually does what Skybox does in AE.

Basically, I have some 360 footage that I 3D-motion tracked in AE using skybox v2. How would I implement a C4D file into this for compositing 3D objects in the scene?

Profile
 
 
Posted: 17 March 2017 09:06 AM   [ Ignore ]   [ # 1 ]  
Moderator
Avatar
Total Posts:  4319
Joined  2011-03-03

Hi jewels,

To my understanding Skybox v2 creates tracking points for fixed positions and those can be connected to your objects.

My assumptions (since I do not own a license of Skybox) is that the camera as well the image cube is moving. I have worked on VR since the early ‘90s and it has become only much more comfortable to work with new tools, but the creation of such hasn’t changed a lot. The key is to understand what is provided by Mettle and how do they translate the motion of the footage. When this is clear the implementation becomes easier to understand

The camera has to sits always in the center of the image cube, as any change of that set up would distort the given image (six images sit on the six sides of a cube -> box). So both move through the 3D space.
Mettle has some tutorials online since a while that represent exactly this. The scene with C4D needs to sit on top of these layers, placed by using the position of tracking points. The key questions for a complete integration/compositing as certainly outside of the scope of a single forums question. How to get the objects place shadow, reflected light, reflections, color bleed, etc into the scene. (In short: basically the ten points of my integration 101 series here on Cineversity)

I could offer to move this question to the Tutorials Request Forum. Please let me know.

All the best

 Signature 

Dr. Sassi V. Sassmannshausen Ph.D.

Photography For C4D Artists: 200 Free Tutorials: Texture, Panorama, HDRI, Camera Projection, etc.
https://www.youtube.com/user/DrSassiLA/playlists

Profile
 
 
Posted: 17 March 2017 09:32 AM   [ Ignore ]   [ # 2 ]  
Total Posts:  2
Joined  2014-12-20

Yes, indeed it is a huge topic. I’m more asking in general, how to set up the C4D file > Cineware > AE & Skybox and what the workflow is. Or on the other hand, How would you bring 360 footage into cinema and track / use as a background to composite 3D on it? This would be more for a 360 video where the camera wasn’t moving at all.

You could certainly move this to the tutorial request as well. I believe Rick made a tutorial of the kitchen Vr one,  but that didn’t talk to overlaying a cineware project onto 360 footage, camera matching, etc. A more advanced tutorial would be 3D motion tracking 360 video for compositing in C4D.

Profile
 
 
Posted: 17 March 2017 10:03 AM   [ Ignore ]   [ # 3 ]  
Moderator
Avatar
Total Posts:  4319
Joined  2011-03-03

Thanks for the details, jewels.

If the practical camera is not moving, then the tracking would not be required for the camera of course. The only parts in a scene that could require tracking are moving objects. I’m not aware that this is an option in Skybox. I haven’t tried it, but if I’m not mistaken Syntheyes can do VR objects.

When you shoot a 360º*180º, an equirectangular, then the camera is in the center point, while the image is on a Sphere for example. The six sided cube works the same, but somehow I’m more happy with a sphere: To trun images into an equirectangular requires sub pixel movement and with that, one lowers the quality. To take this image and convert it into a six side view will do more harm to the image quality. Then use it and render a new format out to use it on an HMD (head mounted display) will be the third conversion already. So, to skip one step is my idea about. Cinema 4D supports this natively: Spherical Projection. All of that if the first generation of footage/image was an equirectangular. There the discussion starts, while the format created there will drastically determine the later steps and with that the quality. Since we might talk about tutorials, this needs to be discussed.

In the moment we have a sphere and the needed projection is set up, we know already that the camera sits in the center. Well, kind of, as VR camera set ups are based on at least two cameras, the nodal point is not in the center, so anything that is too close will not match. Typically the rule of thumb is: one millimeter in nodal point shift, needs a meter distance (camera objects) to balance the problem. Better more distance. With multi camera set ups past the food in diameter, this can cause problems.

Having said all of that, the center point and the projection are quite easy to manual track, as the center is clear and the point where it goes as well. The only thing that isn’t clear is the distance to the camera. While shooting such footage/image, a good survey on location is needed. If the object moves or its size is known, things might be simpler.

The stitching of these images is normally supposed to take all lens distortion out, the reality is not that perfect, but based on equipment and used methods it might come very close. Which means, one could set up a sphere an camera and take 2D images from it, to be used in the Camera Calibrator for example, or in Object tracking in other 3D tracking apps. If the object itself can provide enough parallax, things will get simpler.

Anyway, yes, a longer and detailed tutorial series would be nice, even things changes currently in the practical camera section alone and certainly a lot in the tracking and stitching software, the amount of projects in this area might motivate for such material.

How the six side format will change is also in discussion:
https://blog.google/products/google-vr/bringing-pixels-front-and-center-vr-video/

All the best

 Signature 

Dr. Sassi V. Sassmannshausen Ph.D.

Photography For C4D Artists: 200 Free Tutorials: Texture, Panorama, HDRI, Camera Projection, etc.
https://www.youtube.com/user/DrSassiLA/playlists

Profile