Hi Biagio,
Thanks for your patience.
If I get this right (from the example above, ... I assume I do) you like to use the footage from a practical camera directly in C4D to avoid compositing of CG and practical footage. The practical footage is here and for the following considered as background footage (in opposite to project a part of it into the scene like a partial texture). Background defines here that the footage has the same result ion and is applied 1:1 is size full or partially over the “screen” area. If only the Picture/door area is in discussion, things might be different, but as you mention Syntheyes, the idea is maybe given as discussed in the following. Light to projected areas is not that simple, as shade and light is nothing that is easily mixed among projected and illuminated… think about it: You would illuminate a shadow, to make the shadow vanish when light falls on the shadow, as it is in the footage!
I would not recommend that, based on the following reason:
If you use practical footage to camera project this in the complete scene, you must remove the lens-distortion from the practical footage before you project it, otherwise, the result can be difficult, as the projection isn’t distorted at all (in terms of lens distortion.) This would introduce the first quality loss.
Secondly, the footage projected completely to the whole frame, if possible at all after lens-diostrotion removal, creates the problem that the rendered pixel might be not exactly the projected pixel. This partial or at all. The reason might be less in rounding errors, but more likely in Anti alias and MIP blurring. This lowers the second time the footage quality.
If you got that far without losing too much, keep in mind that the practical footage has already motion blur and perhaps depth of field blur in it. The objects in C4D have that not directly, you need to apply this to the objects in C4D. BUT certainly not again to the objects that carry the projection. Any additional Motion Blur or DOF blur would always be different to the one that the C4D not projected objects carry.
Not really a loss, but mostly underestimated, the color space. If your gamma is baked in your footage that you project, it might be not pulled correctly out (to be linear), in that way you might add this later on again on top of the baked color profile, which supports the CG look.
Noise in the footage. If you have not 100% noise free footage, the projection areas carry noise, the C4D objects by default do not of course. In that way you mix the noise inside the image and if not couterbalanced with Object-buffers and compositing, you get something ugly on the end.
Please have a look in the 54 part Integration series, that will be published here soon. As I believe we have some more points to discuss. I stop here, as I believe I have mentioned enough points, why I would not recommend to Camera project footage in the same format over the complete image, to “slap-compose” C4D objects.
——-
The way to do it, to save the highest quality, is to not touch at all the practical footage. In many cases even the Lens distortion is not removed. Of course to get a proper track from Syntheyes you must care about lens distortion as other wise the tracking result would be sloppy. For any C4D work the tracking must be un-distorted as well for Camera-Projection, always without (!!) lens distortion.
The CG material can be rendered in any way to become then in the Compositing application the right lens-distorsion, noise, color correction and all the little steps that integrate everything, not to mention the need for Light wrap (or Light Glow in some cases), Light interaction and Reflection interaction (in both cases bi-directional) might be needed as well. Thinking of an earlier discussion here on CV2, you might remember that I mentioned that projection misses critical parts for Reflection for example, especially for GI illumination.
In an upcoming series (Making Of JET) I will show how I have created the complete set in C4D as simulation just to gain all the info ration needed, with out showing that simulated set at all as a direct capture in the footage.
Camera projection with practical footage is a longer theme and I work on more material, but as you might see on the many points that I mentioned, that is not just done with a dozen of tutorials. This needs an deep understanding of the material that you work with, and how to examine what works and what not. I use camera mapping since the late ‘90s, so my experience reaches deep in that area. Thanks for your patience, until I can deliver all parts. There are some reasons to prefer still images for projection or specific practical footage for projection. I hope I can buy my next camera soon to supply as well here the some stuff to play with. ;o), standard broad cast cameras (even: HD, 3chip, 10bit/c; 4:2:2, my current one) isn’t working here very well, for that specific purpose.
——-
The need for a compositing application (NUKE for example, well After Effects as well as to certain degree) and the knowledge of integration (which means not only to prepare the C4D scene to deliver all information for the composition… hence Integration is not Compositing, compositing is a sub part of the integration work)
I hope this information above delivers enough to work with to get “Production Quality” for your efforts with Camera-Projection, and I hope as well as that the hundreds of tutorials that “we” will release soon, supports such workflow.
Have a great weekend
Sassi