Moves By Maxon
Posted: 08 April 2020 04:43 PM   [ Ignore ]  
Avatar
Total Posts:  7
Joined  2012-02-21

I’ve installed Moves By Maxon (MBM) in my R21 and iPhone RX.  I’ve been able to successfully create a face capture and load it into R21.  The online MBM videos do a good job of showing how to do those things.

The problem now is knowing how to merge the face capture with the figure you want to merge the face capture with.  The figure I am using is an animated human model from Fuse/Mixamo.  The figure has animation included in the .fbx file I’ve used to import the model into R21.  There is no face animation with this model.  The animated Fuse/Mixamo model works fine in R21.  The question is how to bring the face capture data into the Fuse/Mixamo model.  I have not been able locate any information how to do this.

Also, it is not clear how to work with the :

1) Face Capture Object, and

2) Face Capture - PoseMorph (Moves by Maxon Pose Morph Helper)

It would be very helpful to know how to work with the above to make use of MBM.

Please advise regarding how to use the above and if there are available resources, how to access them.  I have attempted with Maxon Help files and have been unable to locate any information.  Thank you.

Profile
 
 
Posted: 08 April 2020 06:05 PM   [ Ignore ]   [ # 1 ]  
Moderator
Avatar
Total Posts:  8231
Joined  2011-03-03

Hi pacificm,

While the FaceCapture object is active, the Attribute Manager>Object:  will have a help button in the lower area.

Please have a look here:
https://www.cineversity.com/forums/viewthread/3887/

I have one suggestion to get this transferred, no matter what the other mesh is. However, the mesh that was provided in that thread (link above) was too dense to be useful. Here the polygons folded on top of each other. (I hope that makes sense.) In short, I created a joint for each point of the iPhone mesh. I can say with some certainty that I would not suggest that route in general. In the discussed example, the two faces matched nearly perfectly.

The way that I believe is the way to go is based on setting up a PoseMorph on your model, which is shown in my little screen captures. This will allow transferring the parameter/strength of each “pose” to the PoseMorph. To be clear, that will move the slider, not a single point on the receiving mesh.
Once you have decided what poses you like (or perhaps all of them), you have to set them up manually. Which allows as well to find your individual expression of the face.
Here is the clip I have in mind.
https://www.amazon.com/clouddrive/share/utVyF6UfUT1kjyLNCbWxm5Z2egjt3ocygokdlkkVunA

Brainstorming: Something I think about is the option to use a Mesh deformer. To make this work, it needs to have a volume that encloses the face of the character. However, as mentioned above, the mesh should somewhat similar. Since the face is just a portion of a whole figure, a weight map is needed, and the change of position should be adapted. So far, the fastest way was to use Cloth Surface with thickness. I’m not really happy with it so far, so I leave it for now as “brainstorming”.

All the best, and stay safe

 Signature 

Dr. Sassi V. Sassmannshausen Ph.D.
Cinema 4D Mentor since 2004

Photography For C4D Artists: 200 Free Tutorials. Texture, Panorama, HDRI, Camera Projection, etc.
https://www.youtube.com/user/DrSassiLA/playlists

Profile
 
 
Posted: 08 April 2020 08:58 PM   [ Ignore ]   [ # 2 ]  
Moderator
Avatar
Total Posts:  8231
Joined  2011-03-03

P.S.: This is another path which I currently explore. The idea is pretty simple. The first frame of a neutral face is copied and used as a reference. Then the point distance (in X, Y, Z) over the animation is stored in a Vertex Color Tag and updated via Xpresso.

With the data available, it is easy to bake this out as texture. Open EXR linear, of course. 50% is the neutral state, so points move if that changes. R, G, B is X, Y, Z.

A result/example is below, as well as the setup screenshot. The 50% moves the points half the distance, so that needs to be adjusted first. I work on that.
https://www.amazon.com/clouddrive/share/LQS3xcKfRVqI0G5tdrTyX7ynv9aToMa6RgO6ZSMLLJD

There is more development effort needed. No idea when I’m happy with it. If the face is very different, the UV needs to be adjusted, which is kind of a stopper right now.

This is just a preview, a sketch. However, the main idea that I have in mind is to have for each Pose an individual image-based information. Which requires the recording of all poses first. Once this is set into an XYZ (RGB) map, it should help to set up the full PoseMorph.

Preferably the scene file is set to linear all the way, and the correct image for each Pose is given. As a side note, I believe that blurring the XYZ map might help to get smoother results.

Cheers

 Signature 

Dr. Sassi V. Sassmannshausen Ph.D.
Cinema 4D Mentor since 2004

Photography For C4D Artists: 200 Free Tutorials. Texture, Panorama, HDRI, Camera Projection, etc.
https://www.youtube.com/user/DrSassiLA/playlists

Profile
 
 
Posted: 09 April 2020 05:16 PM   [ Ignore ]   [ # 3 ]  
Avatar
Total Posts:  7
Joined  2012-02-21

Hello,

Thanks you for your reply.  Knowing about the Help button was very helpful.  It serves as a very good reference tool. 

The three videos included at https://www.cineversity.com/vidplaylist/moves_by_maxon/moves_by_maxon_overview serve as a great way to get started with Moves.

It would also be helpful if additional videos were added to walk through the steps involved after the motion capture data is transferred to C4D.  I would imagine that more people would make use of Moves if a video showed the process and steps involved from the C4D capture data import to the face capture data transfer to the object. 

This is a request to have such a video made for this purpose. 

Thanks again for your help!

Profile
 
 
Posted: 09 April 2020 06:16 PM   [ Ignore ]   [ # 4 ]  
Moderator
Avatar
Total Posts:  8231
Joined  2011-03-03

You’re very welcome, pacificm, thanks for the reply.

Yes, there certainly should be more about this. Tutorials requests are possible in the Tutorial Suggestion Forum. Should I move the whole thread over there?

It is undoubtedly a compelling option, and with a mini LIDAR now in the Pro iPad, I can see already that this will be a story with a significant future ahead (not that I know of any internal plans nor that I would ever spill the beans.)

Cheers

 Signature 

Dr. Sassi V. Sassmannshausen Ph.D.
Cinema 4D Mentor since 2004

Photography For C4D Artists: 200 Free Tutorials. Texture, Panorama, HDRI, Camera Projection, etc.
https://www.youtube.com/user/DrSassiLA/playlists

Profile
 
 
Posted: 10 April 2020 07:45 AM   [ Ignore ]   [ # 5 ]  
Avatar
Total Posts:  7
Joined  2012-02-21

Hello,

Yes, please do what is necessary to have this be a tutorial request.  Yes, please move the whole thread over there if that would help.

If there is anything else that I can do to make a tutorial request, please let me know.

Moves By Maxon is one of the most exciting developments I’ve experienced in C4D and I would love to do more with it.

Thanks for your help and support!

The Best

Profile
 
 
Posted: 10 April 2020 09:02 AM   [ Ignore ]   [ # 6 ]  
Moderator
Avatar
Total Posts:  8231
Joined  2011-03-03

I will do so, pacificm.

I agree it is an exciting step in a beautiful direction.

Enjoy

 Signature 

Dr. Sassi V. Sassmannshausen Ph.D.
Cinema 4D Mentor since 2004

Photography For C4D Artists: 200 Free Tutorials. Texture, Panorama, HDRI, Camera Projection, etc.
https://www.youtube.com/user/DrSassiLA/playlists

Profile
 
 
Posted: 23 May 2020 03:57 PM   [ Ignore ]   [ # 7 ]  
Total Posts:  3
Joined  2020-05-23

hey dr sassi, sorry but I can’t get to blend my face scan to the actual character , is there an actual tutorial somewhere I can’t seem to find one anywhere on youtube?

Profile
 
 
Posted: 23 May 2020 09:25 PM   [ Ignore ]   [ # 8 ]  
Moderator
Avatar
Total Posts:  8231
Joined  2011-03-03

Hi julopez,

Typically, one uses the PoseMorphs and “designs” one’s own (Pose Morphs) based on the names of the given expressions. The captured values are then animating these PoseMorphs.

A direct transfer to a different mesh is not given.

.

It is pretty clear to me that the direct use of any character is wanted.

I have not seen a tutorial that could transfer the results to any mesh. Well, I have shared a few ideas in the posts above. But they have not really reached the level of production-ready from my point of view.

For a while, I am searching for a straightforward method to transfer this. The result needs more experimentation to streamline it. So far, to move the figure after the transfer, one needs to keep the MbM as well the Locked mesh, and the final mesh under a Null. Again, I would call this an idea in beta. So, test this carefully.

Screen-capture of the idea
https://www.amazon.com/clouddrive/share/NLd8TP910aZMeJLVaa9ewOPYA1EEaSTJ9XDXpat09oW

The next step would be to use PoseMorph Deformer instead of the Xpresso set up.
Scene file
https://www.amazon.com/clouddrive/share/Z1vyylmhB5I13w9XlycMKqrkc9CZxeTaAPYbEApYhAj
In this file, as with all of the point transfers, the Object axis must be positioned in exactly the same way. Points’ world-system is the object axis.

Note that the face is not only based on the expressions, but its very unique topography will also be transplanted/transferred with this as well. To avoid this, a point difference list is needed.

Again, I’m exploring it with my iPad at the moment.

I hope that gives you some ideas.

Enjoy your weekend, stay safe, stay healthy.

Image Attachments
MbM_Face-trans.jpg
 Signature 

Dr. Sassi V. Sassmannshausen Ph.D.
Cinema 4D Mentor since 2004

Photography For C4D Artists: 200 Free Tutorials. Texture, Panorama, HDRI, Camera Projection, etc.
https://www.youtube.com/user/DrSassiLA/playlists

Profile
 
 
   
 
 
‹‹ New Features in R21      motionclips ››