Cineversity

MikeC. - 10 February 2019 01:31 PM

Hi Kent

Thanks for your answer ... hard times for good Plugins ...

Anybody else who could press the Button «Bake BlendShapes» and see what happens?

Greez, Mike C.

Bake BlendShapes will bake the blend shape values onto the strength value of each Pose Morph. If you also have eyes tags on the model then it will also bake the eye positions as well. This should all be fully documented in the help files that ship with CV-AR.

I believe it is a workflow issue, but I would have to spend more time looking into it to see what happens if you have already sculpted your shapes and then press the Bake Button. But from memory I think you may have to bake these before you start your sculpting. If I find some time over the weekend I will take another look.

Kent

OK, a missunderstanding ... it makes sense now ... so it’s a decision you have to do first and it’s not the idea to bake the already made Morphs to use without the Helper-Tag ...

Well, if you have to change a face to another tracking it’s all the way better to use it with the Helper-Tag ...

Thanks Kent ...

Yes I just did a quick test now and can confirm the issue you are seeing. If you already have Pose Morphs in place and then bake the blend shapes the poses are no longer changing. This will require an update to fix this issue as it will need to check and keep any existing poses that are on the Pose Morph. Sorry it took so long for a response on this issue.

Kent

Hey Kent

You do a great job ... and thanks for the test ... 11:30pm ... goto bed ...

Mike C.

Hi there,

I LOVE THIS APP! I was wondering though whats the maximum time I can record on there? It seemed to crash when I tried a 10 min one haha, is there a recommended max length for a single shot?

thanks

polypoint - 27 February 2019 02:35 AM

Are there any smoothing algorithms applied in the recent versions? I used the new version for a face capture and there is a lot of offset between the mesh and the texture image sequence, especially on fast head movements. It seems that the head mesh moves more smoothly, but on the cost of being in accurate sync with the texture frames. Is this something that can be fixed in the settings? Need to get the same results as in version 1128b.

No there aren’t any smooth algorithms applied to the recent version. It may be a bandwidth/procesing speed issue since I think in your test here you are doing 60 FPS? Since in this case it has to record 60 images per second and store those along with capturing the motion. For 60FPS it would be better to just use the Single Image option and only record the very first frame of your face and use that for the texture mapping throughout, this is handled automatically for you by the app and the plugin.

I don’t know what version 1128b is. But if you are referring to the the first release of CV-AR then you can just set the FPS to 30 and that will give you the same results as the 1.0 release.

russs - 11 March 2019 01:51 AM

Hi there,

I LOVE THIS APP! I was wondering though whats the maximum time I can record on there? It seemed to crash when I tried a 10 min one haha, is there a recommended max length for a single shot?

thanks

Sorry that it’s crashing. It shouldn’t be doing that at all. It should be gracefully failing and telling you it has run out of memory for your capture, saving your existing capture in the process.

The length of your capture will depend on the if you are recording with all images or just single image, and if you have 30fsp or 60fps.

If you are recording with All Images then I think 10 mins would most likely run out of internal memory, since it is storing 60 images a second for 10 minutes. So 60*60*10 images.

For long captures consider using the Single Image option to only record the very first image frame for your capture. This will then only store a smaller amount of data just for the face mesh movement itself.

Let me know what settings you are using.

Kent

polypoint - 17 March 2019 02:46 PM

I’m baking the face mesh to PLA for further FX treatment, which works well. As the proper UV mapping gets lost at this point, i want to bake the texture as well. But that doen’t match. Is there a way to get the right mapping of the texture sequence on a baked mesh?

You should try setting the option to use a specific frame for the texture and UVs. Then the UVs will not change during playback. You can also create these files directly from CV-AR with only a single image in them. Just change the option at the top of the App from All Frames to Single Image. Then when you first start recording hold the camera close to your face in a neutral expression to get a good first frame for use as the texture.  When you transfer this capture to C4D it will only have 1 texture image and one set of UVs, and the UVs will not change during playback.

There is no way to bake the images are present. I could write a plugin to do this, but that is out of scope at the moment. Effectively what you want to do is keep the uvs and rebake every image to map to this UV set.

Sorry unfortunately there is no streamlined way to do this. You might be able to bake/remap each image using the BodyPaint/UV tools.

I have my own plugins/tools that I use to do this.

polypoint - 23 March 2019 05:08 AM

Kent Barber - 21 March 2019 06:48 AM

There is no way to bake the images are present. I could write a plugin to do this, but that is out of scope at the moment. Effectively what you want to do is keep the uvs and rebake every image to map to this UV set.

Sorry unfortunately there is no streamlined way to do this. You might be able to bake/remap each image using the BodyPaint/UV tools.

I have my own plugins/tools that I use to do this.

would you mind to share a bit more about your baking process? It would be really helpful to find a solution. It seems that the UVs are dynamically changing, actually simulation a camera mapping from the position of the phone.

I use my own software to remap textures.

https://vimeo.com/295097398

I haven’t automated the process yet, but would be easy enough for me to do. Let me know if you think this would be useful and I could add in an option to convert all images. You could then use the Image Sequence on a material and the UVs would remain static throughout.

sh00rk - 25 May 2019 09:46 PM

Is it possible that data in the plugin somehow has reversed left and right? See my screenshot, Face Mesh shows that left eye is closed, but in the blend shapes data you see that Right Eye Blink is 81% while Left is just 14%

Hi sh00rk,

Thanks for this. Yes you are correct, these values to appear to be swapped. It may be all the eye values. I will look more into this to see where exactly the problem came from.

Thanks again for finding this one.

Best Regards,
Kent

polypoint - 02 July 2019 11:05 AM

Using blend-shapes from CV-AR face mesh for an iPhone X App?

I am wondering if there is a workflow to extract 52 blend-shapes from CV-AR face mesh so that they can be used for a iPhone X App. I understood that the plugin’s main purpose is to record and apply the pose morph data to other meshes inside C4D. However, if the blend shape mesh states are already stored somewhere, it would be useful to access them. Thanks for any hint!

Unfortunately apple doesn’t provide the individual blend shapes as meshes. They only provide the final vertex positions of the face for each frame. This is most likely due to the ML model they are using to generate the mesh itself. My guess would be that there are no actual target blend shape meshes at all internally, and the system is just generating the mesh, each frame, based on the users face and the blend shape values that are fed into the ML model.

With that being said it still may be possible to reverse engineer the blend shapes from a recorded capture. If the actor moved their face through every extreme expression it may be possible to analyse the vertex data along with the blend shape values to determine some base shapes. More research would have to be done however to make this possible, and it is not currently planned.

Does CV AR work with R20 or R21?
I’ve actually installed it in 3 versions and it doesnt show up for any of them.
I am running Sierra 10.12.6 for software purposes.

I have since installed it with R19 on Mojave 10.14 and it still doen’t show up ion my plugins.
I installed it manually in both plug-in folders (as the cineversity app is down right now)

A bit fristrated.

Hi Angie,

Sorry to hear you’re having problems.

CV-AR currently works with the Studio Edition of Cinema 4D for both R19 and R20. What version of Cinema 4D do you have?

Best Regards,
Kent

Hey I’m trying to get this running on R21, on Mac OS Catalina. I moved the files into the plugins folder (by finding it via C4D’s preferences first), quit C4D, restarted, and got a security warning about the .XLIB files, approved them in system settings, restarted C4D, allowed it to open the .XLIB files, and the CV-AR dialog just doesn’t appear in the Extensions menu (but CV-toolbox does).

Anyone able to run this on C4D R21? I know it’s supposed to require studio, but version 21 did away with that.