Hi Annet,
The problem is based on several parts. The first part is that the image size is pretty small. This is relative to the final use and camera setting, but given the mesh density, this is out of balance. If the elements in the image were seen in the rendering left to right, the maximum render resolution would be around 500 pixels. However, the low resolution showcases the underlying problem clearly.
It is essential here to understand the problem that the layer transparency causes. The effect is simply explained: The layer transparency provides only a fraction of the RGB value in the seams around the elements. Which typically sufficient while working in Photoshop. But to apply the transparency values again, it will affect the transparency twice, one time in the color channel, and again in the Alpha Channel.
In VFX, compositing this problem is typically solved in Un-multiply/Premultiply processes. Since the image here is based pretty much on path-based-elements only, the clean edge of the path is destroyed by the low resolution and the idea of Anti Aliasing that photoshop has.
With the path based image file, do the following:
• Set the image size to 16384 pixels (Image, not canvas). (it was only 1024 wide in the provided file.)
• Select the top folder, then with option/alt key down, in the Layer Options, use Merge Visible. let go of the Alt/Option key when done
• With the Cmd/Control key down, click on that new layer.
• While the Selection is active, got to Selection> Save Selection. Then CMD+D to deselect the active ant-selection.
• Go to the Channels and select the alpha (visibility), and the visibility off for anything else.
• In Filter> Others> Minimum 3 pixels.
• Now set the Image size 8192, or leave it in that size you have. Again, the size is relative to the final use in the rendering.
• Save the image and replace it with the one in the scene file.
• Use the Alpha channel. (Use the Layer-set Select Button)
• Set the Anti Alias in Cinema 4D to Best.
It should look much better now. See image
https://www.amazon.com/clouddrive/share/bNKmbckDRREW7rCk0fVj4oyzwzYRf5Ui83FgCJekZP4
(Photoshop files must be saved in PSD/PSB compatible mode (which adds a merged layer), Cinema 4D is not rendering Path based information from Ps. If you use the Select option, please keep this in mind.)
All the best