Maya 2020 fundamentals - modelling the real world
Get halfway through a model and find it's an unworkable mess? Can't add edge loops where you need them? Can't subdivide a mesh properly? If any of this sounds familiar check this course out.
# 1 18-09-2012 , 09:30 PM
Registered User
Join Date: Jun 2008
Location: Los Angeles, CA
Posts: 229

greenscreen transparency issues

This is a rough key test. My character needs to be inside a ball at all times, keyed from greenscreen out of After Effects. I'm using Straight Alpha, not Premultiplied.

This is a render of a surface shader plane in the bg, the ball model and texture I've settled on, and a surface shader w/ keyer alpha image sequence plane for the character, inside the ball.
user added image

This is a photoshop comp - top layer is just the png coming into maya. the other is the tga for the bg plane. Just to show that the key doesn't suck as much as it looks like it does in maya. (the key DOES suck, but that's just cause this is a test)
user added image

Any ideas why non-extreme transparency values are rendering strangely? Things I've done:

Lambert (incandescent + transparency) instead of surface shader
Removed BG plane
Removed Ball object
Removed ALL objects except character
Used TGAs instead of PNGs
Tried Maya Software AND Mental Ray renderers

The other option is to perform the composite in After Effects, but though importing camera motion from maya to ae is pretty straight forward (once you know how to dodge the common issues), it takes quite a bit of time that I won't have once I start rendering. Plus, I don't know how to comp her INSIDE a semi-transparent maya render.


--
Peter Srinivasan
Producer
# 2 19-09-2012 , 05:52 AM
Registered User
Join Date: Jun 2008
Location: Los Angeles, CA
Posts: 229
Update: Switching to a mr_material_x_passes shader, dropping file.color into mr_material_x_passes.additionalColor and the alpha into mr_material_x_passes.cutoutOpacity.

Here's the render. Black lines instead of white... improvement?

Attached Thumbnails

--
Peter Srinivasan
Producer
# 3 19-09-2012 , 06:17 AM
Registered User
Join Date: Jun 2008
Location: Los Angeles, CA
Posts: 229
Next update: it only gives that black outline when the sphere AND BG are visible when using cutout and additional color (I had to turn backface culling on on the sphere shader, otherwise the black outlines remained)! So I'm pretty close to the solution it seems. But not there yet. With this method, removing either the sphere or the BG gives me perfect alpha. So there has to be some kind of transparency sorting issue with multiple layers of transparency.


--
Peter Srinivasan
Producer

Last edited by petersrin; 19-09-2012 at 06:49 AM. Reason: incomplete
# 4 19-09-2012 , 06:58 AM
Registered User
Join Date: Jun 2008
Location: Los Angeles, CA
Posts: 229
Final update so anyone looking for the a solution will have it.

In Render Settings>Mental Ray>Quality>Raytracing (must be active), set Refractions and Max Trace Depth to as many as necessary. In my case, 2 was the lowest value that would work.

That was the key. Once that was set I could keep backface culling off. Apparently culling was simply reducing the number of required refraction traces by enough that the default raytrace settings could handle all the necessary transparency effects. Once I keyed in the right Raytrace quality settings, turning culling off was fine user added image


--
Peter Srinivasan
Producer
# 5 19-09-2012 , 07:41 AM
honestdom's Avatar
The Nurb Herd
Join Date: Oct 2007
Location: London
Posts: 2,381

This is a rough key test. My character needs to be inside a ball at all times, keyed from greenscreen out of After Effects. I'm using Straight Alpha, not Premultiplied.

If you are asking what are the white lines. You partly answered your own question. Your image is unpremultiplied.

# 6 19-09-2012 , 04:19 PM
Registered User
Join Date: Jun 2008
Location: Los Angeles, CA
Posts: 229
I mean, technically with unpremult there will be white lines (or whatever color bg you choose) on the edges in the RGB channels, but that's what the alpha is for. The real issue turned out to be the number of times a ray was allowed to be refracted. Since I was exceeding it, MR just resorted to 0 and 1 for transparency values, thereby making the unpremult lines visible. The final settings removed the unpremult as expected.


--
Peter Srinivasan
Producer
# 7 19-09-2012 , 04:40 PM
honestdom's Avatar
The Nurb Herd
Join Date: Oct 2007
Location: London
Posts: 2,381
haha, sure thing. whatever works for you.

# 8 20-09-2012 , 03:44 AM
Registered User
Join Date: Jun 2008
Location: Los Angeles, CA
Posts: 229
Interesting. I switched back to surface shader just to see why it was giving me the outline and found my answer here: https://forums.cgsociety.org/showthre...r+transparency

Apparently, unlike ALL OTHER SHADERS, the surface shader applies transparency BEFORE outColor, and anything that is non-transparent is black. Then it ADDS the outColor to the entire image, including transparent portions. This satisfactorily explains and solves a number of my issues better than the solution I gave above. Unfortunately, it also seems completely preposterous. Is it some strange attempt to save processing power? Do surface shaders in other programs, renderers, and/or game engines work like that? Or was the link mistaken, but the solution still fine?


--
Peter Srinivasan
Producer
Posting Rules Forum Rules
You may not post new threads | You may not post replies | You may not post attachments | You may not edit your posts | BB code is On | Smilies are On | [IMG] code is On | HTML code is Off

Similar Threads