Ok no more guessing… Here’s a way to calculate alpha gain and alpha offset at least for mental ray for Maya… It should work with most packages though at least the way to calc alpha offset
Quick fix:
If you don’t need to understand what the problem is and how the solution works here’s the values in short
Alpha gain: (Alpha Depth Factor from zbrush * 10) * 2
Alpha offset: the negative (the inverse?) of half the alpha gain.
Example:
Zbrush Alpha Depth Factor = 0.834
Maya alpha gain = 16.68
Maya alpha offset = -8.34
An alternative to using this method is using the set range node and the multiply divide node.
Don’t change anything on the file node.
Connect the out alpha from the file node to the value x in the set range node
Use these settings for the set range node
Min: -1
Max: 1
Old min: 0
Old max: 1
Connect the outvalue.X from the set range to input1.x on the multiply divide node
Connect the outvalue.x from the multiply divide node to your displacement.
Now instead of changing the alpha gain when we want to scale the displacement, change input2.x value on the multiply divide node.
These methods will give you the exact same result but the one using set range is a bit less math.
Now for the ppl who just need to know (like me)
The reason ppl have been getting “bloated” result is because the midpoint or the “no displacement” color of maps made by zbrush is 0,5 or gray instead of 0 or black
THIS HOWEVER IS NOT A BAD THING
Why? Because when modeling in zbrush you are quite likely to push your model inwards, creating negative displacement. If we wanted to get these negative displacements into a map while at the same time having the no displacement color at 0 (black) we would have to have negative color values. This is not supported by any image format I know of. This is why the makers of zbrush have decided to set the zero displacement color to 0.5 or gray.
So what to do? Well you could split the map in one negative and one positive displacement map by using Photoshop to save out al the values above 0.5 to one map and the ones below 0.5 to another.
How ever this poses a new problem and that’s how to get to displacement maps onto a single object. I didn’t find any way to do this in Maya so another way had to be found.
Something that is less that obvious in Maya (and mental ray) is that the displacement node actually can displace negatively. If you want to see for your self add a displacement material without any texture and set the displacement to a negative value. The whole object will shrink.
So what we need to do is subtract half of the values that we got from the map, simple enough just set the alpha offset to -0.5 and you’re done right? Well kinda.
The way alpha gain and alpha offset works is the value from the map is multiplied by the alpha gain and then added to the alpha offset.
Example:
If the value from the map is 0.5,
The AG is 1
And AO is -0.5
The out value would be 0.
This is what we want. Gray in the map is now zero displacement. But what happens if we change the alpha gain?
0.5 from the map
AG is 2
AO is -0.5
(0.5 * 2) - 0.5 = 0.5
Woops… Now gray in the map means a displacement of 0.5. When using AG of 2 we must subtract another 0.5 using the Alpha offset. So for gray to be zero displacement we have to set the alpha offset to -1
This means that the relationship between AO and AG should be
Alpha offset = -(alpha gain/2)
Now we have one more problem. Let’s say we have these settings
Value from map: 1
AG: 1
AO: -0.5
The out value will be 0.5 not 1. To fix this we double the AG and change the AO accordingly.
So:
AG: 2
AO: -1
Now we’ll get an out value of 1 when the value from the map is 1, we’ll get an out value of 0 when the map says 0.5 and we’ll get an out value of -1 when the map says 0.
And that’s all folks