Measure the error in screen space instead of world/object space

[nfrechette]

Currently, the compression error is measured in object space, see here. This is great and it helps approximate the skinning process.

However, it fails to take into account that ultimately, visual vertices are skinned and projected in screen space. This is where the animation accuracy is ultimately visible.

What if instead of measuring the error in object space, we measured it in screen space. Instead of a 3D error threshold, a 2D pixel error threshold would be used for pixel perfect accuracy. Joints would still specify the shell radius but it would be transformed into screen space before measuring the error.

To do this, we could specify the camera information in the compression settings: FOV, screen resolution, etc.

This would allow 3rd person games where the camera never gets close to the animated characters to use fewer bits as less accuracy is needed. Similarly, mobile games with a lower resolution could use fewer bits.

This could also be extended to cinematics where a new track could be provided during compression with the camera information at every key frame. When it zooms in on a character, it would retain more accuracy while other characters further away wouldn't need as much to maintain the same visual quality.

We could also take into account motion blur. Leverage the velocity and its direction to further reduce accuracy where it isn't visible to the naked eye.