PictureTransformPoint computes homogeneous coordinates internally, but fails
to handle intermediate values larger than 16.16. Use 64 bit intermediate
values while computing the final result at 16.16 and only complain if that
result is too large.
PictureTransformIsIdentity was completely wrong -- it was not checking for
identity transforms at all.
RandR matrix computations lose too much precision in fixed point;
computations using the inverted matrix can be as much as 10 pixels off.
Convert them to double precision values and pass those around. These API
changes are fairly heavyweight; the official Render interface remains fixed
point, so the fixed point matrix comes along for the ride everywhere.
This reduces the matrix representation error after inverting a
transformation matrix (although it doesn't eliminate it entirely).
Perhaps we should extend Render to include 64-bit floating point transforms...
The render extension uses many matrix operations internally, this change
exposes those functions to other parts of the server, drivers and
extensions. The change is motivated by the 'transform' additions to the
RandR extension but will likely be useful elsewhere.