Turns out inverting a matrix was the largest chunk of the CoglMatrix
code. By switching to Graphene, a lot of it can go away. The inverse
is still cached in the CoglMatrix struct itself, to preserve the
optimization.
However, switching to graphene_matrix_t to calculate the inverse has
a challenge: float precision. We had to work around it here, and it
needs an explanation.
The way to detect whether a matrix is invertible or not (i.e.
whether it's not a "singular" matrix, or not) is by checking
if the determinant equals 0. So far, so good.
Both graphene_matrix_t and CoglMatrix use single-precision
floats to store their 4x4 matrices. Graphene uses vectorized
operations to optimize determinant calculation, while Cogl
tries to keep track of the matrix type and has special-purpose
determinant functions for different matrix types (the most
common one being a 3D matrix).
Cogl, however, has a fundamentally flawed check for whether
the matrix is invertible or not. Have a look:
```
float det;
…
if (det*det < 1e-25)
return FALSE;
```
Notice that 1e-25 is *way* smaller than FLT_EPSILON. This
check is fundamentally flawed.
"In practice, what does it break?", the reader might ask.
Well, in this case, the answer is opposite of that: Cogl
inverts matrices that should not be invertible. Let's see
an example: the model-view-projection of a 4K monitor. It
looks like this:
```
| +0,002693 +0,000000 +0,000000 +0,000000 |
| +0,000000 -0,002693 +0,000000 +0,000000 |
| +0,000000 +0,000000 +0,002693 +0,000000 |
| -5,169809 +2,908017 -5,036834 +1,000000 |
```
The determinant of this matrix is -0.000000019530306557.
It evidently is smaller than FLT_EPSILON. In this situation,
Cogl would happily calculate the inverse matrix, whereas
Graphene (correctly) bails out and thinks it's a singular
matrix.
This commit works around that by exploiting the maths around
it. The basis of it is:
inverse(scalar * M) = (1/scalar) * M'
which can be extrapolated to:
inverse(M) = scalar * inverse(scalar * M) = M'
In other words, scaling the to-be-inversed matrix, then
scaling the inverse matrix by the same factor, gives us
the desired inverse. In this commit, the scale is calculated
as 1 / (smallest value in the diagonal).
I'm sorry for everyone that has to read through this :(
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1439
ada5e67f7e