mi: fix rounding issues around zero in miPointerSetPosition

Fixes: https://gitlab.freedesktop.org/xorg/xserver/-/issues/577

This patch replaces the instances of trunc in miPointerSetPosition by
floor, thereby removing the incorrect behaviour with subpixel pointer
locations between -1 and 0.

This is the relevant code fragment:

    /* In the event we actually change screen or we get confined, we just
     * drop the float component on the floor
     * FIXME: only drop remainder for ConstrainCursorHarder, not for screen
     * crossings */
    if (x != trunc(*screenx))
        *screenx = x;
    if (y != trunc(*screeny))
        *screeny = y;

The behaviour of this code does not match its comment for subpixel
coordinates between -1 and 0. For example, if *screenx is -0.5, the
preceding code would (correctly) clamp x to 0, but this would not be
detected by this condition, since 0 == trunc(-0.5), leaving *screenx
at -0.5, out of bounds.

This causes undesirable behaviour in GTK3 code using xi2, where negative
subpixel coordinates like this would (to all appearances randomly)
remove the focus from windows aligned with the zero boundary when the
mouse hits the left or top screen boundaries.

The other occurences of trunc in miPointerSetPosition have a more subtle
effect which would prevent proper clamping if there is a pointer limit
at a negative integer rather than at 0. This patch changes these to
floor for consistency.

Signed-off-by: Willem Jan Palenstijn <wjp@usecode.org>
Part-of: <https://gitlab.freedesktop.org/xorg/xserver/-/merge_requests/1451>
(cherry picked from commit 0ee4ed286ea238e2ba2ca57227c3e66aca11f56b)

1 files changed, 4 insertions, 4 deletions

diff --git a/mi/mipointer.c b/mi/mipointer.c
index 8cf003514..8ab814785 100644
--- a/mi/mipointer.c
+++ b/mi/mipointer.c
@@ -622,8 +622,8 @@ miPointerSetPosition(DeviceIntPtr pDev, int mode, double *screenx,
     pPointer = MIPOINTER(pDev);
     pScreen = pPointer->pScreen;
 
-    x = trunc(*screenx);
-    y = trunc(*screeny);
+    x = floor(*screenx);
+    y = floor(*screeny);
 
     switch_screen = !point_on_screen(pScreen, x, y);
 
@@ -701,9 +701,9 @@ miPointerSetPosition(DeviceIntPtr pDev, int mode, double *screenx,
      * drop the float component on the floor
      * FIXME: only drop remainder for ConstrainCursorHarder, not for screen
      * crossings */
-    if (x != trunc(*screenx))
+    if (x != floor(*screenx))
         *screenx = x;
-    if (y != trunc(*screeny))
+    if (y != floor(*screeny))
         *screeny = y;
 
     return pScreen;