summaryrefslogtreecommitdiff
path: root/architecture.html
blob: 6cfa925633e050db1f145d3f927034948a7e32fc (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>

<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8">
<link href="wayland.css" rel="stylesheet" type="text/css">
<script type="text/javascript" src="generated-toc.js"></script>
<title>Wayland</title>
</head>

<body>
<h1><a href="/"><img src="wayland.png" alt="Wayland logo"></a></h1>

<div id="generated-toc" class="generate_from_h2"></div>

<h2>Wayland Architecture</h2>

<p>A good way to understand the wayland architecture and how it is
different from X is to follow an event from the input device to the
point where the change it affects appears on screen.</p>

<p>This is where we are now with X:</p>
<p><img src="x-architecture.png" alt="X architecture diagram"></p>

<ol>
  <li>
    The kernel gets an event from an input device and sends it to X
    through the evdev input driver.  The kernel does all the hard work
    here by driving the device and translating the different device
    specific event protocols to the linux evdev input event standard.
  </li>

  <li>
    The X server determines which window the event affects and sends
    it to the clients that have selected for the event in question on
    that window.  The X server doesn't actually know how to do this
    right, since the window location on screen is controlled by the
    compositor and may be transformed in a number of ways that the X
    server doesn't understand (scaled down, rotated, wobbling, etc).
  </li>

  <li>
    The client looks at the event and decides what to do.  Often the
    UI will have to change in response to the event - perhaps a check
    box was clicked or the pointer entered a button that must be
    highlighted.  Thus the client sends a rendering request back to
    the X server.
  </li>

  <li>
    When the X server receives the rendering request, it sends it to
    the driver to let it program the hardware to do the rendering.
    The X server also calculates the bounding region of the rendering,
    and sends that to the compositor as a <em>damage event</em>.
  </li>

  <li>
    The damage event tells the compositor that something changed in
    the window and that it has to recomposite the part of the screen
    where that window is visible.  The compositor is responsible for
    rendering the entire screen contents based on its scenegraph and
    the contents of the X windows.  Yet, it has to go through the X
    server to render this.
  </li>

  <li>
    The X server receives the rendering requests from the compositor
    and either copies the compositor back buffer to the front buffer
    or does a pageflip.  In the general case, the X server has to do
    this step so it can account for overlapping windows, which may
    require clipping and determine whether or not it can page flip.
    However, for a compositor, which is always fullscreen, this is
    another unnecessary context switch.
  </li>
</ol>

<p>
  As suggested above, there are a few problems with this approach.
  The X server doesn't have the information to decide which window
  should receive the event, nor can it transform the screen
  coordinates to window-local coordinates.  And even though X has
  handed responsibility for the final painting of the screen to the
  compositing manager, X still controls the front buffer and
  modesetting.  Most of the complexity that the X server used to
  handle is now available in the kernel or self contained libraries
  (KMS, evdev, mesa, fontconfig, freetype, cairo, Qt, etc).  In
  general, the X server is now just a middle man that introduces an
  extra step between applications and the compositor and an extra step
  between the compositor and the hardware.
</p>

<p>
  In wayland the compositor <em>is</em> the display server.  We
  transfer the control of KMS and evdev to the compositor.  The
  wayland protocol lets the compositor send the input events directly
  to the clients and lets the client send the damage event directly to
  the compositor:
</p>

<p><img src="wayland-architecture.png" alt="Wayland architecture diagram"></p>

<ol>
  <li>
    The kernel gets an event and sends it to the compositor.  This is
    similar to the X case, which is great, since we get to reuse all
    the input drivers in the kernel.
  </li>

  <li>
    The compositor looks through its scenegraph to determine which
    window should receive the event.  The scenegraph corresponds to
    what's on screen and the compositor understands the
    transformations that it may have applied to the elements in the
    scenegraph.  Thus, the compositor can pick the right window and
    transform the screen coordinates to window-local coordinates, by
    applying the inverse transformations.  The types of transformation
    that can be applied to a window is only restricted to what the
    compositor can do, as long as it can compute the inverse
    transformation for the input events.
  </li>

  <li>
    As in the X case, when the client receives the event, it updates
    the UI in response.  But in the wayland case, the rendering
    happens in the client, and the client just sends a request to the
    compositor to indicate the region that was updated.
  </li>

  <li>
    The compositor collects damage requests from its clients and then
    recomposites the screen.  The compositor can then directly issue
    an ioctl to schedule a pageflip with KMS.
  </li>
</ol>

<h2>Wayland Rendering</h2>

<p>
  One of the details I left out in the above overview is how clients
  actually render under wayland.  By removing the X server from the
  picture we also removed the mechanism by which X clients typically
  render.  But there's another mechanism that we're already using with
  DRI2 under X: <em>direct rendering</em>.  With direct rendering, the
  client and the server share a video memory buffer.  The client links
  to a rendering library such as OpenGL that knows how to program the
  hardware and renders directly into the buffer.  The compositor in
  turn can take the buffer and use it as a texture when it composites
  the desktop.  After the initial setup, the client only needs to tell
  the compositor which buffer to use and when and where it has
  rendered new content into it.
</p>

<p>
  This leaves an application with two ways to update its window
  contents:
</p>

<ol>
  <li>
    Render the new content into a new buffer and tell the compositor
    to use that instead of the old buffer.  The application can
    allocate a new buffer every time it needs to update the window
    contents or it can keep two (or more) buffers around and cycle
    between them.  The buffer management is entirely under application
    control.
  </li>

  <li>
    Render the new content into the buffer that it previously told the
    compositor to use.  While it's possible to just render directly
    into the buffer shared with the compositor, this might race with
    the compositor.  What can happen is that repainting the window
    contents could be interrupted by the compositor repainting the
    desktop.  If the application gets interrupted just after clearing
    the window but before rendering the contents, the compositor will
    texture from a blank buffer.  The result is that the application
    window will flicker between a blank window or half-rendered
    content.  The traditional way to avoid this is to render the new
    content into a back buffer and then copy from there into the
    compositor surface.  The back buffer can be allocated on the fly
    and just big enough to hold the new content, or the application
    can keep a buffer around.  Again, this is under application
    control.
  </li>
</ol>

<p>
  In either case, the application must tell the compositor which area
  of the surface holds new contents.  When the application renders
  directly to the shared buffer, the compositor needs to be noticed
  that there is new content.  But also when exchanging buffers, the
  compositor doesn't assume anything changed, and needs a request from
  the application before it will repaint the desktop.  The idea that
  even if an application passes a new buffer to the compositor, only a
  small part of the buffer may be different, like a blinking cursor or
  a spinner.
</p>

<h2>Hardware Enabling for Wayland</h2>

<p>
  Typically, hardware enabling includes modesetting/display and
  EGL/GLES2.  On top of that, Wayland needs a way to share buffers
  efficiently between processes.  There are two sides to that, the
  client side and the server side.
</p>

<p>
  On the client side we've defined a Wayland EGL platform.  In the EGL
  model, that consists of the native types (EGLNativeDisplayType,
  EGLNativeWindowType and EGLNativePixmapType) and a way to create
  those types.  In other words, it's the glue code that binds the EGL
  stack and its buffer sharing mechanism to the generic Wayland API.
  The EGL stack is expected to provide an implementation of the
  Wayland EGL platform.  The full API is in
  the <a href="https://cgit.freedesktop.org/wayland/wayland/tree/src/wayland-egl.h">wayland-egl.h</a>
  header.  The open source implementation in the mesa EGL stack is
  in <a href="https://cgit.freedesktop.org/mesa/mesa/tree/src/egl/wayland/wayland-egl/wayland-egl.c">wayland-egl.c</a>
  and <a href="https://cgit.freedesktop.org/mesa/mesa/tree/src/egl/drivers/dri2/platform_wayland.c">platform_wayland.c</a>.
</p>

<p>
  Under the hood, the EGL stack is expected to define a
  vendor-specific protocol extension that lets the client side EGL
  stack communicate buffer details with the compositor in order to
  share buffers.  The point of the wayland-egl.h API is to abstract
  that away and just let the client create an EGLSurface for a Wayland
  surface and start rendering.  The open source stack uses
  the <a href="https://cgit.freedesktop.org/mesa/mesa/tree/src/egl/wayland/wayland-drm/wayland-drm.xml">drm</a>
  Wayland extension, which lets the client discover the drm device to
  use and authenticate and then share drm (GEM) buffers with the
  compositor.
</p>

<p>
  The server side of Wayland is the compositor and core UX for the
  vertical, typically integrating task switcher, app launcher, lock
  screen in one monolithic application.  The server runs on top of a
  modesetting API (kernel modesetting, OpenWF Display or similar) and
  composites the final UI using a mix of EGL/GLES2 compositor and
  hardware overlays if available.  Enabling modesetting, EGL/GLES2 and
  overlays is something that should be part of standard hardware
  bringup.  The extra requirement for Wayland enabling is
  the <a href="https://cgit.freedesktop.org/mesa/mesa/tree/docs/specs/WL_bind_wayland_display.spec">EGL_WL_bind_wayland_display</a>
  extension that lets the compositor create an EGLImage from a generic
  Wayland shared buffer.  It's similar to
  the <a href="http://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_image_pixmap.txt">EGL_KHR_image_pixmap</a>
  extension to create an EGLImage from an X pixmap.
</p>

<p>
  The extension has a setup step where you have to bind the EGL
  display to a Wayland display.  Then as the compositor receives
  generic Wayland buffers from the clients (typically when the client
  calls eglSwapBuffers), it will be able to pass the struct wl_buffer
  pointer to eglCreateImageKHR as the EGLClientBuffer argument and
  with EGL_WAYLAND_BUFFER_WL as the target.  This will create an
  EGLImage, which can then be used by the compositor as a texture or
  passed to the modesetting code to use as an overlay plane.  Again,
  this is implemented by the vendor specific protocol extension, which
  on the server side will receive the driver specific details about
  the shared buffer and turn that into an EGL image when the user
  calls eglCreateImageKHR.
</p>

</body>
</html>