Wayland Color Management support proposal GWG V1.4 7th March 2019 ----------------------------------------- Assumptions & aims: ------------------ 1) The compositor needs to be able to map a buffer to any combination of outputs. 2) It should be possible for an application to ensure that color is displayed with highest fidelity necessary to fullfil the application and users purpose. 3) Only the application has full knowledge on how to render color with highest fidelity to a given display. 4) It is acceptable that a surface mapped to more than one output only has maximum fidelity on a single output. 5) It should be possible to use hardware accelleration compositing. 6) Display profiles be ICC based, but restricted to 3 or fewer device channels. 7) Provision for Color Management of HDR displays. 8) It would be nice if default color management was applied for color unaware applications, so that acceptable appearance is maintained when wide gamut and/or HDR displays are being used by such applications. 9) Color management display calibration and profiling applications should be fully supported. This implies supporting APIs to display a top level window on a specific output with a specific size & position, install display color profiles, and honour display calibration curves ('vcgt' tag). Outline: -------- (Based on the similarity to HiDPI aware application support) * Wayland be made 3 channel (i.e. RGB) color profile aware: - Compositor to maintain a register of display color profiles. Profiles have extra associated attributes to trigger any special processing (i.e. HDR) Profiles would be referenced by a handle (i.e. they would be Wayland objects ?) If a system supports separate system and per-user installed display profiles, then the system profiles would be the default if no user display profile is set. A compositor may populate this automatically if it has capability to do so (i.e. based on EDID data), and no explicit profiles have been installed. If there is no installed profile for an output/display, then color management is disabled for that output until a profile is installed. - Maintain a register of standard 3 channel colorspace profiles (i.e. None, sRGB, AdobeRGB, ProPhoto, scRGB, REC2020, P3 etc.) that is available to all clients. - Clients can upload private 3 channel colorspace profiles, which they can also tag buffers with. - Clients can download any registered profiles visible to them. - Buffers are optionaly tagged with a color profile and conversion intent information (see below). A tag of "None" would behave as if the buffers were tagged with the output profile, and no color management would be performed. - If there is a mismatch between a buffer and an output is is mapped to, the compositor will convert between the colorspaces using the buffer intent. If the profiles match, conversion will be skipped. If there is no source color specification, a default profile may be configured (see below). - Intent specifications should include separate source and destination ICC intents (Absolute Colorimetric, Relative Colorimetric, Perceptual, Saturation), and additional information such as black point matching, and possibly HDR options (i.e. source & destination diffuse brightness SDR->HDR, HDR->HDR, HDR-.SDR. Should there be a choice of tone mapping algorithms & associated parameters ??) - There be some means of an application retrieving a display color fidelity preference (output priority order), so that it will know which display colorspace to convert to if it is doing the color management. The client would keep track of what outputs a surface is mapped to by watching wl_surface.enter/leave events. [ Missing - how this interacts with hardware compositing, pixel formats, video range etc. ] * Wayland support for Color Management applications: (It is not so important exactly which protocol these be a part of, as long as they are supported in tandem with the Wayland CM protocol. It would probably be good if they use a Wayland like protocol, and of course they need to share context and coherence with the compositor to identify the output & profile handles, notify changes etc.). - Allow a CM application to install a profile for a given output/display. Attribute of transient or permanent install. A transient install can be removed, restoring the previous permanent profile, and will automatically be removed on client termination. (Note this should share protocol and implementation of client profile upload and download). Some means of the client being notified that the install is complete is needed (the Wayland " sync" mechanism ?) It is up to the compositor implementation as to whether the profile registration is non-volatile, or whether instead it will rely on a 'loader' application (not sure about this aspect being discretionary). - Wayland will honour the calibration curves that may be contained in the 'vcgt' tag of an output/display profile. It is up to the compositor implementation as to how the calibration is honoured, but it should be at least to the same output bit depth of the display hardware CRTC VideoLUTs. i.e. if the CRTC VideoLUTs have an output bit depth of 10 bits/component, then the pixel values after application of the 'vcgt' curves should have 10 bits precision as they are conveyed to the display. If a display profile is installed that is missing a 'vcgt' tag, then linear curves should be assumed. - Information as to the supported 'vcgt' calibration curve input and output bit depth should be available, so that the profiling application can calibrate to the precision that is available. - There be a mechanism for displaying a surface at a particular locations at a particular size on a particular output for displaying color test patches. This should be a surface that cannot be covered by any other, and is not affected by screensavers or power savers. (i.e. similar to media playback in supressing non-activity timeouts.) Color management would operate normally, so that the test patches can be used for either profiling or verification. - There be some means of the user indicating a display color fidelity preference (output order), for the situation where a surface is mirrored or multiply mapped. - There be some means of setting a default source profile and intent information for un-tagged buffers. This should be a choice of "None" or some other registered ICC profile. None would be the same as no color management (i.e. the buffers are assumed to be in display colorspace. This would be the same as non color managed Wayland). This provides default color management for non-color aware applications. Unresolved: - Should the non-volatile behaviour of installed output/display profiles be specified ? It may be better to mandate that the Compositor save the display profile state, so that implementations are consistent, and distros don't have to grapple with different Compositor behavior. - Should it be possible to enable/disable color management on a per output basis ? - Related: should it be possible to de-install an output profile, effectively disabling color management for that output ? (i.e. install "None" ?) - Should installation of an Abstract profile be added for support of "blue shift" and other global color modification applications ? How is this advertised to color aware applications ? How does this work with HDR handling ? Compositing: - Aproximate 1D to/from linear light curves are extractable from the display ICC profiles. [ HDR encoded typicaly uses 12 bits. Assuming an HDR range of 0.001 - 10000 cd/m^2, and a 10% quantization at the lowest level, this gives a linear encoding range of about 1e8. To encode the linear light levels therefore requires either using 16 bit half float or 27 bits of fixed point. If 16 bit half was used, then it would be worth considering using an equation aproximation to the curves based on the Graphics Gems IV, VI.3 pp 401 gain functions: vout = vin/(g - g * vin + 1.0); g >= 0.0 vout = (vin - g * vin)/(1.0 - g * vin); g <= 0.0 where g controls the "gamma" like effect. If 32 bit integer was used, then the encoded to linear conversion could be implemented with a 12 bit/4096 entry 32 bit LUT per channel. For decode a log encoded set of tables could be used, something like (say) 22 x 10 bit/1024 entry 16 bit tables, where the 32 bit value first has it's top set bit identified which selects which of the 22 tables should be used, and then the 10 next bits are an index into the table. The table would contain the nearest quatized 12 bit encoded value. (This is actually equivalent to converting it to half float, and using the exponent to select the table and the significand as the index into the table. Or a half float could be used as an index into a 65526 entry 16 bit table.) ] ========================================================================== Security: * There seems to be no formal set of security aims and principles for Wayland, so how are protocol proposals objectively evaluated ? * No aspect of color management has implications in regard to access to other applications input and output. Profile installation should be per user by default. Global installation would require the application to have elevated privilages that would have to be communicated to the compositor somehow. * There are aspects in regard to integrity - installing a deliberately bad profile could make the display unusable until it is uninstallled. This is similar to other situations in which the display can be disrupted - i.e. resolution change, or a full screen application such as a game. Possible aproaches to mitigate this are: 1) Provide an elevated privilege mechanism. 2) Provide a "profilesane" application that loads a known good profile or disables color management. Perhaps this could be triggered by a certain keystroke sequence ? 3) If compositor doesn't do persistence, then profiles are loaded using a profile loader application, so restarting the display system with the loader disabled will recover display. 4) If compositor does persistence, then it could be responsible for a security dialog along the same lines as changing display resolution - A "request to install ICC profile XXXX, about to switch to it for 5 seconds..." "followed by a "Do you wish to install this profile ?" dialog. * A profiler needs to be able to display and position a top level window at a specified position on the display. This capability may pose a spoofing security threat. The application therefore needs to be given specfic privileges to exercise this functionality. Could a compositor security dialog be used for this too ? How is this capability handled by analogous Wayland protocols, i.e. window manager, touch screen calibration etc. ? [ A wayland security protocol would be the best way to give certain applications CM privalidges ? ] * Adam Jackson notes: "The wayland server is capable of knowing the process on the other end of the socket, and only exposing the color management control protocol to specifically blessed clients." Perhaps this is enough if for a distinct Wayland CM application protocol ?