path: root/blend.txt

The Porter/Duff compositing algebra and blend modes

Porter/Duff

In the Porter/Duff compositing algebra, images are equipped with an
alpha channel that determines on a per-pixel basis whether the image
is there or not. When the alpha channel is 1, the image is fully
there, when it is 0, the image isn't there at all. When it is in
between, the image is partially there. In other words, the alpha
channel describes the the shape of the image, it does not describe
opacity. The way to think of images with an alpha channel is as
irregularly shaped pieces of cardboard, not as colored glass.

Consider these two images:

	 SRC ("triangle")  	DEST ("ampersand")

When we combine them, each pixel of the result can be divided into
four regions:

              	      	 \    /
	         	  \  /
		           \/
      SRC x DEST  	   /\
         		  /  \
	        	 /    \

One region where only the source is present, one where only the
destination is present, one where both are present, and one where
neither is present.

By deciding on what happens in each of the four regions, various
effects can be generated. For example, if the destination-only region
is treated as blank, the source-only region is filled with the source
color, and the 'both' region is filled with the destination color like
this:

			\/
			/\

the effect is as if the destination image is trimmed to match the
source image, and then held up in front of it:

		SRC dest_atop DEST

The Porter/Duff operator that does this is called "Dest Atop".

There are twelve of these operators, each one characterized by its
behavior in the three regions: source, destination and both (the
'neither' region is always blank). The source-only region can be
either 'source' or 'blank'. The destination-only region can be either
'destination' or 'blank'. The 'both' region can be either 'blank',
'source', or 'destination'.

The formula for the operators is a linear combination of the contents
of the four regions, where the weights are the areas of each region:

    A_src * [s] + A_dest * [d] + A_both * [b]

Where [s] is either 0 or the color of the source pixel, [d] either 0
or the color of the destination pixel, and [b] is either 0, the color
of the source pixel, or the color of the destination pixel. The areas
are given by these formulas:

    A_src = as * (1 - ad)
    A_dst = ad * (1 - as)
    A_both = as * ad

The alpha channel of the result is computed in a similar way. There is
full coverage in all regions that are not blank:

    A_src * [as] + A_dest * [ad] + A_both * [ab]

where [as] and [ad] are either 0 or 1 depending on whether the source
and destination regions are present, and where [ab] is 0 when the
'both' region is blank, and 1 otherwise.

Here is a table of all the Porter/Duff operators:

                       [s]     [d]     [b]

	Src		s	0	s
	Atop		0	d	s
	Over		s	d	s
	In		0	0	s
	Out		s	0	0
	Dest		0	d	d
	DestAtop	s	0	d
	DestOver	s	d	d
	DestIn		0	0	d
	DestOut		0	d	0
	Clear		0	0	0
	Xor		s	d	0

And here is how they look:

     [table.png]

It is important to understand that despite being referred to as alpha
blending, and despite alpha often being used to model opacity,
conceptually Porter/Duff is not a way to blend the source and
destination shapes. It is instead a way to overlay, combine and cut
them as if they were pieces of cardboard. The only places where source
and destination pixels are actually blended is where the antialiased
edges meet.


Blending

Photoshop and the Gimp have the concept of layers which are images
stacked on top of each other. In Porter/Duff, stacking images on top
of each other is done with the "Over" operator, which is also what
Photoshop/Gimp use by default to composite layers:

	   <over>

Conceptually, two pieces of cardboard are held up with one in front of
the other. Neither shape is trimmed, and in places where both are
present, only the top layer is visible.

A layer in these programs also has an associated *Blend Mode* which
can be used to modify what happens in places where both are
visible. For example, the 'Color Dodge' blend mode computes a mix of
source and destination according to this formula:

	    B(s, d) = <color dodge formula>

The pixel diagram is this:

       	   Color Dodge diagram

And the result looks like this:

     	   src ColorDodge dest

Unlike with the regular Over operator, in this case there is
substantial chunk of the output where the source and destination are
actually mixed together.

Layers in Photoshop and Gimp are not tailored to each other (except
for layer masks, which we will ignore here), so the compositing of the
layer stack is done with the source-only and destination-only region
set to source and destination respectively. However, there is nothing
in principle stopping us from setting the source-only and
destination-only regions to blank, but keep the blend mode in the
'both' region so that tailoring could be supported alongside
blending. For example, we could set the 'source' region to blank, the
'destination' region to destination, and the 'both' region to
ColorDodge:

	colordodge-dest-diagram.png

The result looks like this:

        ColorDodge dest

Here are all the possibilities:

      dodge-neither dodge-src, dodge-dest, dodge-both,

With three trival blend modes:

       Source:	B(s, d) = s
       Dest:	B(s, d) = d
       Zero:	B(s, d) = 0

we can generate all of the original Porter/Duff operators.

Reinterpreting Porter/Duff in this way leads to a natural
generalization where the blend mode can be chosen from a large set of
formulas. Each such formula gives rise to four new compositing
operators characterized by whether the source and destination are
blank or contain the corresponding pixel color.

The general formula is still an area weighted average:

    A_src * [s] + A_dest * [d] + as * ad * B(s, d)

where [s] and [d] are 0 or the source and destination color
respectively, but where B(s, d) is no longer restricted to one of 0,
s, and d but can instead be chosen from a large set of formulas.

In this way, each new blend mode gives rise to four new compositing
operators defined by whether or the source and destination regions are
present or blank.

The output of the alpha channel is the same as before:

    A_src * [as] + A_dest * [ad] + as * ad * [ab]

except that [ab] is now determined by the blend mode. For the Zero
blend mode there is no coverage in the both region, so [ab] is 0; for
most other, there is full coverage, so [ab] is 1.

Here is a table of the operators that are generated by various blend
modes:

    [ blend table.png ]

The three first rows are the original Porter/Duff operators.


<!--

Alpha Channels


The resulting value corresponds to the amount of light reflected by
the covered part of the pixel, so in order to get the right color
value, we have to divide by alpha. Since divisions are expensive and
since source and destination inputs have to be multiplied with their
corresponding alpha values anyway, pixels are often stored in a format
where all the color channels have been multiplied with the alpha
channel.

With this format, the formula becomes:

    (1 - ad) * [s] + (1 - as) * [d] + as * ad * [b]

where [s] and [d] are the premultiplied source and destination colors,
and [b] is either 0, or one of the premultiplied source or destination
colors divided by their corresponding alpha channel (this division is
then canceled by either the /as/ or the /ad/ in front of [b]).

The alpha channel is computed with the same formula. In this case [s]
is either 0 or as, [d] is either 0 or ad, and [b] is either 0, or
as/as = 1, or ad/ad = 1.

In other words, with premultiplied pixels, this formula:

   (1 - ad) * [s] + (1 - as) * [d] + as * ad * [b]

works for all four channels of a premultiplied pixel.



With the classic Porter/Duff operators the output alpha channel is
computed by plugging 1 into the area weighted average whenever a
region is filled with either source or destination, and 0 whenever a
region is blank. We can also view this as plugging in 1, 1 in the
blend mode formula:

      Source:	B(s, d) = s;	B(1, 1) = 1
      Dest:	B(s, d) = d;	B(1, 1) = 1
      Zero:	B(s, d) = 0;	B(1, 1) = 0

Most of the new blend mode formulas satisfy B(1, 1) = 1, which means
it is tempting to define the alpha computation to be exactly the same
as the color computation, just as it is for the Porter/Duff
operators. As long as B(1, 1) = 1, we get:

    ar = (1 - ad) * as + (1 - as) * ad + as * ad * B(as/as, ad/ad)
       = (1 - ad) * as + (1 - as) * ad + as * ad * B(1, 1)

A very nice result of this is that the ADD operator that simply adds
all four channels can be defined in this framework with the blend
mode:

	Plus:	B(s,d) = s + d

When plugging in 1, 1, you get:

     ar = (1 - ad) * as + (1 - as) * ad + as * ad * B(1, 1)
        = (as - ad * as + ad - as * ad + as * ad * (1 + 1)
	= as + ad

and similarly for the color channels:

     r = (1 - ad) * s + (1 - as) * d + as * ad * (s/as + d/ad)
       = s - s * ad + d - as * d + ad * s + as * d
       = s + d

Unfortunately, there are two annoying exceptions: The blend modes
Exclusion and Difference do not produce 1 when you plug in (s = 1, d =
1), they produce 0. There is some logic to that, because these blend
modes are concerned with the difference between source and
destination, and the difference in coverage in the 'both' region
(where source and destination are both present by definition) would in
fact be zero. Nonetheless, making the alpha channel 0 in the 'both'
region would mean that you couldn't see the blended result, so that's
not a useful option.

The right solution may be instead to define inverted versions:

    Inverted Exclusion:		B(s,d) = 1 - Exclusion(s,d)
				       = 1 - s - d + 2 * s * d

    Inverted Difference:	B(s,d) = 1 - Difference(s,d)
    	     			       = 1 - |s - d|

These will produce the correct alpha channel, and users can still
achieve the non-inverted blend modes by using the inverted blend modes
twice.

Finally, there are four "non-separable" blend modes that don't operate
on a per-channel basis. These instead conceptually convert the source
and destination into the HSL representation and then for each of the
H, S, L channels pick either the source or the destination
value. There is obviously no way to unify this operation into one
formula that applies to all four channels.


Conclusion

It is possible to generalize the Porter/Duff compositing algebra by
allowing more behaviors in the 'both' region of the Porter/Duff
pixel. If the effect on the alpha channel is defined to be B(1,1),
then all four channels can be computed with the same formula:

  (1 - ad) * [s] * S + (1 - as) * [d] * D + as * ad * B(S/as, D/ad)

where [s] and [d] are 1 or 0 depending on whether the source and
destination are drawn, and B is the blend mode. When B(s,d) is chosen
to be 0, s, or d, the formula reduces to the twelve original
Porter/Duff operators.

-->

<!--

CSS compositing spec

There is a proposal for extending CSS
[https://dvcs.w3.org/hg/FXTF/rawfile/tip/compositing/index.html] with
support for blending and compositing that also tries to unify
porter/duff compositing with blend modes. In this scheme a Porter/Duff
operator and a blend mode are chosen orthogonally.

For a given pixel, the source and the destination are blended
according to the chosen blendmode, and then a linear interpolation
between the source and the blended result is computed with the
destination alpha as the interpolation weight. This result is then
used as the source in compositing with the chosen Porter/Duff
operator. The full formula is this:

      Fa * as * [ (1 - ad) * Cs + ad * B(s,d) ] + Fb * ad * Cd

where Fa and Fb are weight factors corresponding to the chosen
Porter/Duff operator.

This formula has some useful properties: If the blend mode is chosen
to be Normal, that is, B(s,d) = Cs, then the formula becomes identical
to whatever Porter/Duff operator is involved. If the Porter/Duff
operator is chosen to be Over, then the formula becomes identical to
the Photoshop/Gimp layer blending. When the operator is Source, the
formula produces the atop-like result shown above.

However, there is no conceptual underlying reason that the formula has
these properties. You can't give a graphical interpretation of it the
way you can with Porter/Duff; the useful properties the formula has
might as well be numerological accidents. In fact, for many operators,
the formula produces nonsensical results.

For example, here is what it looks like when the operator is DEST_OVER
and the blend mode is ColorDodge:

    DEST_OVER / COLOR_DODGE

What is this? It's DEST_OVER, except that the antialiasing is messed
up with a weird purple shadow. There is no situation where this is the
operation you need.

There is also a bunch of operators where the blend mode is simply
ignored: DEST, DEST_IN, DEST_OUT, CLEAR.

The fundamental problem is that it doesn't make any sense to select a
Porter/Duff compositing operators and a blend mode orthogonally. The
Porter/Duff operator and the blend mode both specify what happens in
the region where source and destination overlaps, and if they disagree
on what happens there, it is not clear what happens. The answer to
this is not to pick some formula that happens to produce useful
results in certain special cases.

-->