path: root/blend-wp.txt

Note that the master is currently the server text - copy that before
making changes.



<script type="text/javascript"
  src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>In the Porter/Duff compositing algebra, images are equipped with an alpha channel that determines on a per-pixel basis whether the image is there or not. When the alpha channel is 1, the image is fully there, when it is 0, the image isn't there at all. When it is in between, the image is partially there. In other words, the alpha channel describes the the shape of the image, it does not describe opacity. The way to think of images with an alpha channel is as irregularly shaped pieces of cardboard, not as colored glass. Consider these two images:
<blockquote><img src="wp-content/uploads/2013/03/source.png">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<img src="wp-content/uploads/2013/03/dest.png"></blockquote>

When we combine them, each pixel of the result can be divided into four regions:<blockquote><img src="wp-content/uploads/2013/03/diagram.png"></blockquote>

One region where only the source is present, one where only the destination is present, one where both are present, and one where neither is present.

By deciding on what happens in each of the four regions, various effects can be generated. For example, if the destination-only region is treated as blank, the source-only region is filled with the source color, and the 'both' region is filled with the destination color like this:
<blockquote><img src="wp-content/uploads/2013/03/destatop-diagram.png"></blockquote>
The effect is as if the destination image is trimmed to match the source image, and then held up in front of it:
<blockquote><img src="wp-content/uploads/2013/03/destatop.png"></blockquote>
The Porter/Duff operator that does this is called "Dest Atop".

There are twelve of these operators, each one characterized by its behavior in the three regions: source, destination and both (the 'neither' region is always blank). The source-only region can be either 'source' or 'blank'. The destination-only region can be either 'destination' or 'blank'. The 'both' region can be either 'blank', 'source', or 'destination'.

The formula for the operators is a linear combination of the contents of the four regions, where the weights are the areas of each region:
<blockquote>\(A_\text{src} \cdot [s] + A_\text{dest} \cdot [d] + A_\text{both} \cdot [b]\)</blockquote>

Where \([s]\) is either 0 or the color of the source pixel, \([d]\) either 0 or the color of the destination pixel, and \([b]\) is either 0, the color of the source pixel, or the color of the destination pixel. The areas are given by these formulas:<blockquote>
\(A_\text{src} = \alpha_\text{s} \cdot (1 - \alpha_\text{d})\)
\(A_\text{dst} = \alpha_\text{d} \cdot (1 - \alpha_\text{s})\)
\(A_\text{both} = \alpha_\text{s} \cdot \alpha_\text{d}\)
</blockquote>

The alpha channel of the result is computed in a similar way. There is
full coverage in all regions that are not blank:
<blockquote>
\(A_\text{src} \cdot [\text{as}] + A_\text{dest} \cdot [\text{ad}] + A_\text{both} \cdot [\text{ab}]\)
</blockquote>
where \([\text{as}]\) and \([\text{ad}]\) are either 0 or 1 depending on whether the source and destination regions are present, and where \([\text{ab}]\) is 0 when the 'both' region is blank, and 1 otherwise.

Here is a table of all the Porter/Duff operators:
<blockquote><table>
<tr><td/><td>\([\text{s}]\)</td><td>\([\text{d}]\)</td><td>\([\text{b}]\)</td></tr>
<tr><td>Src</td><td>\(s\)</td><td>\(0\)</td><td>s</td></tr>
<tr><td>Atop</td><td>\(0\)</td><td>\(d\)</td><td>s</td></tr>
<tr><td>Over</td><td>\(s\)</td><td>\(d\)</td><td>s</td></tr>
<tr><td>In</td><td>\(0\)</td><td>\(0\)</td><td>s</td></tr>
<tr><td>Out</td><td>\(s\)</td><td>\(0\)</td><td>\(0\)</td></tr>
<tr><td>Dest</td><td>\(0\)</td><td>\(d\)</td><td>d</td></tr>
<tr><td>DestAtop</td><td>\(s\)</td><td>\(0\)</td><td>d</td></tr>
<tr><td>DestOver</td><td>\(s\)</td><td>\(d\)</td><td>d</td></tr>
<tr><td>DestIn</td><td>\(0\)</td><td>\(0\)</td><td>d</td></tr>
<tr><td>DestOut</td><td>\(0\)</td><td>\(d\)</td><td>\(0\)</td></tr>
<tr><td>Clear</td><td>\(0\)</td><td>\(0\)</td><td>\(0\)</td></tr>
<tr><td>Xor</td><td>\(s\)</td><td>\(d\)</td><td>\(0\)</td></tr>
</table></blockquote>
And here is how they look:
<blockquote><img src="wp-content/uploads/2013/03/table.png"/></blockquote>
It is important to understand that despite being referred to as alpha
blending, and despite alpha often being used to model opacity,
conceptually Porter/Duff is not a way to blend the source and
destination shapes. It is instead a way to overlay, combine and cut
them as if they were pieces of cardboard. The only places where source
and destination pixels are actually blended is where the antialiased
edges meet.

<b>Blending</b>
Photoshop and the Gimp have the concept of layers which are images stacked on top of each other. In Porter/Duff, stacking images on top of each other is done with the "Over" operator, which is also what Photoshop/Gimp use by default to composite layers:
<blockquote><img src="wp-content/uploads/2013/03/over-diagram.png"></blockquote>
Conceptually, two pieces of cardboard are held up with one in front of the other. Neither shape is trimmed, and in places where both are present, only the top layer is visible.

A layer in these programs also has an associated <em>Blend Mode</em> which can be used to modify what happens in places where both are visible. For example, the 'Color Dodge' blend mode computes a mix of source and destination according to this formula:<blockquote>
\(\begin{equation*}
B(s,d)= 
\begin{cases} 0 & \text{if \(d=0\),}
\\
1 & \text{if \(d \ge (1 - s)\),}
\\
d / (1 - s) & \text{otherwise}
\end{cases}
\end{equation*}\)
</blockquote>

The pixel diagram is this:
<blockquote><img src="wp-content/uploads/2013/03/colordodge-diagram.png"></blockquote>
And the result looks like this:
<blockquote><img src="wp-content/uploads/2013/03/colordodge.png"></blockquote>

Unlike with the regular Over operator, in this case there is substantial chunk of the output where the result is actually a mix of the source and destination.

Layers in Photoshop and Gimp are not tailored to each other (except for layer masks, which we will ignore here), so the compositing of the layer stack is done with the source-only and destination-only region set to source and destination respectively. However, there is nothing in principle stopping us from setting the source-only and destination-only regions to blank, but keep the blend mode in the 'both' region so that tailoring could be supported alongside blending. For example, we could set the 'source' region to blank, the 'destination' region to destination, and the 'both' region to ColorDodge:
<blockquote><img src="wp-content/uploads/2013/03/colordodge-dest-diagram.png"/></blockquote>
The result looks like this:
<blockquote><img src="wp-content/uploads/2013/03/colordodge-dest.png"></blockquote>
Here are all the possibilities that involve a ColorDodge blend mode:
<blockquote>
<img src="wp-content/uploads/2013/03/colordodge-none.png"/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<img src="wp-content/uploads/2013/03/colordodge-source.png"/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<img src="wp-content/uploads/2013/03/colordodge-dest.png"/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;<img src="wp-content/uploads/2013/03/colordodge-both.png"/></blockquote>
With three trivial blend modes:
<table>
<tr><td>Source:</td><td>\(B(s, d) = s\)</td></tr>
<tr><td>Dest:</td><td>\(B(s, d) = d\)</td></tr>
<tr><td>Zero:</td><td>\(B(s, d) = 0\)</td></tr>
</table>
we can generate all of the twelve original Porter/Duff operators.

Reinterpreting Porter/Duff in this way leads to a natural generalization where the blend mode can be chosen from a large set of formulas, where each such formula gives rise to four new compositing operators characterized by whether the source and destination are blank or contain the corresponding pixel color.

The general formula is still an area weighted average:
<blockquote>\(A_\text{src} \cdot [s] + A_\text{dest} \cdot [d] + A_\text{both}\cdot B(s, d)\)</blockquote>
where [s] and [d] are the source and destination colors respectively or 0, but where \(B(s, d)\) is no longer restricted to one of \(0\), \(s\), and \(d\), but can instead be chosen from a large set of formulas.

In this way, each new blend mode gives rise to four new compositing operators characterized by whether or the source and destination regions are present or blank.

The output of the alpha channel is the same as before:
<blockquote>\(A_\text{src} \cdot [as] + A_\text{dest} \cdot [ad] + A_\text{both} \cdot [\text{ab}]\)</blockquote>
except that [ab] is now determined by the blend mode. For the Zero blend mode there is no coverage in the both region, so [ab] is 0; for most others, there is full coverage, so [ab] is 1.

Here is a table of the operators that are generated by various blend modes:
<blockquote><img src="wp-content/uploads/2013/03/colordodge-table.png"></blockquote>
The three first rows are the original Porter/Duff operators.


<!--

Alpha Channels


The resulting value corresponds to the amount of light reflected by
the covered part of the pixel, so in order to get the right color
value, we have to divide by alpha. Since divisions are expensive and
since source and destination inputs have to be multiplied with their
corresponding alpha values anyway, pixels are often stored in a format
where all the color channels have been multiplied with the alpha
channel.

With this format, the formula becomes:

    (1 - ad) * [s] + (1 - as) * [d] + as * ad * [b]

where [s] and [d] are the premultiplied source and destination colors,
and [b] is either 0, or one of the premultiplied source or destination
colors divided by their corresponding alpha channel (this division is
then canceled by either the /as/ or the /ad/ in front of [b]).

The alpha channel is computed with the same formula. In this case [s]
is either 0 or as, [d] is either 0 or ad, and [b] is either 0, or
as/as = 1, or ad/ad = 1.

In other words, with premultiplied pixels, this formula:

   (1 - ad) * [s] + (1 - as) * [d] + as * ad * [b]

works for all four channels of a premultiplied pixel.



With the classic Porter/Duff operators the output alpha channel is
computed by plugging 1 into the area weighted average whenever a
region is filled with either source or destination, and 0 whenever a
region is blank. We can also view this as plugging in 1, 1 in the
blend mode formula:

      Source:	B(s, d) = s;	B(1, 1) = 1
      Dest:	B(s, d) = d;	B(1, 1) = 1
      Zero:	B(s, d) = 0;	B(1, 1) = 0

Most of the new blend mode formulas satisfy B(1, 1) = 1, which means
it is tempting to define the alpha computation to be exactly the same
as the color computation, just as it is for the Porter/Duff
operators. As long as B(1, 1) = 1, we get:

    ar = (1 - ad) * as + (1 - as) * ad + as * ad * B(as/as, ad/ad)
       = (1 - ad) * as + (1 - as) * ad + as * ad * B(1, 1)

A very nice result of this is that the ADD operator that simply adds
all four channels can be defined in this framework with the blend
mode:

	Plus:	B(s,d) = s + d

When plugging in 1, 1, you get:

     ar = (1 - ad) * as + (1 - as) * ad + as * ad * B(1, 1)
        = (as - ad * as + ad - as * ad + as * ad * (1 + 1)
	= as + ad

and similarly for the color channels:

     r = (1 - ad) * s + (1 - as) * d + as * ad * (s/as + d/ad)
       = s - s * ad + d - as * d + ad * s + as * d
       = s + d

Unfortunately, there are two annoying exceptions: The blend modes
Exclusion and Difference do not produce 1 when you plug in (s = 1, d =
1), they produce 0. There is some logic to that, because these blend
modes are concerned with the difference between source and
destination, and the difference in coverage in the 'both' region
(where source and destination are both present by definition) would in
fact be zero. Nonetheless, making the alpha channel 0 in the 'both'
region would mean that you couldn't see the blended result, so that's
not a useful option.

The right solution may be instead to define inverted versions:

    Inverted Exclusion:		B(s,d) = 1 - Exclusion(s,d)
				       = 1 - s - d + 2 * s * d

    Inverted Difference:	B(s,d) = 1 - Difference(s,d)
    	     			       = 1 - |s - d|

These will produce the correct alpha channel, and users can still
achieve the non-inverted blend modes by using the inverted blend modes
twice.

Finally, there are four "non-separable" blend modes that don't operate
on a per-channel basis. These instead conceptually convert the source
and destination into the HSL representation and then for each of the
H, S, L channels pick either the source or the destination
value. There is obviously no way to unify this operation into one
formula that applies to all four channels.


Conclusion

It is possible to generalize the Porter/Duff compositing algebra by
allowing more behaviors in the 'both' region of the Porter/Duff
pixel. If the effect on the alpha channel is defined to be B(1,1),
then all four channels can be computed with the same formula:

  (1 - ad) * [s] * S + (1 - as) * [d] * D + as * ad * B(S/as, D/ad)

where [s] and [d] are 1 or 0 depending on whether the source and
destination are drawn, and B is the blend mode. When B(s,d) is chosen
to be 0, s, or d, the formula reduces to the twelve original
Porter/Duff operators.

-->

<!--

CSS compositing spec

There is a proposal for extending CSS
[https://dvcs.w3.org/hg/FXTF/rawfile/tip/compositing/index.html] with
support for blending and compositing that also tries to unify
porter/duff compositing with blend modes. In this scheme a Porter/Duff
operator and a blend mode are chosen orthogonally.

For a given pixel, the source and the destination are blended
according to the chosen blendmode, and then a linear interpolation
between the source and the blended result is computed with the
destination alpha as the interpolation weight. This result is then
used as the source in compositing with the chosen Porter/Duff
operator. The full formula is this:

      Fa * as * [ (1 - ad) * Cs + ad * B(s,d) ] + Fb * ad * Cd

where Fa and Fb are weight factors corresponding to the chosen
Porter/Duff operator.

This formula has some useful properties: If the blend mode is chosen
to be Normal, that is, B(s,d) = Cs, then the formula becomes identical
to whatever Porter/Duff operator is involved. If the Porter/Duff
operator is chosen to be Over, then the formula becomes identical to
the Photoshop/Gimp layer blending. When the operator is Source, the
formula produces the atop-like result shown above.

However, there is no conceptual underlying reason that the formula has
these properties. You can't give a graphical interpretation of it the
way you can with Porter/Duff; the useful properties the formula has
might as well be numerological accidents. In fact, for many operators,
the formula produces nonsensical results.

For example, here is what it looks like when the operator is DEST_OVER
and the blend mode is ColorDodge:

    DEST_OVER / COLOR_DODGE

What is this? It's DEST_OVER, except that the antialiasing is messed
up with a weird purple shadow. There is no situation where this is the
operation you need.

There is also a bunch of operators where the blend mode is simply
ignored: DEST, DEST_IN, DEST_OUT, CLEAR.

The fundamental problem is that it doesn't make any sense to select a
Porter/Duff compositing operators and a blend mode orthogonally. The
Porter/Duff operator and the blend mode both specify what happens in
the region where source and destination overlaps, and if they disagree
on what happens there, it is not clear what happens. The answer to
this is not to pick some formula that happens to produce useful
results in certain special cases.

-->