(For more resources related to this topic, see here.)

When looking at a scene, we may pick up subtle cues from the way colors shift between different image regions. For example, outdoors on a clear day, shadows have a slightly blue tint due to the ambient light reflected from the blue sky, while highlights have a slightly yellow tint because they are in direct sunlight. When we see bluish shadows and yellowish highlights in a photograph, we may get a "warm and sunny" feeling. This effect may be natural, or it may be exaggerated by a filter.

Curve filters are useful for this type of manipulation. A curve filter is parameterized by sets of control points. For example, there might be one set of control points for each color channel. Each control point is a pair of numbers representing the input and output values for the given channel. For example, the pair (128, 180) means that a value of 128 in the given color channel is brightened to become a value of 180. Values between the control points are interpolated along a curve (hence the name, curve filter). In Gimp, a curve with the control points (0, 0), (128, 180), and (255, 255) is visualized as shown in the following screenshot:

making-subtle-color-shifts-curves-img-0

The x axis shows the input values ranging from 0 to 255, while the y axis shows the output values over the same range. Besides showing the curve, the graph shows the line y = x (no change) for comparison.

Curvilinear interpolation helps to ensure that color transitions are smooth, not abrupt. Thus, a curve filter makes it relatively easy to create subtle, natural-looking effects. We may define an RGB curve filter in pseudocode as follows:

dst.b = funcB(src.b) where funcB interpolates pointsB
dst.g = funcG(src.g) where funcG interpolates pointsG
dst.r = funcR(src.r) where funcR interpolates pointsR

For now, we will work with RGB and RGBA curve filters, and with channel values that range from 0 to 255. If we want such a curve filter to produce natural-looking results, we should use the following rules of thumb:

Every set of control points should include (0, 0) and (255, 255). This way, black remains black, white remains white, and the image does not appear to have an overall tint.
As the input value increases, the output value should always increase too. (Their relationship should be monotonically increasing.) This way, shadows remain shadows, highlights remain highlights, and the image does not appear to have inconsistent lighting or contrast.

OpenCV does not provide curvilinear interpolation functions but the Apache Commons Math library does. (See Adding files to the project, earlier in this chapter, for instructions on setting up Apache Commons Math.) This library provides interfaces called UnivariateInterpolator and UnivariateFunction, which have implementations including LinearInterpolator, SplineInterpolator, LinearFunction, and PolynomialSplineFunction. (Splines are a type of curve.) UnivariateInterpolator has an instance method, interpolate(double[] xval, double[] yval), which takes arrays of input and output values for the control points and returns a UnivariateFunction object. The UnivariateFunction object can provide interpolated values via the method value(double x).

API documentation for Apache Commons Math is available at http://commons.apache.org/proper/commons-math/apidocs/.

These interpolation functions are computationally expensive. We do not want to run them again and again for every channel of every pixel and every frame. Fortunately, we do not have to. There are only 256 possible input values per channel, so it is practical to precompute all possible output values and store them in a lookup table. For OpenCV's purposes, a lookup table is a Mat object whose indices represent input values and whose elements represent output values. The lookup can be performed using the static method Core.LUT(Mat src, Mat lut, Mat dst). In pseudocode, dst = lut[src]. The number of elements in lut should match the range of values in src, and the number of channels in lut should match the number of channels in src.

Now, using Apache Commons Math and OpenCV, let's implement a curve filter for RGBA images with channel values ranging from 0 to 255. Open CurveFilter.java and write the following code:

public class CurveFilter implements Filter {
  // The lookup table.
  private final Mat mLUT = new MatOfInt();
  public CurveFilter(
    final double[] vValIn, final double[] vValOut,
    final double[] rValIn, final double[] rValOut,
    final double[] gValIn, final double[] gValOut,
    final double[] bValIn, final double[] bValOut) {
    // Create the interpolation functions.
    UnivariateFunction vFunc = newFunc(vValIn, vValOut);
    UnivariateFunction rFunc = newFunc(rValIn, rValOut);
    UnivariateFunction gFunc = newFunc(gValIn, gValOut);
    UnivariateFunction bFunc = newFunc(bValIn, bValOut);
    // Create and populate the lookup table.
    mLUT.create(256, 1, CvType.CV_8UC4);
    for (int i = 0; i < 256; i++) {
      final double v = vFunc.value(i);
      final double r = rFunc.value(v);
      final double g = gFunc.value(v);
      final double b = bFunc.value(v);
      mLUT.put(i, 0, r, g, b, i); // alpha is unchanged
    }
  }
  @Override
  public void apply(final Mat src, final Mat dst) {
    // Apply the lookup table.
    Core.LUT(src, mLUT, dst);
  }
  private UnivariateFunction newFunc(final double[] valIn,
    final double[] valOut) {
    UnivariateInterpolator interpolator;
    if (valIn.length > 2) {
      interpolator = new SplineInterpolator();
    } else {
      interpolator = new LinearInterpolator();
    }
    return interpolator.interpolate(valIn, valOut);
  }
}

    
        Unlock access to the largest independent learning library in Tech for FREE!
        
            
                Get unlimited access to 7500+ expert-authored eBooks and video courses covering every tech area you can think of.
                Renews at €18.99/month. Cancel anytime

CurveFilter stores the lookup table in a member variable. The constructor method populates the lookup table based on the four sets of control points that are taken as arguments. As well as a set of control points for each of the RGB channels, the constructor also takes a set of control points for the image's overall brightness, just for convenience. A helper method, newFunc, creates an appropriate interpolation function (linear or spline) for each set of control points. Then, we iterate over the possible input values and populate the lookup table.

The apply method is a one-liner. It simply uses the precomputed lookup table with the given source and destination matrices.

CurveFilter can be subclassed to define a filter with a specific set of control points. For example, let's open PortraCurveFilter.java and write the following code:

public class PortraCurveFilter extends CurveFilter {
  public PortraCurveFilter() {
    super(
      new double[] { 0, 23, 157, 255 }, // vValIn
      new double[] { 0, 20, 173, 255 }, // vValOut
      new double[] { 0, 69, 213, 255 }, // rValIn
      new double[] { 0, 69, 218, 255 }, // rValOut
      new double[] { 0, 52, 189, 255 }, // gValIn
      new double[] { 0, 47, 196, 255 }, // gValOut
      new double[] { 0, 41, 231, 255 }, // bValIn
      new double[] { 0, 46, 228, 255 }); // bValOut
  }
}

This filter brightens the image, makes shadows cooler (more blue), and makes highlights warmer (more yellow). It produces flattering skin tones and tends to make things look sunnier and cleaner. It resembles the color characteristics of a brand of photo film called Kodak Portra, which was often used for portraits.

The code for our other three channel mixing filters is similar. The ProviaCurveFilter class uses the following arguments for its control points:

    new double[] { 0, 255 }, // vValIn
    new double[] { 0, 255 }, // vValOut
    new double[] { 0, 59, 202, 255 }, // rValIn
    new double[] { 0, 54, 210, 255 }, // rValOut
    new double[] { 0, 27, 196, 255 }, // gValIn
    new double[] { 0, 21, 207, 255 }, // gValOut
    new double[] { 0, 35, 205, 255 },  // bValIn
    new double[] { 0, 25, 227, 255 }); // bValOut

The effect is a strong, blue or greenish-blue tint in shadows and a strong, yellow or greenish-yellow tint in highlights. It resembles a film processing technique called cross-processing, which was sometimes used to produce grungy-looking photos of fashion models, pop stars, and so on.

For a good discussion of how to emulate various brands of photo film, see Petteri Sulonen's blog at http://www.prime-junta.net/pont/How_to/100_Curves_and_Films/_Curves_and_films.html. The control points that we use are based on examples given in this article.

Curve filters are a convenient tool for manipulating color and contrast, but they are limited insofar as each destination pixel is affected by only a single input pixel. Next, we will examine a more flexible family of filters, which enable each destination pixel to be affected by a neighborhood of input pixels.