Gamma Correction, Dynamic Range Enhancement, and the Canvas

Overview

One good way to start a visual psychophysics experiment might be:

world = Shady.World( canvas=True, backgroundColor=0.5, gamma=2.2 )

This automatically creates a canvas, and configures its background color and gamma correction. “Noisy-bit” dithering, for dynamic range enhancement, is also turned on by default: the default value of ditheringDenominator, for the World and all Stimulus instances, will be automatically set to 255 for most graphics cards, or the appropriate larger 2 ** n - 1 value if your graphics card offers n>8 bits per DAC).

Now, when you create another stimulus (let’s say, a Gabor patch), you’ll most likely want it to have the same backgroundColor, gamma, ditheringDenominator, noiseAmplitude and look-up table as the World and its canvas. One way of ensuring that these properties always match the surrounding World is to use property sharing, and one powerful shortcut is to share the virtual property atmosphere which encompasses all of these linearization- and dynamic-range-related properties. So:

gabor = world.Stimulus(
    signalFunction = Shady.SIGFUNC.SinewaveSignal,
    signalAmplitude = 0.5,
    plateauProportion = 0,
    atmosphere = world,       # matched and linked until further notice
)

That’s equivalent to, but easier to type than:

gabor = world.Stimulus(
    signalFunction = Shady.SIGFUNC.SinewaveSignal,
    signalAmplitude = 0.5,
    plateauProportion = 0,

    backgroundColor = world,
    gamma = world,
    ditheringDenominator = world,
    noiseAmplitude = world,
    lut = world,
)

…and that in turn is equivalent to, but easier to type than:

gabor = world.Stimulus(
    signalFunction = Shady.SIGFUNC.SinewaveSignal,
    signalAmplitude = 0.5,
    plateauProportion = 0,
).LinkPropertiesWithMaster( world,
    'backgroundColor', 'gamma', 'ditheringDenominator', 'noiseAmplitude',
    'lookupTableTextureSize', 'lookupTableTextureSlotNumber', 'lookupTableTextureID',
)

In the following sections, we’ll unpack some of the issues covered above.

Gamma Correction

The mapping from pixel intensity values to physical luminance is not necessarily linear. Instead, it often appears to take the shape of an exponential “gamma” function. Display software must usually correct for this by applying the inverse gamma function to pixel intensities before displaying them: this cancels out the non-linearity and causes pixel luminance to be linear again.

You can apply gamma correction to each Shady Stimulus instance by setting its gamma property, either during construction or subsequently:

# ...
stim = world.Stimulus( gamma=2.2 )   # a scalar works, for setting
                                     # all three color channels the same

# ...
stim.gamma = (2.3, 2.2, 2.1)         # a sequence also works

The standard gamma correction that modern displays are supposed to adopt is the sRGB profile, which is a standard piecewise function that follows the gamma=2.2 exponential curve quite closely (although the exponent it uses in its exponential portion is actually slightly higher). You can tell Shady to use the sRGB correction instead of an exponential function by setting gamma to -1:

# ...
stim.gamma = 1.7    # (1.7, 1.7, 1.7)
stim.gamma = -1     # sRGB

Note that gamma correction will depend on the resolution of your monitor. Displays will only be configured to the sRGB profile at their native resolution, and there will likely be a profile in your monitor’s settings for sRGB specifically. For more about gamma correction, see the Wikipedia article.

Dynamic Range Enhancement

Monitors have a limited dynamic range, which determines how precisely they can present small variations in luminance: close to threshold contrast (not coincidentally) the monitor’s ability to display very small contrasts breaks down due to the discrete number of available pixel intensities.

The limitations on maximum contrast are innate to the display hardware, but there are tricks to work around the constraints on minimum contrast. Shady provides two such techniques: “noisy bit” dithering (which is the recommended approach, enabled by default) and additive noise. Both of these techniques apply spatiotemporal noise to the drawn pixel values of whichever Stimulus objects you apply them to (including the World’s canvas).

“Noisy-bit” dithering (Allard & Faubert 2008) applies a simple stochastic algorithm before converting floating-point RGB pixel intensity values to the discrete integer DAC values that are passed to the monitor hardware. Floating-point RGB values that map to non-integer DAC values are rounded with a weighted probability inversely proportional to their distance from the integer values. For example, every time Shady is instructed to draw a pixel with intensity (0.5, 0.5, 0.5), the desired DAC value on an 8-bit graphics card is 127.5, half-way between two integer values: with noisy-bit dithering on, each color channel will then have a 50% chance of being rounded down to DAC value 127 and a 50% chance of being rounded up to DAC value 128. Similarly, every time Shady is instructed to draw a pixel with intensity 0.25, the target value is 63.75, so the pixel will have a 25% chance of being rounded down to 63 and a 75% chance of being rounded up to 64. This probabilistic conversion is done independently for every color channel in every pixel in every frame, and the resulting noise causes the luminance values to perceptually average to the desired between-DAC value. Noisy-bit dithering is enabled by default. The only property needed to control it is ditheringDenominator, which will be automatically set to the highest DAC value your monitor can produce (usually 255). Other positive values will cause levels of rounding granularity that are not suited to your hardware, and should be avoided. You can negate the value, or set it to 0, to turn dithering off.

Additive noise follows a similar principle to “noisy bit” dithering, but simply adds random noise to the floating-point value of each pixel before it is linearized, looked up in a look-up table, or converted to a discrete DAC value. The resulting noise should again cause the luminance values to perceptually average to the desired luminance and/or color. You can control the strength of this noise by setting the noiseAmplitude property (or its alias, noise). Use negative values for uniform noise, or positive values for Gaussian noise. Noise is computed once for all color channels of the same pixel, but may be scaled separately per channel, so you can set noiseAmplitude to an RGB triplet if you want to tint the noise (or a single value to set all three channels’ noise amplitude the same). Additive noise is useful if you want to perform “bit stealing” (Tyler 1997), which can be accomplished using a look-up table: the bit-stealing technique introduces small-amplitude step changes in chroma which can sometimes become perceptible if their spatial extent is large: noise can effectively break these areas up.

The differences between the two properties are summarized in the table below:

is added before gamma correction
(or look-up table lookup);
is applied after gamma correction;

may be scaled differently in
different color channels;
has the same amplitude on average
in all color channels;
creates noise that is otherwise
perfectly correlated across color
channels;
creates independent noise in each
color channel;

is useful in combination with a
bit-stealing look-up table, or
when you actually want visible
noise;
is recommended for most purposes
(including when visible noise is
used) but is disabled automatically
when a look-up table is in use;
can be scaled arbitrarily, and may
be uniform (when property value is
negative) or Gaussian (when
positive).


only dithers between two nearest
DAC values; the correct property
value (which will be found auto-
matically) is 2 ** bits - 1 where
bits is the bit depth of your
graphics card (usually 8).

The following demos may provide further insight:

  • examples/dynamic-range.py allows you to visualize and interactively explore various dynamic-range-enhancement options.
  • examples/dithering.py performs a numerical sanity-check of our noisy-bit dithering implementation.
  • examples/precision.py performs a quantitative analysis of the effective precision achieved by noisy-bit dithering.
  • examples/noise.py allows you to examine the distribution of random values created by the additive noise effect.
  • examples/fancy-hardware.py illustrates Shady’s support for rendering on specialized vision-science hardware, such as the Bits# or ViewPixx, that can achieve high dynamic range without dithering (see also the bitCombiningMode property of the Shady.World class).

Look-up Table

Instead of using the gamma property to perform automatic gamma-correction, and allowing the ditheringDenominator to perform automatic noisy-bit dithering, you can disable both of these features and take control of linearization and dynamic-range enhancement issues directly yourself, by specifying a look-up table (LUT).

A look-up table is a discrete series of entries corresponding to a discrete (usually large, like 65536) number of ideal-luminance ranges that equally divide up the complete range from 0 to 1. Each entry is a triplet of integers, corresponding to the red, green and blue DACs (for most graphics cards, these will be 8-bit integers).

This is useful only for stimuli whose intensity is one-dimensional (e.g. monochromatic stimuli). In fact, Shady only uses the first color channel (red) to compute indices into the LUT. The output of the LUT will be RGB or RGBA, however. This means that using a LUT is a form of “indexed color” image rendering.

Here is a trivially small example of a 2-bit LUT (i.e. 4 entries) for an 8-bit graphics card (i.e. DAC values go up to 255):

stim.lut = [
  [   0,   0,   0 ],   # ideal luminances 0    - 0.25 map to black
  [ 255,   0,   0 ],   #                  0.25 - 0.5  map to red
  [ 255, 255,   0 ],   #                  0.5  - 0.75 map to yellow
  [ 255, 255, 255 ],   #                  0.75 - 1.0  map to white
]

To attach a LUT to a Stimulus, the easiest way is to call the SetLUT() method or, equivalently, assign to the lut property. You can assign either a Shady.LookupTable instance, or a valid argument to the Shady.LookupTable class constructor (in which case, such an instance will be constructed automatically). This means that in practice you can assign:

  • an existing Shady.LookupTable instance
  • an n-by-3 (or m-by-n-by-3) array of integers (or a nested list that numpy can automatically convert into such an array, as in the example above)
  • a filename of a npy, npz or png file in which you have previously saved a LUT array with Shady.Linearization.SaveLUT()

When you then query the lut property, you will see that its value is a Shady.LookupTable instance. Note that creation of such an instance allocates a texture in OpenGL, so the most efficient use of resources would be to re-use Shady.LookupTable instances wherever appropriate.

Remember that assigning a look-up table disables automatic gamma-correction and noisy-bit dithering. Assigning stim.lut = None or calling stim.SetLUT(None) removes the look-up table and re-enables automatic gamma-correction and noisy-bit dithering.

It is up to you to specify appropriate values for the LUT entries, although Shady does provides a utility for computing them according to one particular strategy: Shady.Linearization.BitStealingLUT() which implements a version of the “bit-stealing” technique (after Tyler 1997).

Bit-stealing allows monochromatic stimuli to be rendered at higher effective dynamic range, by allowing very small chromatic variations: these create luminance levels between the existing strictly-gray levels, while hopefully keeping the chromatic information itself well below the subject’s threshold. The latter point can fail in some circumstances where there is a very gradual change as a function of distance (such as at the outer edges of a Hann window): then it is sometimes possible to see a small step-change in color between large adjacent areas. To break up this effect, it is sometimes useful to add a little noise to the signal (as in the dithering approach, the effect of this noise will be perceptually averaged away over small spatial and temporal scales). We’ve found noiseAmplitude=1e-4 works well.

The examples/dynamic-range.py demo has look-up-table and additive-noise options, and illustrates some of these points.

Canvas

If you create a World:

world = Shady.World()

it starts off filled with a uniform color. You can specify this color in in the constructor call, or manipulate it after construction, via the clearColor attribute:

world.clearColor = [ 1, 0.3, 0.5 ]

Yeesh. Now, this may be sufficient for some purposes. But clearColor is a very simple property that does not change according to your linearization or dynamic-range-enhancement parameters: it is never gamma-corrected, and is always applied completely uniformly, so there can be no dithering.

However, if you’re doing vision science, you’ll probably want both gamma-correction and dynamic-range enhancement in your stimuli. And if you have those things in your stimuli, you’ll probably need them in the backdrop as well—for example, you may need to eliminate the risk that a keen-eyed subject can detect the edge of your stimulus bounding-box because of a just-visible artifact at the boundary between dithered and un-dithered gray regions.

The solution is to create a “canvas”, which is simply a rectangular Stimulus that fills your World. This can be done during World construction:

world = Shady.World( canvas=True )

…or after the fact:

world = Shady.World()
world.MakeCanvas()

Either way, what you get is a Stimulus object with no foreground color, the name 'canvas', and a z value of +1 (i.e. as far as possible away from the camera). In addition, various properties of the canvas are linked to those of the World itself. So if you specify or change any of the following properties of world:

you will actually be affecting the corresponding properties of world.stimuli['canvas']. Indeed, these properties of the World are only placeholders and are ignored during rendering of the empty World itself at the start of each frame. Changes in their values only cause visible effects to the extent that they change Stimulus instances, such as the canvas, that are linked in this way.

Avoiding image artifacts

For all stimuli:

Be aware that you may introduce artifacts due to your graphics card’s linear interpolation between pixel values, whenever you use:

For similar reasons, you should always run your display screen at its native resolution.

For textured stimuli:

Transformations of the carrier signal (via carrierTranslation, carrierRotation and carrierScaling) will also lead to interpolation artifacts as above, if the carrier content comes from a texture, i.e. it is defined by a discrete array of pixels.

For untextured (functionally-generated) stimuli:

If, on the other hand, the carrier content is entirely functionally generated on the GPU functions using the signalFunction, modulationFunction and windowingFunction properties, then you do not need to worry about interpolation artifacts from carrier transformations, because the carrier transformations are applied to the coordinate system before the functions are even evaluated.

You should also check whether a carrier transformation pushes your signal beyond any spatial or spatio-temporal aliasing limits. For example, if you have created an antialiased square-wave signal function as in the examples/custom-functions.py demo, you may think that the function automatically avoids components with fewer than 2 pixels per cycle. But if you then shrink it with a carrierScaling factor < 1.0, you may be back in trouble.

For moving stimuli:

Remember that speed (pixels per second or degrees per second) multiplied by spatial frequency (cycles per pixel or cycles per degree) gives you the flicker frequency of a pixel in Hz (cycles per second). If that is greater than half your screen’s refresh rate (i.e. > 30Hz, for most commercial screens) then you’re into spatio-temporal aliasing territory (that parallel universe where helicopter blades slow down to a standstill and car wheels spin backwards).