I was first made aware of the concept of visual vibrato by Chris Crawford, and it let me to explore some concepts in Using Environmental Noise. Below are some excerpts from the article I read:
Chris Crawford, www.erasmatazz.com, email@example.com
Gaming and Graphics Column, “Artists Against Anatomists”
Pg 8, Vol 36, No 1, Feb 2002, Computer Graphics, ACM Siggraph
[the article starts by talking about how computer generated images used in games and entertainment are often] excellent representations of optical reality, but they just don’t cut the mustard in terms of perceptual reality.
A simple example of the power of perceptual reality can be found in a phenomenon I refer to as "visual vibrato”. In music, vibrato is the deliberate oscillation of a performed note around the specified frequency. A violinist can produce the effect by rapidly quivering the finger on the string as the note is played. The effect is pleasing because it triggers within the human auditory system some discriminators that are keyed to changes in frequency rather than frequency itself, thereby heightening the intensity of the auditory experience.
Much the same thing happens in the human visual system. It’s a difficult phenomenon to notice, because its direct effects are later subtracted out of the perceived visual experience by higher-level processing. The human eye is never at rest; it is constantly darting about, sampling different sectors of the visual field. Any given object is sampled dozens of times per second, with each sampling yielding slightly different results. If we could somehow translate that visual experience at the retinal level into a movie, the object under inspection would have a shimmering quality. Higher up in the visual processing system, that shimmering is filtered out, but much of the resolution and depth that we perceive in an object is derived from this "visual vibrato".
I first encountered this effect 20 years ago while working on a primitive graphical display on the Atari 800 home computer. I wanted to show the face of a character, but my digitized source image had more resolution and pixel depth than my display. Rather than settling for a conventional bit-blit, I tried a dynamic scheme in which each output pixel’s value was determined by a semi-randomly weighted sum of the associated pixels in the source image. In effect, I used time depth as a substitute for pixel depth. I had serendipitously triggered the human visual system’s “visual vibrato,” and the result struck viewers powerfully. The image had a much greater subjective power than any objective calculation would have awarded it.
[a passage here that was less relevant has been left out]
A particularly revealing example of the difference between optical reality and perceptual reality is the depiction of the human face. I am flabbergasted by the brilliant wrongheadedness of the graphics researchers who have captured in algorithmic form the detailed anatomy of the human face, including all the tiny muscles, the effects of skin tone, even pore structure. It is truly impressive work, but it strikes me as a rather like a bridge built with titanium rivets and pig iron girders.
The human visual system boasts a great deal of neural circuitry for facial recognition. It’s hardwired in; infants a few days old can recognise faces. Those recognition algorithms utilize filtering algorithms that ruthlessly discard some visual information and amplify certain other facial features. Eyebrows, eyes and mouth are heavily processed, while noses, ears, cheeks and chins get short shrift. Ideally the graphics algorithms we use should mirror the internal perceptual algorithms in the human visual system.