Sensory cue

A sensory cue is a statistic or signal that can be extracted from the sensory input by a perceiver, that indicates the state of some property of the world that the perceiver is interested in perceiving.

A cue is some organization of the data present in the signal which allows for meaningful extrapolation. For example, Sensory cues include Visual cues, auditory cues, haptic cues, olfactory cues, environmental cues, and so on. Sensory cues are a fundamental part of theories of perception, especially theories of appearance (how things look).

Concept

There are two primary theory sets used to describe the roles of sensory cues in perception. One set of theories are based on the Constructivist theory of perception, while the others are based on the Ecological theory.

Basing his views on the Constructivist theory of perception, Helmholtz (1821-1894) held that the visual system constructs visual percepts through a process of unconscious inference, in which cues are used to make probabilistic inferences about the state of the world. These inferences are based on prior experience, assuming that the most commonly correct interpretation of a cue will continue to hold true.^[1] A visual percept is the final manifestation of this process. Brunswik (1903-1955) later went on to formalize these concepts with the lens model, which breaks the system's use of a cue into two parts: the ecological validity of the cue, which is its likelihood of correlating with a property of the world, and the system's utilization of the cue.^[2] In these theories, accurate perception requires both the existence of cues with sufficiently high ecological validity to make inference possible, and that the system actually utilizes these cues in an appropriate fashion during the construction of percepts.

A second set of theories was posited by Gibson (1904-1979), based on the Ecological theory of perception. These theories held that no inferences are necessary to accomplish accurate perception. Rather, the visual system is able to take in sufficient cues related to objects and their surroundings. This means that a one:one mapping between the incoming cues and the environment they represent can be made. These mappings will be shaped by certain computational constraints; traits known to be common in an organism’s environment.^[3] The ultimate result is the same: a visual precept is manifested by the process.

Cue combination is an active area of research in perception, that seeks to understand how information from multiple sources is combined by the brain to create a single perceptual experience or response. Recent cue recruitment experiments have shown that the adult human visual system can learn to utilize new cues through classical (Pavlovian) conditioning.

Visual Cues

Visual cues are sensory cues received by the eye in the form of light and processed by the visual system during visual perception. Since the visual system is dominant in many species, especially humans, visual cues are a large source of information in how the world is perceived.^[4]

Types of cues

Depth

The ability to perceive the world in three dimensions and estimate the size and distance to an object depends heavily on depth cues. The two major depth cues, Stereopsis and motion parallax, both rely on parallax which is the difference between the perceived position of an object given two different viewpoints. In stereopsis the distance between the eyes is the source of the two different viewpoints, resulting in a Binocular disparity. Motion parallax relies head and body movement to produce the necessary viewpoints.^[5]

Motion

The visual system can detect motion both using a simple mechanism based on information from multiple clusters of neurons as well as by aggregate through by integrating multiple cues including contrast, form, and texture. One major source of visual information when determining self-motion is optic flow. Optic flow not only indicates whether an agent is moving but in which direction and at what relative speed.

Biological Motion

Humans in particular have evolved a particularly keen ability to detect if motion is being generated by biological source, even with point light displays where dots representing the joints of an animal.^[6] Recent research suggests that this mechanism can also reveal the gender, emotional state, and action of a given human light point model.^[7]

Color

The ability to distinguish between colors allows an organism quickly and easily recognize danger since many brightly colored plants and animals pose some kind of threat, usually harboring some kind of toxin. Color also serves as an inferential cue that can prime both motor action^[8] and interpretation of a persuasive message.^[9]

Contrast

Contrast, or the difference in luminance and/or color that helps make an object distinguishable, is important in edge detection and serves as a cue.

Auditory Cues

An auditory cue is a sound signal that represents an incoming sign received through the ears, causing the brain to hear. The results of receiving and processing these cues are collectively known as the sense of hearing, and are the subject of research within the fields of psychology, cognitive science, and neurobiology.

Auditory System

The auditory system of humans and animals allows individuals to assimilate information from the surroundings, represented as sound waves. Sound waves first pass through the pinnae and the auditory canal, the parts of the ear that comprise the outer ear. Sound then reaches the tympanic membrane in the middle ear (also known as the eardrum). The tympanic membrane sets the malleus, incus, and stapes into vibration. The stapes transmits these vibrations to the inner ear by pushing on the membrane covering the oval window, which separates the middle and inner ear. The inner ear contains the cochlea, the liquid-filled structure containing the hair cells. These cells serve to transform the incoming vibration to electrical signals, which can then be transmitted to the brain. The auditory nerve carries the signal generated by the hair cells away from the inner ear and towards the auditory receiving area in the cortex. The signal then travels through fibers to several subcortical structures and then to the primary auditory receiving area in the temporal lobe

Cues for Sound Locating

Humans use several cues to determine the location of a given stimuli, mainly by using the timing difference between ears. These cues allow individuals to identify both the elevation, the height of the stimuli relative to the individual, and azimuth, the angle of the sound relative to the direction the individual is facing.

Interaural Time and Level Difference

Unless a sound is directly in front of or behind the individual, the sound stimuli will have a slightly different distance to travel to reach each ear. This difference in distance causes a slight delay in the time the signal is perceived by each ear. The magnitude of the interaural time difference is greater the more the signal comes from the side of the head. Thus, this time delay allows humans to accurately predict the location of incoming sound cues. Interaural level difference is caused by the difference in sound pressure level reaching the two ears. This is because the head blocks the sound waves for the further ear, causing less intense sound to reach it. This level difference between the two ears allows humans to accurately predict azimuth of an auditory signal. This effect only occurs at sounds that are high frequency.

Spectral Cue

A spectral cue is a monoaural (single ear) cue for locating incoming sounds based on the distribution of the incoming signal. The differences in distribution (or spectrum) of the sound waves are caused by sounds interaction with the head and the outer ear before entering the ear canal.

Principles of Auditory Cue Grouping

The auditory system uses several heuristics to make sense of incoming cues, based on the properties of auditory stimuli that usually occur in the environment. Cue grouping refers to how humans naturally perceive incoming stimuli as organized patterns, based on certain rules.

Onset Time

If two sounds start at different times, they are likely to have originated from different sources. Sounds that originate simultaneously likely come from the same source.

Location

Cues originating at the same, or slowly changing, positions usually have the same source. When two sounds are separated in space, the cue of location (see: sound localization) helps an individual to separate them perceptually. If a sound is moving, it will move continuously. Erratically jumping sound is unlikely to come from the same source.

Similarity of Timbre

Timbre is the tone quality or tone character of a sound, independent of pitch. This helps us distinguish between musical instruments playing the same notes. When hearing multiple sounds, the timbre of each sound will be unchanging (regardless of pitch), and thus we can differentiate between sounds from different sources over time.^[10]

Similarity of Pitch

Pitch refers to the frequency of the sound wave reaching us. Although a single object could produce a variety of pitches overtime, it more likely that it would produce sounds in a similar range. Erratic changes in pitches are more likely to be perceived as originating from different sources.

Auditory Continuity

Similar to the Gestalt principle of good continuation (see: principles of grouping), sound that changes smoothly or remain constant are often produced by the same source. Sound with the same frequency, even when interrupted by other noise, is perceived as continuous. Highly variable sound that is interrupted is perceived as separate.^[11]

Factors Effecting Auditory Cue Perception

The Precedence Effect

When one sound is presented for a long interval before the introduction of a second one originating from a different location, individuals will hear them as two distinct sounds, each originating from the correct location. However, when the delay between the onset of the first and second sound is shortened, listeners are unable to distinguish between the two sounds. Instead, they perceive them as both coming from the location of the lead sound. This effect counteracts the small disparity between the perception of sound caused by the difference in distance between each ear and the source of the auditory stimuli.

The Interaction between Auditory and Visual Cues

There are strong interactions between visual and auditory stimuli. Since both auditory and visual cues provide an accurate source of information about the location of an object, most times there will be minimal discrepancy between the two. However, it is possible to have a disparity in the information provided by the two sets of cues. Visual capture, also known as the ventriloquism effect, occurs when an individual’s visual system locates the source of an auditory stimulus at a different position than where the auditory system locates it. When this occurs, the visual cues will override the auditory ones. The individual will perceive the sound as coming from the location where the object is seen. Audition can also affect visual perception. Research has demonstrated this effect by showing two objects on a screen, one moving diagonally from top-right to bottom-left and the other from top-left to bottom-right, intersecting in the middle. The paths of these identical objects could have been interpreted as crossing over each other, or as bouncing off each other. Without any auditory cue, a vast majority of subjects saw the objects crossing paths and continuing in their original trajectory. But with the addition of a small “click” sound, a majority of subjects perceived the objects as bouncing off each other. In this case, auditory cues help interpret visual cues.^[12]

Haptic Cues

A haptic cue is either a tactile sensation that represents an incoming signal received by the somatic system, or a relationship between tactile sensations which can be used to infer a higher level of information.^[13] The results of receiving and processing these cues are collectively known as the sense of touch, and are the subject of research in the fields of psychology, cognitive science, and neurobiology.

The word “haptic” can refer explicitly to active exploration of an environment (particularly in experimental psychology and physiology), but is often used to refer to the whole of the somesthetic experience.^[14]

Somatosensory System

The somatosensory system assimilates many kinds of information from the environment: temperature, texture, pressure, proprioception, and pain. The signals vary for each of these perceptions, and the receptor systems reflect this: Thermoreceptors, mechanoreceptors, nociceptors, and chemoreceptors.

Haptic Cues in Research

The Interaction between Haptic and Visual Cues

In addition to the interplay of haptic communication and nonverbal communication, haptic cues as primers have been looked at as a means of decreasing reaction time for identifying a visual stimulus.^[15] Subjects were placed in a chair fitted with a back which provided haptic cues indicating where the stimulus would appear on a screen. Valid haptic cues significantly decreased reaction time while invalid cues increased reaction time.^[15]

Use in Technology for the Visually Impaired

Haptic cues are used frequently to allow those who have impaired vision to have access to a greater wealth of information. Braille is a tactile written language which is read via touch, brushing the fingers over the raised patterns. Braille technology is the attempt to extend Braille to digital media and developing new tools to aid in the reading of web pages and other electronic devices often involves a combination of haptic and auditory cues.^[16]

A major issue that different technologies in this area attempt to overcome is sensory overload. The amount of information that can be quickly related via touch is less than that of vision and is limited by current technology. As a result, multi-modal approaches, converting the visual information into both haptic and auditory output, often have the best results. For example, an electronic pen can be drawn across a tablet mapped to the screen and produce different vibrations and sounds depending on what is at that location.^[16]

Olfactory Cues

An olfactory cue is a chemical signal received by the olfactory system that represents an incoming signal received through the nose. This allows humans and animals to smell the chemical signal given off by a physical object. Olfactory cues are extremely important for sexual reproduction, as they trigger mating behavior in many species, as well as maternal bonding and survival techniques such as detecting spoiled food. The results of receiving and processing this information is known as the sense of smell.

Olfactory System

The process of smelling begins when chemical molecules enter the nose and reach the olfactory mucosa, a dime-sized region located in the nasal cavity that contains olfactory receptor neurons. There are 350 types of olfactory receptors, each sensitive to a narrow range of odorants. These neurons send signals to the glomeruli within the olfactory bulb. Each glomerulus collects information from a specific olfactory receptor neuron. The olfactory signal is then conducted to piriform cortex and the amygdala, and then to the orbitalfrontal cortex, where higher level processing of the odor occurs.

Olfactory Memory

Olfactory memory is the recollection of a given smell. Research has found that odor memory is highly persistent and has a high resistance to interference, meaning these memories remain within an individual for long times despite possible interference of other olfactory memories. These memories are mostly explicit, though implicit forms of odor memory do provide some understanding of memory. Mammalian olfactory cues play an important role in the coordination of the mother infant bond, and the following normal development of the offspring. Olfactory memory is especially important for maternal behavior. Studies have shown that the fetus becomes familiar with olfactory cues in the within the uterus. This is demonstrated by research that suggests that newborns respond positively to the smell of their own amniotic fluid, meaning that fetuses learn from these cues in the womb.^[17]

Environmental Cues

Environmental cues are all of the sensory cues that exist in the environment.

With directed attention an environmental cue becomes an attended cue.^[13] However, most environmental cues are assimilated subconsciously, as in visual contextual cuing.

Environmental cues serve as the primary context that shapes how the world is perceived and as such they can prime prior experience to influence memory recall^[18] and decision making.^[19] This has applied use in marketing as there is evidence to suggest a store's atmosphere and layout can influence purchasing behavior.^[20]

Environmental cues play a direct role in mediating the behavior of both plants^[21] and animals. For example, environmental cues, such as temperature change or food availability affect the spawning behavior of fish. In addition to cues generated by the environment itself, cues generated by other agents, such as ant pheromone trails, can influence behavior to indirectly coordinate actions between those agents.

In the study of perception environmental cues play a large role in experimental design since these mechanisms evolved within a natural environment^[22] which gives rise to scene statistics and the desire to create a natural scene. If the experimental environment is too artificial it can damage external validity in an ideal observer experiment that makes use of natural scene statistics.

See also

• Environmental Context Dependent Memory
• Contextual Cuing
• Stimulus (psychology)

References

1. Rogers, edited by William Epstein, Sheena (1995). Perception of space and motion. San Diego: Academic Press. pp. 3–5. ISBN 0080538614.
2. Rogers, edited by William Epstein, Sheena (1995). Perception of space and motion. San Diego: Academic Press. pp. 5–7. ISBN 0080538614.
3. Rogers, edited by William Epstein, Sheena (1995). Perception of space and motion. San Diego: Academic Press. pp. 7–9. ISBN 0080538614.
4. Posner, Michael I.; Nissen, Mary J.; Klein, Raymond M. (March 1976). "Visual dominance: An information-processing account of its origins and significance.". Psychological Review 83 (2): 157–171. doi:10.1037/0033-295X.83.2.157.
5. Steinman, Scott B.; Garzia, Ralph Philip (2000). Foundations of Binocular Vision: A Clinical perspective. McGraw-Hill Professional. pp. 2–5. ISBN 0-8385-2670-5.
6. G. Johansson (1973). "Visual perception of biological motion and a model for its analysis". Percept. Psychophys. 14 (2): 201–211. doi:10.3758/BF03212378.
7. Alaerts, Kaat; Nackaerts, Evelien; Meyns, Pieter; Swinnen, Stephan P.; Wenderoth, Nicole; Valdes-Sosa, Mitchell (June 9, 2011). "Action and Emotion Recognition from Point Light Displays: An Investigation of Gender Differences". PLoS ONE 6 (6): e20989. doi:10.1371/journal.pone.0020989.
8. Schmidt, T.: The finger in flight: Real-time motor control by visually masked color stimuli. In: Psychological Science, Nr. 13, 2002, S. 112-118.
9. Gerend, Mary A.; Sias, Tricia (July 2009). "Message framing and color priming: How subtle threat cues affect persuasion". Journal of Experimental Social Psychology 45 (4): 999–1002. doi:10.1016/j.jesp.2009.04.002.
10. Bregman, Albert (1971). "Primary Auditory Stream Segregation and Perception of Order in Rapid Sequences of Tones". Journal of Experimental Psychology 89 (2): 244–249. doi:10.1037/h0031163. PMID 5567132.
11. Warren, R. M.; Obusek, C. J.; Ackroff, J. M. (9 June 1972). "Auditory Induction: Perceptual Synthesis of Absent Sounds". Science 176 (4039): 1149–1151. doi:10.1126/science.176.4039.1149.
12. Sekuler, Robert; Sekuler, Allison B.; Lau, Renee. "Sound alters visual motion perception". Nature 385 (6614): 308–308. doi:10.1038/385308a0.
13. Goldstein, Bruce E. (2007). Sensation and Perception. Cengage Learning. pp. 5–6. ISBN 978-0-495-60149-4.
14. Robles-De-La-Torre, G. (1 July 2006). "The Importance of the Sense of Touch in Virtual and Real Environments". IEEE Multimedia 13 (3): 24–30. doi:10.1109/MMUL.2006.69.
15. "Validity of Haptic Cues and Its Effect on Priming Visual Spatial Attention" (PDF). 2003. doi:10.1109/HAPTIC.2003.1191265.
16. Jay, Caroline; Stevens, Robert; Hubbold, Roger; Glencross, Mashhuda (1 May 2008). "Using haptic cues to aid nonvisual structure recognition". ACM Transactions on Applied Perception 5 (2): 1–14. doi:10.1145/1279920.1279922.
17. Varendi, H; Porter, RH; Winberg, J (1 September 1997). "Natural odour preferences of newborn infants change over time". Acta Paediatrica 86 (9): 985–990. doi:10.1111/j.1651-2227.1997.tb15184.x.
18. Godden, D; Baddeley, A. (1975). "Context dependent memory in two natural environments". British Journal of Psychology 66 (3): 325–331. doi:10.1111/j.2044-8295.1975.tb01468.x.
19. Elder, Ryan S.; Krishna, Aradhna (2010). "The Effects of Advertising Copy on Sensory Thoughts and Perceived Taste". Journal of Consumer Research 36 (5): 748–56. doi:10.1086/605327.
20. Baker, Julie; Parasuraman, A.; Grewal, Dhruv; Voss, Glenn B. (1 April 2002). "The Influence of Multiple Store Environment Cues on Perceived Merchandise Value and Patronage Intentions". Journal of Marketing 66 (2): 120–141. doi:10.1509/jmkg.66.2.120.18470.
21. Berger, Jonah A.; Heath, Chip (March–April 2010). "Idea Habitats: How the Prevalence of Environmental Cues Influences the Success of Ideas". Cognitive Science 29 (2): 195–221. doi:10.1207/s15516709cog0000_10.
22. Geisler, W. S., & Diehl, R. L. (2003). A Bayesian approach to the evolution of perceptual and cognitive systems. Cognitive Science, 27, 379-402.