Lip reading

Lip-reading, also known as lipreading or speechreading, is a technique of understanding speech by visually interpreting the movements of the lips, face and tongue when normal sound is not available. It relies also on information provided by the context, knowledge of the language, and any residual hearing. Lip-reading is not easy, as this clip demonstrates. Although ostensibly used by deaf and hard-of-hearing people, most people with normal hearing process some speech information from sight of the moving mouth.[1]

Process

Although speech perception is considered to be an auditory skill, it is intrinsically multimodal, since producing speech requires the speaker to make movements of the lips, teeth and tongue which are often visible in face-to-face communication. Information from the lips and face supports aural comprehension [2] and most fluent listeners of a language are sensitive to seen speech actions (see McGurk effect). The extent to which people make use of seen speech actions varies with the visibility of the speech action and the knowledge and skill of the perceiver.

Phonemes and Visemes

The phoneme is the smallest detectable unit of a (spoken) language that changes meaning. /pit/ and /pik/ differ by one phoneme and refer to different concepts. Spoken English has about 38 phonemes. For lipreading, the number of visually distinctive units - visemes - is much smaller, thus several phonemes map onto a few visemes. This is because many phonemes are produced within the mouth and throat, and cannot be seen. These include glottal consonants and most gestures of the tongue. Voiced and unvoiced pairs look identical, such as [p] and [b], [k] and [g], [t] and [d], [f] and [v], and [s] and [z]; likewise for nasalisation (e.g. [m] vs. [b]). Homophenes are words that look similar when lip-read, but which contain different phonemes. Because there are about three times as many phonemes as visemes in English, it is often claimed that only 30% of speech can be lipread. Homophenes are a crucial source of mis-lipreading.

The legend to this puzzle reads "Here is a class of a dozen boys, who, being called up to give their names were photographed by the instantaneous process just as each one was commencing to pronounce his own name. The twelve names were Oom, Alden, Eastman, Alfred, Arthur, Luke, Fletcher, Matthew, Theodore, Richard, Shirmer, and Hisswald. Now it would not seem possible to be able to give the correct name to each of the twelve boys, but if you practice the list over to each one, you will find it not a difficult task to locate the proper name for every one of the boys."[3]

Co-articulation

Visemes can be captured as still images, but speech unfolds in time. The smooth articulation of speech sounds in sequence can mean that mouth patterns may be ‘shaped’ by an adjacent phoneme: the ‘th’ sound in ‘tooth’ and in ‘teeth’ appears very different because of the vocalic context. This feature of dynamic speech-reading affects lip-reading 'beyond the viseme'[4]

How can lip-reading 'work' with so few visemes?

The statistical distribution of phonemes within the lexicon of a language is uneven. While there are clusters of words which are phonemically similar to each other ('lexical neighbors', such as spit/sip/sit/stick...etc.), others are unlike all other words: they are 'unique' in terms of the distribution of their phonemes ('umbrella' may be an example). Skilled users of the language bring this knowledge to bear when interpreting speech, so it is generally harder to identify a heard word with many lexical neighbors than one with few neighbors. Applying this insight to seen speech, some words in the language can be unambiguously lip-read even when they contain few visemes - simply because no other words could possibly 'fit'[5]

Variation in lipreadability and in lip-reading skill

Many factors affect the visibility of a speaking face, including illumination, movement of the head/camera, frame-rate of the moving image and distance from the viewer (see e.g.[6]). Head movement that accompanies normal speech can also improve lip-reading, independently of oral actions.[7] However, when lip-reading connected speech, the viewer's knowledge of the spoken language, familiarity with the speaker and style of speech, and the context of the lip-read material[8] are as important as the visibility of the speaker. While most hearing people are sensitive to seen speech, there is great variability in individual speechreading skill. Good lipreaders are often more accurate than poor lipreaders at identifying phonemes from visual speech.

A simple visemic measure of 'lipreadability' has been questioned by some researchers. The 'phoneme equivalence class' measure takes into account the statistical structure of the lexicon[9] and can also accommodate individual differences in lip-reading ability.[10][11] In line with this, excellent lipreading is often associated with more broad-based cognitive skills including general language proficiency, executive function and working memory.[12][13]

Lip-reading and language learning in hearing infants and children

The first few months

Seeing the mouth plays a role in the very young infant's early sensitivity to speech, and prepares them to become speakers at 1 – 2 years. In order to imitate, a baby must learn to shape their lips in accordance with the sounds they hear; seeing the speaker may help them to do this.[14] Newborns imitate adult mouth movements such as sticking out the tongue or opening the mouth, which could be a precursor to further imitation and later language learning.[15] Infants are disturbed when audiovisual speech of a familiar speaker is desynchronized [16] and tend to show different looking patterns for familiar than for unfamiliar faces when matched to (recorded) voices.[17] Infants are sensitive to McGurk illusions months before they have learned to speak.[18][19] These studies and many more point to a role for vision in the development of sensitivity to (auditory) speech in the first half-year of life.

The next six months; a role for lip-reading in learning a native language

Until around six months of age, most hearing infants are sensitive to a wide range of speech gestures - including ones that can be seen on the mouth - which may or may not later be part of the phonology of their native language. But in the second six months of life, the hearing infant shows perceptual narrowing for the phonetic structure of their own language - and may lose the early sensitivity to mouth patterns that are not useful. The speech sounds /v/ and /b/ which are visemically distinctive in English but not in Castilian Spanish are accurately distinguished in Spanish-exposed and English-exposed babies up to the age of around 6 months. However, older Spanish-exposed infants lose the ability to 'see' this distinction, while it is retained for English-exposed infants.[20] Such studies suggest that rather than hearing and vision developing in independent ways in infancy, multimodal processing is the rule, not the exception, in (language) development of the infant brain.[21]

Early language production: One to two years

Given the many studies indicating a role for vision in the development of language in the pre-lingual infant, ithe effects of congenital blindness on language development are surprisingly small. However, children blind from birth can confuse /m/ and /n/ in their own early production of English words – a confusion rarely seen in sighted hearing children, since /m/ and /n/ are visibly distinctive, but auditorilly confusable.[22] Generally, the role of vision in children aged 1–2 years may be less critical to the production of their native language, since, by that age, they have attained the skills they need to identify and imitate speech sounds. However, hearing a non-native language can shift the child's attention to visual and auditory engagement by way of lipreading and listening in order to process, understand and produce speech.[23]

Lip-reading in childhood

Studies with pre-lingual infants and children use indirect, non-verbal measures to indicate sensitivity to seen speech. Explicit lip-reading can be reliably tested in hearing preschoolers by asking them to 'say aloud what I say silently'.[24] In school-age children, lipreading of familiar closed-set words such as number words can be readily elicited.[25] Individual differences in lip-reading skill, as tested by asking the child to 'speak the word that you lip-read', or by matching a lip-read utterance to a picture,[26] show a relationship between lip-reading skill and age.[27][28]

Lip-reading in hearing adults: lifespan considerations

While lip-reading silent speech poses a challenge for most hearing people, adding sight of the speaker to heard speech improves speech processing under many conditions. The mechanisms for this, and the precise ways in which lip-reading helps, are topics of current research .[29] Seeing the speaker helps at all levels of speech processing from phonetic feature discrimination to interpretation of pragmatic utterances .[30] The positive effects of adding vision to heard speech are greater in noisy than quiet environments,[31] where by making speech perception easier, seeing the speaker can free up cognitive resources, enabling deeper processing of speech content.

As hearing becomes less reliable in old-age people may tend to rely more on lip-reading, and are encouraged to do so. However, greater reliance on lip-reading may not always make good the effects of age-related hearing loss. Cognitive decline in aging may be preceded by and/or associated with measurable hearing loss.[32][33] Thus lipreading may not always be able to fully compensate for the combined hearing and cognitive age-related decrements.

Lip-reading in specific (hearing) populations

A number of studies report anomalies of lipreading in populations with distinctive developmental disorders. Autism : People with autism may show reduced lipreading abilities and reduced reliance on vision in audiovisual speech perception. [34] [35] This may be associated with gaze-to-the-face anomalies in these people. [36] Williams syndrome: People with Williams syndrome show some deficits in speechreading which may be independent of their visuo-spatial difficulties. [37] Specific Language Impairment: Children with SLI are also reported to show reduced lipreading sensitivity, [38] as are people with dyslexia. [39]

Lip-reading and Deafness

"When you are deaf you live inside a well-corked glass bottle. You see the entrancing outside world, but it does not reach you. After learning to lip read, you are still inside the bottle, but the cork has come out and the outside world slowly but surely comes in to you." [40] Debate has raged for hundreds of years over the role of lip-reading ('oralism') compared with other communication methods (most recently, total communication) in the education of deaf people. The extent to which one or other approach is beneficial depends on a range of factors, including level of hearing loss of the deaf person, age of hearing loss, parental involvement and parental language(s). Then there is a question concerning the aims of the deaf person and her community and carers. Is the aim of education to enhance communication generally, to develop sign language as a first language, or to develop skills in the spoken language of the hearing community? Researchers now focus on which aspects of language and communication may be best delivered by what means and in which contexts, given the hearing status of the child and her family, and their educational plans.[41] Bimodal bilingualism (proficiency in both speech and sign language) is one dominant current approach in language education for the deaf child.[42]

Deaf people are often (although not always) better lip-readers than people with normal hearing.[43] Some deaf people practice as professional lipreaders, for instance in forensic lipreading. In deaf people who have a cochlear implant, pre-implant lip-reading skill can predict post-implant (auditory or audiovisual) speech processing.[44] For many deaf people, access to spoken communication can be helped when a spoken message is relayed via a trained, professional lip-speaker.[45][46]

Lipreading and literacy development Children born deaf typically show delayed development of literacy skills[47] which can reflect difficulties in acquiring elements of the spoken language.[48] In particular, reliable phoneme-grapheme mapping may be more difficult for deaf children, who need to be skilled speech-readers in order to master this necessary step in literacy acquisition. Lip-reading skill is associated with literacy abilities in deaf adults and children[49] [50] and training in lipreading may help to develop literacy skills.[51][52]

Cued Speech uses lipreading with accompanying hand shapes that disambiguate the visemic (consonant) lipshape. Cued speech is said to be easier for hearing parents to learn than a sign language, and studies, primarily from Belgium, show that a deaf child exposed to cued speech in infancy can make more efficient progress in learning a spoken anguage than from lipreading alone .[53] The use of cued speech in cochlear implantation for deafness is likely to be positive.[54] A similar approach, involving the use of handshapes accompanying seen speech, is Visual Phonics, which is used by some educators to support the learning of written and spoken language.

Teaching and training in lipreading

The aim of teaching and training in lipreading is to develop awareness of the nature of lipreading, and to practice ways of improving the ability to perceive speech 'by eye'.[55] Lipreading classes, often called lipreading and managing hearing loss classes, are mainly aimed at adults who have hearing loss. The highest proportion of adults with hearing loss have an age related, or noise related loss, and with both these the high frequency sounds are lost first. Since many of the consonants in speech are high frequency sounds, speech becomes distorted. Hearing aids help, but may not cure this. Lipreading classes have been shown to be of benefit in UK studies commissioned by the charity, Action on Hearing Loss in 2012.

Trainers recognise that lipreading is an inexact art. Students are taught to watch the lips, tongue and jaw movements, to follow the stress and rhythm of language, to use their residual hearing, with or without hearing aids, to watch expression and body language, and use their ability to put two and two together. They are taught the lipreaders alphabet, groups of sounds that look alike on the lips (visemes) like p,b,m, or f,v. The aim is to get the gist, so as to have the confidence to join in conversation, and avoid damaging social isolation that often accompanies hearing loss. Lipreading classes are recommended for anyone who struggles to hear in noise, and help to adjust to hearing loss. ATLA ,(the association for teaching lipreading to adults) is the professional association in the UK for qualified lipreading tutors.

Lipreading Tests

Most tests of lipreading were devised to measure individual differences in performing specific speech processing tasks, and to detect changes in performance following training. Lipreading tests have been used with relatively small groups in experimental settings, or as clinical indicators with individual patients and clients. That is, lipreading tests to date have limited validity as markers of lipreading skill in the general population.

Lipreading by machine

Automated lip-reading has been a topic of interest in computational engineering, as well as in science fiction movies science fiction movies. The computational engineer Steve Omohundro ,among others, pioneered its development. In facial animation, the aim is to generate realistic facial actions, especially mouth movements, that simulate human speech actions. Computer algorithms to deform or manipulate images of faces can be driven by heard or written language to achieve this. Systems may be based on detailed models derived from facial movements (motion capture); on anatomical modelling of actions of the jaw, mouth and tongue; or on mapping of known viseme- phoneme properties.[56][57] Facial animation has been used in speechreading training (demonstrating how different sounds 'look').[58] These systems are a subset of speech synthesis modelling which aim to deliver reliable 'text-to-(seen)-speech' outputs. A complementary aim - the reverse of making faces move in speech - is to develop computer algorithms that can deliver realistic interpretations of speech (i.e. a written transcript or audio record) from natural video data of a face in action: this is facial speech recognition. These models too can be sourced from a variety of data.[59] Automatic visual speech recognition from video has been quite successful in distinguishing different languages (from a corpus of spoken language data).[60] Demonstration models, using machine-learning algorithms, have had some success in lipreading speech elements, such as specific words, from video [61] and for identifying hard-to-lipread phonemes from visemically similar seen mouth actions .[62] In future, machine speechreading is likely to make use of advanced machine-learning algorithms and may require training on a large database of speakers (following the successful model for automatic speech recognition).

Uses for machine lipreading could include: automated lipreading of video-only records; automated lipreading of speakers with damaged vocal tracts, and speech processing in face-to-face video (i.e. from videophone data). Automated lipreading may help in processing noisy or unfamiliar speech.[63] Automated lipreading may contribute to biometric person identification, replacing password-based identification.[64][65]

Lipreading and the brain

Are special brain regions and brain circuits implicated in lipreading? If so, it may be possible to identify the neural bases for individual differences in lipreading skill and possibly improve lipreading. Following the discovery that auditory brain regions, including Heschl's gyrus, were activated by seen speech,[66] the neural circuitry for speechreading was shown to include supra-modal processing regions, especially superior temporal sulcus (all parts) as well as posterior inferior occipital-temporal regions including regions specialised for the processing of faces and biological motion.[67] In some but not all studies, activation of Broca's area is reported for speechreading,[68][69] suggesting that articulatory mechanisms can be activated in speechreading.[70] Studies of the time course of audiovisual speech processing showed that sight of speech can prime auditory processing regions in advance of the acoustic signal.[71][72] Better lipreading skill is associated with greater activation in (left) superior temporal sulcus and adjacent inferior temporal (visual) regions in hearing people.[73][74] In deaf people, the circuitry devoted to speechreading appears to be very similar to that in hearing people, with similar associations of (left) superior temporal activation and lipreading skill.[75]

References

  1. ↑ Woodhouse L, Hickson L, Dodd B. 2009 Review of visual speech perception by hearing and hearing-impaired people: clinical implications. International J. Language and Communication Disorders; 44(3):253-70. doi: 10.1080/13682820802090281. Review. PMID 1882111
  2. ↑ Erber NP.1969 Interaction of audition and vision in the recognition of oral speech stimuli. J. Speech Hear Res. 1969 Jun;12(2):423-5. PMID 5808871 [PubMed - indexed for MEDLINE]
  3. ↑ Sam Loyd's Cyclopedia of Puzzles, 1914
  4. ↑ Benguerel AP, Pichora-Fuller MK. 1982 Coarticulation effects in lipreading.J. Speech Hear Res. 1982 Dec;25(4):600-7.PMID 7162162
  5. ↑ Auer ET.2010 Investigating speechreading and deafness. J American Academy of Audiology.21(3):163-8. doi: 10.3766/jaaa.21.3.4.PMID 20211120
  6. ↑ Jordan TR, Thomas SM. 2011 When half a face is as good as a whole: effects of simple substantial occlusion on visual and audiovisual speech perception. Atten. Percept. Psychophys. 73(7):2270-85. doi: 10.3758/s13414-011-0152-4.PMID 21842332
  7. ↑ Thomas SM, Jordan TR.2004 Contributions of oral and extraoral facial movement to visual and audiovisual speech perception. J Exp Psychol Hum Percept Perform. Oct;30(5):873-88.PMID 15462626
  8. ↑ Spehar B, Goebel S, Tye-Murray N. 2015 Effects of Context Type on Lipreading and Listening Performance and Implications for Sentence Processing.J Speech Lang Hear Res. 58(3):1093-102. doi: 10.1044/2015_JSLHR-H-14-0360.PMID 25863923
  9. ↑ Files BT, Tjan BS, Jiang J, Bernstein LE 2015 Visual speech discrimination and identification of natural and synthetic consonant stimuli.Front Psychol. Jul 13;6:878. doi: 10.3389/fpsyg.2015.00878. eCollection 2015.PMID 26217249
  10. ↑ Auer ET Jr, Bernstein LE 1997 Speechreading and the structure of the lexicon: computationally modeling the effects of reduced phonetic distinctiveness on lexical uniqueness. .J Acoust Soc Am. Dec;102(6):3704-10. PMID 9407662
  11. ↑ Feld J1, Sommers M 2011 There Goes the Neighborhood: Lipreading and the Structure of the Mental Lexicon. Speech Commun. Feb;53(2):220-228
  12. ↑ Tye-Murray N, Hale S, Spehar B, Myerson J, Sommers MS. 2014 Lipreading in school-age children: the roles of age, hearing status, and cognitive ability.J Speech Lang Hear Res.Apr 1;57(2):556-65. doi:10.1044/2013_JSLHR-H-12-0273
  13. ↑ Feld JE, Sommers MS.2009 Lipreading, processing speed, and working memory in younger and older adults.J Speech Lang Hear Res.Dec;52(6):1555-65. doi:10.1044/1092-4388(2009/08-0137) Epub 2009 Aug 28. PMID 19717657
  14. ↑ http://www.huffingtonpost.com/2012/01/16/babies-learning-to-talk_n_1209219.html
  15. ↑ Meltzoff AN, Moore MK.1977 Imitation of facial and manual gestures by human neonates. Science. Oct 7;198(4312):74-8.PMID 897687
  16. ↑ Dodd B.1976 Lip reading in infants: attention to speech presented in- and out-of-synchrony.Cognitive Psychology. Oct;11(4):478-84
  17. ↑ Spelke,E. 1976 Infants intermodal perception of events. Cognitive Psychology,8,(4) 553-560
  18. ↑ Burnham D, Dodd B. 2004 Auditory-visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect. Developmental Psychobiology. 45(4):204-20. PMID 15549685
  19. ↑ Rosenblum LD, Schmuckler MA,Johnson JA.1997 The McGurk effect in infants.Perception and Psychophysics 59(3):347-57.PMID 9136265
  20. ↑ Pons F et al. 2009 Narrowing of intersensory speech perception in infancy.Proceedings National Academy of Sciences U S A;106(26):10598-602. doi: 10.1073/pnas.0904134106. PMID 19541648
  21. ↑ Lewkowicz DJ, Ghazanfar AA.2009 The emergence of multisensory systems through perceptual narrowing. Trends in Cognitive Science. Nov;13(11):470-8. doi: 10.1016/j.tics.2009.08.004.
  22. ↑ Mills,A.E. 1987 The development of phonology in the blind child. In B.Dodd & R.Campbell(Eds) Hearing by Eye: the psychology of lipreading, Hove UK, Lawrence Erlbaum Associates
  23. ↑ Infants deploy selective attention to the mouth of a talking face when learning speech. Lewkowicz DJ, Hansen-Tift AM. Proc. Natl Acad Sci U S A. 2012 Jan 31;109(5):1431-6. doi: 10.1073/pnas.1114783109. PMID 22307596
  24. ↑ Davies R1, Kidd E, Lander K. 2009 Investigating the psycholinguistic correlates of speechreading in preschool age children. International J. Language and Communication Disorders 44(2):164-74. doi:10.1080/13682820801997189.
  25. ↑ Dodd B. 1987 The acquisition of lipreading skills by normally hearing children. In B.Dodd & R.Campbell (Eds) Hearing by Eye, Erlbaum NJ pp163-176
  26. ↑ Jerger S et al. 2009,Developmental shifts in children's sensitivity to visual speech: a new multimodal picture-word task.J Exp Child Psychol.;102(1):40-59. doi: 10.1016/j.jecp.2008.08.002. PMID 18829049
  27. ↑ Kyle FE, Campbell R, Mohammed T, Coleman M, MacSweeney M. 2013 Speechreading development in deaf and hearing children: introducing the test of child speechreading. J Speech, Language and Hearing Research 56(2):416-26. doi: 10.1044/1092-4388(2012/12-0039). Epub 2012 Dec 28. PMID 23275416
  28. ↑ Tye-Murray N, Hale S, Spehar B, Myerson J, Sommers MS. 2014 Lipreading in school-age children: the roles of age, hearing status, and cognitive ability. J. Speech Lang Hear Res.57(2):556-65. doi: 10.1044/2013_JSLHR-H-12-0273. PMID 24129010
  29. ↑ Peelle JE, Sommers MS 2015 Prediction and constraint in audiovisual speech perception. Cortex 68:169-81. doi: 10.1016/j.cortex.2015.03.006.
  30. ↑ Campbell R 2008 The processing of audio-visual speech: empirical and neural bases.Philosophical Transactions of the Royal Society of London B Biological Science 363(1493): 1001–1010. Published online doi: 10.1098/rstb.2007.2155 PMCID: PMC2606792
  31. ↑ Sumby WH,Pollack I. 1954 Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, Vol 26, 212-215. http://dx.doi.org/10.1121/1.1907309
  32. ↑ Schmulian Taliaard et al. 2015 The relationship between hearing impairment and cognitive function: A meta-analysis in adults. Clin Otolaryngol. doi: 10.1111/coa.12607. [Epub ahead of print]
  33. ↑ Hung SC et al., 2015 Hearing Loss is Associated With Risk of Alzheimer's Disease: A Case-Control Study in Older People. J Epidemiol. 25(8):517-21. doi: 10.2188/jea.JE20140147.
  34. ↑ Smith EG, Bennetto L.J 2007 Audiovisual speech integration and lipreading in autism. Child Psychol Psychiatry. 48(8):813-21.PMID 17683453
  35. ↑ Irwin JR, Tornatore LA, Brancazio L, Whalen DH.2011Can children with autism spectrum disorders "hear" a speaking face? Child Dev. 82(5):1397-403. doi: 10.1111/j.1467-8624.2011.01619.x. PMID 21790542
  36. ↑ Irwin JR, Brancazio L 2014 Seeing to hear? Patterns of gaze to speaking faces in children with autism spectrum disorders. Front Psychol. 8;5:397. doi: 10.3389/fpsyg.2014.00397. PMID 24847297
  37. ↑ Böhning M, Campbell R, Karmiloff-Smith A. 2002 Audiovisual speech perception in Williams syndrome. Neuropsychologia.40(8):1396-406.PMID 11931944
  38. ↑ Leybaert J, Macchi L, Huyse A, Champoux F, Bayard C, Colin C, Berthommier F. 2014 Atypical audio-visual speech perception and McGurk effects in children with specific language impairment. Front Psychol. ;5:422. doi: 10.3389/fpsyg.2014.00422. eCollection 2014. PMID 24904454
  39. ↑ Mohammed T1, Campbell R, Macsweeney M, Barry F, Coleman M 2006 Speechreading and its association with reading among deaf, hearing and dyslexic individuals.Clinical Linguistics and Phonetics. 20(7-8):621-30.
  40. ↑ Clegg, Dorothy (1953), The Listening Eye: A Simple Introduction to the Art of Lip-reading, Methuen & Company
  41. ↑ http://www.handsandvoices.org/articles/research/v9-2_marschark.htm
  42. ↑ Swanwick, R. 2016 Deaf Children's bimodal bilingualism and education. Language Teaching / Volume 49 / Issue 01 / January 2016, pp 1-34 DOI: http://dx.doi.org/10.1017/S0261444815000348 (About DOI),
  43. ↑ .Bernstein LE, Demorest ME, Tucker PE.(2000) Speech perception without hearing. Perception & Psychophysics. 62(2):233-52
  44. ↑ Bergeson TR1, Pisoni DB, Davis RA.(2005)Development of audiovisual comprehension skills in prelingually deaf children with cochlear implants.Ear & Hearing 26(2):149-64.
  45. ↑ http://www.nidirect.gov.uk/communication-support-for-deaf-people
  46. ↑ http://www.lipspeaker.co.uk
  47. ↑ http://www.nuffieldfoundation.org/reading-and-dyslexia-deaf-children
  48. ↑ C.Mayer 2007 What really matters in the early literacy development of deaf children.J Deaf Stud Deaf Educ.;12(4):411-31. Epub 2007 Jun 12..
  49. ↑ http://www.tandfonline.com/doi/abs/10.1080/02699200500266745?journalCode=iclp20
  50. ↑ Kyle F. E., Harris M. (2010). Predictors of reading development in deaf children: a 3-year longitudinal study. J. Exp. Child Psychol. 107, 229–243 10.1016/j.jecp.2010.04.011
  51. ↑ https://www.ucl.ac.uk/cslir/projects/ongoing/starproject
  52. ↑ http://jslhr.pubs.asha.org/article.aspx?articleid=1795779
  53. ↑ Nicholls GH, Ling D 1982 Cued Speech and the reception of spoken language.J Speech Hear Res.25(2):262-9.
  54. ↑ Leybaert J, LaSasso CJ 2010 Cued speech for enhancing speech perception and first language development of children with cochlear implants.Trends in Amplification. 14(2):96-112. doi: 10.1177/1084713810375567..
  55. ↑ https://www.lipreading.org/lipreading-
  56. ↑ http://www-bcf.usc.edu/~rwalker/Walker/Publications_files/1996_CohenWalker%26Massaro_VisualSpeech.pdf
  57. ↑ http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A318099&dswid=2041
  58. ↑ http://link.springer.com/article/10.1023/B:JADD.0000006002.82367.4f#/page-1
  59. ↑ https://www.uea.ac.uk/computing/visual-speech-synthesis
  60. ↑ http://www.cnet.com/news/lip-reading-computer-can-distinguish-languages/
  61. ↑ https://www.youtube.com/watch?v=Tu2vInqqHX8
  62. ↑ http://www.theguardian.com/business/2016/apr/24/the-innovators-can-computers-be-taught-to-lip-read-artificial-intelligence
  63. ↑ https://www.researchgate.net/profile/David_Bodoff/publication/234819242_An_improved_automatic_lipreading_system_to_enhance_speech_recognition/links/5628b96008ae04c2aeaeb404.pdf
  64. ↑ http://www.asel.udel.edu/icslp/cdrom/vol1/954/a954.pdf
  65. ↑ http://www.planetbiometrics.com-article-details-i-2250
  66. ↑ Calvert GA, Bullmore ET, Brammer MJ et al 1997 Activation of auditory cortex during silent lipreading.Science. 276(5312):593-6.PMID 9110978
  67. ↑ Bernstein LE, Liebenthal E. 2014 Neural pathways for visual speech perception. Front Neurosci. ;8:386. doi: 10.3389/fnins.2014.00386. eCollection 2014. PMID 25520611
  68. ↑ Skipper JI, van Wassenhove V, Nusbaum HC, Small SL. Hearing Lips and Seeing Voices: How Cortical Areas Supporting Speech Production Mediate Audiovisual Speech Perception. Cerebral cortex (New York, NY : 1991). 2007;17(10):2387-2399. doi:10.1093/cercor/bhl147.
  69. ↑ Campbell R, MacSweeney M, Surguladze S, Calvert G, McGuire P, Suckling J, Brammer MJ, David AS. (2001) Brain substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning)Cogn Brain Res. 12(2):233-43. PMID 11587893
  70. ↑ Swaminathan, S., MacSweeney, M., Boyles, R., Waters, D., Watkins, K. E., & Möttönen, R. (2013). Motor excitability during visual perception of known and unknown spoken languages. Brain and Language, 126(1), 1–7. http://doi.org/10.1016/j.bandl.2013.03.002
  71. ↑ Sams M, Aulenko et al. 1991 Seeing Speech: visual information from lip movements modifies activity in the human auditory cortex Neuroscience letters, 127, 141-145
  72. ↑ van Wassenhove V, Grant KW, Poeppel D (2005) Visual speech speeds up the neural processing of auditory speech. Proc Natl Acad Sci U S A. 2005 Jan 25; 102(4):1181-6.
  73. ↑ Hall DA1, Fussell C, Summerfield AQ. 2005 Reading fluent speech from talking faces: typical brain networks and individual differences.J. Cogn Neurosci. 17(6):939-53.
  74. ↑ Bernstein LE, Jiang J, Pantazis D, Lu ZL, Joshi A.(2011) Visual phonetic processing localized using speech and nonspeech face gestures in video and point-light displays. Hum Brain Mapp. 32(10):1660-76. doi: 10.1002/hbm.21139. Epub 2010 Sep 17. PMID 20853377
  75. ↑ Capek CM, Macsweeney M, Woll B, Waters D, McGuire PK, David AS, Brammer MJ, Campbell R. 2008 Cortical circuits for silent speechreading in deaf and hearing people. Neuropsychologia. 46(5):1233-41. doi: 10.1016/j.neuropsychologia.2007.11.026. Epub 2007 Dec 5.PMID 18249420

Bibliography

External links

This article is issued from Wikipedia - version of the Tuesday, April 26, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.