A synaesthesia-like effect in which people 'hear' silent flashes or movement, such as in popular 'noisy GIFs' and memes, could be due to a reduction of inhibition of signals that travel between visual and auditory areas of the brain, according to a new study led by researchers at City, University of London.
The study is the first to provide insight into the brain mechanisms underpinning such auditory sensations also known as a 'visually-evoked auditory response' (aka vEAR or 'visual ear').
Whilst one theory is that areas of the brain responsible for visual and auditory processing normally compete, this research suggests that they may actually cooperate in people who report visual ear.
It was also found that musicians taking part in the study were significantly more likely to report experiencing visual ear than non-musician participants. This could be because musical training may promote joint attention to both the sound of music and the sight of the coordinated movements of the conductor or other musicians.
Dr Elliot Freeman, Principal Investigator on the study and a Senior Lecturer in Psychology at the University said:
"We already knew that some people hear what they see. Car indicator lights, flashing neon shop signs, and people's movements as they walk may all trigger an auditory sensation.
"Our latest study reveals normally-occurring individual differences in how our senses of vision and hearing interact.
"We found that people with 'visual ears' can use both senses together to see and also 'hear' silent motion, while for others hearing is inhibited when watching such visual sequences."
Some neuroscientists believe visual-ear may be a type of synaesthesia, with other examples including music, letters or numbers that can evoke perceptions of colour. However, visual ear appears to be the most prevalent, with as many as 20% of people reporting some experiences of it compared to 4.4 per cent for other types.
The condition has received more attention due to the recent, viral popularity of the 'skipping pylon GIF', and other 'noisy GIFs' depicting silent motion, which in some people evoke very vivid visual ear sensations.
To shed light on what may be going on in the brain when people view such content, the researchers applied a weak alternating current to participants' scalps, using a technique called transcranial Alternating Current Stimulation (tACS), to explore how the visual and auditory parts of the brain interact in those who experience visual ear and those who don't.
The first experiment of the study included 36 healthy participants, including 16 classical musicians from the London Royal College of Music. All were shown auditory and visual 'Morse code' sequences, while electrical simulation (tACS) was applied to either the back of the head (visual areas of the brain) or the sides (auditory areas) using 'alpha-frequency' tACS stimulation. Participants were then classified as visual or non-visual ear depending on whether they reported 'hearing' the silent flashes.
The researchers found that in non-visual ear participants, alpha-frequency stimulation to auditory areas significantly reduced auditory performance but improved visual performance, while the opposite pattern was found for the same frequency of stimulation to visual areas (poorer vision, better audition).
This reciprocal pattern suggests a competitive interaction between visual and auditory brain areas with each normally inhibiting the performance of the other.
However these interactions were strikingly absent in visual ear participants, suggesting that their auditory and visual areas were not competing but cooperating with each other.
A second experiment was conducted to see whether even people without conscious awareness of 'visual ear' sometimes use their auditory brain areas to make purely visual judgements. It found that this might indeed be the case for some, where stimulation to auditory areas of the brain affected accuracy of visual judgements almost as much as stimulating visual areas.
Taken together, the results of these experiments support a popular theory that some kinds of synaesthesia may depend on a disinhibition of pre-existing neural cross-connections between sensory brain areas that are normally inactive. When these connections are disinhibited this may result in conscious awareness of visual ear and other synaesthetic phenomena.
Dr Freeman said:
"We were also interested to find that, on average, participants with visual ear performed better on both visual and auditory tasks than those without. Perhaps their audio-visual cooperation benefits performance because more of the brain is engaged in processing visual stimuli.
"Such cooperation might also benefit musical performance, explaining why so many of the musicians we tested reported experiencing visual ear."
###
The study was published in the Journal of Cognitive Neuroscience.
Dr Freeman is continuing his research in this field with a new online questionnaire, which anyone can complete as a participant.
Notes to editors
To speak to Dr Elliot Freeman or for a copy of the study, please contact Shamim Quadir, Senior Communications Officer at City, University of London on shamim.quadir@city.ac.uk or 0207 040 8782. Watch the 'skipping pylon GIF': https://twitter.com/LisaDeBruine/status/937105553968566272 Watch the 'Bam Bam from the Flinstones' GIF: https://twitter.com/chrisfassnidge/status/939133452053557248 Read the peer-reviewed research:Christopher Fassnidge, Danny Ball, Zainab Kazaz, Synøve Knudsen, Anthony Tipple, Elliot Freeman (in press). Hearing through your eyes: neural basis of audiovisual cross-activation, revealed by transcranial alternating current stimulation. Journal of Cognitive Neuroscience Link to Dr Elliot Freeman's new questionnaire on visual ear: https://cityunilondon.eu.qualtrics.com/jfe/form/SV_1XrRAjC74ScbklT
About City, University of London
City, University of London is a global institution committed to academic excellence, with a focus on business and the professions and an enviable central London location.
It is the top higher education institution in London for student satisfaction (The Complete University Guide), is ranked 18th overall in the United Kingdom (Guardian University Guide) and is among the top five per cent of universities in the world (Times Higher Education World Rankings).
City has around 19,500 students (46% at postgraduate level) from more than 150 countries and staff from over 75 countries. More than 130,000 former students from over 180 countries are members of the City Alumni Network. City's academic range is broadly-based with world-leading strengths in business; law; health sciences; mathematics; computer science; engineering; social sciences; and the arts including journalism and music.
The University's history dates from 1894, with the foundation of the Northampton Institute on what is now the main part of City's campus. In 1966, City was granted University status by Royal Charter and the Lord Mayor of London became its Chancellor. In September 2016, City joined the University of London federation and HRH the Princess Royal became City's Chancellor. Professor Sir Paul Curran has been Vice-Chancellor and President of City since 2010.
Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.
