Action recognition of point-light displays presented with semantically (in)congruent auditory stimuli: behavioral correlates

Abstract

Humans are experts in identifying and understanding other's movements. The visual information is often enough for a very accurate action identification. However, actions usually have an associated sound and thus the integration between visual and auditory modalities can benefit perception. Considering the improved identification driven by the integration of multiple sensory information, researchers began to investigate how such inputs are integrated in a unified percept. Following such line of investigation, we aimed to evaluate if the presence of congruent/incongruent action sounds would improve/impair recognition of the visual Point-Light Display (PLD) of human actions and if non-biological PLDs accompanied by action sounds would bias participants into a false perception of visual human action. Therefore, participants were presented with several masked human and scrambled PLD videos accompanied by sounds that were either biological or a white noise and asked to judge if the video depicted a human figure or not and in the affirmative case, they should name the action. Results showed a significant enhancement for audiovisual biological congruent when compared to the visual biological paired to auditory noise and unimodal visual stimuli. Similarly, a significantly better performance on action recognition occurred for the audiovisual biological congruent condition when compared to the unimodal visual stimuli. Lastly, considering the scramble stimuli we found a significant bias towards the identification of a human figure for the visual stimuli paired to auditory noise when compared to the unimodal visual scramble condition.These findings suggest that adding coincident sounds to a human action visual display impacts human figure identification and action perception in biological PLDs and that insignificant sounds might lead to confound perception of non-biological PLDs.