ArticlePublisher preview available

Identifying Signatures of Perceived Interpersonal Synchrony

Authors:
Eric Novotny at Haworth
Eric Novotny
  • Haworth
Gary Bente at University of Cologne
Gary Bente
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

Interpersonal synchrony serves as a subtle, yet powerful bonding mechanism in social interactions. Problematically, the term ‘synchrony’ has been used to label a variety of distinct aspects of interpersonal coordination, such as postural similarities or movement activity entrainment. Accordingly, different algorithms have been suggested to quantify interpersonal synchrony. Yet, it remains unknown whether the different measures of synchrony represent correlated features of the same perceivable core phenomenon. The current study addresses this by comparing the suitability of a set of algorithms with respect to their association with observers’ judgments of dyadic synchrony and leader-followership. One-hundred fifteen observers viewed computer animations of characters portraying the movements of real dyads who performed a repetitive motor task with instruction to move in unison. Animations were based on full-body motion capture data synchronously collected for both partners during the joint exercise. Results showed most synchrony measures significantly correlated with (a) perceived synchrony and (b) the perceived level of balance of leading/following by each dyad member. Phase synchrony and Pearson correlations were associated most strongly with the observer ratings. This might be typical for intentional, structured forms synchrony such as ritualized group activities. It remains open if these findings also apply to spontaneous forms of synchrony as, for instance, occurring in free-running conversations.
Figures - available from: Journal of Nonverbal Behavior
This content is subject to copyright. Terms and conditions apply.
A preview of this full-text is provided by Springer Nature.
Learn more
Content available from Journal of Nonverbal Behavior
This content is subject to copyright. Terms and conditions apply.
Vol.:(0123456789)
Journal of Nonverbal Behavior (2022) 46:485–517
https://doi.org/10.1007/s10919-022-00410-9
1 3
ORIGINAL PAPER
Identifying Signatures ofPerceived Interpersonal Synchrony
EricNovotny1 · GaryBente2
Accepted: 17 May 2022 / Published online: 30 July 2022
© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022
Abstract
Interpersonal synchrony serves as a subtle, yet powerful bonding mechanism in social
interactions. Problematically, the term ‘synchrony’ has been used to label a variety of dis-
tinct aspects of interpersonal coordination, such as postural similarities or movement activ-
ity entrainment. Accordingly, different algorithms have been suggested to quantify inter-
personal synchrony. Yet, it remains unknown whether the different measures of synchrony
represent correlated features of the same perceivable core phenomenon. The current study
addresses this by comparing the suitability of a set of algorithms with respect to their asso-
ciation with observers’ judgments of dyadic synchrony and leader-followership. One-hun-
dred fifteen observers viewed computer animations of characters portraying the movements
of real dyads who performed a repetitive motor task with instruction to move in unison.
Animations were based on full-body motion capture data synchronously collected for both
partners during the joint exercise. Results showed most synchrony measures significantly
correlated with (a) perceived synchrony and (b) the perceived level of balance of leading/
following by each dyad member. Phase synchrony and Pearson correlations were associ-
ated most strongly with the observer ratings. This might be typical for intentional, struc-
tured forms synchrony such as ritualized group activities. It remains open if these findings
also apply to spontaneous forms of synchrony as, for instance, occurring in free-running
conversations.
Keywords Synchrony· Nonverbal· Perception· Motion capture
Introduction
Behavioral coordination, and more specifically ‘interpersonal synchrony,’ is a common
means of affiliation and bonding among humans (Hove & Risen, 2009; Launay et al.,
2016). Interpersonal synchrony refers to the coordination of body movement rhythms
between individuals in an interaction (Bente & Novotny, 2020; Bernieri, 1988). It can
occur spontaneously in social interactions, promoting rapport (Bernieri, 1988), trust (Lau-
nay etal., 2013), and cooperation (Wiltermuth & Heath, 2009) between the interaction
* Eric Novotny
eric.novotny@uga.edu
1 Grady College ofMass Communication andJournalism, University ofGeorgia, Athens, USA
2 Department ofCommunication, Michigan State University, EastLansing, USA
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
... Burgoon et al. (2015) illustrated using this software to discover patterns of deception in group interaction. Several other methods for analyzing non-verbal dynamics can be found in Novotny and Bente (2022). ...
The future of automated capture of social kinesic signals for psychiatric purposes
Article
Full-text available
  • Aug 2023
This article considers how computer vision can be enlisted for biomedical applications, specifically the measurement, data analytics and treatment of psychiatric disorders. Often, youngsters are too afraid or embarrassed to disclose their emotional and mental problems to human therapists. An AI system can be utilized not only to collect data in a non-threatening ongoing manner and record patient's temporal psychophysiological state but also to analyze and output the periodic results, it may be an efficient and effective means for therapists to plan treatments. We report on various tools for analyzing social kinesic signals for emotional and physiological states. Only one, AVATAR (and its predecessor SPECIES), both records a patient's state and also outputs an analysis that flags problem areas for therapists. In this way, automated tools can augment human observation and judgment.
View
... The development of MEA and the analysis methods described here (see Novotny & Bente, 2022 for other means of analyzing movement data) have yielded exciting insights into nonverbal behavior and synchrony across both clinical and nonclinical contexts. In nonclinical settings, MEA has primarily been used to examine NVS in unacquainted dyads. ...
Friends in Sync? Examining the Relationship Between the Degree of Nonverbal Synchrony, Friendship Satisfaction and Support
Article
Full-text available
  • Jun 2023
  • J NONVERBAL BEHAV
Nonverbal synchrony (NVS), the degree of spontaneous coordination of movements among dyads, has been associated with important social outcomes among romantic and stranger dyads, including the degree of social affiliation. Recently, automated methods, such as Motion Energy Analysis (MEA), have been used to objectively measure NVS. In this study, we examined MEA-quantified NVS among 143 friend dyads and its association with friendship satisfaction, closeness, and support. Friend dyads engaged in two conversations about a problem one friend was experiencing and took turns generating problems. Half the dyads were randomized to a co-rumination condition, where they were given instructions that prompted co-rumination, or a natural condition, where they were prompted to speak about the problem as they naturally would. Friendship satisfaction was measured at baseline while friendship support was measured at baseline and following each conversation using self-report scales. NVS was significantly present above chance during the task, but for each conversation, levels of NVS were not (a) predicted by the degree of friendship satisfaction or support, or (b) predictive of the degree of post-conversation friendship support. Furthermore, exploratory analyses revealed that for individuals who rated their dyad partner as a friend versus a close friend, greater synchrony trended toward predicting lower friendship support during the second conversation. Overall, this study demonstrates that an automated assessment of movement was able to detect NVS among friend dyads during a problem-focused discussion, but raises questions about the role that NVS plays among friends in this context.
View
... Also, a practical reason why the distinction between in-phase and antiphase synchrony has been overlooked is that other major synchrony measures that are supposed to assess temporal coordination, such as cross-recurrence quantification analysis (CRQA; Coco and Dale, 2014) and dynamic time warping (Fujiwara et al., 2022a,b), do not account for phase difference. Indeed, several measures are introduced to the research field (Novotny and Bente, 2022), and different measures have little convergent validity (Altmann et al., 2022), which makes it difficult to see the essence of the problem. Even when cross-correlation was computed, some recent studies have shed light on non-absolute values (Nyman-Salonen et al., 2021;Tourunen et al., 2022), which is promising to be an advantage in understanding antiphase synchrony. ...
Antiphase synchrony increases perceived entitativity and uniqueness: A joint hand-clapping task
Article
Full-text available
  • Mar 2023
In- and antiphase are the dominant patterns identified in the study of synchrony in relative phases. Many previous studies have focused on in-phase synchrony and compared it to asynchrony, but antiphase synchrony has yet not been the subject of much research attention. The limited findings on antiphase synchrony suggest that its role or nature is unclear or unstable in human interaction. To account for this factor, this study examined the possibility that antiphase synchrony simultaneously induced perceived entitativity and uniqueness. The results of an experiment employing a joint hand-clapping task supported this prediction. Further, the elevated feeling of uniqueness in those who experienced antiphase synchrony may have increased the self-other overlap for those who felt oneness with their partner, but it decreased overlap for those who did not. The theoretical implications for synchrony literature are discussed.
View
... Promising approaches to analyze the temporal entrainment of two behavioral vectors in the frequency domain have also been introduced recently using Cross-Wavelet-Transform (Fujiwara and Daibo, 2016). It is important to note that the different algorithms are not just different ways to capture the same phenomenon, but distinctly define what is conceived as coordination or synchrony (Novotny, 2020). Therefore, further studies are needed to comparatively evaluate the different approaches with regard to their effectiveness in predicting the subjective experience or perceptions of rapport in varying contexts (cf. ...
View
... Promising approaches to analyze the temporal entrainment of two behavioral vectors in the frequency domain have also been introduced recently using Cross-Wavelet-Transform (Fujiwara and Daibo, 2016). It is important to note that the different algorithms are not just different ways to capture the same phenomenon, but distinctly define what is conceived as coordination or synchrony (Novotny, 2020). Therefore, further studies are needed to comparatively evaluate the different approaches with regard to their effectiveness in predicting the subjective experience or perceptions of rapport in varying contexts (cf. ...
Beyond Stereotypes: Analyzing Gender and Cultural Differences in Nonverbal Rapport
Article
Full-text available
  • Dec 2020
The current paper addresses two methodological problems pertinent to the analysis of observer studies in nonverbal rapport and beyond. These problems concern: (1) the production of standardized stimulus materials that allow for unbiased observer ratings and (2) the objective measurement of nonverbal behaviors to identify the dyadic patterns underlying the observer impressions. We suggest motion capture and character animation as possible solutions to these problems and exemplarily apply the novel methodology to the study of gender and cultural differences in nonverbal rapport. We compared a Western, individualistic culture with an egalitarian gender-role conception (Germany) and a collectivistic culture with a more traditional gender role conceptions (Middle East, Gulf States). Motion capture data were collected for five male and five female dyadic interactions in each culture. Character animations based on the motion capture data served as stimuli in the observation study. Female and male observers from both cultures rated the perceived rapport continuously while watching the 1 min sequences and guessed gender and cultural background of the dyads after each clip. Results show that masking of gender and culture in the stimuli was successful, as hit rates for both aspects remained at chance level. Further the results revealed high levels of agreement in the rapport ratings across gender and culture, pointing to universal judgment policies. A 2 × 2 × 2 × 2 ANOVA for gender and culture of stimuli and observers showed that female dyads were rated significantly higher on rapport across the board and that the contrast between female and male dyads was more pronounced in the Arab sample as compared to the German sample. nonverbal parameters extracted from the motion capture protocols were submitted to a series of algorithms to identify dyadic activity levels and coordination patterns relevant to the perception of rapport. The results are critically discussed with regard to the role of nonverbal coordination as a constituent of rapport.
View
How and Why People Synchronize: An Integrated Perspective
Article
Full-text available
  • May 2024
  • PERS SOC PSYCHOL REV
Interpersonal synchrony, the alignment of behavior and/or physiology during interactions, is a pervasive phenomenon observed in diverse social contexts. Here we synthesize across contexts and behaviors to classify the different forms and functions of synchrony. We provide a concise framework for classifying the manifold forms of synchrony along six dimensions: periodicity, discreteness, spatial similarity, directionality, leader–follower dynamics, and observability. We also distill the various proposed functions of interpersonal synchrony into four interconnected functions: reducing complexity and improving understanding, accomplishing joint tasks, strengthening social connection, and influencing partners’ behavior. These functions derive from first principles, emerge from each other, and are accomplished by some forms of synchrony more than others. Effective synchrony flexibly adapts to social goals and more synchrony is not always better. Our synthesis offers a shared framework and language for the field, allowing for better cross-context and cross-behavior comparisons, generating new hypotheses, and highlighting future research directions.
View
Classifying interpersonal synchronization states using a data-driven approach: implications for social interaction understanding
Article
Full-text available
  • Jul 2023
This study presents a data-driven approach to identifying interpersonal motor synchrony states by analyzing hand movements captured from a 3D depth camera. Utilizing a single frame from the experiment, an XGBoost machine learning model was employed to differentiate between spontaneous and intentional synchrony modes with nearly 90%\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$90\%$$\end{document} accuracy. Our findings demonstrate a consistent pattern across subjects, revealing that movement velocity tends to be slower in synchrony modes. These insights support the notion that the relationship between velocity and synchrony is influenced by the cognitive load required for the task, with slower movements leading to higher synchrony in tasks demanding higher cognitive load. This work not only contributes to the limited literature on algorithms for identifying interpersonal synchrony but also has potential implications for developing new metrics to assess real-time human social interactions, understanding social interaction, and diagnosing and developing treatment strategies for social deficits associated with conditions such as Autism Spectrum Disorder.
View
Comparing Manual and Automated Coding Methods of Nonverbal Synchrony
Article
Full-text available
  • Dec 2020
The manual coding of nonverbal cues by trained human coders, such as counting the occurrence of a specific behavior or subjective rating about a speaker, is still a major method in the study of nonverbal synchrony. However, its time-consuming nature has been a serious barrier to the development of the field and has made it difficult for new scholars to adopt the technique. Recent advances in automated coding techniques allow researchers to collect nonverbal behavioral data effectively and objectively, but it is unclear how comparable the manual and automated coding methods are. This study, therefore, directly compared both methods of coding in a face-to-face conversation experiment. In the manual coding, a software system C-BAS was employed to count the occurrence of synchrony in gesture, posture, nodding, and other cues. In the automated coding, a cross-correlation analysis and a cross-wavelet coherence analysis were separately performed for the participant’s movement data. The results showed that the manual and automated coding were moderately correlated, and yielded similar significant differences between experimental conditions of conversational involvement in the degree of synchrony. Further, synchrony measured via both coding techniques was significantly associated with post-conversation self-reports. The advantages and disadvantages of both techniques are discussed.
View
Beyond Stereotypes: Analyzing Gender and Cultural Differences in Nonverbal Rapport
Article
Full-text available
  • Dec 2020
The current paper addresses two methodological problems pertinent to the analysis of observer studies in nonverbal rapport and beyond. These problems concern: (1) the production of standardized stimulus materials that allow for unbiased observer ratings and (2) the objective measurement of nonverbal behaviors to identify the dyadic patterns underlying the observer impressions. We suggest motion capture and character animation as possible solutions to these problems and exemplarily apply the novel methodology to the study of gender and cultural differences in nonverbal rapport. We compared a Western, individualistic culture with an egalitarian gender-role conception (Germany) and a collectivistic culture with a more traditional gender role conceptions (Middle East, Gulf States). Motion capture data were collected for five male and five female dyadic interactions in each culture. Character animations based on the motion capture data served as stimuli in the observation study. Female and male observers from both cultures rated the perceived rapport continuously while watching the 1 min sequences and guessed gender and cultural background of the dyads after each clip. Results show that masking of gender and culture in the stimuli was successful, as hit rates for both aspects remained at chance level. Further the results revealed high levels of agreement in the rapport ratings across gender and culture, pointing to universal judgment policies. A 2 × 2 × 2 × 2 ANOVA for gender and culture of stimuli and observers showed that female dyads were rated significantly higher on rapport across the board and that the contrast between female and male dyads was more pronounced in the Arab sample as compared to the German sample. nonverbal parameters extracted from the motion capture protocols were submitted to a series of algorithms to identify dyadic activity levels and coordination patterns relevant to the perception of rapport. The results are critically discussed with regard to the role of nonverbal coordination as a constituent of rapport.
View
Gesture Networks: Introducing Dynamic Time Warping and Network Analysis for the Kinematic Study of Gesture Ensembles
Article
Full-text available
  • Oct 2019
We introduce applications of established methods in time series-and network analysis that we jointly apply here for the kinematic study of gesture ensembles. We define a gesture ensemble as the set of gestures that are produced during discourse by a single person or a group of persons. Here we are interested in how gestures kinematically relate to one another. We employ a bivariate time-series analysis called Dynamic Time Warping (DTW) to assess how similar each gesture is to the other gestures in the ensemble in terms of their velocity profiles (as well as studying multivariate cases with gesture velocity and speech amplitude envelope profiles). By relating each gesture event to all other gesture events produced in the ensemble, we obtain a weighted matrix that essentially represents a network of similarity relationships. We can therefore apply network analysis, which can gauge, for example, how diverse or coherent certain gestures are with respect to the gesture ensemble. We think these analyses promise to be of great value for gesture studies, as we can come to understand how low-level gesture features (kinematics of gesture) relate to the higher-order organizational structures present at the level of discourse.
View
Determining Synchrony Between Behavioral Time Series: An Application of Surrogate Data Generation for Establishing Falsifiable Null-Hypotheses
Article
Full-text available
  • Mar 2018
Synchrony between interacting systems is an important area of nonlinear dynamics in physical systems. Recently psychological researchers from multiple areas of psychology have become interested in nonverbal synchrony (i.e., coordinated motion between two individuals engaged in dyadic information exchange such as communication or dance) as a predictor and outcome of psychological processes. An important step in studying nonverbal synchrony is systematically and validly differentiating synchronous systems from nonsynchronous systems. However, many current methods of testing and quantifying nonverbal synchrony will show some level of observed synchrony even when research participants have not interacted with one another. In this article we demonstrate the use of surrogate data generation methodology as a means of testing new null-hypotheses for synchrony between bivariate time series such as those derived from modern motion tracking methods. Hypotheses generated by surrogate data generation methods are more nuanced and meaningful than hypotheses from standard null-hypothesis testing. We review four surrogate data generation methods for testing for significant nonverbal synchrony within a windowed cross-correlation (WCC) framework. We also interpret the null-hypotheses generated by these surrogate data generation methods with respect to nonverbal synchrony as a specific use of surrogate data generation, which can then be generalized for hypothesis testing of other psychological time series.
View
Why Do We Fall into Sync with Others? Interpersonal Synchronization and the Brain's Optimization Principle
Article
Full-text available
  • Nov 2017
Spontaneous interpersonal synchronization of rhythmic behavior such as gait or clapping is a ubiquitous phenomenon in human interactions, and is potentially important for social relationships and action understanding. Although several authors have suggested a role of the mirror neuron system in interpersonal coupling, the underlying brain mechanisms are not well understood. Here we argue that more general theories of neural computations, namely predictive coding and the Free Energy Principle, could explain interpersonal coordination dynamics. Each brain minimizes coding costs by reducing the mismatch between the representations of observed and own motor behavior. Continuous mutual prediction and alignment result in an overall minimization of free energy, thus forming a stable attractor state.
View
Social Antecedents and Consequences of Behavioral Synchrony
Chapter
  • Oct 2016
Socially situated thought and behaviour are pervasive and vitally important in human society. The social brain has become a focus of study for researchers in the neurosciences, psychology, biology and other areas of behavioural science, and it is becoming increasingly clear that social behaviour is heavily dependent on shared representations. Any social activity, from a simple conversation to a well-drilled military exercise to an exquisitely perfected dance routine, involves information sharing between the brains of those involved. This volume comprises a collection of cutting-edge essays centred on the idea of shared representations, broadly defined. Featuring contributions from established world leaders in their fields and written in a simultaneously accessible and detailed style, this is an invaluable resource for established researchers and those who are new to the field.
View
View
Quantification of nonverbal synchrony using linear time series analysis methods: Lack of convergent validity and evidence for facets of synchrony
Article
  • Oct 2018
  • BEHAV RES METHODS
Nonverbal synchrony describes coordination of the nonverbal behavior of two interacting partners. Additionally, it seems to be important in human interactions, such as during psychotherapy. Currently, there are several options for the automated determination of synchrony based on linear time series analysis methods (TSAMs). However, investigations into whether the different methods measure the same construct have been missing. In this study, N = 84 patient–therapist dyads were videotaped during psychotherapy sessions. Motion energy analysis was used to assess body movements. We applied seven different TSAMs and recorded multiple output scores (average synchrony, maximum synchrony, and frequency of synchrony; in total, N = 16 scores). Convergent validity was examined using correlations of the output scores and exploratory factor analysis. Additionally, two criterion-based validations were conducted: investigations of concordant validity with a more generalized nonlinear method, and of the predictive validity of the synchrony scores for improvement in interpersonal problems at the end of therapy. We found that the synchrony measures only partially correlated with each other. The factor analysis did not support a common-factor model. A three-factor model with a second-order synchrony variable showed the best fit for eight of the selected synchrony scores. Only some synchrony scores were able to predict improvement at the end of therapy. We concluded that the considered TSAMs do not measure the same synchrony construct, but different facets of synchrony: the strength of synchrony of the total interaction, the strength of synchrony during synchronization intervals, and the frequency of synchrony.
View
Using novel methods to examine the role of mimicry in trust and rapport
Conference Paper
  • Jul 2017
Without realising it, people unconsciously mimic each other’s postures, gestures and mannerisms. This ‘chameleon effect’ is thought to play an important role in creating affiliation, rapport and trust. Existing theories propose that mimicry is used as a social strategy to bond with other members of our social groups. There is strong behavioural and neural evidence for the strategic control of mimicry. However, evidence that mimicry leads to positive social outcomes is less robust. In this thesis, I aimed to rigorously test the prediction that mimicry leads to rapport and trust, using novel virtual reality methods with high experimental control. In the first study, we developed a virtual reality task for measuring implicit trust behaviour in a virtual maze. Across three experiments we demonstrated the suitability of this task over existing economic games for measuring trust towards specific others. In the second and third studies we tested the effects of mimicry from virtual characters whose other social behaviours were tightly controlled. In the second study, we found that virtual mimicry significantly increased rapport and this was not affected by the precise time delay in mimicking. In the third study we found this result was not replicated using a strict, pre-registered design, and the effects of virtual mimicry did not change depending on the ingroup or outgroup status of the mimicker. In the fourth study we went beyond mimicry to explore new ways of modelling coordinated behaviour as it naturally occurs in social interactions. We used high-resolution motion capture to record motion in dyadic conversations and calculated levels of coordination using wavelet analysis. We found a reliable pattern of decoupling as well as coordination in people’s head movements. I discuss how the findings of our experiments relate to theories about the social function of mimicry and suggest directions for future research.
View
The Effect of Behavioral Synchrony with Black or White Virtual Agents on Outgroup Trust
Article
  • Feb 2018
  • COMPUT HUM BEHAV
Trust toward outgroup members is generally lower than it is toward ingroup members. Behavioral synchrony with virtual outgroup characters has been identified as a means of improving attitudes toward racial outgroup members, but this effect has not been tested for outgroup trust. We tested the effect of synchrony with an ingroup/outgroup virtual agent on a behavioral measure of outgroup trust. An experiment used an online economic game to obtain pretest and posttest measures of trust. In between these measures, participants played a dance video game on Xbox Kinect. They were randomly assigned to either an ingroup or outgroup agent (black or white) partner. Game score served as a continuous measure of synchrony with the agent. Regression analysis revealed that agent race moderated synchrony's effect on change in outgroup trust. Increased synchrony with an outgroup agent led to increased outgroup trust. Conversely, increased synchrony with an ingroup agent led to decreased outgroup trust. Findings are discussed with respect to implications for using virtual interactions to build outgroup trust in the real world.
View