Motion Energy Analysis (MEA): A Primer on the Assessment of Motion From Video

Abstract

Nonverbal behavior is a central factor influencing the therapeutic relationship. Despite broad agreement on its importance, empirical studies assessing nonverbal behavior in counseling and psychotherapy are relatively scarce and often limited to few cases. One restraining factor may be the resources needed when assessing nonverbal behavior. Movement dynamics are an exemplary aspect of nonverbal behavior that can be captured with computer vision-a discipline concerned with the automated analysis of footage captured on video. One of the simplest methods requiring no special detectors, devices, or markers on patients or therapists is based on the assessment of differences in sequences of pictures (frames) found in video recordings. Algorithms of so-called frame-differencing methods may be implemented on commonly available computers, and they provide a good, straightforward assessment of, for example, patients' and therapists' movement dynamics in counseling and therapy sessions. Frame-differencing methods in psychology date back 36 years, but their use in counseling and psychotherapy research is only recently gaining momentum. In this introductory article, the use of one specific application suitable for the assessment of human motion from archival video material is presented. Motion energy analysis (MEA) is a procedure particularly appropriate for clinicians and researchers who have access to recordings of sessions or who wish to record their own video material. Focusing on the phenomenon of nonverbal synchrony-the coordination of movement between patient and therapist-a step-by-step demonstration of the stages involved in a successful application of MEA in psychotherapy research is provided. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Full text

0
Motion Energy Analysis (MEA). A primer on the assessment of motion from video
Fabian T. Ramseyer
University of Bern, Switzerland
© 2019, American Psychological Association. This paper is not the copy of record and may not
exactly replicate the final, authorit ative version of the article. Please do not copy or cite without
authors' permission. The final article will be available, upon publication, via its DOI:
10.1037/cou0000407
Ramseyer, F. T. (2020). Motion energy analysis (MEA). A primer on the assessment
of motion from video. Journal of Counseling Psychology, 67(4), 536-549.
doi:10.1037/cou0000407
Author Note:
Fabian Tobias Ramseyer, Clinical Psychology and Psychotherapy, University of Bern,
Bern, Switzerland.
Correspondence concerning this article should be addressed to Fabian T. Ramseyer,
Clinical Psychology and Psychotherapy, University of Bern, Gesellschaftsstrasse 49,
CH-3012 Bern, Switzerland.
E-mail: fabian.ramseyer@psy.unibe.ch
This research was supported by a grant from Büro Ramseyer [grant number
BR_2019_003].
The data in this manuscript, instructional movies, and a manual for MEA are available on
the Open Science Framework (OSF) in a project named "Motion Energy Analysis
(MEA)". Link: http://www.osf.io/gkzs3.
The application "MEA" has been freely available online since November 2012 on the
author's webpage. Link: http://www.psync.ch. No previous disseminations of the results
appearing in the manuscript have been made. The ideas regarding nonverbal synchrony
and the use of the application "MEA" have been previously presented in October 2004 at
the "12th Herbstakademie" in Jena, Germany. Results gained with MEA and instructions
regarding the use of MEA have been presented multiple times between the years 2005
and 2018 at conferences organized by the Society for Psychotherapy Research, SPR.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
1
Abstract
Nonverbal behavior is a central factor influencing the therapeutic relationship. Despite broad
agreement on its importance, empirical studies assessing nonverbal behavior in counseling and
psychotherapy are relatively scarce and often limited to few cases. One restraining factor may be
the resources needed when assessing nonverbal behavior. Movement dynamics are an exemplary
aspect of nonverbal behavior that can be captured with computer vision – a discipline concerned
with the automated analysis of footage captured on video. One of the simplest methods requiring
no special detectors, devices, or markers on patients or therapists is based on the assessment of
differences in sequences of pictures (frames) found in video-recordings. Algorithms of so-called
frame-differencing methods may be implemented on commonly available computers, and they
provide a good, straightforward assessment of e.g. patient's and therapist's movement dynamics
in counseling- and therapy sessions. Frame-differencing methods in psychology date back 36
years, but their use in counseling- and psychotherapy research is only recently gaining
momentum. In this introductory article, the use of one specific application suitable for the
assessment of human motion from archival video material is presented. Motion energy analysis
(MEA) is a procedure particularly appropriate for clinicians and researchers who have access to
recordings of sessions or who wish to record their own video material. Focusing on the
phenomenon of nonverbal synchrony – the coordination of movement between patient and
therapist, we provide a step-by-step demonstration of the stages involved in a successful
application of MEA in psychotherapy research.
Keywords: motion energy analysis (MEA), nonverbal synchrony, therapist experience, intake
interview

MOTION ENERGY ANALYSIS (MEA): A PRIMER
2
Public significance statement
An objective, computer-based tool for the assessment of nonverbal behavior shown by dyads in
counseling and psychotherapy is presented. Motion energy analysis (MEA) automatically
quantifies movement dynamics shown by patients and therapists during their verbal interaction.
This aspect of nonverbal behavior provides valuable information on specific characteristics of
the therapeutic relationship and other facets of the therapy process. The practical use of MEA is
illustrated in a sample of psychotherapy intake interviews.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
3
Motion Energy Analysis (MEA). A primer on the assessment of movement from video.
Observable nonverbal behavior is widely regarded as a crucial aspect of the therapeutic
relationship by both researchers and clinicians across schools of therapy. Accordingly, nonverbal
behavior has been on the research agenda for a long time: Kiesler noted that “the most crucial
place to search for relationship is in the nonverbal behavior of the interactants. Nonverbal
communication is the language of emotion and relationship” (1979; p. 303). However, despite
wide agreement bridging the gap between theory and practice, a clear definition and
understanding of what exactly happens in nonverbal behavior and how it relates to the
therapeutic relationship remain elusive. This has not always been the case, as empirical research
on nonverbal behavior in human interaction has witnessed comparatively strong fluctuations
over time, with an intensive phase between 1960 and 1980, impressively documented by the
annotated bibliographies of Davis and colleagues, who summarized an initial total of 931 (Davis,
1972) and 1411 (Davis & Skuipen, 1982) entries of empirical studies in this domain. In
counseling psychology, the assessment of nonverbal behavior, micro-processes or so-called
micro-counseling was a very active area of research (e.g. Fretz, Corn, Tuemmler, & Bellet, 1979;
Ivey, 1973; Ivey, Normington, Miller, Morrill, & Haase, 1968; Truax et al., 1966), that included
specific aspects such as proxemic behavior (Haase & DiMattia, 1970), empathic understanding
(e.g. Fretz, 1966; Haase & Tepper, 1972), posture (Maurer & Tindall, 1983; Hermansson,
Webster, & McFarland, 1988), and general movement patterns (Strong , Taylor, Bratton, &
Loper, 1971). Following this highly productive phase, studies employing measures of nonverbal
behavior have subsequently been reported less regularly, and the term “nonverbal behavior” is
not even part anymore of the index in the current edition of the “Handbook of Psychotherapy and
Behavior Change” (Lambert, 2013; italics added by author). The nonverbal behavior emerging in

MOTION ENERGY ANALYSIS (MEA): A PRIMER
4
dyadic exchange is often crucially different from what is arising at the level of one single person:
In the terminology of dynamic systems theory (Salvatore & Tschacher, 2012), new attractors
(stable states of the system) may emerge (Tschacher & Haken, 2019; Vallacher & Nowak, 2009;
Schiepek, 2009) as soon as two or more individuals engage in the process of conversation or
counseling/psychotherapy (Hayes & Strauss, 1998; Gelo & Salvatore, 2016). The dynamical
systems approach provides an excellent framework for process research in counseling and
psychotherapy (Butner et al., 2017). Interpersonal interaction has been conceptualized as a
coupled dynamic system where the synchronization of sub-systems (the interactants) are
understood as indicators for coupling (Gottman, Swanson, & Swanson, 2002; Steenbeek & van
Geert, 2007). Phenomena of coupling may be observable in synchronized behavior happening in
different communicational modalities (Bernieri & Rosenthal, 1991; De Jaegher & Di Paolo,
2007; Ramseyer & Tschacher, 2006; Shockley, Richardson, & Dale, 2009).
More recently, behavioral manifestations of empathy have again been assessed
empirically, e.g. in the domain of vocal synchrony (Imel et al., 2014; Gaume et al., 2019).
Various technological advancements have renewed research activity in this field (Hall, Horgan,
& Murphy, 2019; Imel, Caperton, Tanana, & Atkins, 2017), and this resumed interest seems
justified in light of the solid fact that the alliance was and remains one of the crucial factors
predicting outcome in psychotherapy (Flückiger, Del Re, Wampold, & Horvath, 2018). If we
restate that nonverbal behavior is one of the critical aspects that influence relationships and
alliance (Chartrand & Lakin, 2013; Schmid Mast, 2007), then the suggestion for more research
in the domain of nonverbal behavior appears entirely reasonable. In the present article, we
describe a tool that allows an easy, quick, and inexpensive assessment of nonverbal behavior,
borrowing algorithms from the domain of computer-vision (Sonka, Hlavac, & Boyle, 2007).

MOTION ENERGY ANALYSIS (MEA): A PRIMER
5
Movement as a signal in human interaction
Movement may be perceived as a so-called honest signal, because – from an evolutionary
point of view (Brüne, 2015; Grammer, Keki, Striebel, Atzmüller, & Fink, 2003) – the social
survival of human individuals and groups has heavily relied on this feature: Successful
navigation in social environments depended on the correct identification and interpretation of
movement signals. Perceiving what others are doing and inferring from these signals what they
may be intending to do is essential (Blake & Shiffrar, 2007). Not only for fight- and flight-
responses, but also because human action communicates essential social aspects such as
intentions and feelings (Grafton, 2009; Blakemore & Decety, 2001). The dynamics of human
motion are well distinguishable, and observers are able to extract a multitude of features from
very rudimentary stimuli, so-called point-light-displays; light-dots attached to the joints of a
person (Clarke, Bradshaw, Field, Hampson, & Rose, 2005). Humans have developed specific
networks for the identification of bodies and bodily expressions (de Gelder et al., 2010).
Assessment of movement: A brief overview of common methods
In the past, many valid solutions for the assessment of movement in recorded or live
interactions (i.e. video analysis, direct assessments) have been developed, as described in
Cornejo, Cuadros, Morales and Paredes (2017) and Delaherche et al. (2012). Early work by
Condon and Ogston (1966) relied on observer-coded movement derived from film-recordings,
while later studies focused more on manual coding of the gestalt of movement (Bernieri &
Rosenthal, 1991; Bernieri, Reznick, & Rosenthal, 1988; Scheflen, 1965) or specific behaviors
(Chartrand & Bargh, 1999; Geerts, Bouhuys, & Van Den Hoofdakker, 1996). In recent years,

MOTION ENERGY ANALYSIS (MEA): A PRIMER
6
there has been a shift toward software-based methods and toward the implementation of e.g.
wearable electronic devices. Computer-vision algorithms and technological solutions such as
accelerometers (Walther et al., 2014), potentiometers (Varlet et al., 2014), magnetic motion
capture systems (Boker et al., 2011), and optical motion capture systems (Beskow, Edlund,
Granström, Gustafson, & House, 2010) have become more common and affordable. In the
remainder of this article, we will focus on one specific algorithm that derives movement
parameters directly from the visual features embedded in video-recordings: Motion Energy
Analysis (MEA).
Motion Energy Analysis: Quantifying movement in film.
Analog film recordings (photochemical methods) are based on the succession of still
images (= frames) captured at a frequency needed to generate the optical illusion of movement.
This principle similarly applies to digital film: Static images are captured (commonly at 25 to 60
frames per second) and stored in a sequence (Jack, 2005). Individual frames of digital film are
made up of an array of pixels (=picture elements) arranged on the image sensor and later shown
on the screen. In a static image/frame, each pixel has a defined color which changes when there
is a change from one frame to the next. Such an alteration in color may occur if an object is
moving. Frame-differencing algorithms rely on the fact that differences between consecutive
frames of a stored sequence may be quantified by comparing one static image with its
predecessor (Sonka, Hlavac, & Boyle, 2007). The difference between frames is then extracted by
summing the number and amount of pixel-change. Frame-differencing methods quantify
movement dynamics: The direction and form of movement are not captured, as the algorithm
only quantifies the degree of change across time. However, by defining a region where

MOTION ENERGY ANALYSIS (MEA): A PRIMER
7
differences should be quantified (= region of interest, ROI), it is possible to discriminate e.g. a
head region of a person indicating head-movement, and an upper-body region indicating
gesturing and movements of the torso (see Figure 1). Frame-differencing programs such as MEA
consecutively calculate the amount of change within a ROI and provide a time-series of this
quantification (see Video "MEA_demo" available on the Open Science Framework, OSF:
https://osf.io/gkzs3/). Raw values of this change may not directly be interpreted, because they
depend on multiple parameters: The size of the ROI, the ratio of foreground versus background,
the resolution of the video (how many pixels), and the noise-to-signal ratio (how good is
movement distinguishable from change in pixels caused by other fluctuations). These parameters
are important insofar as they should be comparable across different sessions within a dataset.
Current implementations in counseling psychology and psychotherapy
The implementation of methods to assess movement based on videotaped interactions has
been introduced to psychology by Watanabe (1983), and was later used in different ethological
and social contexts by Grammer and colleagues (Grammer, Honda, Schmitt, & Juette, 1999).
The first overview of methods and findings in the domain of psychology was presented more
than a decade later (Delaherche et al., 2012), and descriptions of the implementation of such a
tool have been available for some time (Altmann, 2013; Kupper, Ramseyer, Hoffmann,
Kalbermatten, & Tschacher, 2010; Paxton & Dale, 2013a; Reidsma, Nijholt, Tschacher, &
Ramseyer, 2010). Time-series generated by frame-differencing methods are usually evaluated
with further statistical methods, and one useful calculation is concerned with the quantification
of mutual influence across two time-series. In the domain of human interaction, this kind of
bidirectional influence or coordination between interaction partners has been called interactional

MOTION ENERGY ANALYSIS (MEA): A PRIMER
8
synchrony (Bernieri & Rosenthal, 1991), and the term nonverbal synchrony (Tschacher &
Ramseyer, 2017) has been used in a clinical context. In sessions of dyadic psychotherapy,
nonverbal synchrony has been positively associated with relationship quality, self-efficacy and
outcome (Ramseyer & Tschacher, 2011). At the beginning of therapy, dyads with depressive
patients had lower levels of nonverbal synchrony compared to dyads with patients suffering from
anxiety disorders. These differences no longer existed at the end of therapy because synchrony
increased in depressive patients and decreased in anxious patients (Paulick et al., 2018b). The
reduction of interpersonal problems was associated with higher synchrony (Altmann et al.,
2019), and a post-hoc analysis showed that low levels of synchrony were associated with both
non-improvement and drop-out while a medium level of synchrony corresponded to
improvement (Paulick et al., 2018a). A similar association between premature termination and
low movement synchrony was found in patients with social anxiety disorder (Schoenherr et al.,
2019a). A single-case study found clinically important scenes to be associated with higher
synchrony (Kodoma, Hori, Tanaka, & Matsui, 2018). In patients with schizophrenia, nonverbal
synchrony was shown to be differentially related to symptomatology (Kupper, Ramseyer,
Hoffmann, & Tschacher, 2015), and in the context of body-psychotherapy for patients suffering
from schizophrenia, a trend for an increase of synchrony during an interview conducted pre-
therapy versus post-therapy was found (Galbusera, Finn, & Fuchs, 2018). In clinical interviews
with patients suffering from borderline personality disorder and in a group of healthy controls,
nonverbal synchrony was differentially affected by the double-blind administration of Oxytocin
versus Placebo: Oxytocin reduced synchrony in patients while it increased synchrony in healthy
controls (Ramseyer et al., 2019). These studies suggest a trend for a positive association between
nonverbal synchrony and positive aspects of various facets of psychotherapy, as postulated by

MOTION ENERGY ANALYSIS (MEA): A PRIMER
9
the In-Sync model (Koole & Tschacher, 2016). Ramseyer and Tschacher's (2011) finding of an
association between synchrony and relationship quality has not yet been unequivocally replicated
in other psychotherapy settings, and the authors themselves recently failed to find an association
between self-reported relationship quality and nonverbal synchrony across multiple sessions
within dyads (Ramseyer, 2019). Apart from a possible methodological influence by the many
possible parameter settings used in different studies (Schoenherr et al., 2019c), it is also
conceivable that nonverbal synchrony encompasses much more than the factors reported so far.
More empirical work is needed in order to answer this open question satisfyingly. In the
following, we are demonstrating MEA and its application to video-recordings of psychotherapy
sessions.
Practical guidelines
An implementation of the frame-differencing application presented here has been freely
available since 2012 (www.psync.ch). This specific program has been called Motion Energy
Analysis (MEA) and was inspired by the original software developed by Karl Grammer and
colleagues at the University of Vienna, Austria (Grammer et al., 1999). Grammer et al. called
their procedure Motion Energy Detection (MED) or – more broadly – Automatic Movie Analysis
(AMA). Ramseyer and Tschacher (Ramseyer & Tschacher, 2006; 2008) later introduced their
own software-implementation of a similar algorithm under the name Motion Energy Analysis
(MEA). Other implementations of frame-differencing algorithms (Altmann, 2011; Komori,
Maeda, & Nagaoka, 2007; Paxton & Dale, 2013a) are currently available for various software
environments (OpenCV; Bradski & Kaehler, 2000; Matlab, The Mathworks©), and some of
these groups have later adopted the name MEA. The version described in this manuscript is

MOTION ENERGY ANALYSIS (MEA): A PRIMER
10
based on the implementation of MEA by Ramseyer and Tschacher (2011) programmed in a
graphical software environment (Max 8, © Cycling '74, 2019), using elements from the
computer-vision toolset (cv.jit) developed by Pelletier (2019). MEA provides a point-and-click
graphical user interface, making the software accessible to users without prior knowledge of
computer vision, and it does not require any programming skills. On the application's interface
(Figure 1), the researcher is guided through a series of clickable steps that generate a data-file
storing the amount of movement extracted from the video.
Figure 1. User interface of MEA 4.10 for Mac OS X

MOTION ENERGY ANALYSIS (MEA): A PRIMER
11
In the following paragraphs, we describe the 10 steps involved in the generation of time-
series in MEA (the raw data). Our first exemplary analysis will use an artificially created video-
sequence suited for the demonstration of several concepts used with MEA and subsequent
statistical analyses. This sequence is available on the Open Science Framework in a project
called "Motion Energy Analysis (MEA)" (https://osf.io/gkzs3/), and we encourage potential
users to recreate our analyses on their computer. Generally speaking, there are some critical
specifications regarding the nature and quality of video-recordings, because these properties
affect the data generated by frame-differencing algorithms. We will address these video-related
issues first.
Recording video for MEA
There are only minimal basic requirements regarding the usability of video material in
MEA. For a successful analysis of motion from video, the following restrictions have to be
followed:
1. A fixed video camera: MEA is based on differences between video frames. Thus, it is
imperative that only the persons/subjects in question are moving; the rest of the picture should
remain stable throughout the sequence. Only fixed cameras mounted on tripods/walls/ceilings
fulfill this standard. If an operator moves or adjusts the camera during the recording, then MEA
cannot be applied (this includes using the zoom-feature, or changing the aperture). Frame-
differencing methods only quantify change, which implies that although the movement of the
camera would be correctly registered as change, it would be erroneously classified/added to the
movement of a subject.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
12
2. Static background: The scene of the interaction has to remain static; no major changes
in the environment are allowed. Thus, no other subjects should move behind the person to be
analyzed, and the walls/windows in the background should remain without change throughout.
The program will not differentiate between objects moving in the frontal parts of the image (the
therapist) versus objects moving in the background (the therapist's cat chasing a dog).
3. A circumscribed region of movement: If more than one subject’s movement should be
quantified, subjects cannot move in front of each other. The defined regions for the analysis of
movement cannot overlap because the program is not able to detect whether a person
leaves/crosses a region.
4. Stable light conditions: The light should remain relatively stable, i.e. no artificial lights
should be switched off/on. There should be no instances of extreme changes in sunlight and
shade that would affect the appearance of the picture.
5. Codec for digital recording: The possible codecs used in video-cameras are numerous,
and there are no specific restrictions regarding the codec. However, some codecs may induce a
bias, because codecs compress the original video in a specific way, which can lead to regular
changes in the picture that may later be picked up by MEA. These signals induced by the coded
are artifacts that have to be corrected in a later step. It is important to have an awareness of this
possible bias, and it can be spotted when checking the raw data. In previous work employing
MEA, codecs of the type "h.264" have provided good results.
6. Hardware: The specific version or kind of camera (resolution, lens, position), may also
influence how movement is quantified in MEA. As a general rule, the majority of cameras and
lenses will work with MEA, and the point-of-view from the camera may differ from one study to
another. The important factor to keep in mind is the fact that data from one hardware-setup may

MOTION ENERGY ANALYSIS (MEA): A PRIMER
13
not be directly comparable with a different setup (see the comparison of cameras below). The
reason for this potential incompatibility resides in the fact that by reducing three-dimensional
movement onto the two-dimensional plane of a video may result in a biased representation of
movement. A therapist moving in the direction of the camera will (sagittal movement) will
appear as having moved less than the same therapist moving sideways (lateral movement). In the
past, MEA has been applied to split-screen recordings (they offer a favorable signal-to-noise
ratio by maximizing the screen-area taken up by each person), but single- or multi-camera
recordings may be equally analyzed. Simply put, within a sample, one should not mix different
types of recording setups.
Using MEA (Version 4.1)
The following step-by-step guideline applies to MEA for operating systems of Apple's ©
OSX 10.13 and higher, and Microsoft's © Windows 7 and higher. Additional information
regarding version history, installation instructions, and a user-forum are provided on
www.psync.ch. An instructional movie as well as a pdf-manual for MEA are available on the
program's Open Science Framework project (https://www.osf.io/gkzs3).
1. In the first step, a recorded movie is loaded into the program by pointing the
computer’s operating system (OS X / Windows) to the location of the movie-file. Once selected,
the movie begins playing on the central screen.
2. Regions of interest (ROIs) can be drawn directly onto the original movie using a
pencil. Different pencil sizes may be chosen for bigger or more refined regions.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
14
3. If the program has stored ROIs from an analysis run previously, they can (and often
should) be deleted prior to the drawing of regions. ROIs should be adjusted to the size and
location of the subjects in a video.
4. The desired ROI can be selected by clicking its ID number and then drawing the
shape onto the visible screen of the movie. Up to 8 ROIs may be freely chosen. It is strongly
recommended to stick to an identical assignment of regions throughout a dataset because this
will facilitate the automated analysis later on. The adequacy of a ROI may be evaluated at this
point by "zooming" through the sequence using the movie-slider above the screen. Previous
studies have often used one region per subject, often covering the upper body and head down to
the base of the chair. As shown in the demonstration movie, separate regions for the head and the
upper body may be defined, and several studies have shown differential effects of movement
assessed in the head-region versus whole-body movement (Dean, Samson, Newberry, & Mittal,
2018; Kupper et al., 2015; Ramseyer & Tschacher, 2014).
5. Each video may possess different conditions of recording hardware and situational
influences. Thus, the threshold for movement versus no-movement should be adjusted
accordingly. The default threshold of “20” works for many situations; however, researchers are
encouraged to adapt the threshold according to their material. A good visual indicator for the
appropriateness of a setting is provided in the “frame-differenced movie” to the right of the
original movie: In this black-and-white screen, the results of frame-differencing are shown in
real-time. Lowering the threshold (towards zero) results in more pixels appearing in peripheral
regions. A reasonable threshold is characterized by no white “noise” (stray pixels falsely
regarded as movement) outside of ROIs (see Movie "instructions_MEA_4" available on OSF for
a demonstration).

MOTION ENERGY ANALYSIS (MEA): A PRIMER
15
6. After all ROIs have been defined, and a suitable threshold has been found, the data
generated during this process has to be cleared (clear data) and the movie is set back to its
beginning.
7. The analysis may then be started by clicking “Start MEA”. Depending on the
resolution of the original video and the speed of the computer’s processor, the analysis will run
several times faster than real-time (e.g. 8x on a MacPro).
8. An annoying beep is sounded after the completion of the analysis to alert the user for
the next step. The user can now save the raw data in the form of a .txt-file to a destination of
choice (e.g., the hard disk). After saving the file, the next movie may be loaded and analyzed.
9. The raw data generated in steps 1. to 8. should then be carefully checked for artifacts
and anomalies. The package rMEA (Kleinbub & Ramseyer, 2018) provides several utilities for
these checks.
10. The statistical analysis of data generated with MEA may then be completed in an
environment favored by the user. An overview and discussion of currently available algorithms
for the computation of nonverbal synchrony is available in Schoenherr et al. (2019b). We suggest
using the package rMEA (Kleinbub & Ramseyer, 2018) specifically tailored for MEA. It runs in
the statistical software R, and is freely available from the repository CRAN (R Core Team,
2008).
Quantification of nonverbal synchrony using windowed cross-correlations
The most widely used method for the quantification of nonverbal synchrony in behavioral time-
series is based on cross-correlations (Boker, Xu, Rotondo, & King, 2002). Here, we will only
describe the basic concepts and parameters, as the statistical details of these procedures are
available in multiple publications (Altmann, 2011; Boker et al., 2002; Brick & Boker, 2011;

MOTION ENERGY ANALYSIS (MEA): A PRIMER
16
Moulder et al., 2018; Paxton & Dale, 2013a; Schoenherr et al., 2019b; 2019c). Central concepts
and parameters of cross-correlational methods are the following:
stationarity/non-stationarity: Time-series of movement are assumed to change or shift with time.
If cross-correlations were calculated across the entire sequence, these changes (=non-
stationarity) would affect the results. Calculating cross-correlations in segments (also
referred to as windows) takes such changes into account.
lag: Synchronized movement may occur at exactly the same time, but there may also be
instances where one persons’ movement is followed by the other subject's movement with
a certain delay. The duration of this delay is called lag. Cross-correlations use positive and
negative lags in order to capture both directions of influence.
window: The duration or extension of a segment where local stability (=stationarity) may be
assumed.
increment/overlap: The duration that one window is shifted forwards for the calcualtion of the
next cross-correlation. If window-size and increment are identical, windows do not
overlap.
leading: The time-series are shifted in both directions (positive and negative lags), and this
allows a distinction into whose movements were preceding the other's (who moved first?).
pseudosynchrony: A measure of synchrony that would be expected by coincidence. It provides a
good estimate for the strength of the phenomenon because it compares the real associations
found in genuine interactions with chance associations produced by pseudo-interactions
(Bernieri et al., 1988; Ramseyer & Tschacher, 2010; Moulder, Boker, Ramseyer, &
Tschacher, 2018).

MOTION ENERGY ANALYSIS (MEA): A PRIMER
17
Figure 2. Workflow of MEA. A: Quantification of change in video (frame-differencing); B: Time
series of raw data: Y-axis = z-standardized amount of movement, X-axis = time; C: Cross-
correlation plot with corresponding peak visible in raw data (arrow): Y-axis = lags, X-axis =
time; D: Lagplot comparing multiple groups (colored lines: initial = green, novice = blue, middle
= pink, final = turquoise) and pseudosynchrony (grey line): Y-axis = absolute cross-correlations
(= synchrony), X-axis = lags.
For illustrative purposes, we have edited a test-sequence based on a real psychotherapy
interaction: The behavior of one therapist (one side of the original split-screen view from an
intake interview, i.e., excluding the patient) was mirrored onto the other side of the screen (as if
the therapist were talking with himself). This manipulation generates a perfectly synchronized
interaction, since both interaction partners are hereafter moving in an identical way. Multiple

MOTION ENERGY ANALYSIS (MEA): A PRIMER
18
time-delays of movement were then electronically edited into the sequence with video-editing
software (Blackmagic Design, DaVinci Resolve 16), in order to simulate different kinds of
temporal coordination (= synchrony at different lags). These time-delays simulate the more
naturalistic situation of one person moving and then another person moving after a certain
amount of time has passed. Besides fixed temporal delays (0s, 3s, 4s, 8s), we also accelerated
and decelerated one therapist's movement continuously in order to generate "ramps" of varying
time-delays across the sequence. Furthermore, we included a pseudo-interaction by adding
another "personification" of the same therapist taken from a different intake interview. Lastly, we
included a delay of 8s, which is outside the boundaries of the maximum lag analyzed in the
present example (lagSec = ±5s), because this allows us to demonstrate the importance of these
parameters. Figure 3 visualizes how these different features of temporal coordination show up in
color-coded cross-correlation graphs generated in rMEA: Four different sections (A to D)
representative of the specific aspects mentioned above, are visible and separated by vertical
black lines (see Figure 3). A full analysis of the video and the video itself are freely available in
the MEA project-folder on OSF (https://www.osf.io/gkzs3).

MOTION ENERGY ANALYSIS (MEA): A PRIMER
19
Figure 3. Cross-correlation plot (MEAheatmap in rMEA) generated for the video test-sequence.
Four different phases (A to D) separated by vertical black lines. A: Fixed delay of -3s; B:
Acceleration from –3s to zero-delay (0s) and deceleration to +4s; C: Pseudo-interaction; D:
Fixed delay of -8s (not visible, outside maximum lagSec of ±5s).
Averaged, absolute cross-correlation values are written onto the graph for overall synchrony
(values at lag 0); client leading (values on top of graph); and therapist leading (value on bottom
of graph). Further illustrations with different time-lags (lagSec) and window-increments (incSec)
are available at the MEA-project on OSF: https://www.osf.io/gkzs3.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
20
Clinical Demonstration of MEA in psychotherapy
We demonstrate the use of MEA with a study assessing nonverbal synchrony in a clinical
sample from an ambulatory psychotherapy clinic at the University of Bern, Switzerland. The
anonymized analysis of video-recordings with MEA is part of the general research activity
declared and approved in the consent form signed by all patients included in this sample. The
cantonal ethics committee of Bern, Switzerland has approved this use of recorded video material.
Sessions consist of intake interviews with durations of at least 60 minutes, with a majority lasting
90 minutes. The focus of these extended interviews is on understanding the reason for the client's
appointment and a detailed assessment of the personal life history.
Sample
Given the fact that this manuscript should serve as a guideline that brings to life a novel
approach for process research in counseling and psychotherapy, we will use a rather peculiar
dataset to achieve this aim: The author of the approach provides data from his intake interviews,
which is normally not unproblematic for clinical analyses, but in the present case, it will allow an
evaluation of possible (self-referential) effects induced by the assessment of synchrony. The
sample thus contains many sessions from one single therapist (age at first interview = 40 years,
experience = ca. 600 hours of psychotherapy) who conducted N = 103 interviews [sample T]
between the years 2015 and 2018. Clients were n = 57 female (55.3%), and 29.1% (30) suffered
from affective disorders, 16.5% (17) from anxiety disorders, 5.8% (6) from adjustment disorders,
11.7% (12) from other diagnoses, and 36.9% (38) had no clinical diagnosis. In this large group
without a clinical diagnosis, 14 patients (13.6%) were not diagnosed because of drop-out after
the intake interview. Another n = 20 interviews conducted by 20 therapists in training [sample N]

MOTION ENERGY ANALYSIS (MEA): A PRIMER
21
(age = 30.2 years, experience = ca. 0 to 100 hours of psychotherapy) were added to the one
therapist. They were comparable in terms of setting and procedure, and their inclusion was
intended to serve as a reference for possible between- vs. within-therapist effects. In this small
addition, female patients were overrepresented (75.0%), and fewer patients suffered from
affective disorders (16.7%), but more were diagnosed with anxiety disorders (22.2%).
Furthermore, patients without a diagnosis (33.3%) were at a similar level, but the level of drop-
out was higher at 35% (7).
During the initial minutes of the interview, video-recording, financial details, and other
logistics were covered before commencing with the relevant subject, which was usually started
with the question asking clients what motivated them to call for an appointment. For the analysis
presented here, we skipped the first ten minutes of each interview, in order to leave out the
section concerned with organizational issues. Furthermore, we wanted to keep settings for all
interviews constant and limited the analysis to 45 minutes, i.e. a segment between minute 10
(into the interview) and minute 55 was eventually analyzed with MEA.
Practical recommendations
When analyzing a large number of videos in MEA, the researcher has to decide how to
deal with the beginning and end of interactions in videos. In many cases, sequences contain
preparatory steps (starting the recording equipment, showing patients to the room). These
sections may or may not be related to the planned analysis of movement and synchrony. They
can be deleted by trimming the sequence in video-editing software, or the exact start of the
interaction can be noted (number of frames before the interaction began) for subsequent deletion
of unwanted data points in the text-file generated by MEA.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
22
Explorative analyses
First of all, we were interested whether synchrony could be distinguished from
associations of movement that one would assume by pure coincidence. This comparison with
pseudosynchrony (see definition above) is interesting insofar, as it allows an evaluation of the
strength of the phenomenon, and it also facilitates the comparison of nonverbal synchrony across
different samples: If the procedure used to compute pseudosynchrony is comparable across
samples, then the effect-size of synchrony may be compared even if essential factors such as
technical details are different (e.g. cameras, angle of the recording, influences from the setting),
because the bootstrap-data used for the quantification of pseudosynchrony always comprises
these peculiarities of the original sample. Apart from this basic question/check, we made use of
the special case in "sample T", where the factor “therapist” remained constant across multiple
years and many interviews: We thus explored a) the intraindividual temporal stability of
nonverbal synchrony, b) effects of training/experience, c) effects of gender, and d) differences
between an experienced therapist and therapists in training ("sample N").
Practical recommendations for the calculation of synchrony
There are numerous ways to calculate nonverbal synchrony from time-series of motion
energy data (raw data generated in MEA), and empirical comparisons of different configurations
have shown that these parameters may map on different aspects of the construct of nonverbal
synchrony (for a review, see Schoenherr et al., 2019c). The method which has been historically
used most often is based on windowed cross-correlations (Boker et al., 2002) and an adapted,
modified version of this method for the use in psychotherapy data has been described in detail
elsewhere (Ramseyer & Tschacher, 2011). Generally speaking, time-series of two individuals are

MOTION ENERGY ANALYSIS (MEA): A PRIMER
23
cross-correlated at different time-lags. This means that we are also considering delayed
(=lagged) associations of movement activity that exist besides simultaneous movement (=lag0).
Cross-correlations are windowed (calculated in separate segments) because the segmentation
allows for temporal changes within sessions (so-called non-stationarity of synchrony). For the
analysis presented here, we will follow the parameters used by Ramseyer and Tschacher (2011).
In rMEA (Kleinbub & Ramseyer, 2018), the time-delayed associations between movements of
patient and therapist, was set to a maximum of five seconds (lag-length, lagSec = ±5 seconds).
The segment-size was set at 1 minute (window, winSec = 60 seconds), and the windows were
non-overlapping (window-increment, incSec = 60 seconds). Pseudosynchrony was generated by
assembling a bootstrap dataset with N = 3000 pseudo-interactions. As a general rule, we
recommend using the graphical visualizations available in rMEA for a visual inspection of the
appropriateness of parameters. A graphical representation of the stages from video-analysis with
MEA to the calculation of synchrony is provided in Figure 2.
Results
We limit this section to the findings suggested in our explorative hypotheses mentioned
above, and invite readers to explore the dataset in our repository at the Open Science Framework
(https://osf.io/gkzs3/). In the combined sample (N = 123; samples T & N), nonverbal synchrony
was clearly distinct from pseudosynchrony: The comparison of real interactions with "fabricated"
pseudo-interactions provided an effect size of Cohen’s d = 1.02. Differentiating ROIs into body-
synchrony (without the head-region) provided a lower effect-size (d = 0.89), which was also true
for head-synchrony in isolation (d = 0.87). The combination of gender within interviews affected
nonverbal synchrony [F(2,120) = 4.45; p = .014], i.e. both female and male same-sex dyads

MOTION ENERGY ANALYSIS (MEA): A PRIMER
24
showed higher synchrony than the mixed-sex dyads. Considering only the initial 15 minutes of
the interview for comparison with results reported by Ramseyer & Tschacher (2011; d = 0.51
1
)
and Paulick et al. (2018a; d = 1.67
2
), resulted in an effect-size of d = 0.83 for the entire sample
(N = 123). Novice therapists showed higher synchrony than the more experienced therapist both
across 45 minutes of the interview (dnovice = 1.42; dexpert = 0.99) as well as in the initial 15
minutes (dnovice = 1.21; dexpert = 0.76). Categorizing the combined sample (T & N) into four
groups of intake interviews ("initial" = first 20 interviews, "final" = last 20 interviews, "middle"
= 63 interviews between "initial" and "final"; "novice" = 20 interviews by different novice
therapists) showed significant differences between these categories [F(3,119) = 2.97; p = .035].
Using the middle-group from the experienced therapist (n = 63) as a reference group, indicated
that the initial 20 interviews were characterized by higher synchrony [t(119) = 2.03; p = .045],
while the final 20 interviews were characterized by lower synchrony [t(119) = -2.41; p = .017].
The evolution of synchrony within an experienced therapist (sample T), showed a trend
for a decrease of synchrony across the four years (2015 to 2018) conducting intake interviews
[F(3,98) = 2.46; p = .067], which was also evident by correlating the consecutive number of his
interviews with synchrony [r(102) = -.275; p = .005]. Again, same-sex dyads (male-male, mm)
were characterized by higher synchrony than mixed-sex dyads (female-male, fm) [Mmm = 1.31;
SDmm = 1.25; Mfm = 0.73; SDfm = 1.13; t(101) = 2.47; p = .015; d = 0.49]. For a further
exploration of sample T, mixed models using "dyad" nested in "therapist" as random factor and
"time", "gender", and the interaction term "time X gender" as fixed effects were run for nine
different configurations: Models were based on combinations of ROI (head, body, combined),
1
Pseudosynchrony calculated in rMEA by permutation of dyads, identical to samples T & N
2
Pseudosynchrony calculated by within-dyad permutation.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
25
and direction of influence (overall-synchrony, patient-leading, therapist-leading). The quantified
synchrony-values were entered as dependent effects. Corrected Akaike Information criteria was
used to assess best model fit. Effects in all models pointed in the same direction, i.e. the nine
different configurations of ROIs and "direction of influence" provided comparable results. For
reasons of comparability and simplicity, here we are only reporting the model with parameters
previously used in our work. These included overall synchrony, and a ROI for the entire upper
body-region: This model indicated that a) synchrony decreased with experience ["time": F(1,99)
= 8.31; p = .005], and that b) synchrony was higher in same-sex dyads ["gender": F(1,99) = 5.56;
p = .020], and that c) there was an interaction for "time X gender" [F(1,99) = 6.11; p = .015], i.e.
synchrony decreased significantly more in same-sex dyads. A scatterplot for this model is
depicted in Figure 4, Panel A. For illustrative purposes, and as a reference, we also added the
distribution of sample N to Figure 4 (Panel B). The best fitting model (AICc) for the 9 models
contained overall-synchrony assessed in the body-region. A complete table with details of all
models is available on our project's site on the Open Science Framework
(https://www.osf.io/gkzs3).

MOTION ENERGY ANALYSIS (MEA): A PRIMER
26
Figure 4. Left panel (A): Scatterplot of nonverbal synchrony from one therapist across four years
of conducting interviews (X-axis: Serial number of intake interview sorted in temporal order).
Right panel (B): Mean and distribution of n = 20 interviews conducted by novice therapists (no
temporal order).
Discussion
This primer demonstrated the use of MEA, a frame-differencing method, for the
assessment of movement in video-recorded clinical interactions. A dataset of intake interviews
was provided in order to show the potential of MEA in a clinical sample. Nonverbal synchrony
was shown to change with therapist experience, and the strength of synchrony depended on the

MOTION ENERGY ANALYSIS (MEA): A PRIMER
27
gender-combination of the dyad. The awareness of the assessment of nonverbal synchrony did
not increase the phenomenon: Across time, the therapist became less synchronized with his
clients. This finding is surprising insofar as the authors would have expected the opposite:
Having studied the phenomenon for many years and assuming that synchrony is predominantly a
positive indicator of relationship quality and outcome (Koole & Tschacher, 2016), the therapist
assumed to be positively biased believing that high synchrony is an entirely positive aspect of
nonverbal behavior. Even though there was no deliberate attempt at increasing nonverbal
synchrony from the part of the therapist, the observed decrease was contrary to expectations. A
possible explanation for this decrease may be derived from the literature on therapist training
(Hill, Spiegel, Hoffman, Kivlighan Jr, & Gelso, 2017). By accumulating clinical experience, a
therapist may become less worried of e.g. possible drop-out of clients, and this may result that
the therapist will be more relaxed regarding a patient's decision for or against therapy. In other
words, the therapist may be less courting the patient's favor during an intake interview, and this
more laid-back attitude is reflected in less synchronized movement. Such a change would fit
evidence cited by Hill and Knox (2013), who reported that beginning trainees learn to give less
unsolicited advice, talk less, and interrupt less. An experienced therapist could thus become more
restrained regarding her/his movement dynamics. While the effect for "time" was consistently
found in 8 out of 9 models with different configurations of ROI and "direction of influence", the
interaction-effect of "time X gender" reached statistical significance in only 4 out of 9 models
tested. In other words, depending on the decision where (= ROI) and how (= direction of
influence) synchrony was measured may lead to different results (for an extended discussion of
this issue, see Schoenherr et al., 2019c). In the dataset presented here, this finding could be
interpreted in multiple ways, and – taking into account the demonstrative nature of the present

MOTION ENERGY ANALYSIS (MEA): A PRIMER
28
analysis – we are limiting the number of possible explanations regarding the effect of gender on
synchrony: First of all, in previous empirical work concerning the gender-combination and
synchrony, we encounter mixed findings: Some studies did not find effects of gender (Paulick et
al., 2018a, 2018b; Paxton & Dale, 2013b; Nelson, Grahe, & Ramseyer, 2016) or only a
marginally significant effect (Schoenherr et al., 2019a), while several studies explicitly included
same-sex dyads only (Fujiwara, Kimura, & Daibo, 2019; Lozza et al., 2018; Ramseyer &
Tschacher, 2011; Tschacher et al., 2014). In the specific case of the present dataset with only one
therapist interacting with either female or male patients, we cannot say whether the effect of
gender may be due to idiosyncratic characteristics of this therapist, or whether this is a finding
that may generalize to other therapists. However, if we resort to "sample N", the 20 novice
therapists provide some evidence for a similar pattern, because synchrony in matched dyads (n =
15) was higher (M = 1.73; SD = 2.29) than in mixed dyads (n = 5; M = 0.49; SD = 0.91), still
keeping in mind that the small sample precludes clear conclusions [t(18) = 1.16; p = .261; d =
0.60]. Furthermore, we think that the differential associations reported here speak for the
empirical potential of the procedures demonstrated in this article: The combination of MEA with
windowed, lagged cross-correlations provides a rich body of data for psychotherapy and
counseling process research. Assessing the dynamic quality of movement in clients and
therapists taps into a quality that resides outside a subject's conscious control, and our data
suggested that synchrony does not increase even when therapists have conscious knowledge of
its assessment. This quality – especially the index of nonverbal synchrony resulting from the
temporal coordination between interactants – has been associated with a range of relevant factors
in counseling and psychotherapy, and this was also the case in the special sample used for the
demonstration reported here. Further empirical work will show if and how the knowledge gained

MOTION ENERGY ANALYSIS (MEA): A PRIMER
29
by the automated analysis of patients' and therapists’ movement dynamics may subsequently be
applied in a practical way.
Limitations
Although using MEA is straightforward and requires little to no training, this simplicity
contains a certain danger: MEA provides a quantification of frame-differences irrespective of the
adequacy of the video-material. Recordings with e.g. unstably mounted cameras or with
changing backgrounds will result in time-series that are not immediately recognizable as biased
or as artifacts. Researchers have to pay careful attention regarding violations of the technical
limits of MEA. Homogeneity in terms of hardware (using the same camera) and in the recording
setting (keeping environmental factors stable such as light, background) will reduce this danger,
but it is recommended to periodically evaluate a sample's time-series with the actual movement-
behavior visible in videos. Apart from these technological risks, another – more severe –
limitation of the approach should not be forgotten: Even though one may define specific ROIs in
a video, MEA is not able to qualitatively assess movement in these regions; the program is blind
to the direction of movement and to the location of movement within a ROI. If a researcher
chooses a ROI that covers the entire body, MEA will not distinguish between head-movement
and feet-movement, and it will not quantify whether the subject moves in the direction of the
other subject or away from it. This limitation may be partially overcome by defining multiple
ROIs, but MEA is not able to dynamically adjust the position of ROIs. In other words: A client
that touches her/his face very often will show higher head-movement than another client not
touching her/his face. The definition of ROIs should thus be carefully considered, and their
appropriateness checked. In the area of computer-vision, there are many algorithms and

MOTION ENERGY ANALYSIS (MEA): A PRIMER
30
programs available that are capable of overcoming the limits of MEA mentioned above. These
tools will become more accessible in the future and may easily surpass MEA.
Open questions and future directions
Even though more refined tools for the assessment of movement are becoming available,
we believe that frame-differencing methods still provide a very simple and efficient way to
quantify an aspect of nonverbal behavior that is otherwise hard to access. The simplicity of the
algorithm and the breadth of different video-formats and recording setups suited for MEA make
it an appealing choice for video-material in archives or material recorded without a specific
intention for the quantification of nonverbal synchrony. MEA takes up the tradition of assessing
nonverbal behavior which has a long history in counseling research (Fretz, 1966), and we hope
that with further dissemination, the empirical analysis of nonverbal behavior will reclaim its
rightful place in counseling and psychotherapy research. In the future, MEA and the assessment
of nonverbal synchrony could even be used for ongoing (e.g. live) monitoring in psychotherapy,
or in the assessment and/or training of new therapists. MEA enables access to a quality that only
exists at the level of the dyad, and which usually goes unnoticed by individuals involved in an
interaction. The fact that this facet of interpersonal dynamics has been shown to be associated
with important aspects of psychotherapy hopefully motivates other researchers to explore more
of this hidden dimension.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
31
References
Altmann, U. (2011). Investigation of movement synchrony using windowed cross-lagged
regression. In A. Esposito, A. Vinciarelli, K. Vicsi, C. Pelachaud, & A. Nijholt (Eds.),
Analysis of verbal and nonverbal communication and enactment. The processing issues
(Vol. 6800, pp. 335-345). Berlin, Germany: Springer. http://dx.doi.org/10.1007/978-3-
642-25775-9_31
Altmann, U. (2013). Synchronisation nonverbalen Verhaltens. [Synchronization of Nonverbal
Behavior]. Berlin, Germany: Springer.
Altmann, U., Schoenherr, D., Paulick, J., Deisenhofer, A. K., Schwartz, B., Rubel, J.
A., . . . Strauss, B. (2019). Associations between movement synchrony and outcome in
patients with social anxiety disorder: Evidence for treatment specific effects.
Psychotherapy Research. Advance online publication.
http://dx.doi.org/10.1080/10503307.2019.1630779
Bernieri, F. J., & Rosenthal, R. (1991). Interpersonal coordination: Behavior matching and
interactional synchrony. In R. S. Feldman & B. Rime (Eds.), Fundamentals of nonverbal
behavior. (pp. 401-432). New York, NY: Cambridge University Press.
Bernieri, F. J., Reznick, S., & Rosenthal, R. (1988). Synchrony, pseudosynchrony, and
dissynchrony: Measuring the entrainment process in mother-infant interactions. Journal
of Personality and Social Psychology, 54, 243-253. http://dx.doi.org/10.1037/0022-
3514.54.2.243.
Beskow, J., Edlund, J., Granström, B., Gustafson, J., & House, D. (2010). Face-to-face
interaction and the KTH cooking show. In A. Esposito, N. Campbell, C. Vogel, A.
Hussain, & A. Nijholt (Eds.), Development of multimodal interfaces: Active listening and

MOTION ENERGY ANALYSIS (MEA): A PRIMER
32
synchrony (pp. 157-168). Berlin, Germany: Springer. http://dx.doi.org/10.1007/978-3-
642-12397-9_13
Blake, R., & Shiffrar, M. (2007). Perception of human motion. Annual Review of Psychology,
58, 47-73. http://dx.doi.org/10.1146/annurev.psych.57.102904.190152
Blakemore, S. J., & Decety, J. (2001). From the perception of action to the understanding of
intention. Nature Reviews. Neuroscience, 2, 561-567. http://dx.doi.org/10.1038/35086023
Boker, S. M., Xu, M., Rotondo, J. L., & King, K. (2002). Windowed cross-correlation and peak
picking for the analysis of variability in the association between behavioral time series.
Psychological Methods, 7, 338-355. http://dx.doi.org/10.1037/1082-989X.7.3.338
Boker, S. M., Cohn, J. F., Theobald, B. J., Matthews, I., Mangini, M., Spies, J. R., . . . Brick, T.
R. (2011). Something in the way we move: Motion dynamics, not perceived sex,
influence head movements in conversation. Journal of Experimental Psychology: Human
Perception and Performance, 37, 874-891. http://dx.doi.org/10.1037/a0021928
Bradski, G., & Kaehler, A. (2000). OpenCV. Dr. Dobb’s journal of software tools, 3. Retrieved
from http://opensource-robotics.tokyo.jp/ros.org/wiki.ros.org/
Brick, T. R., & Boker, S. M. (2011). Correlational methods for analysis of dance movements.
Dance Research, 29(supplement), 283-304. http://dx.doi.org/10.3366/drs.2011.0021
Brüne, M. (2015). Textbook of evolutionary psychiatry and psychosomatic medicine: The origins
of psychopathology. Oxford, UK: Oxford University Press.
Butner, J. E., Deits-Lebehn, C., Crenshaw, A. O., Wiltshire, T. J., Perry, N. S., Kent de Grey, R.
G., . . . Baucom, B. R. W. (2017). A multivariate dynamic systems model for
psychotherapy with more than one client. Journal of Counseling Psychology, 64, 616-
625. http://dx.doi.org/10.1037/cou0000238

MOTION ENERGY ANALYSIS (MEA): A PRIMER
33
Chartrand, T. L., & Bargh, J. A. (1999). The chameleon effect: the perception-behavior link and
social interaction. Journal of Personality and Social Psychology, 76, 893-910.
http://dx.doi.org/10.1037/0022-3514.76.6.893
Chartrand, T. L., & Lakin, J. L. (2013). The antecedents and consequences of human behavioral
mimicry. Annual Review of Psychology, 64, 285-308. http://dx.doi.org/10.1146/annurev-
psych-113011-143754
Clarke, T. J., Bradshaw, M. F., Field, D. T., Hampson, S. E., & Rose, D. (2005). The perception
of emotion from body movement in point-light displays of interpersonal dialogue.
Perception, 34, 1171-1180. http://dx.doi.org/10.1068/p5203
Condon, W. S., & Ogston, W. D. (1966). Sound film analysis of normal and pathological
behavior patterns. Journal of Nervous and Mental Disease, 143, 338-347.
http://dx.doi.org/10.1097/00005053-196610000-00005
Cornejo, C., Cuadros, Z., Morales, R., & Paredes, J. (2017). Interpersonal coordination:
Methods, achievements, and challenges. Frontiers in Psychology, 8, 1685.
http://dx.doi.org/10.3389/fpsyg.2017.01685
Cycling '74. (2019). Max 8. [computer software]. Retrieved from http://www.cycling74.com
Davis, M. (1972). Understanding body movement. New York, NY: Arno Press.
Davis, M., & Skuipen, J. (1982). Body movement and nonverbal communication. Bloomington,
IN: Indiana University Press.
de Gelder, B., Van den Stock, J., Meeren, H. K. M., Sinke, C. B. A., Kret, M. E., & Tamietto, M.
(2010). Standing up for the body. Recent progress in uncovering the networks involved in
the perception of bodies and bodily expressions. Neuroscience and Biobehavioral
Reviews, 34, 513-527. http://dx.doi.org/10.1016/j.neubiorev.2009.10.008

MOTION ENERGY ANALYSIS (MEA): A PRIMER
34
De Jaegher, H., & Di Paolo, E. (2007). Participatory sense-making. Phenomenology and the
Cognitive Sciences, 6, 485-507. http://dx.doi.org/10.1007/s11097-007-9076-9
Dean, D. J., Samson, A. T., Newberry, R., & Mittal, V. A. (2018). Motion energy analysis
reveals altered body movement in youth at risk for psychosis. Schizophrenia Research,
200, 35-41. http://dx.doi.org/10.1016/j.schres.2017.05.035
Delaherche, E., Chetouani, M., Mahdaoui, A., Saint-Georges, C., Viaux, S., & Cohen, D. (2012).
Interpersonal synchrony: A survey of evaluation methods across disciplines. IEEE
Transactions On Affective Computing, 3, 349-365. http://dx.doi.org/10.1109/T-
AFFC.2012.12
Flückiger, C., Del Re, A. C., Wampold, B. E., & Horvath, A. O. (2018). The alliance in adult
psychotherapy: A meta-analytic synthesis. Psychotherapy (Chicago, Ill.), 55, 316-340.
http://dx.doi.org/10.1037/pst0000172
Fretz, B. R. (1966). Postural movements in a counseling dyad. Journal of Counseling
Psychology, 13, 335-343. http://dx.doi.org/10.1037/h0023716
Fretz, B. R., Corn, R., Tuemmler, J. M., & Bellet, W. (1979). Counselor nonverbal behaviors and
client evaluations. Journal of Counseling Psychology, 26, 304-311.
http://dx.doi.org/10.1037/0022-0167.26.4.304
Fujiwara, K., Kimura, M., & Daibo, I. (2019). Gender differences in synchrony: Females in sync
during unstructured dyadic conversation. European Journal of Social Psychology. 49,
1042-1054. http://dx.doi.org/10.1002/ejsp.2587

MOTION ENERGY ANALYSIS (MEA): A PRIMER
35
Galbusera, L., Finn, M. T., & Fuchs, T. (2018). Interactional synchrony and negative symptoms:
An outcome study of body-oriented psychotherapy for schizophrenia. Psychotherapy
Research, 28, 457-469. http://dx.doi.org/10.1080/10503307.2016.1216624
Gaume, J., Hallgren, K. A., Clair, C., Schmid Mast, M., Carrard, V., & Atkins, D. C. (2019).
Modeling empathy as synchrony in clinician and patient vocally encoded emotional
arousal: A failure to replicate. Journal of Counseling Psychology 66, 341-350.
http://dx.doi.org/10.1037/cou0000322
Geerts, E., Bouhuys, N., & Van Den Hoofdakker, R. H. (1996). Nonverbal attunement between
depressed patients and an interviewer predicts subsequent improvement. Journal of
Affective Disorders, 40, 15-21. http://dx.doi.org/10.1016/0165-0327(96)00037-7
Gelo, O. C., & Salvatore, S. (2016). A dynamic systems approach to psychotherapy: A meta-
theoretical framework for explaining psychotherapy change processes. Journal of
Counseling Psychology, 63, 379-395. http://dx.doi.org/10.1037/cou0000150
Gottman, J., Swanson, C., & Swanson, K. (2002). A general systems theory of marriage:
Nonlinear difference equation modeling of marital interaction. Personality and Social
Psychology Review, 6, 326-340. http://dx.doi.org/10.1207/S15327957PSPR0604_07
Grafton, S. T. (2009). Embodied cognition and the simulation of action to understand others.
Annals of the New York Academy of Sciences, 1156, 97-117.
http://dx.doi.org/10.1111/j.1749-6632.2009.04425.x
Grammer, K., Honda, R., Schmitt, A., & Juette, A. (1999). Fuzziness of nonverbal courtship
communication unblurred by motion energy detection. Journal of Personality and Social
Psychology, 77, 487-508. http://dx.doi.org/10.1037/0022-3514.77.3.487

MOTION ENERGY ANALYSIS (MEA): A PRIMER
36
Grammer, K., Keki, V., Striebel, B., Atzmüller, M., & Fink, B. (2003). Bodies in motion: a
window to the soul. In E. Voland & K. Grammer (Eds.), Evolutionary aesthetics (pp.
295-324). Heidelberg, Germany: Springer.
Hall, J. A., Horgan, T. G., & Murphy, N. A. (2019). Nonverbal communication. Annual Review
of Psychology, 70, 271-294. http://dx.doi.org/10.1146/annurev-psych-010418-103145
Haase, R. F., & DiMattia, D. J. (1970). Proxemic behavior: Counselor, administrator, and client
preference for seating arrangement in dyadic interaction. Journal of Counseling
Psychology, 17, 319-325. http://dx.doi.org/10.1037/h0029662
Haase, R. F., & Tepper, D. T. (1972). Nonverbal components of empathic communication.
Journal of Counseling Psychology, 19, 417-424. http://dx.doi.org/10.1037/h0033188
Hayes, A. M., & Strauss, J. L. (1998). Dynamic systems theory as a paradigm for the study of
change in psychotherapy: An application to cognitive therapy for depression. Journal of
Consulting and Clinical Psychology, 66, 939-947. http://dx.doi.org/10.1037/0022-
006X.66.6.939
Hermansson, G. L., Webster, A. C., & McFarland, K. (1988). Counselor deliberate postural lean
and communication of facilitative conditions. Journal of Counseling Psychology, 35,
149-153. http://dx.doi.org/10.1037/0022-0167.35.2.149
Hill, C. E., & Knox, S. (2013). Training and supervision in psychotherapy. In M. J. Lambert
(Ed.), Handbook of Psychotherapy and Behavior Change (pp. 775-811). Hoboken, NJ:
Wiley.
Hill, C. E., Spiegel, S. B., Hoffman, M. A., Kivlighan Jr, D. M., & Gelso, C. J. (2017). Therapist
expertise in psychotherapy revisited. The Counseling Psychologist, 45, 7-53.
http://dx.doi.org/10.1177/0011000016641192

MOTION ENERGY ANALYSIS (MEA): A PRIMER
37
Imel, Z. E., Barco, J. S., Brown, H. J., Baucom, B. R., Baer, J. S., Kircher, J. C., & Atkins, D. C.
(2014). The association of therapist empathy and synchrony in vocally encoded arousal.
Journal of Counseling Psychology, 61, 146-153. http://dx.doi.org/10.1037/a0034943
Imel, Z. E., Caperton, D. D., Tanana, M., & Atkins, D. C. (2017). Technology-enhanced human
interaction in psychotherapy. Journal of Counseling Psychology, 64, 385-393.
http://dx.doi.org/10.1037/cou0000213
Ivey, A. E. (1973). The clinician as teacher of interpersonal skills. Let’s give away what we’ve
got. Massachusetts, MA: Massachusetts Univ., Amherst. Human Relations.
Ivey, A. E., Normington, C. J., Miller, C. D., Morrill, W. H., & Haase, R. F. (1968).
Microcounseling and attending behavior: An approach to prepracticum counselor
training. Journal of Counseling Psychology, 15, 1-12.
http://dx.doi.org/10.1037/h0026129
Jack, K. (2005). Video demystified: A handbook for the digital engineer (4th ed. ed. Vol.
Demystifying technology series). Amsterdam; The Netherlands: Elsevier.
Kiesler, D. J. (1979). An interpersonal communication analysis of relationship in psychotherapy.
Psychiatry, 42, 299-311. http://dx.doi.org/10.1080/00332747.1979.11024034
Kleinbub, J., R., & Ramseyer, F. (2018). rMEA: Synchrony in Motion Energy Analysis (MEA)
Time-Series. Retrieved from https://CRAN.R-project.org/package=rMEA
Kodama, K., Hori, K., Tanaka, S., & Matsui, H. (2018). How interpersonal coordination can
reflect psychological counseling: An exploratory study. Psychology, 9, 1128.
http://dx.doi.org/10.4236/psych.2018.95070

MOTION ENERGY ANALYSIS (MEA): A PRIMER
38
Koole, S. L., & Tschacher, W. (2016). Synchrony in psychotherapy: A review and an integrative
framework for the therapeutic alliance. Frontiers in Psychology, 7, 862.
http://dx.doi.org/10.3389/fpsyg.2016.00862
Komori, M., Maeda, K., & Nagaoka, C. (2007). A video-based quantification method of body
movement synchrony: An application for dialogue in counseling. Japanese Journal of
Interpersonal and Social Psychology, 7, 41-48.
Kupper, Z., Ramseyer, F., Hoffmann, H., Kalbermatten, S., & Tschacher, W. (2010). Video-
based quantification of body movement during social interaction indicates the severity of
negative symptoms in patients with schizophrenia. Schizophrenia Research, 121, 90-100.
http://dx.doi.org/10.1016/j.schres.2010.03.032
Kupper, Z., Ramseyer, F., Hoffmann, H., & Tschacher, W. (2015). Nonverbal synchrony in
social interactions of patients with schizophrenia indicates socio-communicative deficits.
Plos One, 10, e0145882. http://dx.doi.org/10.1371/journal.pone.0145882
Lambert, M. J. (Ed.). (2013). Bergin and Garfield’s handbook of psychotherapy and behavior
change. Hoboken, NJ: Wiley.
Lozza, N., Spoerri, C., Ehlert, U., Kesselring, M., Hubmann, P., Tschacher, W., & La Marca, R.
(2018). Nonverbal synchrony and complementarity in unacquainted same-sex dyads: A
comparison in a competitive context. Journal of Nonverbal Behavior, 42, 179-197.
http://dx.doi.org/10.1007/s10919-018-0273-8
Maurer, R. E., & Tindall, J. H. (1983). Effect of postural congruence on client’s perception of
counselor empathy. Journal of Counseling Psychology, 30, 158-163.
http://dx.doi.org/10.1037/0022-0167.30.2.158

MOTION ENERGY ANALYSIS (MEA): A PRIMER
39
Moulder, R. G., Boker, S. M., Ramseyer, F., & Tschacher, W. (2018). Determining synchrony
between behavioral time series: An application of surrogate data generation for
establishing falsifiable null-hypotheses. Psychological Methods, 23, 757-773.
http://dx.doi.org/10.1037/met0000172
Nelson, A. A., Grahe, J. E., & Ramseyer, F. (2016). Interacting in flow: An analysis of rapport-
based behavior as optimal experience. SAGE Open, 6, 1-11.
http://dx.doi.org/10.1177/2158244016684173
Paulick, J., Deisenhofer, A.-K., Ramseyer, F., Tschacher, W., Boyle, K., Rubel, J., & Lutz, W.
(2018a). Nonverbal synchrony: A new approach to better understand psychotherapeutic
processes and drop-out. Journal of Psychotherapy Integration, 28, 367-384.
http://dx.doi.org/10.1037/int0000099
Paulick, J., Rubel, J. A., Deisenhofer, A., Schwartz, B., Thielemann, D., Altmann, U., . . . Lutz,
W. (2018b). Diagnostic features of nonverbal synchrony in psychotherapy: Comparing
depression and anxiety. Cognitive Therapy and Research, 42, 539-551.
http://dx.doi.org/10.1007/s10608-018-9914-9
Paxton, A., & Dale, R. (2013a). Frame-differencing methods for measuring bodily synchrony in
conversation. Behavior Research Methods, 45, 329-343.
http://dx.doi.org/10.3758/s13428-012-0249-2
Paxton, A., & Dale, R. (2013b). Argument disrupts interpersonal synchrony. The Quarterly
Journal of Experimental Psychology, 66, 2092-2102.
http://dx.doi.org/10.1080/17470218.2013.853089
Pelletier, J-M. (2019). cv.jit. Computer vision for Jitter. [computer software]. Retrieved from:
https://jmpelletier.com/cvjit/

MOTION ENERGY ANALYSIS (MEA): A PRIMER
40
R Core Team (2008). R: A language and environment for statistical computing. Vienna, Austria:
R Foundation for Statistical computing. Retrieved from http://www.R-project.org
Ramseyer, F. T. (2019). Exploring nonverbal synchrony in psychotherapy dyads: The
idiographic perspective provides a different picture. Psychotherapy Research. Advance
online publication. http://dx.doi.org/10.1080/10503307.2019.1676932
Ramseyer, F., & Tschacher, W. (2006). Synchrony: A core concept for a constructivist approach
to psychotherapy. Constructivism in the Human Sciences, 11, 150-171.
Ramseyer, F., & Tschacher, W. (2008). Synchrony in dyadic psychotherapy sessions. In S.
Vrobel, O. E. Rössler, & T. Marks-Tarlow (Eds.), Simultaneity: Temporal Structures and
Observer Perspectives (pp. 329-347). Singapore: World Scientific.
http://dx.doi.org/10.1142/9789812792426_0020
Ramseyer, F., & Tschacher, W. (2010). Nonverbal synchrony or random coincidence? How to
tell the difference. In A. Esposito, N. Campbell, C. Vogel, A. Hussain, & A. Nijholt
(Eds.), Development of Multimodal Interfaces: Active Listening and Synchrony (pp. 182-
196). Berlin, Germany: Springer. http://dx.doi.org/10.1007/978-3-642-12397-9_15
Ramseyer, F., & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: Coordinated
body-movement reflects relationship quality and outcome. Journal of Consulting and
Clinical Psychology, 79, 284-295. http://dx.doi.org/10.1037/a0023419
Ramseyer, F., & Tschacher, W. (2014). Nonverbal synchrony of head-and body-movement in
psychotherapy: different signals have different associations with outcome. Frontiers in
Psychology, 5, 979. http://dx.doi.org/10.3389/fpsyg.2014.00979
Ramseyer, F., Ebert, A., Roser, P., Edel, M-A., Tschacher, W., & Brüne, M. (2019). Exploring
nonverbal synchrony in borderline personality disorder: A double-blind placebo-

MOTION ENERGY ANALYSIS (MEA): A PRIMER
41
controlled study using oxytocin. British Journal of Clinical Psychology. Advance online
publication. http://dx.doi.org/ 10.1111/bjc.12240
Reidsma, D., Nijholt, A., Tschacher, W., & Ramseyer, F. (2010). Measuring multimodal
synchrony for human-computer interaction. In A. Sourin (Ed.), 2010 International
Conference on Cyberworlds (pp. 67-71). IEEE Computer Society Press.
http://dx.doi.org/10.1109/CW.2010.21
Salvatore, S., & Tschacher, W. (2012). Time dependency of psychotherapeutic exchanges: the
contribution of the theory of dynamic systems in analyzing process. Frontiers in
Psychology, 3, 253. http://dx.doi.org/10.3389/fpsyg.2012.00253
Scheflen, A. E. (1965). Quasi-courtship behavior in psychotherapy. Psychiatry, 28, 245-257.
http://dx.doi.org/10.1080/00332747.1965.11023433
Schiepek, G. (2009). Complexity and nonlinear dynamics in psychotherapy. European Review,
17, 331-356. http://dx.doi.org/10.1017/S1062798709000763
Schmid Mast, M. (2007). On the importance of nonverbal communication in the physician-
patient interaction. Patient Education and Counseling, 67, 315-318.
http://dx.doi.org/10.1016/j.pec.2007.03.005
Schoenherr, D., Paulick, J., Strauss, B. M., Deisenhofer, A. K., Schwartz, B., Rubel, J.
A., . . . Altmann, U. (2019a). Nonverbal synchrony predicts premature termination of
psychotherapy for social anxiety disorder. Psychotherapy (Chicago, Ill.). Advance online
publication. http://dx.doi.org/10.1037/pst0000216
Schoenherr, D., Paulick, J., Strauss, B. M., Deisenhofer, A. K., Schwartz, B., Rubel, J.
A., . . . Altmann, U. (2019b). Identification of movement synchrony: Validation of

MOTION ENERGY ANALYSIS (MEA): A PRIMER
42
windowed cross-lagged correlation and -regression with peak-picking algorithm. PLoS
One, 14(2), e0211494. http://dx.doi.org/10.1371/journal.pone.0211494
Schoenherr, D., Paulick, J., Worrack, S., Strauss, B. M., Rubel, J. A., Schwartz, B., . . . Altmann,
U. (2019c). Quantification of nonverbal synchrony using linear time series analysis
methods: Lack of convergent validity and evidence for facets of synchrony. Behavior
Research Methods, 51, 361-383. http://dx.doi.org/10.3758/s13428-018-1139-z
Shockley, K., Richardson, D. C., & Dale, R. (2009). Conversation and coordinative structures.
Topics in Cognitive Science, 1, 305-319. http://dx.doi.org/10.1111/j.1756-
8765.2009.01021.x
Sonka, M., Hlavac, V., & Boyle, R. (2007). Image Processing, Analysis, and Machine Vision.
Toronto, Canada: Thomson Learning.
Steenbeek, H. W., & van Geert, P. L. C. (2007). A theory and dynamic model of dyadic
interaction: Concerns, appraisals, and contagiousness in a developmental context.
Developmental Review, 27, 1 - 40. http://dx.doi.org/10.1016/j.dr.2006.06.002
Strong, S.-R., Taylor, R.-G., Bratton, J.-C., & Loper, R.-G. (1971). Nonverbal behavior and
perceived counselor characteristics. Journal of Counseling Psychology, 18, 554-561.
http://dx.doi.org/10.1037/h0031754
Truax, C. B., Wargo, D. G., Frank, J. D., Imber, S. D., Battle, C. C., Hoehn-Saric, R., . . . Stone,
A. (1966). Therapist empathy, genuineness, and warmth and patient therapeutic outcome.
Journal of Consulting Psychology, 30, 395-401. http://dx.doi.org/10.1037/h0023827
Tschacher, W., & Haken, H. (2019). The process of psychotherapy. Berlin, Germany: Springer.

MOTION ENERGY ANALYSIS (MEA): A PRIMER
43
Tschacher, W., & Ramseyer, F. (2017). Nonverbal synchrony. In A. Wenzel (Ed.), The Sage
Encyclopedia of Abnormal and Clinical Psychology (p. 2298). Thousand Oaks, CA:
SAGE Publications. http://dx.doi.org/10.4135/9781483365817.n922
Tschacher, W., Rees, G. M., & Ramseyer, F. (2014). Nonverbal synchrony and affect in dyadic
interactions. Frontiers in Psychology, 5, 1323.
http://dx.doi.org/10.3389/fpsyg.2014.01323
Vallacher, R. R., & Nowak, A. (2009). The dynamics of human experience: Fundamentals of
dynamical social psychology. In S. J. Guastello, M. Koopmans, & D. Pincus (Eds.),
Chaos and Complexity in Psychology: The Theory of Nonlinear Dynamical Systems (pp.
370-401). New York, NY: Cambridge University Press.
Varlet, M., Marin, L., Capdevielle, D., Del-Monte, J., Schmidt, R., Salesse, R., . . . Raffard, S.
(2014). Difficulty leading interpersonal coordination: towards an embodied signature of
social anxiety disorder. Frontiers in Behavioral Neuroscience, 8, 29.
http://dx.doi.org/10.3389/fnbeh.2014.00029
Walther, S., Ramseyer, F., Horn, H., Strik, W., & Tschacher, W. (2014). Less structured
movement patterns predict severity of positive syndrome, excitement, and
disorganization. Schizophrenia Bulletin, 40, 585-591.
http://dx.doi.org/10.1093/schbul/sbt038
Watanabe, T. (1983). A study of motion-voice synchronization. Bulletin of JSME, 26, 2244-
2250. http://dx.doi.org/10.1299/jsme1958.26.2244