Figure - available from: Frontiers in Psychology
This content is subject to copyright.
Source publication
Recent controversies about the level of replicability of behavioral research analyzed using statistical inference have cast interest in developing more efficient techniques for analyzing the results of psychological experiments. Here we claim that complementing the analytical workflow of psychological experiments with Machine Learning-based analysi...
Citations
... However, this raises a natural question: what are the advantages of ML compared to some professional statistical software such as IBM's SPSS? In other words, what is the difference between statistics and ML? Orrù et al. argued that statistical methods focus on whether the results are "real" effects or caused by noise (P value testing is usually the most commonly used method for this approach); the ML method treats data rulers as unknowns and focuses on predicting unobserved results or future behavior [51] . In other words, ML is more suitable for discovering patterns from data, while statistical methods are more suitable for verifying the discovered patterns. ...
This article reviews the psychological and neuroscience achievements in concept learning since 2010 from the perspectives of individual learning and social learning, and discusses several issues related to concept learning, including the assistance of machine learning about concept learning. In terms of individual learning, current evidence shows that the brain tends to process concrete concepts through typical features (shared features); and for abstract concepts, semantic processing is the most important cognitive way. In terms of social learning, interpersonal neural synchrony (INS) is considered the main indicator of efficient knowledge transfer (such as teaching activities between teachers and students), but this phenomenon only broadens the channels for concept sources and does not change the basic mode of individual concept learning. Ultimately, this article argues that the way the human brain processes concepts depends on the concept's own characteristics, so there are no "better" strategies in teaching, only more "suitable" strategies.
... Specifically, ML approaches focus on learning statistical functions from multidimensional data sets, including many potential predictors, and their linear and nonlinear interactions, to make generalizable predictions about individuals, providing crucial information about potential risk factors. Therefore, ML can produce meaningful, accurate, replicable, and generalizable models that can improve current theoretical models and be easily integrated into clinical care (Dwyer et al., 2018;Orrù et al., 2020;Yarkoni & Westfall, 2017 Note. When a model ID includes a letter (e.g., 33a, 33b), it represents that the data set included more than one PPU measure, and thus, all PPU measures were considered outcome variables in separate models. ...
This study suggests that the top five predictors of problematic pornography use (PPU) were frequency of
use, emotional avoidance pornography use motivation, stress reduction pornography use motivation, moral incongruence, and sexual shame. These findings provide empirically based key insights to develop effective, scientifically driven prevention and intervention programs for PPU that are currently absent from the literature and health care systems.
... Specifically, ML approaches focus on learning statistical functions from multidimensional data sets, including many potential predictors, and their linear and nonlinear interactions, to make generalizable predictions about individuals, providing crucial information about potential risk factors. Therefore, ML can produce meaningful, accurate, replicable, and generalizable models that can improve current theoretical models and be easily integrated into clinical care (Dwyer et al., 2018;Orrù et al., 2020;Yarkoni & Westfall, 2017 Note. When a model ID includes a letter (e.g., 33a, 33b), it represents that the data set included more than one PPU measure, and thus, all PPU measures were considered outcome variables in separate models. ...
Problematic pornography use (PPU) is the most common manifestation of the newly introduced compulsive sexual behavior disorder diagnosis in the 11th revision of the International Classification of Diseases. Research related to PPU has proliferated in the past two decades, but most prior studies were characterized by several shortcomings (e.g., using homogenous, small samples), resulting in crucial knowledge gaps and a limited understanding concerning empirically based risk factors for PPU. This study aimed to identify the most robust risk factors for PPU using a preregistered study design. Independent laboratories’ 74 preexisting self-report data sets (Nparticipants = 112,397; Ncountries = 16) were combined to identify which factors can best predict PPU using an artificial intelligence-based method (i.e., machine learning). We conducted random forest models on each data set to examine how different sociodemographic, psychological, and other characteristics predict PPU, and combined the results of all data sets using random-effects meta-analysis with meta-analytic moderators (e.g., community vs. treatment-seeking samples). Predictors explained 45.84% of the variance in PPU scores. Out of the 700+ potential predictors, 17 variables emerged as significant predictors across data sets, with the top five being (a) pornography use frequency, (b) emotional avoidance pornography use motivation, (c) stress reduction pornography use motivation, (d) moral incongruence toward pornography use, and (e) sexual shame. This study is the largest and most integrative data analytic effort in the field to date. Findings contribute to a better understanding of PPU’s etiology and may provide deeper insights for developing more efficient, cost-effective, empirically based directions for future research as well as prevention and intervention programs targeting PPU.
... Machine learning algorithms have seen extensive usage in the health and medical fields, but have seen far less adoption in the psychology and behavioral sciences. Researchers in the field of psychological analysis are increasingly gravitating toward the use of machine learning from statistical inferences [14]. Consequently, it is frequently utilized as a strong approach for sorting through enormous volumes of healthcare data [15]. ...
In contemporary society, depression has emerged as a prominent mental disorder that exhibits exponential growth and exerts a substantial influence on premature mortality. Although numerous research applied machine learning methods to forecast signs of depression. Nevertheless, only a limited number of research have taken into account the severity level as a multiclass variable. Besides, maintaining the equality of data distribution among all the classes rarely happens in practical communities. So, the inevitable class imbalance for multiple variables is considered a substantial challenge in this domain. Furthermore, this research emphasizes the significance of addressing class imbalance issues in the context of multiple classes. We introduced a new approach Feature group partitioning (FGP) in the data preprocessing phase which effectively reduces the dimensionality of features to a minimum. This study utilized synthetic oversampling techniques, specifically Synthetic Minority Over-sampling Technique (SMOTE) and Adaptive Synthetic (ADASYN), for class balancing. The dataset used in this research was collected from university students by administering the Burn Depression Checklist (BDC). For methodological modifications, we implemented heterogeneous ensemble learning stacking, homogeneous ensemble bagging, and five distinct supervised machine learning algorithms. The issue of overfitting was mitigated by evaluating the accuracy of the training, validation, and testing datasets. To justify the effectiveness of the prediction models, balanced accuracy, sensitivity, specificity, precision, and f1-score indices are used. Overall, comprehensive analysis demonstrates the discrimination between the Conventional Depression Screening (CDS) and FGP approach. In summary, the results show that the stacking classifier for FGP with SMOTE approach yields the highest balanced accuracy, with a rate of 92.81%. The empirical evidence has demonstrated that the FGP approach, when combined with the SMOTE, able to produce better performance in predicting the severity of depression. Most importantly the optimization of the training time of the FGP approach for all of the classifiers is a significant achievement of this research.
... The six features were entered into three machine-learning algorithms: logistic regression, KNN, and random forest classifier. We chose multiple algorithms to prevent the selection of only the highest-performing model by chance, checking whether there is substantial variation in classification accuracy across various classifiers 80 . Moreover, we chose these specific algorithms to test the robustness of our classification performance because they are based on various underlying assumptions. ...
... Leveraging machine-learning models allows for prediction of individual behaviors rather than generalization of a group's collective behavior 81 . To ensure the out-of-sample generalization, the standard K-fold cross-validation technique (K = 10) was employed 80 . This technique is critical, mainly when one works with limited data samples that require out-of-sample accuracy. ...
... Traditional statistical methods often focus on group-level variations and may lack the precision required for individual-case analysis. In contrast, machine-learning models offer a robust framework for detailed assessments of an individual's likelihood of being truthful or deceptive-a crucial aspect in the forensic domain, in which individual accuracy at the single-subject level is required 80 . ...
In this study, we propose an approach to detect deception during investigative interviews by integrating response latency and error analysis with the unexpected question technique. Sixty participants were assigned to an honest (n = 30) or deceptive group (n = 30). The deceptive group was instructed to memorize the false biographical details of a fictitious identity. Throughout the interviews, participants were presented with a randomized sequence of control, expected, and unexpected open-ended questions about identity. Responses were audio recorded for detailed examination. Our findings indicate that deceptive participants showed markedly longer latencies and higher error rates when answering expected (requiring deception) and unexpected questions (for which premeditated deception was not possible). Longer response latencies were also observed in participants attempting deception when answering control questions (which necessitated truthful answers). Moreover, a within-subject analysis highlighted that responding to unexpected questions significantly impaired individuals’ performance compared to answering control and expected questions. Leveraging machine-learning algorithms, our approach attained a classification accuracy of 98% in distinguishing deceptive and honest participants. Additionally, a classification analysis on single response levels was conducted. Our findings underscore the effectiveness of merging response latency metrics and error rates with unexpected questioning as a robust method for identity deception detection in investigative interviews. We also discuss significant implications for enhancing interview strategies.
... Machine learning has been used to automate parts of scientific research, allowing for increasingly larger datasets to be used and analyzed (Rudin and Wagstaff 2014). The use of machine learning in psychology is still relatively new but offers intriguing possibilities (Dwyer et al. 2018;Orrù et al. 2020;Yarkoni and Westfall 2017). There are several ways in which artificial intelligence can be used in cognitive testing, and experts in the fields of cognitive science, computer science, and engineering are currently pioneering these avenues. ...
The block design test (BDT) has been used for over a century in research and clinical contexts as a measure of spatial cognition, both as a singular ability and as part of more comprehensive intelligence assessment. Traditionally, the BDT has been scored using methods that do not reflect the full potential of individual differences that could be measured by the test. Recent advancements in technology, including eye-tracking, embedded sensor systems, and artificial intelligence, have provided new opportunities to measure and analyze data from the BDT. In this methodological review, we outline the information that BDT can assess, review several recent advancements in measurement and analytic methods, discuss potential future uses of these methods, and advocate for further research using these methods.
... Una novedad de este estudio refiere a la utilización de algoritmos de machine learning (aprendizaje automático). Desde hace algunos años estos algoritmos se han popularizado en psicología, especialmente con propósitos predictivos (Yarkoni & Westfall, 2017), ya que permiten descubrir patrones y desarrollar modelos predictivos en una variedad de circunstancias y campos de aplicación de la psicología, tales como la psicometría, la psicología experimental, el diagnóstico, el tratamiento, el seguimiento y la atención personalizada de los pacientes (Dey, 2016;Dhall et al., 2020;Dwyer et al, 2018;Jacobucci & Grimm, 2020;Koul et al., 2018;Lin et al., 2020;Orrù et al., 2020;Shatte et al., 2019). Por ejemplo, algunas investigaciones que usaron algoritmos de aprendizaje automático permitieron identificar rasgos de personalidad a través de posteos en redes sociales (Bleidorn & Hopwood, 2019;Park et al., 2015), o de la música preferida según los likes en Facebook (Nave et al., 2018). ...
El objetivo de este estudio fue verificar un modelo predictivo de rasgos de personalidad positivos y negativos tomando como criterio el bienestar psicológico mediante la implementación de algoritmos de machine learning. Participaron 2038 sujetos adultos (51.9 % mujeres). Para la recolección de datos se utilizó: Big Five Inventory y Mental Health Continuum-Short Form. Además, para evaluar los rasgos positivos y negativos de personalidad se utilizaron los ítems ya validados de los modelos de rasgos positivos (HFM) y negativos (BAM) de forma conjunta. A partir de los hallazgos encontrados se pudo verificar que la eficacia predictiva del modelo testeado de rasgos positivos y negativos derivados de un enfoque léxico resultó superior a la capacidad predictiva de los rasgos normales de personalidad para la predicción del bienestar.
... However, others have suggested that selecting items to retain for an abbreviated measure could be conceptualized as a feature selection task, which differentiates itself from factor analytic strategies by focusing on the impact each item has on model predictions (Gonzalez, 2020;Kuhn & Johnson, 2013). Defining the importance of an item as its contribution to model predictions, rather than its reliability as a measure of a latent construct, lends itself to machine learning approaches which aim to predict relevant outcomes rather than to make statistical inferences (Orrù et al., 2020). Shapley Additive Explanations (SHAP) values are a measure of feature importance that describe the impact of each feature (e.g., item) on model output. ...
... For example, younger respondents are known to have different patterns of response bias than older participants, possibly related to differences in social desirability (Kozma & Stones, 1988;Soubelet & Salthouse, 2011). Another concern is that the choice of algorithm used in machine learning is justified by its performance (Orrù et al., 2020), though its interpretability may be limited. In the present study, multilayer neural network models outperformed logistic regression and support vector machine models in terms of classification accuracy, but neural network models, particularly those with multiple layers, are more difficult to interpret in regard to how each item leads to a particular prediction, than logistic regression or support vector machine models. ...
Creating abbreviated measures from lengthy questionnaires is important for reducing respondent burden while improving response quality. Though factor analytic strategies have been used to guide item retention for abbreviated questionnaires, item retention can be conceptualized as a feature selection task amenable to machine learning approaches. The present study tested a machine learning-guided approach to item retention, specifically item-level importance as measured by Shapley values for the prediction of total score, to create abbreviated versions of the Penn State Worry Questionnaire (PSWQ) in a sample of 3,906 secondary school students. Results showed that Shapley values were a useful measure for determining item retention in creating abbreviated versions of the PSWQ, demonstrating concordance with the full PSWQ. As item-level importance varied based on the proportion of the worry distribution predicted (e.g., high versus low PSWQ scores), item retention is dependent on the intended purpose of the abbreviated measure. Illustrative examples are presented.
... To address the limitations of the statistical procedures that are commonly used in empirical research, neuroscience and experimental psychology have started to pay great attention to machine learning (ML) approaches (Orrù, Monaro, Conversano, Gemignani, & Sartori, 2020;Glaser, Benjamin, Farhoodi, & Kording, 2019). When carefully applied, ML techniques improve the generalizability and robustness of results (Scheinost et al., 2019;Woo, Chang, Lindquist, & Wager, 2017). ...
Changes in error processing are observable in a range of anxiety-related disorders. Numerous studies, however, have reported contradictory and nonreplicating findings, thus the exact mapping of brain response to errors (i.e., error-related negativity [ERN]; error-related positivity, Pe) onto specific anxiety symptoms remains unclear. In this study, we collected 16 self-reported scores of anxiety dimensions and obtained spatial features of EEG recordings from 171 individuals. We then used machine learning to (1) identify symptoms that are central for elevated ERN/Pe and (2) estimate the generalizability of traditional statistical approaches. ERN was associated with rumination, threat overestimation, and inhibitory intolerance of uncertainty. Pe was associated with rumination, prospective intolerance of uncertainty, and behavioral inhibition. Our findings emphasize that not only the amplitude of ERN but also other sources of brain signal variance encode information relevant to individual differences in error processing. The results of the generalizability check reveal the need for a change in result-validation methods to move toward robust findings that reflect stable individual differences and clinically useful biomarkers. Our study benefits from the use of machine learning to improve the generalizability of results.
... Machine learning is an umbrella term for methods that learn patterns from data through automated model build" (Van Lissa, 2023, p. 2). Although ML methods are still a novelty in psychological research, its use is growing fast in all areas of psychological research (Orrù et al., 2020), as educational psychology (Levy et al., 2020;Luan and Tsai, 2021), clinical psychology (Dwyer et al., 2018), and social psychology (Kumar et al., 2019). Because these techniques used here are still not very widely known, they are shortly described in a foot note 2 . ...
The aim of the present study was to assess the impact of past traumatic war experiences on preadolescents in the Gaza Strip, which could be useful for psychological intervention with current and future child victims. Participants were 521 preadolescents from United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA) schools, aged 11 and 13 Years old. Sections I to IV from Iraqi Version-Arabic of Harvard Trauma Questionnaire was used to assess trauma experiences and Post-Traumatic Stress Disorder (PTSD). The results show that the preadolescents in the Gaza Strip witnessed the destruction of their homes and the murder of family members and friends. A quarter of the individuals assessed either suffered torture or witnessed others undergoing it, including sexual assaults. Almost half of them experienced a lack of food and clean water. The traumatic and torture experiences seriously affected preadolescents' mental health as 26.29% met criteria for the diagnosis of PTSD. The data analysis revealed two PTSD modalities, with the severity of impact depending on whether social implications were involved. Further research is required to check whether these two modalities fit to PTSD and complex PTSD. Understanding the effects of past wars on preadolescents in Gaza and distinguishing between different PTSD types could enhance comprehension of the impacts of current attacks on child victims. It can also aid in determining the type of intervention needed to minimize the impact on the mental health of Palestinian youth, enhancing their resilience through psychological and social support.
