Figure 1 - available via license: CC BY
Content may be subject to copyright.
Example time courses of the different types of noise that are generated by fmrisim. Each plot represents a voxel's activity for each type of noise. Full-size DOI: 10.7717/peerj.8564/fig-1
Source publication
With advances in methods for collecting and analyzing fMRI data, there is a concurrent need to understand how to reliably evaluate and optimally use these methods. Simulations of fMRI data can aid in both the evaluation of complex designs and the analysis of data. We present fmrisim, a new Python package for standardized, realistic simulation of fM...
Context in source publication
Similar publications
We present a novel Python package for the uncertainty and sensitivity analysis of computational models. The mathematical background is based on the non-intrusive generalized polynomial chaos method allowing one to treat the investigated models as black box systems, without interfering with their legacy code. Pygpc is optimized to analyze models wit...
We introduce an open source python framework named PHS - Parallel Hyperparameter Search to enable hyperparameter optimization on numerous compute instances of any arbitrary python function. This is achieved with minimal modifications inside the target function. Possible applications appear in expensive to evaluate numerical computations which stron...
Citations
... To generate more realistic simulations, we employed the fmrisim module of the BrainIAK package [22,16]. We firstly extract noise parameters from one participant's fMRI data provided by DataSpace (http://arks.princeton.edu/ ...
Functional magnetic resonance imaging (fMRI) is widely used in neuroscience research. FMRI data is noisy; improving denoising methods could lead to novel discoveries. Here, we introduce and evaluate a denoising method (DeepCor) which utilizes deep generative models to disentangle and remove noise. DeepCor outperforms CompCor (a state-of-the art denoising approach) on a variety of simulated datasets. In addition, DeepCor enhances differences in connectivity between brain networks in real datasets.
... It is made We extended this synthetic dataset by adding noise and downsampling to test the Mapper algorithm under higher noise regimes and with lower temporal resolution acquisitions. For adding noise to our signal, we first generated white noise using the 'brainiak' package (Ellis et al. 2020) based on the extracted properties of the input dataset: drift noise, autoregressive/moving-average noise, and system noise. Then, based on the desired Signal-To-Noise ratio (SNR), we scaled and added the generated noise to each voxel activation of the dataset. ...
Capturing and tracking large-scale brain activity dynamics holds the potential to understand cognition. Previously, tools from Topological Data Analysis, especially Mapper, have been successfully used to mine brain activity dynamics at the highest spatiotemporal resolutions. Even though it is a relatively established tool within the field of Topological Data Analysis, Mapper results are highly impacted by parameter selection. Given that non-invasive human neuroimaging data (e.g., from fMRI) is typically fraught with artifacts and no gold standards exist regarding 'true' state transitions, we argue for a thorough examination of Mapper parameter choices to reveal their impact better. Using synthetic data (with known transition structure) and real fMRI data, we explore a variety of parameter choices for each Mapper step, thereby providing guidance and heuristics for the field. We also release our parameter-exploration toolbox as a software package to make it easier for scientists to investigate and apply Mapper on any dataset.
... Consequently, an experimenter may (incorrectly) conclude that a brain region that is well predicted by the downsampled features is selective for the slowly varying information it captures like discourse, when, in fact, the brain region merely responds to the rate of words. In other cases, it may be important to model several different sources of noise, which has been pursued in other work simulating fMRI data (Ellis et al., 2020). Current neuroimaging modalities also have low signalto-noise ratio, limiting the predictive performance of computational models. ...
Language neuroscience currently relies on two major experimental paradigms: controlled experiments using carefully hand-designed stimuli, and natural stimulus experiments. These approaches have complementary advantages which allow them to address distinct aspects of the neurobiology of language, but each approach also comes with drawbacks. Here we discuss a third paradigm—in silico experimentation using deep learning-based encoding models—that has been enabled by recent advances in cognitive computational neuroscience. This paradigm promises to combine the interpretability of controlled experiments with the generalizability and broad scope of natural stimulus experiments. We show four examples of simulating language neuroscience experiments in silico and then discuss both the advantages and caveats of this approach.
... The process of simulating fMRI is non-trivial. There are packages to simulate fMRI data in python Ellis et al. (2020), MATLAB Erhardt et al. (2012), and R Welvaert et al. (2011); each of these packages focus on generating the functional time series for a given voxel in the brain. Here, we are interested in the analysis of the functional connectivity matrix, and so propose a novel simulation method by constructing underlying functional connectivity matrices from real data and then perturbing them with known, implanted signals. ...
In this paper, we propose the use of geodesic distances in conjunction with multivariate distance matrix regression, called geometric-MDMR, as a powerful first step analysis method for manifold-valued data. Manifold-valued data is appearing more frequently in the literature from analyses of earthquake to analysing brain patterns. Accounting for the structure of this data increases the complexity of your analysis, but allows for much more interpretable results in terms of the data. To test geometric-MDMR, we develop a method to simulate functional connectivity matrices for fMRI data to perform a simulation study, which shows that our method outperforms the current standards in fMRI analysis.
... BrainIAK also includes a detailed set of tutorials [22] that are didactic in nature; the tutorials include very detailed steps and helper functions that facilitate learning and implementing some of the methods, including materials relevant to running on HPC clusters. Scientists can also use BrainIAK's simulator [23] to create model-based patterns of activity at the voxel level, without going through the expensive and time-consuming process of data collection. The package is released with an open-source license and is free to use on a variety of platforms. ...
... Instead, the notebooks we provide here are integrated with the BrainIAK documentation, provide an overview of the technique, and allow users to quickly access code snippets for the method. Also, the notebooks include methods that are not covered in the tutorials such as Bayesian RSA [14,15], TFA [17], IEMs [18,19], BrainIAK's simulator [23], and matrix-normal models [24]. All example notebooks are available at https://github.com/brainiak/brainiak-aperture, ...
... To fill in this gap, we developed fmrisim [23], an opensource Python package for simulating realistic fMRI data. fmrisim linearly combines a number of noise sources, inspired by biology and MRI physics, which are tuned in a data-driven fashion to match specific fMRI data that is provided as an input. ...
Functional magnetic resonance imaging (fMRI) offers a rich source of data for studying the neural basis of cognition. Here, we describe the Brain Imaging Analysis Kit (BrainIAK), an open-source, free Python package that provides computationally optimized solutions to key problems in advanced fMRI analysis. A variety of techniques are presently included in BrainIAK: intersubject correlation (ISC) and intersubject functional connectivity (ISFC), functional alignment via the shared response model (SRM), full correlation matrix analysis (FCMA), a Bayesian version of representational similarity analysis (BRSA), event segmentation using hidden Markov models, topographic factor analysis (TFA), inverted encoding models (IEMs), an fMRI data simulator that uses noise characteristics from real data (fmrisim), and some emerging methods. These techniques have been optimized to leverage the efficiencies of high-performance compute (HPC) clusters, and the same code can be seamlessly transferred from a laptop to a cluster. For each of the aforementioned techniques, we describe the data analysis problem that the technique is meant to solve and how it solves that problem; we also include an example Jupyter notebook for each technique and an annotated bibliography of papers that have used and/or described that technique. In addition to the sections describing various analysis techniques in BrainIAK, we have included sections describing the future applications of BrainIAK to real-time fMRI, tutorials that we have developed and shared online to facilitate learning the techniques in BrainIAK, computational innovations in BrainIAK, and how to contribute to BrainIAK. We hope that this manuscript helps readers to understand how BrainIAK might be useful in their research.
... BrainIAK also includes a detailed set of tutorials [22] that help the novice or expert user learn and implement the methods, including materials relevant to running on HPC clusters. Scientists can also use BrainIAK's simulator [23] 4 to create model based patterns of activity at the voxel level, without going through the expensive and time consuming process of data collection. The package is released with an open source license and is free to use on a variety of platforms. ...
... To fill in this gap, we developed fmrisim [23], an open-source Python package for simulating realistic fMRI data. fmrisim linearly combines a number of noise sources, inspired by biology and MRI physics, that are tuned in a data-driven fashion to match specific fMRI data that is provided as an input. ...
... Through an iterative fitting procedure, the noise properties of the simulation are updated to optimize the match of the simulated data to the real data ( Figure 8). We previously validated that this fitting procedure produces accurate simulations of real data [23]. We have used fmrisim to evaluate the power of different experimental design parameters [23], and also to evaluate the efficacy of new analysis methods [78,79]. ...
Functional magnetic resonance imaging (fMRI) offers a rich source of data for studying the neural basis of cognition. Here, we describe the Brain Imaging Analysis Kit (BrainIAK), an open-source, free Python package that provides computationally-optimized solutions to key problems in advanced fMRI analysis. A variety of techniques are presently included in BrainIAK: intersubject correlation (ISC) and intersubject functional connectivity (ISFC), functional alignment via the shared response model (SRM), full correlation matrix analysis (FCMA), a Bayesian version of representational similarity analysis (BRSA), event segmentation using hidden Markov models, topographic factor analysis (TFA), inverted encoding models (IEM), an fMRI data simulator that uses noise characteristics from real data (fmrisim), and some emerging methods. These techniques have been optimized to leverage the efficiencies of high performance compute (HPC) clusters, and the same code can be seamlessly transferred from a laptop to a cluster. For each of the aforementioned techniques , we describe the data analysis problem that the technique is meant to solve, and how it solves that problem; we also include an example Jupyter notebook for each technique and an annotated bibliography of papers that have used and/or described that technique. In addition to the sections describing various analysis techniques in BrainIAK, we have included sections describing the future applications of BrainIAK to real-time fMRI, tutorials that we have developed and shared online to facilitate learning the techniques in BrainIAK, computational innovations in BrainIAK, and how to contribute to BrainIAK. We hope that this manuscript helps readers to understand how BrainIAK might be useful in their research.
... To confirm that running a searchlight in functional space can capture distributed information throughout the brain, we performed the RSA described above on simulated data with a known spatial signal distribution. In particular, we simulated fMRI data [19] where the signal from the units in the first fully connected layer of AlexNet was inserted into random voxels with varying degrees of spatial smoothness. Our results indicated that the relative improvement afforded by the functional searchlight is best when the signal is highly distributed throughout the brain (Fig 4). ...
... We evaluated this interpretation by simulating fMRI data that varied in how broadly information was distributed across the brain and repeating the AlexNet analysis in the 'Sherlock' experiment. The simulation was performed using the fmrisim package in BrainIAK [19]. In particular, we simulated realistic fMRI noise from participant templates [28] and chose a set of voxels in the simulated brain to insert signal. ...
The extent to which brain functions are localized or distributed is a foundational question in neuroscience. In the human brain, common fMRI methods such as cluster correction, atlas parcellation, and anatomical searchlight are biased by design toward finding localized representations. Here we introduce the functional searchlight approach as an alternative to anatomical searchlight analysis, the most commonly used exploratory multivariate fMRI technique. Functional searchlight removes any anatomical bias by grouping voxels based only on functional similarity and ignoring anatomical proximity. We report evidence that visual and auditory features from deep neural networks and semantic features from a natural language processing model, as well as object representations, are more widely distributed across the brain than previously acknowledged and that functional searchlight can improve model-based similarity and decoding accuracy. This approach provides a new way to evaluate and constrain computational models with brain activity and pushes our understanding of human brain function further along the spectrum from strict modularity toward distributed representation.
... Finally, it is also possible that other replacement methods could enhance the quality and usability of synthetic data, while limiting privacy disclosure. For example, reality-based synthetic fMRI data have been used to facilitate power analyses and experiment design (Ellis et al., 2020). Deep learning models have also been developed to create synthetic brain images that appear realistic (Bermudez et al., 2018;Calimeri et al., 2017) to augment training datasets for machine learning methods to perform diagnostic tasks more accurately (Shorten and Khoshgoftaar, 2019). ...
Scientific transparency, data exploration, and education are advanced through data sharing. However, risk for disclosure of personal information and institutional data sharing regulations can impede human subject/patient data sharing and thus limit open science initiatives. Sharing fully synthetic data is an alternative when it is not possible to share real or observed data. Here we describe a data sharing approach that borrows principles and methods from multiple imputation to replace observed values with synthetic values, thereby creating fully synthetic neuroimaging data that accurately represents the covariance structure of the observed dataset. Predictor tables composed of demographic, site, behavioral and total intracranial volume (ICV) variables from 264 pediatric cases were used to create synthetic predictor tables, which were then used to synthesize gray matter images derived from T1-weighted data. The synthetic predictor tables demonstrated pooled variance and statistical estimates that closely approximated the observed data, as reflected in measures of efficiency and statistical bias. Similarly, the synthetic gray matter data accurately represented the variance and voxel-level associations with predictor variables (age, sex, verbal IQ, and ICV). The magnitude and spatial distribution of gray matter effects in the observed imaging data were replicated in the pooled results from the synthetic datasets. This approach for generating fully synthetic neuroimaging data has widespread potential for data sharing, including replication, new discovery, and education. Fully synthetic MRI data can enable data-sharing because it accurately represents patterns of variance in the original data, while diminishing the risk of privacy disclosures that can accompany neuroimaging data sharing.
... As well as helping ensure that researchers follow their pre-specified analysis plan, Registered Reports also reduce publication bias as results are published regardless of statistical significance. While the pre-registration of detailed analysis plans for neuroimaging studies is less straightforward, recent recommendations aim at increasing the transparency and reproducibility of neuroimaging research [110] and tools have recently emerged to assist this process, such as pre-registration templates [111] and fMRI data simulation [112]. Second, future clinical studies should aim at recruiting both male and female participants to improve generalizability. ...
Reports on the modulatory role of the neuropeptide oxytocin on social cognition and behavior have steadily increased over the last two decades, stimulating considerable interest in its psychiatric application. Basic and clinical research in humans primarily employs intranasal application protocols. This approach assumes that intranasal administration increases oxytocin levels in the central nervous system via a direct nose-to-brain route, which in turn acts upon centrally-located oxytocin receptors to exert its behavioral effects. However, debates have emerged on whether intranasally administered oxytocin enters the brain via the nose-to-brain route and whether this route leads to functionally relevant increases in central oxytocin levels. In this review we outline recent advances from human and animal research that provide converging evidence for functionally relevant effects of the intranasal oxytocin administration route, suggesting that direct nose-to-brain delivery underlies the behavioral effects of oxytocin on social cognition and behavior. Moreover, advances in previously debated methodological issues, such as pre-registration, reproducibility, statistical power, interpretation of non-significant results, dosage, and sex differences are discussed and integrated with suggestions for the next steps in translating intranasal oxytocin into psychiatric applications.
... In addition, investigators have implemented an important part of fMRI validation: software to synthesize realistic fMRI responses data, some for generic task activations [10][11][12][13], others more specifically targeted, for example in studies of scotomas [14,15], the impact of eye movements [16], or temporal integration [17]. See here for a review of functional MRI simulations [18]. ...
Neuroimaging software methods are complex, making it a near certainty that some implementations will contain errors. Modern computational techniques (i.e., public code and data repositories, continuous integration, containerization) enable the reproducibility of the analyses and reduce coding errors, but they do not guarantee the scientific validity of the results. It is difficult, nay impossible, for researchers to check the accuracy of software by reading the source code; ground truth test datasets are needed. Computational reproducibility means providing software so that for the same input anyone obtains the same result, right or wrong. Computational validity means obtaining the right result for the ground-truth test data. We describe a framework for validating and sharing software implementations, and we illustrate its usage with an example application: population receptive field (pRF) methods for functional MRI data. The framework is composed of three main components implemented with containerization methods to guarantee computational reproducibility. In our example pRF application, those components are: (1) synthesis of fMRI time series from ground-truth pRF parameters, (2) implementation of four public pRF analysis tools and standardization of inputs and outputs, and (3) report creation to compare the results with the ground truth parameters. The framework was useful in identifying realistic conditions that lead to imperfect parameter recovery in all four pRF implementations, that would remain undetected using classic validation methods. We provide means to mitigate these problems in future experiments. A computational validation framework supports scientific rigor and creativity, as opposed to the oft-repeated suggestion that investigators rely upon a few agreed upon packages. We hope that the framework will be helpful to validate other critical neuroimaging algorithms, as having a validation framework helps (1) developers to build new software, (2) research scientists to verify the software’s accuracy, and (3) reviewers to evaluate the methods used in publications and grants.
