Table 1 - uploaded by Hannu Oja
Content may be subject to copyright.
Source publication
Deflation-based FastICA, where independent components (IC's) are extracted one-by-one, is among the most popular methods for estimating an unmixing matrix in the indepen-dent component analysis (ICA) model. In the literature, it is often seen rather as an algorithm than an estimator related to a certain objective function, and only recently has its...
Similar publications
The small-scale wave-like structure (SSWS) of F region bottomside plasma density was proposed to be an important seeding for equatorial plasma bubble (EPB) generation, and employed in theoretical simulations of EPBs in recent years. The seeding role of SSWS, however, is waiting to be demonstrated by observation. Here we present two cases of SSWS an...
Citations
... In the case of the deflation approach, the independent components are estimated successively one by one and after every iteration step, the Gram-Schmidt orthogonalization is performed. The major drawback of this approach is that estimation errors of the first vectors are cumulated in the subsequent ones by the orthogonalization [19]. The symmetric variant of the FastICA Borowicz EURASIP Journal on Advances in Signal Processing (2020) 2020: 39 Page 3 of 23 algorithm [20] estimates the components in parallel. ...
... We assume that the expectations in (17), (18) exist and are finite. Therefore, they can be approximated by sums, and the vector x in (19) can be replaced with an observation matrix X of size 4×m consisting of m mixture vectors stacked in column-wise order. The vectors (19), (20) can be replaced with transformed data matrices Y − k , Y + k ∈ R 4×m as well. ...
... Therefore, they can be approximated by sums, and the vector x in (19) can be replaced with an observation matrix X of size 4×m consisting of m mixture vectors stacked in column-wise order. The vectors (19), (20) can be replaced with transformed data matrices Y − k , Y + k ∈ R 4×m as well. Thus, by using the matrix-vector notation, we can redefine the contrast functions in (17) and (18) as follows: ...
Independent component analysis (ICA) is a popular technique for demixing multichannel data. The performance of a typical ICA algorithm strongly depends on the presence of additive noise, the actual distribution of source signals, and the estimated number of non-Gaussian components. Often, a linear mixing model is assumed and source signals are extracted by data whitening followed by a sequence of plane (Jacobi) rotations. In this article, we develop a novel algorithm, based on the quaternionic factorization of rotation matrices and the Newton-Raphson iterative scheme. Unlike conventional rotational techniques such as the JADE algorithm, our method exploits 4×4 rotation matrices and uses approximate negentropy as a contrast function. Consequently, the proposed method can be adjusted to a given data distribution (e.g., super-Gaussians) by selecting a suitable non-linear function that approximates the negentropy. Compared to the widely used, the symmetric FastICA algorithm, the proposed method does not require an orthogonalization step and is more accurate in the presence of multiple Gaussian sources.
... This filter was adapted from the default filters in the python package MNE (Gramfort et al., 2013, p. v 0.0.17.1). Eye movements were corrected using independent components analysis (ICA: Jung et al., 2000) with the deflation-based FastICA algorithm (Hyvärinen & Oja, 1997;Nordhausen, Ilmonen, Mandal, Oja, & Ollila, 2011;Miettinen, Nordhausen, Oja, & Taskinen, 2014). After this, we rejected segments containing a voltage difference of over 100 in a time window of 150 ms or containing a voltage step of over 50 / . ...
Several studies (e.g., Wicha et al., 2003; DeLong et al., 2005) have shown that readers use information from the sentential context to predict nouns (or some of their features), and that predictability effects can be inferred from the EEG signal in determiners or adjectives appearing before the predicted noun. While these findings provide evidence for the pre-activation proposal, recent replication attempts together with inconsistencies in the results from the literature cast doubt on the robustness of this phenomenon. Our study presents the first attempt to use the effect of gender on predictability in German to study the pre-activation hypothesis, capitalizing on the fact that all German nouns have a gender and that their preceding determiners can show an unambiguous gender marking when the noun phrase has accusative case. Despite having a relatively large sample size (of 120 subjects), both our preregistered and exploratory analyses failed to yield conclusive evidence for or against an effect of pre-activation. The sign of the effect is, however, in the expected direction: the more unexpected the gender of the determiner, the larger the negativity. The recent, inconclusive replication attempts by Nieuwland et al. (2018) and others also show effects with signs in the expected direction. We conducted a Bayesian random-effects meta-analysis using our data and the publicly available data from these recent replication attempts. Our meta-analysis shows a relatively clear but very small effect that is consistent with the pre-activation account and demonstrates a very important advantage of the Bayesian data analysis methodology: we can incrementally accumulate evidence to obtain increasingly precise estimates of the effect of interest.
... In the case of the deflation approach, the independent components are estimated successively one by one and after every iteration step, the Gram-Schmidt orthogonalization is performed. The major drawback of this approach is that estimation errors of the first vectors are cumulated in the subsequent ones by the orthogonalization [19]. The symmetric variant of the FastICA algorithm [20] estimates the components in parallel. ...
... We assume that the expectations in (17), (18) exist and are finite. Therefore, they can be approximated by sums, and the vector x in (19) can be replaced with an observation matrix X of size 4 × m consisting of m mixture vectors stacked in column-wise order. The vectors (19), (20) can be replaced with transformed data matrices Y − k , Y + k ∈ R 4×m as well. ...
... Therefore, they can be approximated by sums, and the vector x in (19) can be replaced with an observation matrix X of size 4 × m consisting of m mixture vectors stacked in column-wise order. The vectors (19), (20) can be replaced with transformed data matrices Y − k , Y + k ∈ R 4×m as well. Thus, by using the matrix-vector notation, we can redefine the contrast functions in (17) and (18) as follows: ...
Independent component analysis (ICA) is a popular technique for demixing multi-channel data. The performance of a typical ICA algorithm strongly depends on the presence of additive noise, the actual distribution of source signals, and the estimated number of non-Gaussian components. Often a linear mixing model is assumed and source signals are extracted by data whitening followed by a sequence of plane (Jacobi) rotations. In this article, we develop a novel algorithm, based on the quaternionic factorization of rotation matrices and the Newton-Raphson iterative scheme. Unlike conventional rotational techniques such as the JADE algorithm, our method exploits $4 \times 4$ rotation matrices and uses approximate negentropy as a contrast function. Consequently, the proposed method can be adjusted to a given data distribution (e.g. super-Gaussians) by selecting a suitable non-linear function that approximates the negentropy. Compared to the widely-used, the symmetric FastICA algorithm, the proposed method does not require an orthogonalization step and is more accurate in the presence of multiple Gaussian sources.
... A preliminary estimate of U (such as the FOBI discussed later in this review) can, however, be used to find the initial values that provide the correct extraction order with a high probability. See Hyvärinen and Oja (1997), Hyvärinen (1997), Ollila (2010), Nordhausen, Ilmonen, Mandal, Oja, andVirta (2017) for further details. ...
... Note that the above results also suggest the use of different G-functions for different components and the optimization of the order in which the components are extracted. For adaptive choices of G and the optimization of the extraction order inspired by these findings, see Koldovský and Tichavský (2015); Miettinen, Nordhausen, Oja, and Taskinen (2014a); Nordhausen, Ilmonen, et al. (2011), and references therein. ...
Independent component analysis (ICA) is a data analysis tool that can be seen as a refinement of principal component analysis or factor analysis. ICA recovers the structures in the data which stay hidden if only the covariance matrix is used in the analysis. The ICA problem is formulated as a latent variable model where the observed variables are linear combinations of unobserved mutually independent non‐Gaussian variables. The goal is to recover linear transformations back to these latent independent components (ICs). As a statistical tool, the unmixing procedure is expressed as a functional in a relevant semiparametric model which further allows a careful formulation of the inference problem and the comparison of competing estimation procedures. For most approaches, the ICs are found in two steps, (a) by standardizing the random vector and then (b) by rotating the standardized vector to the ICs. In the projection pursuit, the ICs can be found either one‐by‐one or simultaneously and this is discussed in detail when the convex combination of the squared third and fourth cumulants is used as a projection index. Alternative projection indices and their use are also explained. The classical fourth‐order blind identification (FOBI) and joint approximate diagonalization of eigenmatrices (JADE) are described as well. The statistical tools for the comparison of consistent and asymptotically multivariate normal unmixing matrix estimates are discussed. Finally, recent extensions for times series, matrix‐ and tensor‐valued and functional data are reviewed.
This article is categorized under:
• Statistical and Graphical Methods of Data Analysis > Modeling Methods and Algorithms
• Statistical Models > Multivariate Models
• Statistical and Graphical Methods of Data Analysis > Dimension Reduction
• Statistical and Graphical Methods of Data Analysis > Information Theoretic Methods
... ESTIMATORS The limiting variances and the asymptotic multinormality of the deflation-based and symmetric FastICA unmixing matrix estimators were found quite recently in [17], [15], [21] and [22]. In this section, we review these findings and derive the results for the squared symmetric FastICA estimator. ...
... The requirements for the function G and the conventional choices of it are discussed in Section II-E. The deflation-based FastICA functional Γ d satisfies the following p estimating equations [17], [15]: Definition 2. The deflation-based FastICA functional Γ d = (γ d 1 , . . . , γ d p ) T solves the estimating equations ...
... r, are implemented in R package fICA [11]. The extraction order of the components is highly important not only for the affine equivariance of the estimate, but also for its efficiency. In the deflationary approach, accurate estimation of the first components can be shown to have a direct impact on accurate estimation of the last components as well. [15] discussed the extraction order and the estimation efficiency and introduced the so-called reloaded deflation-based FastICA, where the extraction order is based on the minimization of the sum of the asymptotic variances, see Section III. [13] discussed the estimate that uses different G-functions for different components. Different versi ...
In this paper we study the theoretical properties of the deflation-based
FastICA method, the original symmetric FastICA method, and a modified symmetric
FastICA method, here called the squared symmetric FastICA. This modification is
obtained by replacing the absolute values in the FastICA objective function by
their squares. In the deflation-based case this replacement has no effect on
the estimate since the maximization problem stays the same. However, in the
symmetric case a novel estimate with unknown properties is obtained. In the
paper we review the classic deflation-based and symmetric FastICA approaches
and contrast these with the new squared symmetric version of FastICA. We find
the estimating equations and derive the asymptotical properties of the squared
symmetric FastICA estimator with an arbitrary choice of nonlinearity.
Asymptotic variances of the unmixing matrix estimates are then used to compare
their efficiencies for large sample sizes showing that the squared symmetric
FastICA estimator outperforms the other two estimators in a wide variety of
situations.
... The generalized symmetric FastICA algorithm with finite sample size consists of iterating (23) until convergence. We shall refer to algorithm (23) as the empirical FastICA. ...
... The method of estimating equation and M-estimator [30] is a powerful tool to solve problems of this kind, see [23]- [26] for some earlier results based on this method. ...
... Many researchers have studied the asymptotic behavior of FastICA [12]- [14], [16], [22], [23], [34], [35]. However, most of the work were dedicated to the one-unit version of the algorithm, which is much easier to deal with. ...
This contribution deals with the FastICA algorithm in the domain of Independent Component Analysis (ICA). The focus is on the asymptotic behavior of the generalized symmetric variant of the algorithm. The latter has already been shown to possess the potential to achieve the Cramér-Rao Bound (CRB) by allowing the usage of different nonlinearity functions in its implementation. Although the FastICA algorithm along with its variants are among the most extensively studied methods in the domain of ICA, a rigorous study of the asymptotic distribution of the generalized symmetric FastICA algorithm is still missing. In fact, all the existing results exhibit certain limitations. Some ignores the impact of data standardization on the asymptotic statistics; others are only based on heuristic arguments. In this work, we aim at deriving general and rigorous results on the limiting distribution and the asymptotic statistics of the FastICA algorithm. We begin by showing that the generalized symmetric FastICA optimizes a function that is a sum of the contrast functions of traditional one-unit FastICA with a correction of the sign. Based on this characterization, we established the asymptotic normality and derived a closed-form analytic expression of the asymptotic covariance matrix of the generalized symmetric FastICA estimator using the method of estimating equation and M-estimator. Computer simulations are also provided, which support the theoretical results.
... In engineering literature , these ICA methods have originally been formulated and considered as algorithms only and 465 therefore the rigorous analysis and comparison of theirstatistical properties have been missing until very recently. The statistical properties of the deflation-based FastICA method were derived in Ollila (2010) and Nordhausen et al. (2011a). The asymptotical behavior of FOBI estimate was considered in Ilmonen et al. (2010a). ...
... Using equation (2) in Nordhausen et al. (2011) and Slutsky's Theorem, the above equation re-600 duces to √ ...
In the independent component analysis it is assumed that the components of
the observed random vector are linear combinations of latent independent
components, and the aim is then to estimate the linear transformations back to
independent components. Traditional methods to find estimates of an unmixing
matrix in engineering literature such as FOBI (fourth order blind
identification), JADE (joint approximate diagonalization of eigenmatrices) and
FastICA are based on various uses of fourth moments but the statistical
properties of these estimates are not well known. This paper describes in
detail the independent component functionals based on fourth moments through
corresponding optimization problems, estimating equations and algorithms and,
for the first time, provides the full treatment and comparison of the limiting
statistical properties of these estimates. Wide efficiency studies indicate
that JADE and symmetric version of FastICA perform better than their
competitors in most cases and, in certain cases, provide asymptotically
equivalent estimates.
... Remark 5. Although the asymptotic error of the FastICA algorithm has already been studied by quite a few researchers [5,6,7,8], many of the results presented in this contribution, notably expressions (11)-(13) established in Theorem 1 and (19)-(22) in Theorem 3, are new. Example 1. ...
This contribution summarizes the results on the asymptotic performance of
several variants of the FastICA algorithm. A number of new closed-form
expressions are presented.
... Note also, that choosing either α = 0 or α = 1 makes the proposed method equivalent to the so-called deflation-based FastICA (Hyvärinen, 1999) with the projection indices |γ(u T k x st )| and |κ(u T k x st )|, respectively. For general results concerning deflation-based FastICA using absolute values see also Ollila (2010); Nordhausen et al. (2011); Miettinen et al. (2014a). The affine equivariance of the procedure given in Definition 4.1.1 ...
... wherê Γ k = 3αˆh3αˆ 3αˆh 3kˆT3kˆ 3kˆT 3k +4(1−α) ˆ h 4kˆT4kˆ 4kˆT 4k . Then, using Equation (5) from Nordhausen et al. (2011) we get the identity ...
... Next, using Equation (3) from Nordhausen et al. (2011) separately forˆTforˆ forˆT 3k andˆTandˆ andˆT 4k gives the following two identities. ...
The independent component model is a latent variable model where the
components of the observed random vector are linear combinations of latent
independent variables. The aim is to find an estimate for a transformation
matrix back to independent components. In moment-based approaches third
cumulants are often neglected in favor of fourth cumulants, even though both
approaches have similar appealing properties. This paper considers the joint
use of third and fourth cumulants in finding independent components. First,
univariate cumulants are used as projection indices in search for independent
components (projection pursuit). Second, multivariate cumulant matrices are
jointly used to solve the problem. The properties of the estimates are
considered in detail through corresponding optimization problems, estimating
equations, algorithms and asymptotic statistical properties. Comparisons of the
asymptotic variances of different estimates in wide independent component
models show that in most cases symmetric projection pursuit approach using both
third and fourth squared cumulants is a safe choice.
... Depending on where one starts from, the algorithm may stop at any critical point instead of the global maximum. It is then remarkable that extracting the sources in a different order changes the unmixing matrix estimate more than just the permutation [17], [15]. To be precise in our notation, we therefore write W (U , X; g) for the estimate that is provided by the FastICA algorithm for the data X with the initial value U init = U and the nonlinearity g. ...
... The statistical properties of the deflation-based FastICA estimator were rigorously discussed only recently in [15], [16] and [17]. Let X = (x 1 , . . . ...
... The following theorem extends Theorem 1 in [15], allowing different nonlinearity functions for different source components . The proof is similar to the proof in [15]. ...
Deflation-based FastICA is a popular method for independent component analysis. In the standard deflation-based approach the row vectors of the unmixing matrix are extracted one after another always using the same nonlinearities. In practice the user has to choose the nonlinearities and the efficiency and robustness of the estimation procedure then strongly depends on this choice as well as on the order in which the components are extracted. In this paper we propose a novel adaptive two-stage deflation-based FastICA algorithm that (i) allows one to use different nonlinearities for different components and (ii) optimizes the order in which the components are extracted. Based on a consistent preliminary unmixing matrix estimate and our theoretical results, the algorithm selects in an optimal way the order and the nonlinearities for each component from a finite set of candidates specified by the user. It is also shown that, for each component, the best possible nonlinearity is obtained by using the log-density function. The resulting ICA estimate is affine equivariant with a known asymptotic distribution. The excellent performance of the new procedure is shown with asymptotic efficiency and finite-sample simulation studies.
