Content uploaded by Peter Benner
Author content
All content in this area was uploaded by Peter Benner on Nov 21, 2015
Content may be subject to copyright.
A preview of the PDF is not available
Parametric Model Order Reduction Accelerated by Subspace Recycling
Abstract and Figures
Many model order reduction methods for parameterized systems need to construct a projection matrix V which requires computing several moment matrices of the parameterized systems. For computing each moment matrix, the solution of a linear system with multiple right-hand sides is required. Furthermore, the number of linear systems increases with both the number of moment matrices used and the number of parameters in the system. Usually, a considerable number of linear systems has to be solved when the system includes more than two parameters. The standard way of solving these linear systems in case sparse direct solvers are not feasible is to use conventional iterative methods such as GMRES or CG. In this paper, a fast recycling algorithm is applied to solve the whole sequence of linear systems and is shown to be much more efficient than the standard iterative solver GMRES as well as the newly proposed recycling method MKR-GMRES from. As a result, the computation of the reduced-order model can be significantly accelerated.
Figures - uploaded by Peter Benner
Author content
All figure content in this area was uploaded by Peter Benner
Content may be subject to copyright.
... We explore the usage of recycling BiCGSTAB for parametric model order reduction (PMOR) [5,11] that requires solution of systems of the form (1.1). We show about 40% reduction in iteration count when using recycling as compared to not using recycling. ...
... One practical application where such a challenge arises is micro-electro-mechanical systems (MEMS) design [12,7]. The goal of parametric model order reduction (PMOR) [5,11] is to generate a reduced model such that parametric dependence, as in the original model, is preserved (or retained). ...
Krylov subspace recycling is a process for accelerating the convergence of
sequences of linear systems. Based on this technique we have recently developed
the recycling BiCG algorithm. We now generalize and extend this recycling
theory to BiCGSTAB. Recycling BiCG focuses on efficiently solving sequences of
dual linear systems, while the focus here is on efficiently solving sequences
of single linear systems (assuming non-symmetric matrices for both recycling
BiCG and recycling BiCGSTAB).
As compared to other methods for solving sequences of single linear systems
with non-symmetric matrices (e.g., recycling variants of GMRES), BiCG based
recycling algorithms, like recycling BiCGSTAB, have the advantage that they
involve a short-term recurrence, and hence, do not suffer from storage issues
and are also cheaper with respect to the orthogonalizations.
We modify the BiCGSTAB algorithm to use a recycle space, which is built from
left and right approximate eigenvectors. Using our algorithm for a parametric
model order reduction example gives good results. We show about 40% savings in
iterations, and demonstrate that solving the problem without recycling leads to
(about) a 35% increase in runtime.
... Recycling techniques for iterative methods have been considered for multiple Krylov solvers and a wide range of applications, but mainly for square system matrices and for well-posed problems [38,45,49,30,1,44,28,29,13,32]. Augmented LSQR methods have been described in [3,2] for well-posed least-squares problems that require many LSQR iterations. ...
... Many problems in design involve sequences of slowly changing linear systems, for which recycling is highly efficient, for example, in topology optimization and other structural optimization problems [28,34,42,153,157], and aerodynamic shape optimization [33,81]. Recycling has also been used to compute reduced order models (for a range of applications) [5,6,57,58,111]. Another important area is nonlinear optimization, such as nonlinear least-squares, for example, in tomography [90,100,111,130] and blind deconvolution [78]. ...
This survey concerns subspace recycling methods, a popular class of iterative methods that enable effective reuse of subspace information in order to speed up convergence and find good initial vectors over a sequence of linear systems with slowly changing coefficient matrices, multiple right‐hand sides, or both. The subspace information that is recycled is usually generated during the run of an iterative method (usually a Krylov subspace method) on one or more of the systems. Following introduction of definitions and notation, we examine the history of early augmentation schemes along with deflation preconditioning schemes and their influence on the development of recycling methods. We then discuss a general residual constraint framework through which many augmented Krylov and recycling methods can both be viewed. We review several augmented and recycling methods within this framework. We then discuss some known effective strategies for choosing subspaces to recycle before taking the reader through more recent developments that have generalized recycling for (sequences of) shifted linear systems, some of them with multiple right‐hand sides in mind. We round out our survey with a brief review of application areas that have seen benefit from subspace recycling methods.
... Recycling techniques for iterative methods have been considered for multiple Krylov solvers and a wide range of applications, but mainly for square system matrices and for well-posed problem [24,30,31,18,1,29,16,17,8,20]. Augmented LSQR methods have been described in [3,2] for well-posed leastsquares problems that require many LSQR iterations. ...
Iterative hybrid projection methods have proven to be very effective for solving large linear inverse problems due to their inherent regularizing properties as well as the added flexibility to select regularization parameters adaptively. In this work, we develop Golub-Kahan-based hybrid projection methods that can exploit compression and recycling techniques in order to solve a broad class of inverse problems where memory requirements or high computational cost may otherwise be prohibitive. For problems that have many unknown parameters and require many iterations, hybrid projection methods with recycling can be used to compress and recycle the solution basis vectors to reduce the number of solution basis vectors that must be stored, while obtaining a solution accuracy that is comparable to that of standard methods. If reorthogonalization is required, this may also reduce computational cost substantially. In other scenarios, such as streaming data problems or inverse problems with multiple datasets, hybrid projection methods with recycling can be used to efficiently integrate previously computed information for faster and better reconstruction. Additional benefits of the proposed methods are that various subspace selection and compression techniques can be incorporated, standard techniques for automatic regularization parameter selection can be used, and the methods can be applied multiple times in an iterative fashion. Theoretical results show that, under reasonable conditions, regularized solutions for our proposed recycling hybrid method remain close to regularized solutions for standard hybrid methods and reveal important connections among the resulting projection matrices. Numerical examples from image processing show the potential benefits of combining recycling with hybrid projection methods.
... Because of the structure and size of the systems, the solves are typically done iteratively [9]. In recent work [2,16,17], the authors investigate the use of Krylov subspace recycling for these systems to generate the model-order reduction (MOR) basis. Recycling for shape based inversion for DOT was investigated in [21], but the goal was solving the sequence of systems in the inversion process, rather than for use in computing global basis matrices for use with MOR. ...
In nonlinear imaging problems whose forward model is described by a partial differential equation (PDE), the main computational bottleneck in solving the inverse problem is the need to solve many large-scale discretized PDEs at each step of the optimization process. In the context of absorption imaging in diffuse optical tomography, one approach to addressing this bottleneck proposed recently (de Sturler, et al, 2015) reformulates the viewing of the forward problem as a differential algebraic system, and then employs model order reduction (MOR). However, the construction of the reduced model requires the solution of several full order problems (i.e. the full discretized PDE for multiple right-hand sides) to generate a candidate global basis. This step is then followed by a rank-revealing factorization of the matrix containing the candidate basis in order to compress the basis to a size suitable for constructing the reduced transfer function. The present paper addresses the costs associated with the global basis approximation in two ways. First, we use the structure of the matrix to rewrite the full order transfer function, and corresponding derivatives, such that the full order systems to be solved are symmetric (positive definite in the zero frequency case). Then we apply MOR to the new formulation of the problem. Second, we give an approach to computing the global basis approximation dynamically as the full order systems are solved. In this phase, only the incrementally new, relevant information is added to the existing global basis, and redundant information is not computed. This new approach is achieved by an inner-outer Krylov recycling approach which has potential use in other applications as well. We show the value of the new approach to approximate global basis computation on two DOT absorption image reconstruction problems.
... In [5], the recycling BiCG algorithm is proposed and applied effectively to a parametric model order reduction example, while recycling BiCGSTAB is used (also for parametric model reduction) in [3]. More discussion of recycling Krylov subspace methods specifically applied to model reduction applications can be found in [35,36]. We give results for two sets of matrices, Rail and Flow, which can be found in [14] along with additional information. ...
Preconditioners are generally essential for fast convergence in the iterative solution of linear systems of equations. However, the computation of a good preconditioner can be expensive. So, while solving a sequence of many linear systems, it is advantageous to recycle preconditioners, that is, update a previous preconditioner and reuse the updated version. In this paper, we introduce a simple and effective method for doing this. Although our approach can be used for matrices changing slowly in any way, we focus on the important case of sequences of the type $(s_k\textbf{E}(\textbf{p}) + \textbf{A}(\textbf{p}))\textbf{x}_k = \textbf{b}_k$, where the right hand side may or may not change. More general changes in matrices will be discussed in a future paper. We update preconditioners by defining a map from a new matrix to a previous matrix, for example the first matrix in the sequence, and combine the preconditioner for this previous matrix with the map to define the new preconditioner. This approach has several advantages. The update is entirely independent from the original preconditioner, so it can be applied to any preconditioner. The possibly high cost of an initial preconditioner can be amortized over many linear solves. The cost of updating the preconditioner is more or less constant and independent of the original preconditioner. There is flexibility in balancing the quality of the map with the computational cost. In the numerical experiments section we demonstrate good results for several applications.
A parametrized model in addition to the control and state-space variables depends on time-independent design parameters, which essentially define a family of models. The goal of parametrized model reduction is to approximate this family of models. In this paper, a reduction method for linear time-invariant (LTI) parametrized models is presented, which constitutes the development of a recently proposed reduction approach. Reduced order models are computed based on the finite number of frequency response samples of the full order model. This method uses a semidefinite relaxation, while enforcing stability on the reduced order model for all values of parameters of interest. As a main theoretical statement, the relaxation gap (the ratio between upper and lower bounds) is derived, which validates the relaxation. The proposed method is flexible in adding extra constraints (e.g., passivity can be enforced on reduced order models) and modifying the objective function (e.g., frequency weights can be added to the minimization criterion). The performance of the method is validated on a numerical example.
SUMMARY A fast computational technique that speeds up the process of parametric macro-model extraction is proposed. An efficient starting point is the technique of parametric model order reduction (PMOR). The key step in PMOR is the computation of a projection matrix V, which requires the computation of multiple moment matrices of the underlying system. In turn, for each moment matrix, a linear system with multiple right-hand sides has to be solved. Usually, a considerable number of linear systems must be solved when the system includes more than two free parameters. If the original system is of very large size, the linear solution step is computationally expensive. In this paper, the subspace recycling algorithm outer generalized conjugate residual method combined with generalized minimal residual method with deflated restarting (GCRO-DR), is considered as a basis to solve the sequence of linear systems. In particular, two more efficient recycling algorithms, G-DRvar1 and G-DRvar2, are proposed. Theoretical analysis and simulation results show that both the GCRO-DR method and its variants G-DRvar1 and G-DRvar2 are very efficient when compared with the standard solvers. Furthermore, the presented algorithms overcome the bottleneck of a recently proposed subspace recycling method the modified Krylov recycling generalized minimal residual method. From these subspace recycling algorithms, a PMOR process for macro-model extraction can be significantly accelerated. Copyright © 2013 John Wiley & Sons, Ltd.
A thermal imaging system based on a commercial uncooled microbolometric array of 384 × 288 elements has been developed and produced. The temperature and spatial resolutions of the system are <0.08°C and 0.96 mrad, respectively. The system has been designed to monitor the technical state of thermal power industry objects. Owing to the “open architecture” and the modular structure of the hardware and software, the system can be adapted for any thermal diagnostic task.
Parametric model order reduction (PMOR) has received a tremendous amount of attention in recent years. Among the first approaches considered, mainly in system and control theory as well as computational electromagnetics and nanoelectronics, are methods based on multi-moment matching. Despite numerous other successful methods, including the reduced-basis method (RBM), other methods based on (rational, matrix, manifold) interpolation, or Kriging techniques, multi-moment matching methods remain a reliable, robust, and flexible method for model reduction of linear parametric systems. Here we propose a numerically stable algorithm for PMOR based on multi-moment matching. Given any number of parameters and any number of moments of the parametric system, the algorithm generates a projection matrix for model reduction by implicit moment matching. The implementation of the method based on a repeated modified Gram-Schmidt-like process renders the method numerically stable. The proposed method is simple yet efficient. Numerical experiments show that the proposed algorithm is very accurate.
A modification is given of the GMRES iterative method for nonsymmetric systems of linear equations. The new method deflates eigenvalues using Wu and Simon's thick restarting approach. It has the efficiency of implicit restarting, but is simpler and does not have the same nu-merical concerns. The deflation of small eigenvalues can greatly improve the convergence of restarted GMRES. Also, it is demonstrated that using harmonic Ritz vectors is important, because then the whole subspace is a Krylov subspace that contains certain important smaller subspaces.
We present a deflated version of the conjugate gradient algorithm for solving linear systems. The new algorithm can be useful in cases when a small number of eigenvalues of the iteration matrix are very close to the origin. It can also be useful when solving linear systems with multiple right-hand sides, since the eigenvalue information gathered from solving one linear system can be recycled for solving the next systems and then updated.
Many problems in engineering and physics require the solution of a large sequence of linear systems. We can reduce the cost of solving subsequent systems in the sequence by recycling information from previous systems. We overview two different approaches. For several model problems, we demonstrate that we can reduce the iteration count required to solve a linear system by a factor of two. We consider both Hermitian and non-Hermitian problems, and present numerical experiments to illustrate the effects of subspace recycling.
Projection techniques are developed for computing approximate solutions to linear systems of the form Axn = bn, for a sequence n = 1, 2,…, e.g. arising from time discretization of a partial differential equation. The approximate solutions are based upon previous solutions, and can be used as initial guesses for iterative solution of the system, resulting in significantly reduced computational expense. Examples of two- and three-dimensional incompressible Navier-Stokes calculations are presented in which xn represents the pressure at time level tn, and A is a consistent discrete Poisson operator. In flows containing significant dynamic activity, these projection techniques lead to as much as a two-fold reduction in solution time.
The simulation of slowly varying transient electric high-voltage fields and magnetic fields requires the repeated and successive solution of high-dimensional linear algebraic systems of equations with identical or near-identical system matrices and different right-hand side vectors. For these solution processes which are required within implicit time integration schemes and nonlinear (quasi-)Newton–Raphson methods an iterative multiple right-hand side (mrhs) scheme is used which recycles vector subspaces resulting from previous preconditioned conjugate gradient iteration runs. The combination of this scheme with a subspace projection extrapolation start value generation scheme is discussed. Numerical results for three-dimensional electric and magnetic field simulations are presented and the efficiency of the new schemes re-using eigenvector information from previous iteration processes with different tolerance criteria are compared to those of standard conjugate gradient iterations.
A robust algorithm for computing reduced-order models of parametric systems is proposed. Theoretical considerations suggest that our algorithm is more robust than previous algorithms based on explicit multi-moment matching. Moreover, numerical simulation results show that the proposed algorithm yields more accurate approximations than traditional non-parametric model reduction methods and parametric model reduction methods based on explicitly computing moments. (© 2008 WILEY-VCH Verlag GmbH & Co. KGaA, Weinheim)
Recently the GMRESR method for the solution of linear systems of equations has been introduced by Vuik and Van der Vorst (1991). Similar methods have been proposed by Axelsson and Vassilevski (1991) and Saad (1993) (FGMRES1). GMRESR involves an outer and an inner method. The outer method is GCR, which is used to compute the optimal approximation over a given set of search vectors in the sense that the residual is minimized. The inner method is GMRES, which computes a new search vector by approximately solving the residual equation. This search vector is then used by the outer algorithm to compute a new approximation. However, the optimality of the approximation over the space of search vectors is ignored in the inner GMRES algorithm. This leads to suboptimal corrections to the solution in the outer algorithm. Therefore, we propose to preserve the orthogonality relations of GCR in the inner GMRES algorithm. This gives optimal corrections to the solution and also leads to solving the residual equation in a smaller subspace and with an “improved” operator, which should also lead to faster convergence. However, this involves using Krylov methods with a singular, nonsymmetric operator. We will discuss some important properties of this. We will show by experiments that in terms of matrix-vector products, this modification (almost) always leads to better convergence. Because we do more orthogonalizations, it does not always give an improved performance in time. This depends on the costs of the matrix-vector products relative to the costs of the orthogonalizations. Of course, we can also use methods other than GMRES as the inner method. Methods with short recurrences like BiCGSTAB seem especially interesting. The experimental results indicate that, especially for such methods, it is advantageous to preserve the orthogonality in the inner method.
In this paper we present a preconditioned recycled Krylov-subspace method to accelerate a recently developed approach for ac and noise analysis of linear periodically-varying communication circuits. Examples are given to show that the combined method can be used to analyse switching filter frequency response, mixer 1/f noise frequency translation, and amplifier intermodulation distortion. In addition, it is shown that for large circuits the preconditioned recycled Krylov-subspace method is up to forty times faster than the standard optimized direct methods