Abstract
This monograph is based on a series of 10 lectures at Ohio State University at Columbus, March 23–27, 1987, sponsored by the Conference Board of the Mathematical Sciences and the National Science Foundation. The selection of topics is quite personal and, together with the talks of the other speakers, the lectures represent a story, as I saw it in March 1987, of many of the interesting things that statisticians can do with splines. I told the audience that the priority order for topic selection was, first, obscure work of my own and collaborators, second, other work by myself and students, with important work by other speakers deliberately omitted in the hope that they would mention it themselves. This monograph will more or less follow that outline, so that it is very much slanted toward work I had some hand in, although I will try to mention at least by reference important work by the other speakers and some of the attendees. The other speakers were (in alphabetical order), Dennis Cox, Randy Eubank, Ker-Chau Li, Douglas Nychka, David Scott, Bernard Silverman, Paul Speckman, and James Wendelberger. The work of Finbarr O'Sullivan, who was unable to attend, in extending the developing theory to the non-Gaussian and nonlinear case will also play a central role, as will the work of Florencio Utreras.
... This means that these estimators are not so good for estimating models of data that fluctuate in the subintervals, because these estimators will return the estimation results with large value of mean squared errors (MSEs). On the other hand, spline estimators, especially the smoothing spline estimator, have the ability to handle these problems because the splines consider not only the goodness of fit factor but also the smoothness factor [1,12]. Further, the spline estimators such as the smoothing spline, M-type spline, truncated spline, penalized spline, least square spline, linear spline, and B-spline estimators are more flexible than other estimators to use for estimating the nonparametric regression functions, especially for prediction and interpretation purposes [12,13]. ...
... On the other hand, spline estimators, especially the smoothing spline estimator, have the ability to handle these problems because the splines consider not only the goodness of fit factor but also the smoothness factor [1,12]. Further, the spline estimators such as the smoothing spline, M-type spline, truncated spline, penalized spline, least square spline, linear spline, and B-spline estimators are more flexible than other estimators to use for estimating the nonparametric regression functions, especially for prediction and interpretation purposes [12,13]. These splines have been used and developed widely in several cases by many researchers. ...
... The smoothing spline estimator provides good fitting results because it can overcome the curve of data, which has a sharp decrease and increase pattern that results a relatively smooth curve. Also, using the smoothing spline provides several advantages in that it has unique statistical properties, enables visual interpretation, can handle smooth data and functions, and can readily handle data that change at certain subintervals [1,12,13]. Meanwhile, the Fourier series estimator is a popular smoothing technique in nonparametric regression modeling. ...
In data analysis using a nonparametric regression approach, we are often faced with the problem of analyzing a set of data that has mixed patterns, namely, some of the data have a certain pattern and the rest of the data have a different pattern. To handle this kind of datum, we propose the use of a mixed estimator. In this study, we theoretically discuss a developed estimation method for a nonparametric regression model with two or more response variables and predictor variables, and there is a correlation between the response variables using a mixed estimator. The model is called the multiresponse multipredictor nonparametric regression (MMNR) model. The mixed estimator used for estimating the MMNR model is a mixed estimator of smoothing spline and Fourier series that is suitable for analyzing data with patterns that partly change at certain subintervals, and some others that follow a recurring pattern in a certain trend. Since in the MMNR model there is a correlation between responses, a symmetric weight matrix is involved in the estimation process of the MMNR model. To estimate the MMNR model, we apply the reproducing kernel Hilbert space (RKHS) method to penalized weighted least square (PWLS) optimization for estimating the regression function of the MMNR model, which consists of a smoothing spline component and a Fourier series component. A simulation study to show the performance of proposed method is also given. The obtained results are estimations of the smoothing spline component, Fourier series component, MMNR model, weight matrix, and consistency of estimated regression function. In conclusion, the estimation of the MMNR model using the mixed estimator is a combination of smoothing spline component and Fourier series component estimators. It depends on smoothing and oscillation parameters, and it has linear in observation and consistent properties.
... It increase model flexibility by allowing target functions to have different degrees of smoothness on different intervals. It has been heavily studied in existing literature including statistical properties and practical applications (Eubank 1988;Wahba 1990; Boor, C. DE, 1978DE, , 2001). ...
... Smoothing splines (Eubank 1988;Wahba 1990) use spline functions with all distinct observation locations as knots. To avoid overfitting, a penalty for the roughness of the function is added. ...
... Kimeldorf and Wahba (1971) used a reproducing kernel Hilbert space to solve smoothing problems and some of the results showed that spline interpolation and smoothing is equivalent to prediction. Detailed descriptions of smoothing splines were summarized in Wahba (1990) and Eubank (1999). Some other studies on smoothing spline can be found in the literature. ...
In areas such as spatial analysis and time series analysis, it is essential to understand
and quantify spatial or temporal heterogeneity. In this dissertation, we focus on
a spatially varying coefficient model, in which spatial heterogeneity is accommodated
by allowing the regression coefficients to vary in a given spatial domain. We propose a
model selection method for spatially varying coefficient models using penalized bivariate
splines. It uses bivariate splines defined on triangulation to approximate nonparametric
varying coefficient functions and minimizes the sum of squared errors with local
penalty on L2 norms of spline coefficients for each triangle. Our method partitions the
region of interest using triangulation and provides efficient approximation of irregular
domains. In addition, we propose an efficient algorithm to obtain the proposed estimator
using the local quadratic approximation. We also establish the consistency of estimated
nonparametric coefficient functions and the estimated null regions. Moreover, we develop
model confidence regions as the inference tool to quantify the uncertainty of the
estimated null regions. The numerical performance of the proposed method is evaluated
in both simulation case and real data analysis.
... Kernel methods are among the most successful models in machine learning, particularly due to their inherently non-linear and non-parametric nature, that nonetheless allows for a sound theoretical analysis. Kernels have been used extensively in regression (Kimeldorf and Wahba 1971;Wahba 1990) and classification (Cortes and Vapnik 1995;Mika et al. 1999). Since representation learning, or finding suitable features, is a key challenge is many scientific fields, we believe there is considerable scope for developing such models in these fields. ...
... Kernel methods in the supervised setting are well established and previous works offer rigorous theoretical analysis (Wahba 1990;Schölkopf and Smola 2002;Bartlett and Mendelson 2002). In this section, we show that the proposed kernel methods for contrastive SSL as well as for the reconstruction setting can be analysed in a similar fashion, and we provide generalisation error bounds for each of the proposed models. ...
Unsupervised and self-supervised representation learning has become popular in recent years for learning useful features from unlabelled data. Representation learning has been mostly developed in the neural network literature, and other models for representation learning are surprisingly unexplored. In this work, we introduce and analyze several kernel-based representation learning approaches: Firstly, we define two kernel Self-Supervised Learning (SSL) models using contrastive loss functions and secondly, a Kernel Autoencoder (AE) model based on the idea of embedding and reconstructing data. We argue that the classical representer theorems for supervised kernel machines are not always applicable for (self-supervised) representation learning, and present new representer theorems, which show that the representations learned by our kernel models can be expressed in terms of kernel matrices. We further derive generalisation error bounds for representation learning with kernel SSL and AE, and empirically evaluate the performance of these methods in both small data regimes as well as in comparison with neural network based models.
... 1. The dimension d of DS cannot be relatively large, since the time complexity of most algorithms constructing MLR models grows exponentially with d [15,16]; ...
This paper introduces a new type of regression methodology named as Convex-Area-Wise Linear Regression(CALR), which separates given datasets by disjoint convex areas and fits different linear regression models for different areas. This regression model is highly interpretable, and it is able to interpolate any given datasets, even when the underlying relationship between explanatory and response variables are non-linear and discontinuous. In order to solve CALR problem, 3 accurate algorithms are proposed under different assumptions. The analysis of correctness and time complexity of the algorithms are given, indicating that the problem can be solved in $o(n^2)$ time accurately when the input datasets have some special features. Besides, this paper introduces an equivalent mixed integer programming problem of CALR which can be approximately solved using existing optimization solvers.
... We expect that this will produce a map with an appropriate information content given the precision of the data. This method is analogous to fitting a spline to a one-or twodimensional dataset, where there is a single regularisation parameter that gives an optimal cross-validation score (Wahba 1990). ...
Eclipse mapping uses the shape of the eclipse of an exoplanet to measure its two-dimensional structure. Light curves are mostly composed of longitudinal information, with the latitudinal information only contained in the brief ingress and egress of the eclipse. This imbalance can lead to a spuriously confident map, where the longitudinal structure is constrained by out-of-eclipse data and the latitudinal structure is wrongly determined by the priors on the map. We present a new method to address this issue. The method tests for the presence of an eclipse mapping signal by using k-fold cross-validation to compare the performance of a simple mapping model to the null hypothesis of a uniform disk. If a signal is found, the method fits a map with more degrees of freedom, optimising its information content. The information content is varied by penalising the model likelihood by a factor proportional to the spatial entropy of the map, optimised by cross-validation. We demonstrate this method for simulated datasets then apply it to three observational datasets. The method identifies an eclipse mapping signal for JWST MIRI/LRS observations of WASP-43b but does not identify a signal for JWST NIRISS/SOSS observations of WASP-18b or Spitzer Space Telescope observations of HD 189733b. It is possible to fit eclipse maps to these datasets, but we suggest that these maps are overfitting the eclipse shape. We fit a new map with more spatial freedom to the WASP-43b dataset and show a flatter east-west structure than previously derived.
... Furthermore, the TPS-GLM allow modeling non-linear joint interaction effects due to some covariates, as well as the effects of coordinates in spatial data, making them a useful tool to model dynamic pattern in different scientific areas, such as environment, agronomy, ecology, and so on. Some of the main works related to thin-plate spline technique are Duchon [2,3], Bookstein [4], and Chen et al. [5], while in the context of statistical modeling, Wahba [6], Green and Silverman [7], Wood [8], and Moraga et al. [9], can be mentioned, among others. However, it is well known that diagnostic analysis is a fundamental process in all statistical modeling for any data set. ...
Thin-Plate Spline Generalized Linear Models (TPS-GLMs) are an extension of Semiparametric Generalized Linear Models (SGLMs), because they allow a smoothing spline to be extended to two or more dimensions. This class of models allows modeling a set of data in which it is desired to incorporate the non-linear joint effects of some covariates to explain the variability of a certain variable of interest. In the spatial context, these models are quite useful, since they allow the effects of locations to be included, both in trend and dispersion, using a smooth surface. In this work, we extend the local influence technique for the TPS-GLM model in order to evaluate the sensitivity of the maximum penalized likelihood estimators against small perturbations in the model and data. We fit our model through a joint iterative process based on Fisher Scoring and weighted backfitting algorithms. In addition, we obtained the normal curvature for the case-weight perturbation and response variable additive perturbation schemes, in order to detect influential observations on the model fit. Finally, two data sets from different areas (agronomy and environment) were used to illustrate the methodology proposed here.
... Going back at least to [54,30,62], RKHS are by now indispensable tools in statistics and machine learning, in particular to kernel-based regression or the kernel support vector machine. We refer to [72,65,67,16,63,36] for an overview of current developments in this area. Studying optimal transport in RKHS is not new; see e.g. ...
Optimal transport and the Wasserstein distance $\mathcal{W}_p$ have recently seen a number of applications in the fields of statistics, machine learning, data science, and the physical sciences. These applications are however severely restricted by the curse of dimensionality, meaning that the number of data points needed to estimate these problems accurately increases exponentially in the dimension. To alleviate this problem, a number of variants of $\mathcal{W}_p$ have been introduced. We focus here on one of these variants, namely the max-sliced Wasserstein metric $\overline{\mathcal{W}}_p$. This metric reduces the high-dimensional minimization problem given by $\mathcal{W}_p$ to a maximum of one-dimensional measurements in an effort to overcome the curse of dimensionality. In this note we derive concentration results and upper bounds on the expectation of $\overline{\mathcal{W}}_p$ between the true and empirical measure on unbounded reproducing kernel Hilbert spaces. We show that, under quite generic assumptions, probability measures concentrate uniformly fast in one-dimensional subspaces, at (nearly) parametric rates. Our results rely on an improvement of currently known bounds for $\overline{\mathcal{W}}_p$ in the finite-dimensional case.
... In the subsequent step, it is crucial to objectively select appropriate values for the hyper-parameters t and t since the solution is highly dependent on these parameters. Different approaches, like the L-curve [68] and the generalized cross-validation (GCV) [69,70], have been employed for hyper-parameter selection. Here, we estimate hyper-parameter values using the GCV method by minimizing the cost function defined in Eq. (11): ...
One of the most important needs in neuroimaging is brain dynamic source imaging with high spatial and temporal resolution. EEG source imaging estimates the underlying sources from EEG recordings, which provides enhanced spatial resolution with intrinsically high temporal resolution. To ensure identifiability in the underdetermined source reconstruction problem, constraints on EEG sources are essential. This paper introduces a novel method for estimating source activities based on spatio-temporal constraints and a dynamic source imaging algorithm. The method enhances time resolution by incorporating temporal evolution of neural activity into a regularization function. Additionally, two spatial regularization constraints based on \({L}_{1}\) and \({L}_{2}\) norms are applied in the transformed domain to address both focal and spread neural activities, achieved through spatial gradient and Laplacian transform. Performance evaluation, conducted quantitatively using synthetic datasets, discusses the influence of parameters such as source extent, number of sources, correlation level, and SNR level on temporal and spatial metrics. Results demonstrate that the proposed method provides superior spatial and temporal reconstructions compared to state-of-the-art inverse solutions including STRAPS, sLORETA, SBL, dSPM, and MxNE. This improvement is attributed to the simultaneous integration of transformed spatial and temporal constraints. When applied to a real auditory ERP dataset, our algorithm accurately reconstructs brain source time series and locations, effectively identifying the origins of auditory evoked potentials. In conclusion, our proposed method with spatio-temporal constraints outperforms the state-of-the-art algorithms in estimating source distribution and time courses.
... Different existing methods can accomplish this task. In our experiments, the regression methods that provided the best accuracy are Reduced Basis Function (RBF) interpolation [59,60] and second-order polynomial sparse approximation. In the former, the function is modeled as ...
Stable partitioned techniques for simulating unsteady fluid-structure interaction (FSI) are known to be computationally expensive when high added-mass is involved. Multiple coupling strategies have been developed to accelerate these simulations, but often use predictors in the form of simple finite-difference extrapolations. In this work, we propose a non-intrusive data-driven predictor that couples reduced-order models of both the solid and fluid subproblems, providing an initial guess for the nonlinear problem of the next time step calculation. Each reduced order model is composed of a nonlinear encoder-regressor-decoder architecture and is equipped with an adaptive update strategy that adds robustness for extrapolation. In doing so, the proposed methodology leverages physics-based insights from high-fidelity solvers, thus establishing a physics-aware machine learning predictor. Using three strongly coupled FSI examples, this study demonstrates the improved convergence obtained with the new predictor and the overall computational speedup realized compared to classical approaches.
... Green et al. [24] computed convergence rates of the Laplacian smoothing estimator, which can be viewed as a discrete approximation of Problem (C). For further references, see, for example, de Boor [16], Dierckx [17], Green and Silverman [23], Györfi et al. [27], Wahba [71], and Wegman and Wright [74]. For Problem (A), when d � 1, the existence and uniqueness of the solution are established by Schoenberg [60], and the convergence rates are derived by van de Geer [68]. ...
We consider the problem of estimating an unknown function f* and its partial derivatives from a noisy data set of n observations, where we make no assumptions about f* except that it is smooth in the sense that it has square integrable partial derivatives of order m. A natural candidate for the estimator of f* in such a case is the best fit to the data set that satisfies a certain smoothness condition. This estimator can be seen as a least squares estimator subject to an upper bound on some measure of smoothness. Another useful estimator is the one that minimizes the degree of smoothness subject to an upper bound on the average of squared errors. We prove that these two estimators are computable as solutions to quadratic programs, establish the consistency of these estimators and their partial derivatives, and study the convergence rate as n increases to infinity. The effectiveness of the estimators is illustrated numerically in a setting where the value of a stock option and its second derivative are estimated as functions of the underlying stock price.
... By adjusting λ and the placement of knots, the degree of smoothing can be controlled, allowing flexibility in capturing complex relationships in the data. For further insights, additional details can be found in [5] or [7]. ...
The analysis of odds ratio curves is a valuable tool in understanding the relationship between continuous predictors and binary outcomes. Traditional parametric regression approaches often assume specific functional forms, limiting their flexibility and applicability to complex data. To address this limitation and introduce more flexibility, several smoothing methods may be applied, and approaches based on splines are the most frequently considered in this context. To better understand the effects that each continuous covariate has on the outcome, results can be expressed in terms of splines-based odds ratio (OR) curves, taking a specific covariate value as reference. In this paper, we introduce an R package, flexOR, which provides a comprehensive framework for pointwise nonparametric estimation of odds ratio curves for continuous predictors. The package can be used to estimate odds ratio curves without imposing rigid assumptions about their underlying functional form while considering a reference value for the continuous covariate. The package offers various options for automatically choosing the degrees of freedom in multivariable models. It also includes visualization functions to aid in the interpretation and presentation of the estimated odds ratio curves. flexOR offers a user-friendly interface, making it accessible to researchers and practitioners without extensive statistical backgrounds.
... Regresi semiparametrik adalah metode statistika untuk mendapatkan pemodelan antara variabel respon dan prediktor dengan sebagian diketahui bentuk fungsi dan sebagian lagi tidak diketahui bentuk fungsinya dari pola data [10] Model terbaik semiparametrik B Spline merupakan titik knot optimal pada nilai GCV minimum, maka perlu dilakukan perbandingan nilai GCV di setiap titik knot dan orde. Secara teoritis, GCV mempunyai sifat optimal asimptotik yang tidak terdapat pada metode lain [11]. Kriteria GCV dirumuskan sebagai Persamaan 25. ...
p dir="ltr"> The increases of the world gold and crude oil prices have a big role as a main factor that effect composite stock price index, the effect can make investors to buy stock from Bursa Efek Indonesia. Regression semiparametric used in this research for a purpose to get combined parametric and nonparametric with B Spline approach. B Spline is a development of spline to overcome weaknesses in making singular matrix at a high order spline with many knot points and close together. Variable parametric component is composite stock price index with crude oil price, and variable nonparametric component is composite stock price index with gold price that got obtained from January 2015 until December 2022. The result from this research is best regression semiparametric B-Spline modelling can be obtained using some combination of order and knot points. The optimal point is obtained on 2nd order using 4 knot point (1.135;1.319,15;1.320,75;1.323,25) with a minimum GCV value is 100.227,8. The best measure of goodness with a coefficient of determination value (R-Square) obtained a value 78,8%, because the value is more than 67% make it as a strong model. MAPE value is 3,37% that has a value less than 10 %, make this model have a perfect forecasting ability.
Keywords: Gold; Crude Oil; Composite Stock Price Index; Semiparametric B Spline; GCV </p
... In other fields of human movement science, a different approach for physiological signal reconstruction, which relies on smoothing splines (de Boor, 2001;Eubank, 1999;Medved, 2001;Schumaker, 2007;Wahba, 1990), has gained traction. As in the filtering approach, the splines approach ensures that the signal and (at least) its first and second derivatives are reconstructed as continuous curves (in conformity with the smoothness property of biological signals; cf. ...
Purpose
We compare two signal smoothing and differentiation approaches: a frequently used approach in the speech community of digital filtering with approximation of derivatives by finite differences and a spline smoothing approach widely used in other fields of human movement science.
Method
In particular, we compare the values of a classic set of kinematic parameters estimated by the two smoothing approaches and assess, via regressions, how well these reconstructed values conform to known laws about relations between the parameters.
Results
Substantially smaller regression errors were observed for the spline smoothing than for the filtering approach.
Conclusion
This result is in broad agreement with reports from other fields of movement science and underpins the superiority of splines also in the domain of speech.
... The objective is to select the points that belong to the active set w.r.t some criteria [2]. One of these approaches is the subset of regressors (SR) proposed by G. Wahba [20], which reduces the complexity to O(m 2 n) for initialization and O(m) and O(m 2 ) for the prediction of the mean and variance for a new test point. This reduction in the complexity is accomplished by approximating the form of the predictive distribution, specially the predictive mean. ...
... The smoothing parameter p controls the tradeoff between the smoothness of the spline and its weighted agreement with the measurements and is analogous to the cutoff frequency in FBP. Interestingly, if the weights wj are identical, the spline smoothing would correspond to a low-pass Butterworth filter of order 2 times the order of the derivative in the second term and a half-power frequency that is a function of p (Fessler 1993, Wahba 1990). Based on the above, the weights were varied inversely proportional to the optical depth values as follows: ...
An isooctane spray from a high-pressure multi-hole GDI injector (Bosch HDEV6) was characterized by means of optical extinction tomography, relying on parallel illumination by a focused-shadowgraph setup. The tests were carried out in air at ambient conditions at an injection pressure of 300 bar. Extinction images of the spray were acquired over a 180-degree angular range in 1-degree increments. The critical issues of optical extinction tomography of sprays, related to the strong light extinction by the dense liquid core of fuel jets, were addressed. To mitigate artifacts arising from the reconstruction process, the extinction data were subjected to spatially-variant filtering steps of both the raw and post-log data, before being analytically inverted through the inverse Radon transform. This made it possible to process extinction data for very large optical depths. A nearly complete three-dimensional reconstruction of the spray was obtained, providing significant details of the spray morphology and the internal structure of the jets throughout the spray development. The different phases of the atomization process from the near-field to far-field regions of the spray were observed.
... We take α * as the solution to the problem in (13), whose feasibility is guaranteed by representer theorem [36] and the non-emptiness of the feasible set (f is feasible for (12)). Therefore, mt(x) = (α * ) T K Xx is the optimal solution to (12). ...
Efficient global optimization is a widely used method for optimizing expensive black-box functions. In this paper, we study the worst-case oracle complexity of the efficient global optimization problem. In contrast to existing kernel-specific results, we derive a unified lower bound for the oracle complexity of efficient global optimization in terms of the metric entropy of a ball in its corresponding reproducing kernel Hilbert space. Moreover, we show that this lower bound nearly matches the upper bound attained by non-adaptive search algorithms, for the commonly used squared exponential kernel and the Matérn kernel with a large smoothness parameter ν\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\nu $$\end{document}. This matching is up to a replacement of d/2 by d and a logarithmic term logRϵ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\log \frac{R}{\epsilon }$$\end{document}, where d is the dimension of input space, R is the upper bound for the norm of the unknown black-box function, and ϵ\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon $$\end{document} is the desired accuracy. That is to say, our lower bound is nearly optimal for these kernels.
... where C denotes the optimization metric (normalized mutual information or normalized cross-correlation) as a function of the moving image (I M ), the fixed image (I F ), and the set of parameters of the image transformation ( m), which are translation and rotation parameters for WRR, and the parameters of a deformation field for the WDR (Sederberg and Parry 1986). The function ( ) R m is a penalty term (bending energy) that enforces the smoothness of the transformation (S and Wahba 2006). The optimal transformation field T n (for image n) is then applied to the coordinates of the lesion masks L n associated with each of the N images in the series, generating a registered lesion mask L . ...
Objective: Manual analysis of individual cancer lesions to assess disease response is clinically impractical and requires automated lesion tracking methodologies. However, no methodology has been developed for whole-body individual lesion tracking, across an arbitrary number of scans, and acquired with various imaging modalities. Approach: This study introduces a lesion tracking methodology and benchmarked it using 23 68Ga-DOTATATE PET/CT and PET/MR images of eight neuroendocrine tumor patients. The methodology consists of six steps: (1) Alignment of multiple scans via image registration, (2) Body-part labeling, (3) Automatic lesion-wise dilation, (4) Clustering of lesions based on local lesion shape metrics, (5) Assignment of lesion tracks, and (6) Output of a lesion graph. Registration performance was evaluated via landmark distance, lesion matching accuracy was evaluated between each image pair, and lesion tracking accuracy was evaluated via identical track ratio. Sensitivity studies were performed to evaluate the impact of lesion dilation (fixed vs. automatic dilation), anatomic location, image modalities (inter- vs. intra-modality), registration mode (direct vs. indirect registration), and track size (number of time-points and lesions) on lesion matching and tracking performance. Main Results: Manual contouring yielded 956 lesions, 1,570 lesion-matching decisions, and 493 lesion tracks. The median residual registration error was 2.5 mm. The automatic lesion dilation led to 0.90 overall lesion matching accuracy, and an 88% identical track ratio. The methodology is robust regarding anatomic locations, image modalities, and registration modes. The number of scans had a moderate negative impact on the identical track ratio (94% for 2 scans, 91% for 3 scans, and 81% for 4 scans). The number of lesions substantially impacted the identical track ratio (93% for 2 nodes vs. 54% for ≥5 nodes). Significance: The developed methodology resulted in high lesion-matching accuracy and enables automated lesion tracking in PET/CT and PET/MR.
... This interpolation technique offers a compromise between local interpolation methods such as IDW and global interpolation methods such as kriging, by allowing the resultant DEM values to follow abrupt changes in terrain which include streams, ridges, and cliffs, thus preserving the topographical continuity (Pavlova 2017). In fact, ANUDEM algorithm is a modified version based of the discretization of TPS (thin plate spline) algorithm, in which the roughness penalty J(f) has been modified to allow the fitted DEM to follow abrupt changes in the topographic surface, such as streams and ridges (Wahba 1990). ...
The purpose of this study was to evaluate and improve the vertical accuracy of three free global digital elevation models (DEMs)—the Shuttle Radar Topography Mission (SRTM), the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER), and the Forest and Buildings Removed Copernicus DEM (FABDEM). The vertical accuracy of the DEMs was evaluated through a survey of 3551 points using a Real-Time Kinematic global positioning system (RTK-GPS) in Aïn Leuh, Morocco. The findings indicated that the SRTM and FABDEM DEMs have a root mean square error (RMSE) of ± 7.25 m and 6.53 m, respectively, surpassing the ASTER DEM (± 8.12 m). Interpolation methods were then employed to correct and improve the DEMs using the surveyed GPS data at 15 m and 30 m spatial resolutions. Five interpolators were used: Inverse Distance Weighting (IDW), Natural Neighbor, Spline, ANUDEM, and ordinary kriging (OK). By integrating the GPS data with the DEMs, the interpolation methods were able to improve the accuracy of the models. However, it is worth noting that ANUDEM algorithm outperforms other algorithms in terms of accuracy and the surface's natural appearance of topography. Furthermore, our findings indicate that resampling to a 15-m spatial resolution achieves an appropriate balance between improving the DEM’s quality and revealing topographic features that weren’t discernible in the raw DEMs. The suggested approach resulted in a 49% reduction in errors compared to the raw DEMs. The RMSE values for the resampled DEMs were ± 3.6 m and ± 3.3 m for the SRTM + GPS and FABDEM + GPS, respectively.
... The main advantage of using SVM in an oil pollution detection missions is effective processing of high amount of data from different sensors of multi-sensor payloads. SVM finds a hyperplane that is a hyperplane in a feature space induced by a kernel K (the kernel defines a dot product in that space (Wahba, 1990)). The advantage of using SVM methods for oil spill detection is reliability to perform dividing data into two classes and possibility to create a model on small data sets. ...
The paper presents the results of the research on oil spill detection using machine learning methods such as Support Vector Machine (SVM) for classification of infrared images and Logistic regression for water quality parameters. This paper focuses on real time detection of oil spills using infrared images and water quality data obtained by RPA equipped with multi-sensor payload. The developed Naïve Bayes (NB), SVM and Logistic regression classification models for prediction of oil spill have been successfully tested in real experiment conditions. All developed classification models were tuned using grid search method and main tuning parameters to determine the optimal parameters. The proposed complex algorithm for identification of oil spills using infrared images and water quality parameters is evaluated by experiments in real environment conditions. The proposed algorithm is based on the binary SVM and NB classification of infrared images and the classification of water quality parameters using the machine learning method logistic regression allows to rapidly and with high accuracy identify any oil pollution of water. Proposed complex algorithm achieves higher accuracy and efficiency; moreover, the developed machine learning models will further reduce the probability of human error and save man-hours of work.
... Here we www.nature.com/scientificreports/ optimize using the Generalized Cross Validation (GCV) 39 . The GCV measures the predictive error of the fitted surface by omitting each data point in turn and summing the square of the discrepancy of each data point from a surface fitted using all other data points. ...
The 2021 summer upwelling season off the United States Pacific Northwest coast was unusually strong leading to widespread near-bottom, low-oxygen waters. During summer 2021, an unprecedented number of ship- and underwater glider-based measurements of dissolved oxygen were made in this region. Near-bottom hypoxia, that is dissolved oxygen less than 61 µmol kg⁻¹ and harmful to marine animals, was observed over nearly half of the continental shelf inshore of the 200-m isobath, covering 15,500 square kilometers. A mid-shelf ribbon with near-bottom, dissolved oxygen less than 50 µmol kg⁻¹ extended for 450 km off north-central Oregon and Washington. Spatial patterns in near-bottom oxygen are related to the continental shelf width and other features of the region. Maps of near-bottom oxygen since 1950 show a consistent trend toward lower oxygen levels over time. The fraction of near-bottom water inshore of the 200-m isobath that is hypoxic on average during the summer upwelling season increases over time from nearly absent (2%) in 1950–1980, to 24% in 2009–2018, compared with 56% during the anomalously strong upwelling conditions in 2021. Widespread and increasing near-bottom hypoxia is consistent with increased upwelling-favorable wind forcing under climate change.
The impact load transfer caused by the landing of a carrier aircraft is a complicated problem in engineering, especially for a deck structure with a known area but an unknown location. This is a crucial structural dynamics inverse problem. The traditional dynamics inverse problem is generally to divide and calibrate the known region through Green’s function, locate the region through the minimum norm of the load data obtained from the measurement response inversion, and obtain the time history. However, this method is mostly aimed at single-peak impacts and cannot deal with impact load migration, that is, with multipeak impacts in the time domain. Therefore, based on the time-slice method, this paper presents a method of locating the impact load migration trajectory and time history inversion for deck structures. A mathematical model is constructed independently through the segmentation and identification of the response, combining the Tikhonov regularization method and the generalized cross-validation (GCV) regularization operator to achieve region calibration and time history inversion and then locating the migration trajectory using a load similarity criterion, thus overcoming the limitation that traditional methods cannot achieve multipeak recognition. At the same time, invalid data are avoided through slice screening, and the accuracy of trajectory recognition is greatly improved. The effectiveness and accuracy of the method are verified based on the influence of different slice lengths on the identification results. The influence of different incidence angles on the results is discussed, and the universality of the method is verified.
We use a Gaussian Process Regression (GPR) strategy to analyze different types of curves that are commonly encountered in parametric eigenvalue problems. We employ an offline-online decomposition method. In the offline phase, we generate the basis of the reduced space by applying the proper orthogonal decomposition (POD) method on a collection of pre-computed, full-order snapshots at a chosen set of parameters. Then we generate our GPR model using four different Matérn covariance functions. In the online phase, we use this model to predict both eigenvalues and eigenvectors at new parameters. We then illustrate how the choice of each covariance function influences the performance of GPR. Furthermore, we discuss the connection between Gaussian Process Regression and spline methods and compare the performance of the GPR method against linear and cubic spline methods. We show that GPR outperforms other methods for functions with a certain regularity.
In contrast to the existing literature focusing on post-disaster regional impacts, we illustrate how the perception of disaster exposure affects regional population flows through household location decisions using a quantitative spatial economics model. More importantly, the quantitative spatial economics model helps identify critical drivers for regional migration that motivate the subsequent empirical analyses. A generalized additive model is applied to US county-level data to capture the nonlinear impact of disaster exposure on migration. The regional migration is not responsive to small and moderate disaster exposures. However, counties subject to severe disaster exposure experience significantly slower net inmigration.
Drinking water supplies located along the coast of Indonesia face susceptibility to saltwater pollution due to seawater intrusion, including in the coastal area of Batang, Indonesia. Therefore, this study aims to determine the community’s ability to access water in the Batang coastal area following the seawater intrusion phenomenon. First, an interpolation in ArcGIS based on the electrical conductivity measurements of 40 wells was done to identify and visualize areas affected by seawater intrusion. Furthermore, we interviewed 116 respondents and employed the contingency valuation method to calculate and analyze the public willingness to pay (WTP) for improving drinking water sources contaminated with saltwater due to seawater intrusion. The results showed that the groundwater near the coast was already contaminated with salt water, with the highest electrical conductivity value reaching 3999 μmhos/cm. Furthermore, the economic valuation analysis results show that the expected WTP of the entire population for enhancing water quality is IDR 17,830 (~ 1.1€). The expected WTP of the population affected and not affected by the intrusion is IDR 31,150 (~ 2€) and IDR 14,800 (~ 0.9€), respectively. Compared with the minimum wage in Batang in 2022, i.e., IDR 2,132,535, the expected WTP for the entire population, the population affected, and not affected by the intrusion, were 0.84%, 1.46%, and 0.69%, respectively. This study offers valuable insights for future investigations and serves as a foundational reference for local governments and communities in their efforts to enhance water quality in natural ecosystems through more comprehensive and efficient strategies.
The self-sustained interactions between a flexible film and periodic vortices epitomize the spirit of fish swimming and flag flapping in nature, involving intricate patterns of flow–structure coupling. Here, we comprehensively investigate the multiple coupling states of a film in the cylinder wake mainly with experiments, complemented by theoretical solutions and nonlinear dynamical analyses. Four regimes of film motion states are identified in the parameter space spanned by the reduced velocity and the length ratio. These regimes are (i) keeping stationary, (ii) deflection flutter, (iii) hybrid flutter and (iv) periodic large-amplitude flapping, each governed by a distinct coupling mechanism, involving regular and irregular Kármán vortices, local instability of the elongated shear layers and 2P mode vortex shedding. The film futtering in regimes (ii) and (iii) is substantiated to be chaotic and bears a resemblance to the ‘entraining state’ of fish behind an obstacle in the river. The periodic flapping in regime (iv) manifests itself in an amalgam of standing and travelling waves, and has intrinsic relations to the ‘Kármán gaiting’ of fish in periodic vortices. With the spatiotemporal reconstruction for the periodic flapping, we procure the energy distributions on the film, revealing the energy transfer processes between the film and the large-scale vortices. The findings unequivocally indicate that the flow–structure interaction during the energy-release stage of the film is more intense than that during the energy-extraction stage. Given the similarities, the mathematical and physical methods presented in this work are also applicable to the research on biological undulatory locomotion.
Image-based virtual try-on technology provides a better shopping experience for online customers and holds immense commercial value. However, existing methods face challenges in accurately aligning garments and preservation of garment texture details when dealing with challenging body poses or complex target clothing. Furthermore, these methods have been unable to adaptively fuse and generate images based on different body parts in a refined manner, struggling to generate and retain high-quality details of body parts, resulting in limited quality of try-on results. To address these issues, we propose a novel virtual try-on network named Context-Aware Enhanced Virtual Try-On Network (CAE-VTON). The key ideas of our method are as follows: (1) Introducing a Multi-Scale Neighborhood Consensus Warp Module (MNCWM) with matching filtering capability that is sensitive to small semantic differences, which generates highly accurate garment alignment results and coupled natural try-on generation results. (2) Proposing a fabric deformation energy smoothness loss to constrain local deformations of clothing, thus preserving complex details in garments. (3) Designing a Body Reconstruction Module (BRM) that adaptively generates and retains exposed skin areas of the body. (4) Introducing a novel try-on generation module called Context-Adaptive Awareness-Enhanced Try-on Module (CAAETM) that integrates all components and utilizes target semantic label map to adaptively generate the final try-on results for different body parts. We evaluate our model on the VITON-HD and VITON datasets and find that our method achieves state-of-the-art performance in qualitative and quantitative evaluations for virtual try-on.
We consider the problem of estimating an unknown function [Formula: see text] and its partial derivatives from a noisy data set of n observations, where we make no assumptions about [Formula: see text] except that it is smooth in the sense that it has square integrable partial derivatives of order m. A natural candidate for the estimator of [Formula: see text] in such a case is the best fit to the data set that satisfies a certain smoothness condition. This estimator can be seen as a least squares estimator subject to an upper bound on some measure of smoothness. Another useful estimator is the one that minimizes the degree of smoothness subject to an upper bound on the average of squared errors. We prove that these two estimators are computable as solutions to quadratic programs, establish the consistency of these estimators and their partial derivatives, and study the convergence rate as [Formula: see text]. The effectiveness of the estimators is illustrated numerically in a setting where the value of a stock option and its second derivative are estimated as functions of the underlying stock price.
This paper introduces a new nonparametric estimator of the regression based on local quasi-interpolation spline method. This model combines a B-spline basis with a simple local polynomial regression, via blossoming approach, to produce a reduced rank spline like smoother. Different coefficients functionals are allowed to have different smoothing parameters (bandwidths) if the function has different smoothness. In addition, the number and location of the knots of this estimator are not fixed. In practice, we may employ a modest number of basis functions and then determine the smoothing parameter as the minimizer of the criterion. In simulations, the approach achieves very competitive performance with P-spline and smoothing spline methods. Simulated data and a real data example are used to illustrate the effectiveness of the method proposed in this paper.
We approximate d-variate periodic functions in weighted Korobov spaces with general weight parameters using n function values at lattice points. We do not limit n to be a prime number, as in currently available literature, but allow any number of points, including powers of 2, thus providing the fundamental theory for construction of embedded lattice sequences. Our results are constructive in that we provide a component-by-component algorithm which constructs a suitable generating vector for a given number of points or even a range of numbers of points. It does so without needing to construct the index set on which the functions will be represented. The resulting generating vector can then be used to approximate functions in the underlying weighted Korobov space. We analyse the approximation error in the worst-case setting under both the \(L_2\) and \(L_{\infty }\) norms. Our component-by-component construction under the \(L_2\) norm achieves the best possible rate of convergence for lattice-based algorithms, and the theory can be applied to lattice-based kernel methods and splines. Depending on the value of the smoothness parameter \(\alpha \), we propose two variants of the search criterion in the construction under the \(L_{\infty }\) norm, extending previous results which hold only for product-type weight parameters and prime n. We also provide a theoretical upper bound showing that embedded lattice sequences are essentially as good as lattice rules with a fixed value of n. Under some standard assumptions on the weight parameters, the worst-case error bound is independent of d.
Tikhonov Regularization is the most widely used method for geophysical inversion problems. The result of previous and current research has shown that how to estimate the regularization parameter has a dramatic effect on inversion results. In the present research, conventional methods, including L-curve, Discrepancy principle, GCV, and ACB are compared with an innovative technique called Unbiased Predictive Risk Estimator (UPRE) in the inversion of 2D magnetotelluric data. For this purpose, MT2DInvMatlab is applied as the main program. It uses the Levenberg–Marquardt method as the inversion core and the ACB method to estimate the regularization parameter. Then, this program was developed in a way that it could estimate the regularization parameter using all of the above-mentioned methods. Next, a relatively complex model consisting of two layers and three blocks was used as a synthetic model. Comparing the results of all methods in TM, TE, and joint modes showed that the UPRE method, which previously provided desirable results in the inversion of potential field data in terms of convergence rate, time, and accuracy of results, here along with the ACB method, presented more acceptable results in the same indicators. Therefore, these two methods were used in a geothermal case in the North-West of Iran as a real test. In this case, the UPRE presented results at the same level as the ACB method and better than it in terms of some indicators. So, the UPRE method, especially in large-scale problems, could be a suitable alternative to the ACB method.
The processing of potential field datasets requires many steps; one of them is the inverse modeling of potential field data. Using a measurement dataset, the purpose is to evaluate the physical and geometric properties of an unidentified model in the subsurface. Because of the ill-posedness of the inverse problem, the determination of an acceptable solution requires the imposition of a regularization term to stabilize the inversion process. We also need a regularization parameter that determines the comparative weights of the stabilization and data fit terms. This work offers an evaluation of automated strategies for the estimation of the regularization parameter for underdetermined linear inverse problems. We look at the methods of generalized cross validation, active constraint balancing (ACB), the discrepancy principle, and the unbiased predictive risk estimator. It has been shown that the ACB technique is superior by applying the algorithms to both synthetic data and field data, which produces density models that are representative of real structures and demonstrate the method’s supremacy. Data acquired over the chromite deposit in Camaguey, Cuba, are utilized to corroborate the procedures for the inversion of experimental data. The findings gathered from the three-dimensional inversion of gravity data from this region demonstrate that the ACB approach gives appropriate estimations of anomalous density structures and depth resolution inside the subsurface.
A data-driven model is compared to classical equation-driven approaches to investigate its ability to predict quantity of interest and their uncertainty when studying airfoil aerodynamics. The focus is on autoencoders and the effect of uncertainties due to the architecture, the hyperparamaters and the choice of the training data (internal or model-form uncertainties). Comparisons with a Gaussian Process regression approach clearly illustrate the autoencoder advantage in extracting useful information on the prediction confidence even in the absence of ground truth data. Simulations accounting for internal uncertainties are also compared to the impact of the variability induced by uncertain operating conditions (external uncertainties) showing the importance of accounting for the total uncertainty when establishing prediction confidence.
We investigate the asymptotic behaviour of gradient boosting algorithms when the learning rate converges to zero and the number of iterations is rescaled accordingly. We mostly consider L2-boosting for regression with linear base learner as studied in Bühlmann and Yu (2003) and analyze also a stochastic version of the model where subsampling is used at each step (Friedman, 2002). We prove a deterministic limit in the vanishing learning rate asymptotic and characterize the limit as the unique solution of a linear differential equation in an infinite dimensional function space. Besides, the training and test error of the limiting procedure are thoroughly analyzed. We finally illustrate and discuss our result on a simple numerical experiment where the linear L2-boosting operator is interpreted as a smoothed projection and time is related to its number of degrees of freedom.
We introduce a novel framework for the classification of functional data supported on nonlinear, and possibly random, manifold domains. The motivating application is the identification of subjects with Alzheimer’s disease from their cortical surface geometry and associated cortical thickness map. The proposed model is based upon a reformulation of the classification problem as a regularized multivariate functional linear regression model. This allows us to adopt a direct approach to the estimation of the most discriminant direction while controlling for its complexity with appropriate differential regularization. Our approach does not require prior estimation of the covariance structure of the functional predictors, which is computationally prohibitive in our application setting. We provide a theoretical analysis of the out-of-sample prediction error of the proposed model and explore the finite sample performance in a simulation setting. We apply the proposed method to a pooled dataset from Alzheimer’s Disease Neuroimaging Initiative and Parkinson’s Progression Markers Initiative. Through this application, we identify discriminant directions that capture both cortical geometric and thickness predictive features of Alzheimer’s disease that are consistent with the existing neuroscience literature.
Understanding how neural networks learn features, or relevant patterns in data, for prediction is necessary for their reliable use in technological and scientific applications. In this work, we presented a unifying mathematical mechanism, known as Average Gradient Outer Product (AGOP), that characterized feature learning in neural networks. We provided empirical evidence that AGOP captured features learned by various neural network architectures, including transformer-based language models, convolutional networks, multi-layer perceptrons, and recurrent neural networks. Moreover, we demonstrated that AGOP, which is backpropagation-free, enabled feature learning in machine learning models, such as kernel machines, that apriori could not identify task-specific features. Overall, we established a fundamental mechanism that captured feature learning in neural networks and enabled feature learning in general machine learning models.
Dynamic models have been successfully used in producing estimates of HIV epidemics at the national level due to their epidemiological nature and their ability to estimate prevalence, incidence, and mortality rates simultaneously. Recently, HIV interventions and policies have required more information at sub-national levels to support local planning, decision-making and resource allocation. Unfortunately, many areas lack sufficient data for deriving stable and reliable results, and this is a critical technical barrier to more stratified estimates. One solution is to borrow information from other areas within the same country. However, directly assuming hierarchical structures within the HIV dynamic models is complicated and computationally time-consuming. In this article, we propose a simple and innovative way to incorporate hierarchical information into the dynamical systems by using auxiliary data. The proposed method efficiently uses information from multiple areas within each country without increasing the computational burden. As a result, the new model improves predictive ability and uncertainty assessment.
Deep learning has achieved unprecedented success in recent years. This approach essentially uses the composition of nonlinear functions to model the complex relationship between input features and output labels. However, a comprehensive theoretical understanding of why the hierarchical layered structure can exhibit superior expressive power is still lacking. In this paper, we provide an explanation for this phenomenon by measuring the approximation efficiency of neural networks with respect to discontinuous target functions. We focus on deep neural networks with rectified linear unit (ReLU) activation functions. We find that to achieve the same degree of approximation accuracy, the number of neurons required by a single‐hidden‐layer (SHL) network is exponentially greater than that required by a multi‐hidden‐layer (MHL) network. In practice, discontinuous points tend to contain highly valuable information (i.e., edges in image classification). We argue that this may be a very important reason accounting for the impressive performance of deep neural networks. We validate our theory in extensive experiments.
ResearchGate has not been able to resolve any references for this publication.