No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Projection Pursuit Regression
Journal of the American Statistical Association
Abstract
A new method for nonparametric multiple regression is presented. The procedure models the regression surface as a sum of general smooth functions of linear combinations of the predictor variables in an iterative manner. It is more general than standard stepwise and stagewise regression procedures, does not require the definition of a metric in the predictor space, and lends itself to graphical interpretation.
... Other classes of functions which enable us to achieve a better rate of convergence results include single index models (also called ridge functions in some literature), where m(x) = g a T x x ∈ R d for some a ∈ R d and g : R → R (cf., e.g., Härdle and Stoker [21], Härdle, Hall and Ichimura [22], Yu and Ruppert [48], Kong and Xia [36] and Lepski and Serdyukova [37]); and projection pursuit, where m(x) = r l=1 g l a T l x x ∈ R d for some r ∈ N, a l ∈ R d and g l : R → R (l = 1, . . . , r) (cf., e.g., Friedman and Stuetzle [18], Huber [26], Jones [29,30], Hall [20], Zhao and Atkeson [49] and Ben-Ari and Steinberg [10]). In Section 22.3 in Györfi et al. [19] it is shown that suitably defined (nonlinear) least squares estimates in a (p, C)-smooth projection pursuit model achieve the univariate rate of convergence n −2p/(2p+1) up to some logarithmic factor. ...
... The estimates in Horowitz and Mammen [25] and the one for projection pursuit in Section 22.3 in Györfi et al. [19] are nonlinear (penalized) least squares estimates; therefore, it is unclear how they can be computed exactly in practice. Friedman and Stuetzle [18] described easily implementable estimates for projection pursuit, but in their definition several heuristic simplifications are used, and as a consequence it is unclear whether for these estimates any rate of convergence result can be shown. ...
... λ j is chosen approximately as IQR of a sample of size 100 of m(X), and we use the values λ 1 = 2.20, λ 2 = 1.96, λ 3 = 2.85, and λ 4 = 1.59. From distribution defined by (18) we generate a sample of size n = 100, 200, 400 and apply our newly proposed neural network regression estimate and compare our results to that of six alternative regression estimates on the same data. Then we compute the L 2 errors of these estimates approximately by using the empirical L 2 error ε L2,N (·) on an independent sample of X of sizeN = 10,000. ...
... Machine learning (ML) algorithms build a model based on sample data, known as training data, in order to make predictions or decisions without being explicitly programmed to do so [6]. As the proposed projection (PP) was proposed for the first time by the scientist (Kruska, 1969) and this method is used in order to find the important projections For the data [20]. That the term projection indicates that we are looking for the data that has been projected, and the term Pursuit means continuing successively to find a good projection for the purpose of performing the regression until obtaining the least possible error, the method of successive projections (PP) was used It is widely used in high-dimensional data in order to separate data sources randomly [15]. ...
... The scientist (Friedman, 1981) later expanded the idea and added successive projection regression (PPR) [20], and the (PPR) method is considered one of the statistical techniques that are concerned with finding the most important projections in the multidimensional data, and with finding each projection the data shrinks by means of the composite [21]. The length of the projection, and the process is repeated to find good projections until the best projections are obtained, and the most important advantage of it is that it is one of the few methods that can overcome the problem of dimensionality or the curse of dimensions (curse of dimensionality) resulting from the space of high dimensions, and that the basic idea of the successive projection regression (PPR) is to model the surface of the regression as a sum of non-linear functions of the linear combinations of the variables, which is expressed by the following function [19] : PPR is a generalization of the Additive Model, and it deals with models that are in the following form: ...
... The scientist (Friedman, 1981) later expanded the idea and added successive projection regression (PPR) [20], and the (PPR) method is considered one of the statistical techniques that are concerned with finding the most important projections in the multidimensional data, and with finding each projection the data shrinks by means of the composite [21]. The length of the projection, and the process is repeated to find good projections until the best projections are obtained, and the most important advantage of it is that it is one of the few methods that can overcome the problem of dimensionality or the curse of dimensions (curse of dimensionality) resulting from the space of high dimensions, and that the basic idea of the successive projection regression (PPR) is to model the surface of the regression as a sum of non-linear functions of the linear combinations of the variables, which is expressed by the following function [19] : PPR is a generalization of the Additive Model, and it deals with models that are in the following form: ...
In this study, the research aims to use some methods that deal with several independent variables with a dependent variable, where two methods were used to deal with, which is the method of slice inverse regression (SIR), which is considered a non-classical method, and two methods of machine learning, which is (TLBO, PSO), which is most popular of the teaching methods machine learning, the work of (SIR), (TLBO, PSO) is based on making reduced linear combinations of a partial set of the original explanatory variables, which may suffer from the problem of heterogeneity and the problem of multicollinearity between most of the explanatory variables. These new combinations of linear compounds resulting from the two methods will reduce the largest number of explanatory variables to reach one or more new dimensions called the effective dimension. The root mean square error criterion will be used to compare the two methods to indicate the preference of the methods.
... The measurement of the efficiency or performance of homogeneous production or services, namely decision-making units (DMUs), has long been a hotspot or subject of efficiency of small and medium-sized samples that do not obey normal distribution; a better model should be established with a better generalization ability and robustness. On the other hand, projection pursuit regression (PPR) is particularly applied to modeling with nonlinear, small, and medium-sized samples that do not obey normal distribution [25][26][27][28]. Therefore, this paper introduces PPR into efficiency research to establish a DEA-PPR combined model to obtain more reliable and robust results. ...
... PPR is consistent with the BPNN model regarding the nonlinear approximation ability, especially for the nonlinear modeling of small and medium-sized samples that do not obey normal distribution [25][26][27][28]44,45]. The constraint condition of the PPR model is that the sum of the squares of the weights of all independent variables equals one. ...
... The PPR model was proposed by Friedman and Stuetzle [25]. Because its constraint is that the sum of the squares of the weights of each independent variable is equal to 1, over-training is less likely to occur. ...
Data envelopment analysis (DEA) is a leading approach in performance analysis and discovering newer benchmarks, and the traditional DEA models cannot forecast the future efficiency of decision-making units (DMUs). Machine learning, such as the artificial neural networks (ANNs), support vector machine/regression (SVM/SVR), projection pursuit regression (PPR), etc., have been viewed as beneficial for managers in predicting system behaviors. PPR is especially suitable for small and non-normal distribution samples, the usual cases in DEA analysis. This paper integrates DEA and PPR to cover the shortcomings we faced while using DEA and DEA-BPNN, DEA-SVR, etc. This study explores the advantages of combining these complementary methods into an integrated performance measurement and prediction model. Firstly, the DEA approach is used to evaluate and rank the efficiency of DMUs. Secondly, we establish two DEA-PPR combined models to describe the DEA efficiency scores (also called the production function) and the DEA-efficient frontier function. The first combined model’s input variables are input–output indicators in the DEA model, and the output variable is the DEA efficiency. In the second model, its input variables are input or output indicators in the DEA model, and the output variable is the optimal input indicator for input-oriented DEA or the output indicator for output-oriented DEA. We conducted positive research on two examples with actual data and virtual small, medium-sized, and large samples. Compared with the DEA-BPNN and DEA-SVR models, the results show that the DEA-PPR combined model has more vital global optimization ability, better convergence, higher accuracy, and a simple topology. The DEA-PPR model can obtain robust results for both small and large cases. The DEA-BPNN and DEA-SVR models cannot obtain robust results for small and medium-sized samples due to overfitting. For large samples, the DEA-PPR model outperforms DEA-BPNN, DEA-SVR, etc. The DEA-PPR combined model possesses better suitability, applicability, and reliability than the DEA-BPNN model, the DEA-SVR model, etc.
... Without loss of generality, assume that x i is partitioned into x T i(1) = (x i1 , x i2 , . . . , x iq ) and x T i (2) = (x i(q+1) , . . . , x ip ) for some q ∈ {1, . . . ...
... It is assumed that the response and the covariates are centered, and thus the intercept term is omitted without loss of generality. There are several approaches for the estimation of non-parametric additive models, including the back-fitting technique (see [2]), simultaneous estimation and optimization [3][4][5][6], mixed model approach [1,7,8], and Boosting approach [9,10]. Ref. [5] has presented a review of some of these methods, up to 2006, and [11] has performed several comparisons between these techniques. ...
Determining the predictor variables that have a non-linear effect as well as those that have a linear effect on the response variable is crucial in additive semi-parametric models. This issue has been extensively investigated by many researchers in the area of semi-parametric linear additive models, and various separation methods are proposed by the authors. A popular issue that might affect both estimation and separation results is the existence of outliers among the observations. In order to address this lack of sensitivity towards extreme observations, robust estimating approaches are frequently applied. We propose a robust method for simultaneously identifying the linear and nonlinear components of a semi-parametric linear additive model, even in the presence of outliers in the observations. Additionally, this model is sparse in that it may be used to determine which explanatory variables are ineffective by giving accurate zero estimates for their coefficients. To assess the effectiveness of the proposed method, a comprehensive Monte Carlo simulation study is conducted along with an application to investigate the dataset, which includes Boston property prices dataset.
... In addition it is not always apriori clear which model space to use. The nonparametric additive model [13] has been seen as one of the most direct models for multivariate nonparametric learning problems. There, we assume features do not interact, or more formally that the regression function takes the following additive form: ...
... For example, ψ (2,2,2) (x) = 2 3/2 3 k=1 cos(πx k ) is excluded since it varies in all three dimensions. We formalize this using an infinite value for the index in our rule (13). ...
... It is highly flexible and designed to capture the nonlinear relationship between the response and predictor variable via the smooth function ( (⋅)). The smooth functions for each response-predictor variable pair are estimated simultaneously via the maximized penalized likelihood method [19]. Ultimately, the smooth functions are all added together to formulate a simple yet powerful GAM model as shown in Eq. (3). ...
... Ultimately, the smooth functions are all added together to formulate a simple yet powerful GAM model as shown in Eq. (3). For detailed mathematical information on the algorithm, the reader is referred to Hastie and Tibshirani [14] and Friedman and Stuetzle [19]. The model can be expressed as: ...
... A second extension is by transforming the variables. This is done in, for example, additive models (Friedman and Stuetzle (1981); Hastie and Tibshirani (1990); Winsberg and Ramsay (1980)) and Optimal Scaling regression (OS-regression) (Young et al., 1976;Gifi, 1990; Van der Kooij and Meulman, 1999). The predictor variables are transformed using either a parametric or a nonparametric function. ...
... This type of algorithm is referred to as alternating least squares in the psychometric literature (Gifi, 1990;Young et al., 1976), since the least squares solution is calculated by alternating the estimation of optimal quantifications and model coefficients for one variable at a time. In the statistical literature it is called backfitting and has been extensively used to fit Additive Models and GAMs (Friedman and Stuetzle, 1981;Hastie and Tibshirani, 1990). A variety of other terms is present in the literature, like component-wise update and block relaxation, but it is currently usually referred to as coordinate descent. ...
In Generalized Linear Models (GLMs) it is assumed that there is a linear effect of the predictor variables on the outcome. However, this assumption is often too strict, because in many applications predictors have a nonlinear relation with the outcome. Optimal Scaling (OS) transformations combined with GLMs can deal with this type of relations. Transformations of the predictors have been integrated in GLMs before, e.g. in Generalized Additive Models. However, the OS methodology has several benefits. For example, the levels of categorical predictors are quantified directly, such that they can be included in the model without defining dummy variables. This approach enhances the interpretation and visualization of the effect of different levels on the outcome. Furthermore, monotonicity restrictions can be applied to the OS transformations such that the original ordering of the category values is preserved. This improves the interpretation of the effect and may prevent overfitting. The scaling level can be chosen for each individual predictor such that models can include mixed scaling levels. In this way, a suitable transformation can be found for each predictor in the model. The implementation of OS in logistic regression is demonstrated using three datasets that contain a binary outcome variable and a set of categorical and/or continuous predictor variables.
... Further related work -With its origins on the classical projection pursuit method (Friedman and Tukey, 1974;Friedman and Stuetzle, 1981), there is an extensive literature dedicated to designing and analysing efficient algorithms to train multi-index models models, such as Isotronic Regression for single-index (Brillinger, 1982;Kalai and Sastry, 2009;Kakade et al., 2011) and Sliced Inverse Regression in the multi-index case (Dalalyan et al., 2008;Yuan, 2011;Fornasier et al., 2012;Babichev and Bach, 2018). The case p = 1 has seen a lot of interest recently. ...
Multi-index models -- functions which only depend on the covariates through a non-linear transformation of their projection on a subspace -- are a useful benchmark for investigating feature learning with neural networks. This paper examines the theoretical boundaries of learnability in this hypothesis class, focusing particularly on the minimum sample complexity required for weakly recovering their low-dimensional structure with first-order iterative algorithms, in the high-dimensional regime where the number of samples is $n=\alpha d$ is proportional to the covariate dimension $d$. Our findings unfold in three parts: (i) first, we identify under which conditions a \textit{trivial subspace} can be learned with a single step of a first-order algorithm for any $\alpha\!>\!0$; (ii) second, in the case where the trivial subspace is empty, we provide necessary and sufficient conditions for the existence of an {\it easy subspace} consisting of directions that can be learned only above a certain sample complexity $\alpha\!>\!\alpha_c$. The critical threshold $\alpha_{c}$ marks the presence of a computational phase transition, in the sense that no efficient iterative algorithm can succeed for $\alpha\!<\!\alpha_c$. In a limited but interesting set of really hard directions -- akin to the parity problem -- $\alpha_c$ is found to diverge. Finally, (iii) we demonstrate that interactions between different directions can result in an intricate hierarchical learning phenomenon, where some directions can be learned sequentially when coupled to easier ones. Our analytical approach is built on the optimality of approximate message-passing algorithms among first-order iterative methods, delineating the fundamental learnability limit across a broad spectrum of algorithms, including neural networks trained with gradient descent.
... (Hoerl and Kennard, 1970) and is implemented in the package glmnet (Friedman et al., 2010). Projection pursuit regression (PPR) is a non-parametric smoother (Friedman and Stuetzle, 1981) and is available as a core function in R (Team, 2023). Support vector machines (SVM) (Cortes and Vapnik, 1995) is a non-linear kernel based algorithm, whose implementation in the package e1071 (Meyer et al., 2023) was employed. ...
The aim of this analysis is to predict whether an NBA player will be active in the league for at least 10 years so as to be qualified for NBA's full retirement scheme which allows for the maximum benefit payable by law. We collected per game statistics for players during their second year, drafted during the years 1999 up to 2006, for which information on their career longetivity is known. By feeding these statistics of the sophomore players into statistical and machine learning algorithms we select the important statistics and manage to accomplish a satisfactory predictability performance. Further, we visualize the effect of each of the selected statistics on the estimated probability of staying in the league for more than 10 years.
... (Hoerl and Kennard, 1970) and is implemented in the package glmnet (Friedman et al., 2010). Projection pursuit regression (PPR) is a non-parametric smoother (Friedman and Stuetzle, 1981) and is available as a core function in R (Team, 2023). Support vector machines (SVM) (Cortes and Vapnik, 1995) is a non-linear kernel based algorithm, whose implementation in the package e1071 (Meyer et al., 2023) was employed. ...
The aim of this analysis is to predict whether an NBA player will be active in the league for at least 10 years so as to be qualified for NBA's full retirement scheme which allows for the maximum benefit payable by law. We collected per game statistics for players during their second year, drafted during the years 1999 up to 2006, for which information on their career longetivity is known. By feeding these statistics of the sophomore players into statistical and machine learning algorithms we select the important statistics and manage to accomplish a satisfactory predictability performance. Further, we visualize the effect of each of the selected statistics on the estimated probability of staying in the league for more than 10 years.
... The low-complexity structure described in (6) is common in various models. Besides the additive model, similar structures can also be observed in other statistical inference models, e.g., the single index model (Hardle, Hall, and Ichimura 1993), the projection pursuit model (Friedman and Stuetzle 1981). Moreover, recent research (Chen, Wang, and Yang 2023) has shown that operators associated with well-known PDEs, including the Poisson, parabolic, and Burgers equation, exhibit this structure or its variants. ...
Deep reinforcement learning (RL) has shown remarkable success in specific offline decision-making scenarios, yet its theoretical guarantees are still under development. Existing works on offline RL theory primarily emphasize a few trivial settings, such as linear MDP or general function approximation with strong assumptions and independent data, which lack guidance for practical use. The coupling of deep learning and Bellman residuals makes this problem challenging, in addition to the difficulty of data dependence. In this paper, we establish a non-asymptotic estimation error of pessimistic offline RL using general neural network approximation with C-mixing data regarding the structure of networks, the dimension of datasets, and the concentrability of data coverage, under mild assumptions. Our result shows that the estimation error consists of two parts: the first converges to zero at a desired rate on the sample size with partially controllable concentrability, and the second becomes negligible if the residual constraint is tight. This result demonstrates the explicit efficiency of deep adversarial offline RL frameworks. We utilize the empirical process tool for C-mixing sequences and the neural network approximation theory for the Holder class to achieve this. We also develop methods to bound the Bellman estimation error caused by function approximation with empirical Bellman constraint perturbations. Additionally, we present a result that lessens the curse of dimensionality using data with low intrinsic dimensionality and function classes with low complexity. Our estimation provides valuable insights into the development of deep offline RL and guidance for algorithm model design.
... In our case the relation between response and predictor is complicated, for which linear parametric regression analysis gives ambiguous results. Therefore, a nonparametric regression analysis, which considers assumptions on the regression surface was utilized (Friedman and Stuetzle, 1981). As a result, the alternating conditional expectations (ACE) approach, which was developed by Breiman and Friedman (1985), is utilized to measure the optimal transformations of dependent and predictor variables. ...
Lakes, the most widespread inland water bodies in the globe, are highly susceptible to change in trophic state due
to external factors. Changing hydro-climatic conditions and land cover changes (LCC) can cause lake water
quality deterioration. This study establishes the quantitative relationship between variability in the water quality
index and changes in hydro-climatic and LCC variables. Water quality is represented by the Forel-ule index (FUI)
whereas the hydro-climatic variables considered in this study are lake bottom layer temperature (lblt), lake total
layer temperature (ltlt), precipitation, runoff, evaporation, lake skin temperature (lskt), surface wind speed and
air temperature. The LCC is quantified by lower and higher level leaf area index (Lv-lai and Hv-lai). FUI has a
positive relationship with surface wind speed, precipitation, runoff, ltlt, lblt, and LCC and a negative relationship
with evaporation, lskt, and air temperature with 95% confidence level over most parts of the Lake. The temporal
correlation is also apparent from the long-term trend pattern. A significant decreasing trend is observed in FUI
and lake bottom layer temperature (lblt). In contrast, an insignificant increasing trend is observed in air tem�perature and lake skin temperature (lskt). The changes in LCC, runoff, precipitation, and surface wind speed is
insignificant between 2000 and 2020. Moreover, the phase composites of FUI and hydro-climatic and LCC
variables derived from multichannel singular spectrum analysis (MSSA) show strong seasonal modulation of
water quality by hydro-climatic and LCC variables. The annual cycle represented by the first two eigenmodes
(except wind speed which is represented by the second and third eigenmodes) accounts for between 27.41%
(wind speed) to 52.32% (precipitation) of the total joint spatiotemporal variability of FUI and the driving var�iables. The convergent cross-mapping (CCM) analysis shows that cross-map skill (ρ2
) is increased with increasing
library length (L) and time delay (τ), which suggests significant causal effects of hydro-climatic and LCC variables
on FUI and the lagged causation is consistent with maximum values of ρ2
. The significant feedback of FUI to
changes in hydro-climatic and LCC variables shows the possibility of hindcast/forecast of the historical/future
status of water quality from hydro-climatic and LCC variables. As a result, a multivariate nonlinear regression
model (MNWQFM) is developed to forecast the lake water quality index from the hydro-climatic and LCC var�iables. The model has high performance with R2 of 83.6% and root means square error (RMSE) of 0.15 in FUI.
... The solution procedure is similar to the principle of maximum entropy (McElreath 2018). Derived from GLMs and the additive model (Friedman and Stuetzle 1981), Hastie and Tibshirani (1987) developed generalized additive models (GAMs), where the linear response variable depends linearly on unknown smooth functions. The focus of interest in GAMs is on the inferences about these smooth functions, which are usually approximated using a linear combination of polynomials, penalized splines (Eilers and Marx 1996), or wavelet functions (Antoniadis and Fan 2001). ...
In semiparametric regression, traditional methods such as mixed generalized additive models (GAM), computed via Laplace approximation or variational approximation using penalized marginal likelihood estimation, may not achieve sparsity and unbiasedness simultaneously, and may sometimes suffer from convergence problems. To address these issues, we propose an estimator for semiparametric generalized additive models based on the marginal likelihood. Our approach provides sparsity estimates and allows for statistical inference. To estimate and select variables, we use the smoothly clipped absolute deviation penalty (SCAD) within the framework of variational approximation. We also propose efficient iterative algorithms to obtain estimations. Simulation results support our theoretical characteristics, and we demonstrate that our method is more effective than the original variational approximations framework and many other penalized methods under certain conditions. Moreover, applications with actual data further demonstrate the superior performance of the proposed method.
... The effect of variables on CPUE was examined by means of GAM techniques (Hastie and Tibshirani, 1986). The additive models and their generalizations make available to use many nonparametric models which are essential in regression analysis, when the linearity assumption does not engage well (Friedman and Stuetzle, 1981;Hastie and Tibshirani, 1986;Amodio et al., 2014). Additionally, the advantages (i.e. ...
In this study, we applied generalized additive model to investigate the influence of spatial temporal variables and vessel length on catch per unit-effort (CPUE) of Atlantic bluefin tuna (ABFT) purse seine fishery using catch and effort data from commercial logbooks and field surveys from 1992 to 2006. The vessel lengths of sampled purse seines ranged from 20 to 64 m. The number of ABFT caught within each operation varied between 1 and 2000. A total of 386 CPUE values for ABFT were calculated 0.05 and 60 t ⋅ (haul day)-1 with mean CPUE of 5.51 ± 0.54 t ⋅ (haul day)-1. Although the sea surface temperature had little influence on the CPUE, the effect of time and spatial variables, vessel length and salinity was found as significant. In conclusion, the spatial dynamics of ABFT should be considered if the impact of fisheries on the ecosystem is to be reduced.
... Research has shown that it has the same nonlinear approximation ability as BPNN. Still, it is especially suitable for small and medium sample data modeling that does not obey the ordinary distribution law [20][21][22][41][42][43][44][45][46]. Due to PPR, the model of independent variable weight sum is equal to 1 for multiple independent variables with collaborative constraints. ...
The drastic fluctuations in pork prices directly affect the sustainable development of pig farming, agriculture, and feed processing industries, reducing people’s happiness and sense of gain. Although there have been extensive studies on pork price prediction and early warning in the literature, some problems still need further study. Based on the monthly time series data of pork prices and other 11 influencing prices (variables) such as beef, hog, piglet, etc., in China from January 2000 to November 2023, we have established a project pursuit auto-regression (PPAR) and a hybrid PPAR (H-PPAR) model. The results of the PPAR model study show that the monthly pork prices in the lagged periods one to three have an important impact on the current monthly pork price. The first lagged period has the largest and most positive impact. The second lagged period has the second and a negative impact. We built the H-PPAR model using the 11 independent variables (prices), including the prices of corn, hog, mutton, hen’s egg, and beef in lagged period one, the piglet’s price in lagged period six, and by deleting non-important variables. The results of the H-PPAR model show that the hog price in lagged period one is the most critical factor, and beef price and the other six influencing variables are essential factors. The model’s performance metrics show that the PPAR and H-PPAR models outperform approaches such as support vector regression, error backpropagation neural network, dynamic model average, etc., and possess better suitability, applicability, and reliability. Our results forecast the changing trend of the monthly pork price and provide policy insights for administrators and pig farmers to control and adjust the monthly pork price and further enhance the health and sustainable development of the hog farming industry.
... Although it is straightforward to transform selection estimates on PCs back to the original trait space, the resulting estimates of selection on the original traits are biased (Chong et al., 2018). Projection pursuit regression (Friedman & Stuetzle, 1981) can also help solve interpretability problems by defining orthogonal axes of the multivariate phenotype that maximize the explained variation in fitness (Schluter & Nychka, 1994). However, when this approach is implemented in a reduced data space, the estimates of selection are also biased. ...
The breeder’s equation, Δz¯=Gβ , allows us to understand how genetics (the genetic covariance matrix, G) and the vector of linear selection gradients β interact to generate evolutionary trajectories. Estimation of β using multiple regression of trait values on relative fitness revolutionized the way we study selection in laboratory and wild populations. However, multicollinearity, or correlation of predictors, can lead to very high variances of and covariances between elements of β , posing a challenge for the interpretation of the parameter estimates. This is particularly relevant in the era of big data, where the number of predictors may approach or exceed the number of observations. A common approach to multicollinear predictors is to discard some of them, thereby losing any information that might be gained from those traits. Using simulations, we show how, on the one hand, multicollinearity can result in inaccurate estimates of selection, and, on the other, how the removal of correlated phenotypes from the analyses can provide a misguided view of the targets of selection. We show that regularized regression, which places data-validated constraints on the magnitudes of individual elements of β, can produce more accurate estimates of the total strength and direction of multivariate selection in the presence of multicollinearity and limited data, and often has little cost when multicollinearity is low. We also compare standard and regularized regression estimates of selection in a reanalysis of three published case studies, showing that regularized regression can improve fitness predictions in independent data. Our results suggest that regularized regression is a valuable tool that can be used as an important complement to traditional least-squares estimates of selection. In some cases, its use can lead to improved predictions of individual fitness, and improved estimates of the total strength and direction of multivariate selection.
... The relationship between blood pressure and MCAv was then characterized using PPR. Briefly, PPR is a nonparametric, multiple-regression method that iteratively sums a linear combination of ridge terms (Friedman & Stuetzle, 1981). A single ridge function was used as utilizing more than one can reduce the ability to derive physiological interpretations from the data due to potential interactions between ridge functions (Taylor et al., 2014). ...
To compare the construct validity and between‐day reliability of projection pursuit regression (PPR) from oscillatory lower body negative pressure (OLBNP) and squat‐stand maneuvers (SSMs). Nineteen participants completed 5 min of OLBNP and SSMs at driven frequencies of 0.05 and 0.10 Hz across two visits. Autoregulatory plateaus were derived at both point‐estimates and across the cardiac cycle. Between‐day reliability was assessed with intraclass correlation coefficients (ICCs), Bland–Altman plots with 95% limits of agreement (LOA), coefficient of variation (CoV), and smallest real differences. Construct validity between OLBNP‐SSMs were quantified with Bland–Altman plots and Cohen's d . The expected autoregulatory curve with positive rising and negative falling slopes were present in only ~23% of the data. The between‐day reliability for the ICCs were poor‐to‐good with the CoV estimates ranging from ~50% to 70%. The 95% LOA were very wide with an average spread of ~450% for OLBNP and ~350% for SSMs. Plateaus were larger from SSMs compared to OLBNPs (moderate‐to‐large effect sizes). The cerebral pressure‐flow relationship is a complex regulatory process, and the “black‐box” nature of this system can make it challenging to quantify. The current data reveals PPR analysis does not always elicit a clear‐cut central plateau with distinctive rising/falling slopes.
... Projection Pursuit Regression (PPR) is a statistical method used for constructing regression models [49]. The core idea of PPR is to optimize non-linear projection functions to simplify the relationship between the target variable and the independent variables in the projected space. ...
Chlorophyll content is highly susceptible to environmental changes, and monitoring these changes can be a crucial tool for optimizing crop management and providing a foundation for research in plant physiology and ecology. This is expected to deepen our scientific understanding of plant ecological adaptation mechanisms, offer a basis for improving agricultural production, and contribute to ecosystem management. This study involved the collection of hyperspectral data, image data, and SPAD data from jujube leaves. These data were then processed using SG smoothing and the isolated forest algorithm, following which eigenvalues were extracted using a combination of Pearson’s phase relationship method and the Partial Least Squares Regression–continuous projection method. Subsequently, seven methods were employed to analyze the results, with hyperspectral data and color channel data used as independent variables in separate experiments. The findings indicated that the integrated BPNN-RF-Ridge Regression algorithm provided the best results, with an R2 of 0.8249, MAE of 2.437, and RMSE of 2.9724. The inclusion of color channel data as an independent variable led to a 3.2% improvement in R2, with MAE and RMSE increasing by 1.6% and 3.9%, respectively. These results demonstrate the effectiveness of integrated methods for the determination of chlorophyll content in jujube leaves and underscore the potential of using multi-source data to improve the model fit with a minimal impact on errors. Further research is warranted to explore the application of these findings in precision agriculture for jujube yield optimization and income-related endeavors, as well as to provide insights for similar studies in other plant species.
... 1. Multivariate Adaptive Regression Spline (gcvEarth) (Milborrow, 2020) 2. Partial Least Squares Regression (pls) (Mevik et al., 2020) 3. Bayesian Generalized Linear Model (bayesglm) (Gelman et al., 2008;Gelman & Su, 2020) 4. Elastic Net Regression ( glmnet) (Friedman et al., 2010) 5. Cubist (cubist) (Kuhn & Quinlan, 2020) 6. Projection Pursuit Regression (ppr) (Friedman & Stuetzle, 1981) 7. Gaussian Process with Radial Basis Function Kernel ( gaussprRadial) (Karatzoglou et al., 2004;Williams & Rasmussen, 1996) 8. Support Vector Machine with Radial Kernel (svmRadial) (Hearst et al., 1998;Karatzoglou et al., 2004) Firstly the dataset was randomly split into training and test sets using an 80:20 ratio, and 10-fold cross-validation was used. After training the five different models, their performances were evaluated by computing the cross-validated absolute errors. ...
In typical machine learning frameworks, model selection is of fundamental importance: commonly, multiple models have to be trained and compared in order to identify the one with the best predictive performances. The aim of this study is to provide a new tool to improve the model selection process, allowing the user to identify the algorithm which significantly outperforms the other candidates. It proposes a robust model selection procedure based on a multi‐aspect permutation test which makes it possible to detect differences in both location and variability between two paired samples of prediction errors. A new extension of the nonparametric combination (NPC) methodology is therefore introduced and is integrated with an appropriate ranking procedure in order to deal with the comparison of C≥2$$ C\ge 2 $$ candidate models. A simulation study is conducted to evaluate the performances of this testing procedure in 2‐sample and C$$ C $$‐sample problems, by generating data from various well‐known distributions and simulating several possible null and alternative scenarios. The adoption of the proposed technique in machine learning model selection problems is then discussed by means of multiple real data applications. These applications confirm what emerges from the simulation study: the introduced NPC‐based approach performs well under several different scenarios and represents a valuable tool for robust machine learning model selection.
... Additive regression as a nonparametric regression technique [10], enhances the performance of weak prediction models based on a specified criterion. It achieves this by aggregating contributions obtained from other models. ...
Predicting sunspot numbers presents ongoing challenges in forecasting , including non-stationary patterns and unclear fluctuation dynamics. This study compares traditional methods and hybrid models, incorporating machine learning techniques, to predict monthly mean sunspot numbers (MMSNs) from January 1, 1900, to December 31, 2022. Among the traditional methods, ARIMA(5,0,4) demonstrated performance with an MSE of 580.949, RMSE of 24.103, MAE of 17.19, and MAPE of 0.511. However, the proposed hybrid model, which combines ARIMA(5,0,4) with additive regression (AR) using Regression by Discretization (RegbyDisc) based on J48, achieved markedly superior forecasting accuracy with an MSE of 114.653, RMSE of 10.708, MAE of 6.441, and MAPE of 0.438. We employed this hybrid model to forecast sunspot numbers from
... The projection pursuit model, a mathematical, statistical model proposed by Kruskal in 1972 for nonlinear and informal high-dimensional data, was used instead to make the weight determination of complex data robust and objectively reasonable. By projecting high-dimensional data into low-dimensional subspaces, the projection pursuit model can match optimal values [48,49], the basic steps for which were as follows. ...
Simple Summary
African swine fever (ASF) has a significant impact on the pig industry, leading to drastic fluctuations in the supply and price of live pigs on the market, leading to substantial economic losses for farmers. As China enters the ‘post-ASF’ era of regular epidemic prevention and control, the behavior of farmers becomes crucial for effectively preventing and controlling major animal epidemics. To analyze the current situation and factors influencing farmers’ epidemic prevention and control behaviors, survey data were empirically analyzed. The findings indicate that the overall level of biosecurity in pig farms is low. Additionally, factors such as technical training, farm size, income share, production organization, and government inspection significantly influence farmers’ adoption of biosecurity measures.
Abstract
Effective biosecurity measures are crucial in controlling and preventing major pig diseases, ultimately ensuring farm income and social stability. This study extracted data from 205 farmer surveys in Sichuan Province, China, to construct a biosecurity index system for pig farms. The biosecurity levels of pig farms were evaluated using a projection pursuit method to identify weak areas. The Tobit model was then utilized to determine the factors that influenced the biosecurity levels. The results indicated that the overall biosecurity levels of the pig farms were low. The study found that the average biosecurity score among farms was 0.61, with a minimum score of 0.37 and a maximum score of 0.89 (on a scale of 0 to 1). These results suggest that there are significant differences in biosecurity levels among the farms. The study also found that the scores for first-level indicators related to breeding environment management, as well as second-level indicators related to personnel management and awareness of African swine fever prevention and control, were significantly lower than scores for other indicators in the farmers’ biosecurity systems. This study investigated the factors influencing biosecurity on farms and found that technical training, farm size, income share, production organization, and government inspections had a significant impact on the level of biosecurity implemented. This study emphasizes the significance of biosecurity in enhancing pig farm biosecurity and its role in improving farm resilience to major animal diseases like African swine fever. It also provides valuable insights for policymakers to make informed decisions regarding related policies.
... The GAMs are particularly useful for exploratory data analysis to allow the data to "speak for themselves" (Yee, 2015). GAMs have resulted from additive models (Friedman & Stuetzle, 1981) and have been introduced by Hastie and Tibshirani (1990). GAM framework was extended further by Wahba (1990), Eilers and Marx (1996), Ruppert et al. (2003), Reiss and Ogden (2009), Wood (2000, 2003, 2004, 2008. ...
In the twenty first century horizontal price transmission has become the topic of a great interest in applied microeconomics research in terms of the perspective of understanding on how geographically separated markets function. The paper provides detailed review of applied research in the field of the spatial price transmission modelling, also the most popular econometric models have been discussed in the light of the main advantages and disadvantages with a special focus on nonlinear techniques. Being in line with the last studies on non-linear time series models of spatial agri-food price transmission and market integration, we introduce non-parametric technique of generalized additive modelling in order to give evidence of agri-food market integration efficiency and non-linear patterns in price linkages. The results of our empirical approach may contribute to the knowledge about market efficiency and competitiveness as well as provide information to policymakers. KEYWORDS: Horizontal price transmission, market integration, nonlinear time series , generalized additive model
ABSTRAKT Téma horizontálnej cenovej transmisie sa z hľadiska pochopenia fungovania geograficky oddelených trhov stala v oblasti aplikovaného mikroekonomického výskumu významnou najmä v dvadsiatom prvom storočí. Tento príspevok poskytuje detailný prehľad apliko-vaného výskumu v oblasti modelovania priestorového prenosu cien, diskutuje aplikované ekonometrické modely s prezentáciou ich hlavných výhod a nevýhod s osobitným zamer-aním na nelineárne modelovacie techniky. V súlade s poslednými štúdiami o nelineárnych modeloch časových radov priestorového prenosu agropotravinárskych cien a trhovej inte-grácie je predstavená neparametrická metóda zovšeobecneného aditívneho modelovania (GAM, z angl. generalized additive modelling). Táto metóda umožňuje overenie efek-tívnosti integrácie agropotravinárskych trhov a odhalenie nelinearít v cenových prepoje-niach. Výsledky nášho empirického prístupu môžu prispieť k poznaniu efektívnosti trhov a konkurencieschopnosti, ako aj poskytnúť informácie tvorcom politík. Kľúčové slová: Horizontálny prenos cien, trhová integrácia, nelineárne časové rady, zovšeobecnený aditívny model JEL CLASSIFICATION: Q110, C510
The view expressed in the WP and language revision is those of the author.
Layout by: Sergei Kharin, Cand. Sc. Institute of Economic Research SAS/Ekonomický ústav SAV, v.v.i., Šancová 56, 811 05 Bratislava, www.ekonom.sav.sk
CONTACT: sergei.kharin@savba.sk ©Institute of Economic Research SAS/Ekonomický ústav SAV, v.v.i., Bratislava 2023
In developing countries, especially China, large-scale commercial complexes are the current trend in commercial real estate. Compared with other similar buildings, scientific site selection is very important for the smooth construction and efficient operation of these complexes. However, there is still a lack of a targeted evaluation index system and quantitative evaluation methods. Therefore, this paper put forward the evaluation index system and method of large-scale commercial-complex location based on a projection pursuit model. First, this paper comprehensively considered the environmental, social, and economic factors, and used secondary and tertiary indicators to systematically establish an evaluation index system. This index system effectively dealt with the complex problem of its site selection. Compared with the traditional multi-attribute evaluation method based on expert advice, the evaluation method of the improved projection pursuit model based on a sparrow search algorithm constructed in this paper was to mine key information from the evaluation data, which could evaluate the site suitability of large commercial complexes more scientifically and objectively. In addition, this paper made a detailed empirical study of the Joy City project in Nanchang, China. The research results found the key factors affecting the site selection of the project and determined that the site-selection evaluation result of the project was medium. The research results of this paper provide the scientific and objective decision-making basis for the development enterprises of large commercial complexes to reduce site-selection risk and improve investment efficiency.
Shapley values originated in cooperative game theory but are extensively used today as a model-agnostic explanation framework to explain predictions made by complex machine learning models in the industry and academia. There are several algorithmic approaches for computing different versions of Shapley value explanations. Here, we consider Shapley values incorporating feature dependencies, referred to as conditional Shapley values, for predictive models fitted to tabular data. Estimating precise conditional Shapley values is difficult as they require the estimation of non-trivial conditional expectations. In this article, we develop new methods, extend earlier proposed approaches, and systematize the new refined and existing methods into different method classes for comparison and evaluation. The method classes use either Monte Carlo integration or regression to model the conditional expectations. We conduct extensive simulation studies to evaluate how precisely the different method classes estimate the conditional expectations, and thereby the conditional Shapley values, for different setups. We also apply the methods to several real-world data experiments and provide recommendations for when to use the different method classes and approaches. Roughly speaking, we recommend using parametric methods when we can specify the data distribution almost correctly, as they generally produce the most accurate Shapley value explanations. When the distribution is unknown, both generative methods and regression models with a similar form as the underlying predictive model are good and stable options. Regression-based methods are often slow to train but quickly produce the Shapley value explanations once trained. The vice versa is true for Monte Carlo-based methods, making the different methods appropriate in different practical situations.
In cancer studies, it is commonplace that a fraction of patients participating in the study are cured, such that not all of them will experience a recurrence, or death due to cancer. Also, it is plausible that some covariates, such as the treatment assigned to the patients or demographic characteristics, could affect both the patients’ survival rates and cure/incidence rates. A common approach to accommodate these features in survival analysis is to consider a mixture cure survival model with the incidence rate modeled by a logistic regression model and latency part modeled by the Cox proportional hazards model. These modeling assumptions, though typical, restrict the structure of covariate effects on both the incidence and latency components. As a plausible recourse to attain flexibility, we study a class of semiparametric mixture cure models in this article, which incorporates two single-index functions for modeling the two regression components. A hybrid nonparametric maximum likelihood estimation method is proposed, where the cumulative baseline hazard function for uncured subjects is estimated nonparametrically, and the two single-index functions are estimated via Bernstein polynomials. Parameter estimation is carried out via a curated expectation-maximization algorithm. We also conducted a large-scale simulation study to assess the finite-sample performance of the estimator. The proposed methodology is illustrated via application to two cancer datasets.
We present convergence estimates of two types of greedy algorithms in terms of the entropy numbers of underlying compact sets. In the first part, we measure the error of a standard greedy reduced basis method for parametric PDEs by the entropy numbers of the solution manifold in Banach spaces. This contrasts with the classical analysis based on the Kolmogorov [Formula: see text]-widths and enables us to obtain direct comparisons between the algorithm error and the entropy numbers, where the multiplicative constants are explicit and simple. The entropy-based convergence estimate is sharp and improves upon the classical width-based analysis of reduced basis methods for elliptic model problems. In the second part, we derive a novel and simple convergence analysis of the classical orthogonal greedy algorithm for nonlinear dictionary approximation using the entropy numbers of the symmetric convex hull of the dictionary. This also improves upon existing results by giving a direct comparison between the algorithm error and the entropy numbers.
Groundwater resources are vital to ecosystems and livelihoods. Excessive groundwater withdrawals can cause groundwater levels to decline1–10, resulting in seawater intrusion¹¹, land subsidence12,13, streamflow depletion14–16 and wells running dry¹⁷. However, the global pace and prevalence of local groundwater declines are poorly constrained, because in situ groundwater levels have not been synthesized at the global scale. Here we analyse in situ groundwater-level trends for 170,000 monitoring wells and 1,693 aquifer systems in countries that encompass approximately 75% of global groundwater withdrawals¹⁸. We show that rapid groundwater-level declines (>0.5 m year⁻¹) are widespread in the twenty-first century, especially in dry regions with extensive croplands. Critically, we also show that groundwater-level declines have accelerated over the past four decades in 30% of the world’s regional aquifers. This widespread acceleration in groundwater-level deepening highlights an urgent need for more effective measures to address groundwater depletion. Our analysis also reveals specific cases in which depletion trends have reversed following policy changes, managed aquifer recharge and surface-water diversions, demonstrating the potential for depleted aquifer systems to recover.
The Interstellar Boundary Explorer (IBEX) has been observing the outer heliosphere and its interactions with the very local interstellar medium (VLISM) via measurements of energetic neutral atoms (ENAs) for over 14 yr. We discovered the IBEX Ribbon—a structure completely unanticipated by any prior theory or model—that almost certainly resides beyond the heliopause in the VLISM. We also characterized the other major source of heliospheric ENAs, the globally distributed flux (GDF), produced largely in the heliosheath between the termination shock and heliopause. In this study, we make three major new contributions. First, we validate, provide, and analyze the most recent 3 yr of IBEX-Hi (0.5–6 keV FWHM) data (2020–2022) for the first time. Second, we link these observations to the prior 11 yr of observations, exploring long-term variations. Finally, we provide the first IBEX team-validated Ribbon/GDF separation scheme and separated maps. Because of the uncertainty in separating different line-of-sight integrated sources, we provide not just best guess (median) maps, but also maps with upper and lower reasonable values of Ribbon and GDF fluxes, along with bounding fluxes that add the uncertainties to the upper and lower values. This allows theories and models to be compared with a range of possible values that the IBEX team believes are consistent with data. These observations, along with the reanalysis of the prior 11 yr of IBEX-Hi data, provide new insights and even further develop our detailed understanding of the heliosphere’s interaction with the local interstellar medium unlocked by IBEX.
The results of an analysis of the mathematical dependences of the parameters of subtle impact damage on samples made of polymer composite materials with an impact-sensitive luminescent smart coating applied to the surface on the magnitude of the impact energy, constructed according to visual-measurement control data, are presented. The control was carried out both with and without the use of an ultraviolet lamp. It was revealed that the parameters of the detected damage (the area of the dent on the surface of the samples) are not related by a simple linear relationship with the impact energy, which indicates complex processes of destruction of the structure of the samples, including the manifestation of an abrupt transition from an elastic response to an inelastic response of the indented structure when the critical energy value is exceeded blow.
Biased population samples pose a prevalent problem in the social sciences. Therefore, we present two novel methods that are based on positive-unlabeled learning to mitigate bias. Both methods leverage auxiliary information from a representative data set and train machine learning classifiers to determine the sample weights. The first method, named maximum representative subsampling (MRS), uses a classifier to iteratively remove instances, by assigning a sample weight of 0, from the biased data set until it aligns with the representative one. The second method is a variant of MRS – Soft-MRS – that iteratively adapts sample weights instead of removing samples completely. To assess the effectiveness of our approach, we induced artificial bias in a public census data set and examined the corrected estimates. We compare the performance of our methods against existing techniques, evaluating the ability of sample weights created with Soft-MRS or MRS to minimize differences and improve downstream classification tasks. Lastly, we demonstrate the applicability of the proposed methods in a real-world study of resilience research, exploring the influence of resilience on voting behavior. Through our work, we address the issue of bias in social science, amongst others, and provide a versatile methodology for bias reduction based on machine learning. Based on our experiments, we recommend to use MRS for downstream classification tasks and Soft-MRS for downstream tasks where the relative bias of the dependent variable is relevant.
This study examines the effect of land cover, vegetation health, climatic forcings, elevation heat loads, and terrain characteristics (LVCET) on land surface temperature (LST) distribution over West Africa (WA). We employ fourteen machine-learning models, which preserve nonlinear relationships, to downscale LST and other predictands while preserving the geographical variability of WA. Our results showed that the random forest model performs best in downscaling predictands. This is important for the sub-region since it has limited access to mainframes to power multiplex machine-learning algorithms. In contrast to the northern regions, the southern regions consistently exhibit healthy vegetation. Also, areas with unhealthy vegetation coincide with hot LST clusters. The positive Normalized Difference Vegetation Index (NDVI) trends in the Sahel underscore rainfall recovery and subsequent Sahelian greening. The southwesterly winds cause the upwelling of cold waters, lowering LST in southern WA and highlighting the cooling influence of water bodies on LST. Identifying regions with elevated LST is paramount for prioritizing greening initiatives, and our study underscores the importance of considering LVCET factors in urban planning. Topographic slope-facing angles, heat loads, and diurnal anisotropic heat all contribute to variations in LST, emphasizing the need for a holistic approach when designing resilient and sustainable landscapes.
In projection pursuit regression (PPR), a univariate response variable is approximated by the sum of M “ridge functions,” which are flexible functions of one-dimensional projections of a multivariate input variable. Traditionally, optimization routines are used to choose the projection directions and ridge functions via a sequential algorithm, and M is typically chosen via cross-validation. We introduce a novel Bayesian version of PPR, which has the benefit of accurate uncertainty quantification. To infer appropriate projection directions and ridge functions, we apply novel adaptations of methods used for the single ridge function case (M=1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$M=1$$\end{document}), called the Bayesian Single Index Model; and use a Reversible Jump Markov chain Monte Carlo algorithm to infer the number of ridge functions M. We evaluate the predictive ability of our model in 20 simulated scenarios and for 23 real datasets, in a bake-off against an array of state-of-the-art regression methods. Finally, we generalize this methodology and demonstrate the ability to accurately model multivariate response variables. Its effective performance indicates that Bayesian Projection Pursuit Regression is a valuable addition to the existing regression toolbox.
In this paper, we consider variable selection for single-index models via martingale difference divergence. The important covariates are selected by maximizing penalized martingale difference divergence objective functions with fixed tuning parameters. To choose tuning parameters, we propose to use a BIC criterion. The consistency of the variable selection procedure with LASSO, SCAD and ALASSO penalties and asymptotic properties of the resulting estimators are established. The performance of the proposed procedure is assessed through extensive simulation studies. Finally, we apply the proposed method to real data sets.
This paper introduces a Factor Augmented Sparse Throughput (FAST) model that utilizes both latent factors and sparse idiosyncratic components for nonparametric regression. The FAST model bridges factor models on one end and sparse nonparametric models on the other end. It encompasses structured nonparametric models such as factor augmented additive models and sparse low-dimensional nonparametric interaction models and covers the cases where the covariates do not admit factor structures. Via diversified projections as estimation of latent factor space, we employ truncated deep ReLU networks to nonparametric factor regression without regularization and to a more general FAST model using nonconvex regularization, resulting in factor augmented regression using neural network (FAR-NN) and FAST-NN estimators respectively. We show that FAR-NN and FAST-NN estimators adapt to the unknown low-dimensional structure using hierarchical composition models in nonasymptotic minimax rates. We also study statistical learning for the factor augmented sparse additive model using a more specific neural network architecture. Our results are applicable to the weak dependent cases without factor structures. In proving the main technical result for FAST-NN, we establish a new deep ReLU network approximation result that contributes to the foundation of neural network theory. Our theory and methods are further supported by simulation studies and an application to macroeconomic data.
Motivated by the computational difficulties incurred by popular deep learning algorithms for the generative modeling of temporal densities, we propose a cheap alternative that requires minimal hyperparameter tuning and scales favorably to high-dimensional problems. In particular, we use a projection-based optimal transport solver [Meng et al.,Advances in Neural Information Processing Systems (Curran Associates, 2019), Vol. 32] to join successive samples and, subsequently, use transport splines (Chewi et al., 2020) to interpolate the evolving density. When the sampling frequency is sufficiently high, the optimal maps are close to the identity and are, thus, computationally efficient to compute. Moreover, the training process is highly parallelizable as all optimal maps are independent and can, thus, be learned simultaneously. Finally, the approach is based solely on numerical linear algebra rather than minimizing a nonconvex objective function, allowing us to easily analyze and control the algorithm. We present several numerical experiments on both synthetic and real-world datasets to demonstrate the efficiency of our method. In particular, these experiments show that the proposed approach is highly competitive compared with state-of-the-art normalizing flows conditioned on time across a wide range of dimensionalities.
This study examines the effect of land cover, vegetation health, climatic forcings, elevation heat loads and terrain characteristics (LVCET) on land surface temperature (LST) distribution in West Africa (WA). We employed fourteen machine-learning models, which preserve nonlinear relationships to downscale LST while preserving WA's geographical variability. Our results showed that the simple random model was the best in downscaling predictands. This is important for the sub-region since its access to mainframes, which could power more multiplex machine-learning algorithms, is limited. The yearly vegetation health based on the Normalized Difference Vegetation Index (NDVI) and self-organized maps (SOM) indicates constant healthy vegetation in most southern areas but unhealthy vegetation in the northern area. Locations where we found unhealthy vegetation coincided with the hot LST clusters as categorized by SOM. Also, the southwest winds cause the upwelling of cold waters, lowering LST in southern WA.
This emphasizes the cooling influence of water bodies on LST. Identifying high LST locations is vital to prioritizing places for greening. A high heat load and diurnal anisotropic heat might translate to a relatively high LST depending on the topographic slope-facing angle. Therefore, urban planners should consider the joint attribution of LST dynamics to LVCET while planning landscapes.
Patients with dysautonomia often experience symptoms such as dizziness, syncope, blurred vision and brain fog. Dynamic cerebral autoregulation, or the ability of the cerebrovasculature to react to transient changes in arterial blood pressure, could be associated with these symptoms.
In this narrative review, we go beyond the classical view of cerebral autoregulation to discuss dynamic cerebral autoregulation, focusing on recent advances pitfalls and future directions.
Following some historical background, this narrative review provides a brief overview of the concept of cerebral autoregulation, with a focus on the quantification of dynamic cerebral autoregulation. We then discuss the main protocols and analytical approaches to assess dynamic cerebral autoregulation, including recent advances and important issues which need to be tackled.
The researcher or clinician new to this field needs an adequate comprehension of the toolbox they have to adequately assess, and interpret, the complex relationship between arterial blood pressure and cerebral blood flow in healthy individuals and clinical populations, including patients with autonomic disorders.
Recent papers have used machine learning architecture to fit low-order functional ANOVA models with main effects and second-order interactions. These GAMI (GAM + Interaction) models are directly interpretable as the functional main effects and interactions can be easily plotted and visualized. Unfortunately, it is not easy to incorporate the monotonicity requirement into the existing GAMI models based on boosted trees, such as EBM (Lou et al. 2013) and GAMI-Lin-T (Hu et al. 2022). This paper considers models of the form $f(x)=\sum_{j,k}f_{j,k}(x_j, x_k)$ and develops monotone tree-based GAMI models, called monotone GAMI-Tree, by adapting the XGBoost algorithm. It is straightforward to fit a monotone model to $f(x)$ using the options in XGBoost. However, the fitted model is still a black box. We take a different approach: i) use a filtering technique to determine the important interactions, ii) fit a monotone XGBoost algorithm with the selected interactions, and finally iii) parse and purify the results to get a monotone GAMI model. Simulated datasets are used to demonstrate the behaviors of mono-GAMI-Tree and EBM, both of which use piecewise constant fits. Note that the monotonicity requirement is for the full model. Under certain situations, the main effects will also be monotone. But, as seen in the examples, the interactions will not be monotone.
In this study, the non-hypothetical projection pursuit regression (NH-PPR) is proposed. The proposed NH-PPR model can predict the hydration heat based on the four cement phases, FA, SL, cement fineness and hydration time. The NH-PPR model is proposed by using the multiple layer iteration method and the non-hypothetical and non-parametric ridge functions to enhance accuracy and solve the problems caused by the parameter selection and the subjective hypothesis. The modeling data set is applied to train model, the testing data set is regressed and fitted into the model, and then the obtained results are compared with the BP model. To further validate the proposed model, another published data set is used to obtain a higher degree of confidence in the prediction. It is shown that the proposed model obtains the better accuracy, stability and versatility, and avoids the parameter selection and subjective hypothesis.
An algorithm for the analysis of multivariate data is presented and is discussed in terms of specific examples. The algorithm seeks to find one-and two-dimensional linear projections of multivariate data that are relatively highly revealing.
An algorithm for the analysis of multivariate data is presented and is discussed in terms of specific examples. The algorithm seeks to find one- and two-dimensional linear projections of multivariate data that are relatively highly revealing.
The cross validation mean square error technique is used to determine the correct degree of smoothing, in fitting smoothing solines to discrete, noisy observations from some unknown smooth function. Monte Cario results snow amazing success in estimating the true smooth function as well as its derivative.
A dependent variable is some unknown function of independent variables plus an error component. If the magnitude of the error could be estimated with minimal assumptions about the underlying functional dependence, then this could be used to judge goodness-of-fit and as a means of selecting a subset of the independent variables which best determine the dependent variable. We propose a procedure for this purpose which is based on a data-directed partitioning of the space into subregions and a fitting of the function in each subregion. The behavior of the procedure is heuristically discussed and illustrated by some simulation examples.
Let $M_0$ be a normal linear regression model and let $M_1,\cdots, M_K$ be distinct proper linear submodels of $M_0$. Let $\hat k \in \{0,\cdots, K\}$ be a model selection rule based on observed data from the true model. Given $\hat k$, let the unknown parameters of the selected model $M_{\hat k}$ be fitted by the maximum likelihood method. A loss function is introduced which depends additively on two parts: (i) a measure of the difference between the fitted model $M_{\hat k}$ and the true model; and (ii) a measure $C_{\hat k}$ of the "complexity" of the selected model. A natural model selection rule $\bar{k}$, which minimizes an empirical version of this loss, is shown to be admissible and very nearly Bayes.
Let $(X, Y)$ be a pair of random variables such that $X$ is $\mathbb{R}^d$-valued and $Y$ is $\mathbb{R}^{d'}$-valued. Given a random sample $(X_1, Y_1), \cdots, (X_n, Y_n)$ from the distribution of $(X, Y)$, the conditional distribution $P^Y(\bullet \mid X)$ of $Y$ given $X$ can be estimated nonparametrically by $\hat{P}_n^Y(A \mid X) = \sum^n_1 W_{ni}(X)I_A(Y_i)$, where the weight function $W_n$ is of the form $W_{ni}(X) = W_{ni}(X, X_1, \cdots, X_n), 1 \leqq i \leqq n$. The weight function $W_n$ is called a probability weight function if it is nonnegative and $\sum^n_1 W_{ni}(X) = 1$. Associated with $\hat{P}_n^Y(\bullet \mid X)$ in a natural way are nonparametric estimators of conditional expectations, variances, covariances, standard deviations, correlations and quantiles and nonparametric approximate Bayes rules in prediction and multiple classification problems. Consistency of a sequence $\{W_n\}$ of weight functions is defined and sufficient conditions for consistency are obtained. When applied to sequences of probability weight functions, these conditions are both necessary and sufficient. Consistent sequences of probability weight functions defined in terms of nearest neighbors are constructed. The results are applied to verify the consistency of the estimators of the various quantities discussed above and the consistency in Bayes risk of the approximate Bayes rules.
The greatest or least value of a function of several variables is to be found when the variables are restricted to a given region. A method is developed for dealing with this problem and is compared with possible alternatives. The method can be used on a digital computer, and is incorporated in a program for Mercury.
Van Hove Analysis of the Reactions n -p 4 ? r -n -n + p and n -p -+ n+?r+n-at 16 GeV/C
- References Ballam
- J Chadwick
- G B Guiragossian
- W B Leith
- D W G S Morigasu
REFERENCES BALLAM, J.. CHADWICK, G.B.. GUIRAGOSSIAN. Z.C.G., JOHN-SON, W.B., LEITH, D. W.G.S., and MORIGASU, J. (1971 ). " Van Hove Analysis of the Reactions n -p 4 ? r -n -n + p and n -p -+ n+?r+n-at 16 GeV/C, " Physics Review, 4, 1946-1947.
Smoothing Tech-niques for Curve Estimation
- T Gasser
GASSER, T., and ROSENBLATT, M. (eds.) (1979), " Smoothing Tech-niques for Curve Estimation, " in Lecture Notes in Mathematics 757, New York: Springer-Verlag.
Multivariate Model Building: The Validation of a Search Strategy Uni-versity of Michigan, Ann Arbor Nonparametric Regression and Its Applications Admissible Selection of an Accurate and Parsimonious Normal Linear Regression Model Annals ofStnrisrics, 9. in press
- J Sonquist
- C J Stone
SONQUIST, J. (1970), " Multivariate Model Building: The Validation of a Search Strategy, " Report, Institute for Social Research, Uni-versity of Michigan, Ann Arbor. STONE, C.J. (1977), " Nonparametric Regression and Its Applications " (with discussion), Annuls of Starisrics, 5, 595-645. (1981), " Admissible Selection of an Accurate and Parsimonious Normal Linear Regression Model, " Annals ofStnrisrics, 9. in press. TUKEY, J.W. (1977), EDA Explorutory Data Analysis, Reading, Mass.: Addison-Wesley.
- Cleveland W. S.