Figure 1 - uploaded by Ullrich Heilemann
Content may be subject to copyright.
Source publication
The major focus of this paper is to determine whether the accur acy of German macroeconomic forecasts has improved over time. We examine 1-year-ahead forecasts of real GDP and inflation for 1967 to 2001 made by three major Germ an forecasting groups and the OECD. We examine the accuracy of the forecasts over the entire period and in three sub-perio...
Contexts in source publication
Context 1
... main focus of our analysis is on the question of whether the German forecasts have improved over time. Nevertheless, we summarize the results for the entire period , 1967- 2001. The forecasts and the actual data of growth and inflation are shown in Figure 1, and the results of the accuracy analysis are in Table 1. The MAE of the growth forecasts is about 1.2 percentage points. This was about 40% of the mean absolute ...
Context 2
... the impact that wage inflation and the oil-shocks had on inflation in the first half of the 1970s can be observed, but the statistics decline steadily towards a limit afterwards. The most plausible explanation is that exogenous inflation impulses and internal inflation behavior simply had normalized (see Figure 1) and forecasters have been able to forecast accurately in this ...
Similar publications
In this paper I conducted a simple experiment to using Artificial Neural Network in time-series forecasting, by combining First order Markov Switching Model and K-means algorithms, the forecasting performance of machine learning has outperformed the benchmark of time-series inflation rate forecasting. The paper reveal the potential of ANN forecasti...
Confronted by a slowing economy, the Reserve Bank of India has undertaken steps to revive it. These measures, however, run the risk of worsening current high levels of inflation. This paper examines certain aspects of India's financial system that have contributed to this situation. It argues that unduly low yields on Government bonds have prevente...
Suriname's overall economic performance has been relatively weak throughout the 1980's and much of the 1990's, with episodes of economic contraction. Moreover, during the 1990's there were at least two periods with near-hyperinflation. However, stabilization measures since 2001 and favorable commodity prices have improved Suriname's economic perfor...
Citations
... Additionally, it might be of interest from a monetary policy perspective, which institution ranks high in a list of forecasters since both FED and ECB conduct a survey of professional forecasters (see, for example, Meyler 2020; Rich and Tracy 2021, for the ECB). Finally, comparing forecast accuracy across countries (Heilemann and Müller 2018;Heilemann and Stekler 2013) might also give valuable insights, for example, in analyzing possible lower bounds of accuracy. ...
We rank the quality of German macroeconomic forecasts using various methods for 17 regular annual German economic forecasts from 14 different institutions for the period from 1993 to 2019. Using data for just one year, rankings based on different methods correlate only weakly with each other. Correlations of rankings calculated for two consecutive years and a given method are often relatively low and statistically insignificant. For the total sample, rank correlations between institutions are generally relatively high among different criteria. We report substantial long-run differences in forecasting quality, which are mostly due to distinct average forecast horizons. In the long-run, choosing the criterion to rank the forecasters is of minor importance. Rankings based on recession years and normal periods are similar. The same does hold for rankings based on real-time vs revised data.
... A large number of existing studies have examined the accuracy and efficiency of German macroeconomic forecasts (see e.g. Heilemann and Stekler 2013;Fritsche and Tarassow 2017;Döpke et al. 2019, and the literature cited therein). Prior research suggests three key insights. ...
... Döpke et al. 2010;Krüger and Hoss 2012). Second, forecast errors seem to be stable on average over decades which are neither increasing nor decreasing in tendency (Heilemann and Stekler 2013). Third, no forecaster's performance is uniformly superior (Döpke and Fritsche 2006), and there are not significant institutional differences in accuracy across a long time horizon (Döhrn and Schmidt 2011). 1 Recently, another forecast evaluation approach, which uses qualitative text as data, has become increasingly popular. ...
... for Germany (Heilemann and Stekler 2013;Döpke et al. 2019). The ME is nearly zero, indicating unbiased forecasts. ...
Based on German business cycle forecast reports covering 10 German institutions for the period 1993-2017, the paper analyses the information content of German forecasters' narratives for German business cycle forecasts. The paper applies textual analysis to convert qualitative text data into quantitative sentiment indices. First, a sentiment analysis utilizes dictionary methods and text regression methods, using recursive estimation. Next, the paper analyses the different characteristics of sentiments. In a third step, sentiment indices are used to test the efficiency of numerical forecasts. Using 12-month-ahead fixed horizon forecasts, fixed-effects panel regression results suggest some informational content of sentiment indices for growth and inflation forecasts. Finally, a forecasting exercise analyses the predictive power of sentiment indices for GDP growth and inflation. The results suggest weak evidence, at best, for in-sample and out-of-sample predictive power of the sentiment indices.
... Nevertheless, increases in computing power and a better understanding of how to separate signal from noise should lead to some improvements in forecast accuracy. However, this does not appear to have been the case, at least for macroeconomic forecasting (Fildes & Stekler, 2002;Heilemann & Stekler, 2013;Stekler, 2007). ...
... A large body of literature has addressed the accuracy and efficiency of German macroeconomic forecasts (see e.g. Heilemann and Stekler, 2013;Fritsche and Tarassow, 2017;Döpke et al., 2019, and the literature cited therein). To sum up the general results, three key insights can be concluded. ...
... Döpke et al. (2010) and Krüger and Hoss (2012)). Second, there is no obvious tendency of the forecast errors to increase or decrease (Heilemann and Stekler, 2013). Third, no forecaster's performance is uniformly superior (Döpke and Fritsche, 2006), and there are not significant institutional differences in accuracy across a long time horizon (Döhrn and Schmidt, 2011). ...
... Table 1 provides an overview of some standard measures of forecast evaluation (see for example Fildes and Stekler, 2002) for the pooled data of the introduced sample. On the whole, the error measures correspond to previous forecast evaluation studies for Germany (Heilemann and Stekler, 2013;Döpke et al., 2019). The ME is nearly zero, indicating unbiased forecasts. ...
Based on German business cycle forecast reports covering 10 German
institutions for the period 1993–2017, the paper analyses the information content of German forecasters’ narratives for German
business cycle forecasts. The paper applies textual analysis to convert
qualitative text data into quantitative sentiment indices. First, a sentiment analysis utilizes dictionary methods and text regression
methods, using recursive estimation. Next, the paper analyses the
different characteristics of sentiments. In a third step, sentiment indices are used to test the efficiency of numerical forecasts. Using
12-month-ahead fixed horizon forecasts, fixed-effects panel regression
results suggest some informational content of sentiment indices for
growth and inflation forecasts. Finally, a forecasting exercise analyses
the predictive power of sentiment indices for GDP growth and inflation. The results suggest weak evidence, at best, for in-sample and out-of-sample predictive power of the sentiment indices.
... The research on macroeconomic forecasts published by German economic research institutes has a long tradition in the scientific community with early analyses by Neumann and Buscher (1980) and Kirchgässner (1984). Today, the research topics in this field are manifold, including studies on forecast revisions (Kirchgässner and Müller 2006), forecast accuracy (Heilemann and Stekler 2013), external assumptions of forecasts (Engelke et al. 2019), forecaster rankings (Kirchgässner 1993;Sinclair et al. 2016), or the economic value of forecasts (Döpke et al. 2018). Most of these studies focus on the analysis of GDP and inflation forecasts by means of panel-based models (Döpke and Fritsche 2006;Müller et al. 2019) or time series models (Kirchgässner and Savioz 2001). ...
This study contributes to research on the nonparametric evaluation of German trade forecasts. To this end, I compute random classification and regression forests to analyze the optimality of annual German export and import growth forecasts from 1970 to 2017. A forecast is considered as optimal if a set of predictors, which models the information set of a forecaster at the time of forecast formation, has no explanatory power for the corresponding (sign of the) forecast error. I analyze trade forecasts of four major German economic research institutes, a collaboration of German economic research institutes, and one international forecaster. For trade forecasts with a horizon of half-a-year, I cannot reject forecast optimality for all but one forecaster. In the case of a forecast horizon of one year, forecast optimality is rejected in more cases if the underlying loss function is assumed to be quadratic. Allowing for a flexible loss function results in more favorable assessment of forecast optimality.
... Nevertheless, increases in computing power and a better understanding of how to separate signal from noise should lead to some improvements in forecast accuracy. However, this does not appear to have been the case, at least for macroeconomic forecasting (Fildes & Stekler, 2002;Heilemann & Stekler, 2013;Stekler, 2007). ...
This paper provides a non-systematic review of the progress of forecasting in social settings. It is aimed at someone outside the field of forecasting who wants to understand and appreciate the results of the M4 Competition, and forms a survey paper regarding the state of the art of this discipline. It discusses the recorded improvements in forecast accuracy over time, the need to capture forecast uncertainty, and things that can go wrong with predictions. Subsequently, the review classifies the knowledge achieved over recent years into (i) what we know, (ii) what we are not sure about, and (iii) what we don’t knowIn the first two areas, we explore the difference between explanation and prediction, the existence of an optimal model, the performance of machine learning methods on time series forecasting tasks, the difficulties of predicting non-stable environments, the performance of judgment, and the value added by exogenous variables. The article concludes with the importance of (thin and) fat tails, the challenges and advances in causal inference, and the role of luck.
... We reexamine the efficiency of growth and inflation forecasts of four leading German economic research institutes during the sample period from 1970 to 2016. Our research adds to significant earlier work on various aspects of growth and inflation forecasts for Germany (for early studies, see [9,26,31] among others). 1 In recent studies, Heilemann and Stekler [22] study the time-varying accuracy of growth and inflation forecasts. Kirchgässner and Müller [27] highlight the implications of costly forecast revisions. ...
We use Bayesian additive regression trees to reexamine the efficiency of growth and inflation forecasts for Germany. To this end, we use forecasts of four leading German economic research institutes for the sample period from 1970 to 2016. We reject the strong form of forecast efficiency and find evidence against the weak form of forecast efficiency for longer-term growth and longer-term inflation forecasts. We cannot reject weak efficiency of short-term growth and inflation forecasts and of forecasts disaggregated at the institute level. We find that Bayesian additive regression trees perform significantly better than a standard linear efficiency-regression model in terms of forecast accuracy.
... • Second, there is no obvious tendency of forecast errors to in-or decrease over time. Heilemann and Stekler (2013) analyse the long-term development of forecast accuracy of German GDP growth and inflation forecasts from 1967 to 2010 and come to a rather sobering conclusion: Enhancements of in form of small forecast errors in the 1980s and 1990s appeared to be only temporary in nature and are largely driven by a low inflation and growth variance in these periods. The authors summarize that neither technical (e.g. ...
Based on a panel of annual data for 17 growth and inflation forecasts from 14 institutions for Germany, we analyse forecast accuracy for the periods before and after the Great Recession, including measures of directional change accuracy based on Receiver Operating Curves (ROC). We find only small differences on forecast accuracy between both time periods. We test whether the conditions for forecast rationality hold in both time periods. We document an increased cross-section variance of forecasts and a changed correlation between inflation and growth forecast errors after the crisis, which might hint to a changed forecaster behaviour. This is also supported by estimated loss functions before and after the crisis, which suggest a stronger incentive to avoid overestimations (growth) and underestimations (inflation) after the crisis. Estimating loss functions for a 10-year rolling window also reveal shifts in the level and direction of loss asymmetry and strengthens the impression of a changed forecaster behaviour after the Great Recession.
... The results in Table 6 broadly confirm the findings of previous studies, for example, Heilemann and Stekler (2013). As regards the mean absolute errors, the growth predictions show values of near or slightly under one percentage point, while inflation rate projections reveal an error of roughly half of a percentage point. ...
We evaluate the economic value of business cycle forecasts for potential investors on financial markets as opposed to statistical measures of forecasts accuracy. Taking Germany as an example, and based on annual data ranging from 1990 to 2016 covering 16 institutions and 18 different forecasts, we calculate the value of portfolios that actively react to business cycle forecasts and compare these values with the value of portfolios that are passively managed. We find that actively managed portfolios do not systemically outperform passively managed ones, while statistical measures suggest that forecasts are better than naive ones. Furthermore, statistical and economic measures of forecast quality often correlate negatively across forecasters. One main reason for the difference between the two views on forecast quality is that the economic value of a correct forecast changes during time. We check the robustness of our results by applying several trading rules referring to business cycle forecasts.
... However, the forecasting credibility of many central institutions was shattered in the recent volatile years, mainly because of their inability to foresee the economic turning point and an underestimation of the amplitude of the changes that arrived (Stekler, 2008; Frankel & Schreger, 2013; Wickens, 2014). Indeed, available empirical evidence suggests that predictions issued by some 2 central institutions are often marked by systemic bias (Daníelson, 2008; Campbell & Murphy, 2006) and display difficulties with incremental and over-time improvement (Öller & Barot, 2000; Heilemann & Stekler, 2003). It further reveals a strong mutual interconnection (correlation) between forecasts prepared by the central institutions and their supra-national patrons (Öller, 2000; Marinheiro, 2011). ...
The paper deals with the accuracy of the real GDP growth forecasts produced by two Czech non-governmental institutions: the Czech-Moravian Confederation of Trade Unions (CMKOS) and the Czech Banking Association (CBA) in the years 2007-2014 and 2011-2014 respectively. Utilizing a research method composed of simple (AFE), scale-dependent (RMSE) as well as relative (MASE) error measures, we found out that (i) CMKOS predictions achieved a lower forecasting error on average, beginning with a notable overestimation in the first turnover point from growth to decline (2008-2009), yet followed by gradual improvement resulting in superior accuracy over set benchmarks (Ministry of Finance, Czech National Bank, OECD) in later years (2010-2014). The CBA predictions, on the other hand, exhibited (ii) a high level of interconnection with official bodies (MF, CNB), but with overall inferior forecasting accuracy, despite the shorter time frame (2011-2014). Overall, the study suggests that of the two surveyed non-governmental bodies, only CMKOS forecasts represent a viable alternative to the official predictions published by the Ministry of Finance or the Czech National Bank, as CBA forecasts were found to be a less accurate satellite of these bodies.
