nep-for New Economics Papers
on Forecasting
Issue of 2020‒03‒30
six papers chosen by
Rob J Hyndman
Monash University

  1. On bootstrapping tests of equal forecast accuracy for nested models By Firmin Doko Tchatoka; Qazi Haque
  2. Betting markets for English Premier League results and scorelines: evaluating a simple forecasting model By J. James Reade; Carl Singleton; Leighton Vaughan Williams
  3. Does More Expert Adjustment Associate with Less Accurate Professional Forecasts? By Franses, Ph.H.B.F.; Welz, M.
  4. Forecasting in a complex environment: Machine learning sales expectations in a Stock Flow Consistent Agent-Based simulation model By Ermanno Catullo; Mauro Gallegati; Alberto Russo
  5. On bootstrapping tests of equal forecast accuracy for nested models By Firmin Doko Tchatoka; Qazi Haque
  6. Early Childhood Education and Life-cycle Health By Jorge Luis Garcia; James J. Heckman

  1. By: Firmin Doko Tchatoka; Qazi Haque
    Abstract: The asymptotic distributions of the recursive out-of-sample forecast accuracy test statistics depend on stochastic integrals of Brownian motion when the models under comparison are nested. This often complicates their implementation in practice because the computation of their asymptotic critical values is costly. Hansen and Timmermann (2015, Econometrica) propose a Wald approximation of the commonly used recursive F-statistic and provide a simple characterization of the exact density of its asymptotic distribution. However, this characterization holds only when the larger model has one extra predictor or the forecast errors are homoscedastic. No such closed-form characterization is readily available when the nesting involves more than one predictor and heteroskedasticity is present. We first show both the recursive F-test and its Wald approximation have poor finite-sample properties, especially when the forecast horizon is greater than one. We then propose a hybrid bootstrap method consisting of a block moving bootstrap (which is nonparametric) and a residual based bootstrap for both statistics, and establish its validity. Simulations show that our hybrid bootstrap has good finite-sample performance, even in multi-step ahead forecasts with heteroscedastic or autocorrelated errors, and more than one predictor. The bootstrap method is illustrated on forecasting core inflation and GDP growth.
    Keywords: Out-of-sample forecasts, HAC estimator, Moving block bootstrap, Bootstrap consistency
    JEL: C12 C15 C32
    Date: 2020–03
  2. By: J. James Reade (Department of Economics, University of Reading); Carl Singleton (Department of Economics, University of Reading); Leighton Vaughan Williams (Nottingham Business School, Nottingham Trent University, UK)
    Abstract: Using betting odds from two recent seasons of English Premier League football matches, we evaluate probability and point forecasts generated from a standard statistical model of goal scoring. The bookmaker odds show significant evidence of the favourite-longshot bias for exact scorelines, which is not generally present for match results. We find evidence that the scoreline probability forecasts from the model are better than what the odds of bookmakers imply, based on forecast encompassing regressions. However, when we apply a simple betting strategy using point forecasts from the model, there are no substantial or consistent financial returns to be made over the two seasons. In other words, there is no evidence from this particular statistical model that the result, scoreline, margin of victory or total goals betting markets are on average inefficient.
    Keywords: Forecasting, Statistical modelling, Regression models, Prediction markets
    JEL: C53 G14 G17 L83
    Date: 2020–03–20
  3. By: Franses, Ph.H.B.F.; Welz, M.
    Abstract: Professional forecasters can rely on an econometric model to create their forecasts. It is usually unknown to what extent they adjust an econometric model‐based forecast. In this paper we show, while making just two simple assumptions, that it is possible to estimate the persistence and variance of the deviation of their forecasts from forecasts from an econometric model. A key feature of the data that facilitates our estimates is that we have forecast updates for the same forecast target. An illustration to consensus forecasters who give forecasts for GDP growth, inflation and unemployment for a range of countries and years suggests that the more a forecaster deviates from a prediction from an econometric model, the less accurate are the forecasts.
    Keywords: professional forecasters, econometric model, expert adjustment, forecast accuracy
    JEL: C53
    Date: 2020–01–01
  4. By: Ermanno Catullo (Research Department, Link Campus University, Rome, Italy); Mauro Gallegati (Department of Management, Università Politecnica delle Marche, Acona, Italy); Alberto Russo (Department of Management, Università Politecnica delle Marche, Ancona, Italy and Department of Economics, Universitat Jaume I, Castellón, Spain)
    Abstract: The aim of this paper is to investigate how different degrees of sophistication in agents’ behavioural rules may affect individual and macroeconomic performances. In particular, we analyze the effects of introducing into an agentbased macro model firms that are able to formulate effective sales forecasts by using machine learning. These techniques are able to provide predictions that are unbiased and present a certain degree of accuracy, especially in the case of a genetic algorithm. We observe that machine learning allows firms to increase profits, though this result in a declining wage share and a smaller long-run growth rate. Moreover, the predictive methods are able to formulate expectations that remain unbiased when shocks are not massive, thus providing firms with forecasting capabilities that to a certain extent may be consistent with the Lucas Critique.
    Keywords: agent-based model, machine learning, genetic algorithm, forecasting, policy shocks
    JEL: C63 D84 E32 E37
    Date: 2020
  5. By: Firmin Doko Tchatoka (School of Economics, The University of Adelaide); Qazi Haque (Economics Discipline, Business School, University of Western Australia and Centre for Applied Macroeconomic Analysis, Australian National University)
    Keywords: Out-of-sample forecasts; HAC estimator; Moving block bootstrap; Bootstrap consistency
    JEL: C12 C15 C32
    Date: 2020
  6. By: Jorge Luis Garcia (Clemson University); James J. Heckman (The University of Chicago)
    Abstract: This paper forecasts the life-cycle treatment effects on health of a high-quality early childhood program. Our predictions combine microsimulation using non-experimental data with experimental data from a midlife long-term follow-up. The follow-up incorporated a full epidemiological exam. The program mainly benefits males and significantly reduces the prevalence of heart disease, stroke, cancer, and mortality across the life-cycle. For men, we estimate an average reduction of 3.8 disability-adjusted years (DALYs). The reduction in DALYs is relatively small for women. The gain in quality-adjusted life years (QALYs) is almost enough to offset all of the costs associated with program implementation for males and half of program costs for women.
    Keywords: early childhood education, life-cycle health, long-term forecasts, program evaluation, randomized trials
    JEL: I10 J13 I28 C93
    Date: 2020–03

This nep-for issue is ©2020 by Rob J Hyndman. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.
General information on the NEP project can be found at For comments please write to the director of NEP, Marco Novarese at <>. Put “NEP” in the subject, otherwise your mail may be rejected.
NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.