nep-for New Economics Papers
on Forecasting
Issue of 2016‒09‒04
three papers chosen by
Rob J Hyndman
Monash University

  1. Predicting Fed Forecasts By Neil R. Ericsson
  2. Which Market Indicators Best Forecast Recessions? By Travis J. Berge; Nitish R. Sinha; Michael Smolyansky
  3. Predicting Experimental Results: Who Knows What? By Stefano DellaVigna; Devin Pope

  1. By: Neil R. Ericsson
    Abstract: Print{{p}}Table 1: Keywords in the FOMC Minutes, Their Assessment, and the Corresponding FOMC Minutes Index.{{p}}Keywords Assessment FOMC Minutes Index (percent per annum){{p}}Strong, robust, considerable, upbeat, brisk, surge Strong growth 4.0{{p}}Normal, solid, steady Normal growth 3.4{{p}}Modest, moderate, sustainable Modest growth 2.8{{p}}Slow, gradual, subdued, muted Slow growth 2.1{{p}}Unclear, mixed Unclear 1.5{{p}}Decelerating, stabilizing, ongoing adjustment, leveling out Decelerating growth 0.9{{p}}Continued weakness, sluggish, slack, below potential Continued weakness 0.3{{p}}Declining, deteriorating Decline -0.4{{p}}Recession, contraction, sharp and widespread decline Recession -1.0{{p}}February 12, 2016{{p}}Predicting Fed Forecasts{{p}}Neil R. Ericsson1{{p}}Monetary policy decisions by the Fed's Federal Open Market Committee (FOMC) have attracted considerable attention in recent years,{{p}}especially with quantitative easing through large-scale asset purchases, the introduction of forward guidance, and December's "lift-off"{{p}}after seven years of a near-zero federal funds rate. The FOMC's decisions are based in part on the Greenbook forecasts, which are{{p}}economic forecasts produced by the Federal Reserve Board's staff and which are presented to the FOMC prior to their policy meetings.{{p}}This note shows that the minutes of the FOMC meetings--and the information in those minutes about the Greenbook forecasts--provide{{p}}valuable insights into the decision-making process of the FOMC.{{p}}Recent analysis by Herman Stekler and Hilary Symington lays the foundation for these results.2 Stekler and Symington constructed{{p}}indexes that quantify the FOMC's views about the U.S. economy, as expressed in the minutes of the FOMC's meetings for 2006–2010.{{p}}This note compares these indexes with publicly available Greenbook forecasts, including the recently released 2010 Greenbook{{p}}forecasts. The indexes very closely track the Greenbook forecasts of the current-quarter and one-quarter-ahead U.S. real GDP growth{{p}}rates--particularly so for the sixteen forecasts in 2010, even though those forecasts were not available when the indexes were{{p}}constructed. Stekler and Symington's indexes thus provide a proximate and relatively accurate mechanism for inferring Greenbook{{p}}forecasts, well in advance of the public release of the Greenbook. The Greenbook is not released to the public until more than five years{{p}}after it is presented to the FOMC, whereas the minutes of an FOMC meeting are published just three weeks after the meeting itself.{{p}} Background: Stekler and Symington's Indexes{{p}}Stekler and Symington employ a focused textual analysis of the minutes for the 40 FOMC meetings during 2006–2010, a period that{{p}}spans the financial crisis and leads into the Great Recession. From their textual analysis, Stekler and Symington construct quantitative{{p}}indexes that gauge the FOMC's views on the current and future strength of the U.S. economy, as expressed in the FOMC minutes{{p}}themselves. The indexes are scaled such that they correspond to real GDP growth rates in percent per annum.{{p}} Source:{{p}}Stekler and{{p}}Symington{{p}}(2015, Tables{{p}}2 and 4).{{p}}To design{{p}}their{{p}}indexes,{{p}}Stekler and{{p}}Symington{{p}}examine{{p}}certain{{p}}sections of{{p}}the minutes{{p}}that discuss:{{p}}the{{p}}current economic outlook, typically in a paragraph or paragraphs beginning "The information reviewed at the ... meeting{{p}}suggested that ... "; and{{p}}i.{{p}}the future economic outlook, typically in a paragraph or paragraphs beginning "In their discussion of the economic situation{{p}}and outlook, meeting participants ... ".{{p}}ii.{{p}}In these sets of paragraphs, Stekler and Symington search for select keywords that characterize views on the outlook. Keywords range{{p}}from "strong" and "robust" for a very optimistic outlook to "recession" and "contraction" for a very pessimistic one. From the frequencies{{p}}of occurrence of the keywords, Stekler and Symington create two indexes, one for the current outlook and one for the future outlook.{{p}}These indexes are called FOMC Minutes Indexes (or FMIs) below. Table 1 lists the keywords, Stekler and Symington's assessment of{{p}}those keywords for economic growth, and the corresponding values for the FMI.{{p}}From a broader perspective, three steps lead to the FMIs.{{p}} FRB: IFDP Note: Predicting Fed Forecasts http://www.federalreserve.gov/econresdata/notes/ifdp-notes/2016/predict...{{p}}1 of 5 2/16/2016 12:16 PM{{p}}Figure 1: The current-outlook and future-outlook FMIs, the Greenbook forecasts of the current-quarter and{{p}}one-quarter-ahead U.S. real GDP growth rates, and the differentials between the FMI and Greenbook forecasts.{{p}}Accessible version{{p}}Figure 2: The truncation-adjusted FOMC Minutes Index for the current outlook, the Greenbook forecast of the U.S.{{p}}real GDP growth rate in the current quarter, and ±1 standard error bands for the FMI's predictions in 2010.{{p}}Meeting of the FOMC. The FOMC meets to discuss monetary policy, with the Greenbook forecasts and certain qualitative{{p}}and quantitative information in hand.{{p}}a.{{p}}b. Writing of the FOMC minutes. Minutes of the meeting are then prepared and made public.3{{p}}Quantification of the FOMC minutes. From textual analysis of the FOMC minutes, Stekler and Symington quantify the tone of{{p}}the FOMC's discussion about the current and future outlook of the U.S. economy and calibrate that quantification to{{p}}forecasts of the GDP growth rate, thereby generating the FMI.4{{p}}c.{{p}}The Greenbook forecasts are provided to the FOMC participants, prior to the FOMC meeting, and hence the FMI--through the FOMC's{{p}}policymaking process--may depend on the Greenbook forecasts. This characterization applies to the FMIs for both the current outlook{{p}}and the future outlook.{{p}}Comparion{{p}}of the FMI{{p}}with the{{p}}Greenbook{{p}}Forecast{{p}}It is thus of{{p}}interest to{{p}}compare the{{p}}FMI directly{{p}}with the{{p}}Greenbook{{p}}forecast, as{{p}}in Figure 1.{{p}}Figure 1a{{p}}plots two{{p}} "nowcasts":{{p}}the FMI for{{p}}the current{{p}}outlook, and{{p}}the{{p}}Greenbook{{p}}forecast for{{p}}the current{{p}}quarter.{{p}}Figure 1b{{p}}plots two forecasts: the FMI for the future outlook, and the Greenbook forecast for one quarter ahead. Figure 1c plots the difference{{p}}between the FMI for the current outlook and the Greenbook forecast for the current quarter; and Figure 1d plots the difference between{{p}}the FMI for the future outlook and the Greenbook forecast for one quarter ahead. In Figure 1 (and likewise in Figures 2 and 3 below), the{{p}}horizontal axis specifies the date of the FOMC meeting to which a Greenbook is submitted and from which an FMI is constructed. The{{p}}graphs in Figure1 show that the FMI and the Greenbook forecasts are generally very close numerically, whether for the current outlook{{p}}or for the future outlook.{{p}}That said, the FMI does deviate markedly from the Greenbook forecast in December 2008, January 2009, and March 2009: see the{{p}}boxed-in areas in Figure 1. For these FOMC meetings, the FMI is at its minimum (= –1.0% per annum, which is the most pessimistic{{p}}outlook allowed in Stekler and Symington's framework), whereas the Greenbook forecasts are typically much more negative. This{{p}}discrepancy between the FMI and the Greenbook forecast arises because Stekler and Symington's FMI as constructed from Table 1{{p}}cannot be more negative than –1%, thereby truncating the distribution of the forecast implicit in the FOMC minutes. Thus, in the next{{p}}section, these three meetings are treated separately from the other meetings when "predicting" the Greenbook forecasts from the FMI.5{{p}}Predicting Greenbook Forecasts{{p}}As Figure 1 implies, the FMI very closely approximates the Greenbook forecast, once accounting for the FMI's truncation in December{{p}}2008, January 2009, and March 2009. This close relationship between the FMI and the Greenbook forecast presents a special{{p}}opportunity for "predicting" Greenbook forecasts, conditional on the FMI, noting that the minutes of an FOMC meeting are publicly{{p}}available three weeks after the meeting, whereas the Greenbook forecasts are not released to the public until more than five years after{{p}}the Greenbook itself is presented to the FOMC. When Stekler and Symington calculated FMIs for 2006–2010, they did not have the{{p}}2010 Greenbook forecasts available, so their FMIs can be used to predict the 2010 Greenbook forecasts. Moreover, with the recent{{p}}release of the 2010 Greenbooks, those predictions from the FMIs can now be assessed against the actual 2010 Greenbook forecasts.6{{p}}Figure 2{{p}}plots the{{p}}truncation-adjusted{{p}}current-{{p}} FRB: IFDP Note: Predicting Fed Forecasts http://www.federalreserve.gov/econresdata/notes/ifdp-notes/2016/predict...{{p}}2 of 5 2/16/2016 12:16 PM{{p}}Accessible version{{p}}Figure 3: The truncation-adjusted FOMC Minutes Index for the future outlook, the Greenbook forecast of the U.S.{{p}}real GDP growth rate one quarter ahead, and ±1 standard error bands for the FMI's predictions in 2010.{{p}}Accessible version{{p}}Table 2: Root mean squared errors and other summary statistics for the deviation between the Greenbook forecast{{p}}of the U.S. real GDP growth rate and the FMI, for both current-quarter and one-quarter-ahead forecast horizons.{{p}}Forecast horizon{{p}}Sample period{{p}}RMSE Mean Standard deviation{{p}}Current quarter{{p}}2006-2009 0.96 -0.26 0.94{{p}}2010 0.65 -0.14 0.68{{p}}2006-2010 0.90 -0.23 0.88{{p}}outlook FMI,{{p}}along with{{p}}the{{p}}Greenbook{{p}}forecast of{{p}}the current{{p}}quarter's{{p}}U.S. real{{p}}GDP growth{{p}}rate. The{{p}}FMI and the{{p}}Greenbook{{p}}forecast are{{p}}very close{{p}}numerically,{{p}}with{{p}}deviations{{p}}between{{p}}them being{{p}}small,{{p}}typically less{{p}}than 1% per{{p}}annum and{{p}}often less{{p}}than 0.5%{{p}}per annum.{{p}}Figure 3{{p}}plots the{{p}}truncation-adjusted{{p}}future-outlook{{p}}FMI{{p}}and the{{p}}Greenbook{{p}}forecast for{{p}}one quarter{{p}}ahead. Their{{p}}deviations{{p}}are small as{{p}}well.{{p}}Figures 2{{p}}and 3 also{{p}}include ±1{{p}}standard{{p}}error bands{{p}}for the FMI{{p}}in 2010, with{{p}}those bands derived from the properties of the FMIs and the Greenbook forecasts over 2006–2009. For each FMI (current-outlook or{{p}}future-outlook), only one of the eight Greenbook forecasts in 2010 lies outside those bands--significantly fewer outliers than the{{p}}one-in-three expected for ±1 standard error bands. The 2010 predictions range between 2.1% and 3.4% for both the current outlook and{{p}}the future outlook, with somewhat different dynamics for the two forecast horizons. The forecast standard error is under 1%.{{p}}Note. Units{{p}}are quarterly{{p}}rates, in{{p}}percent per{{p}}annum.{{p}}In light of{{p}}the{{p}}discussion{{p}}above, Table{{p}}2{{p}}numerically{{p}} FRB: IFDP Note: Predicting Fed Forecasts http://www.federalreserve.gov/econresdata/notes/ifdp-notes/2016/predict...{{p}}3 of 5 2/16/2016 12:16 PM{{p}}Forecast horizon{{p}}Sample period{{p}}RMSE Mean Standard deviation{{p}}One quarter ahead{{p}}2006-2009 0.84 -0.09 0.85{{p}}2010 0.55 0.31 0.49{{p}}2006-2010 0.79 -0.00 0.80{{p}}assesses{{p}}the{{p}}discrepancies between the FMIs and the Greenbook forecasts over three periods: 2006–2009, 2010, and 2006–2010. Root mean{{p}}squared errors (RMSEs) are less than 1% per annum for all sample periods and for both forecast horizons. Notably, the{{p}}one-quarter-ahead RMSEs are smaller than the current-quarter RMSEs. That is, Stekler and Symington's indexes are more accurate at{{p}}inferring the one-quarter-ahead Greenbook forecast than the current-quarter Greenbook forecast. Also, both FMIs are much more{{p}}accurate at inferring the Greenbook forecasts for 2010 than they are at inferring the Greenbook forecasts for 2006–2009.{{p}}Remarks{{p}}Several observations are germane. First, textual analysis--such as that employed by Stekler and Symington--is common in the literature.{{p}}Similar examples include Boukus and Rosenberg (2006), who assess the roles of different themes in the FOMC's minutes; and Meade,{{p}}Burk, and Josselyn (2015), who calculate the changing frequencies of different quantitative words in the FOMC's minutes to ascertain{{p}}the diversity of views among the FOMC members and participants.{{p}}Second, Stekler and Symington's analysis is novel by quantifying qualitative text from the minutes on the outlook. By contrast, Meade,{{p}}Burk, and Josselyn focus on the disparity of views in the minutes, rather than on some central tendency of views. Banternghansa and{{p}}McCracken (2009, 2014) likewise focus on the disparity of views, albeit as measured by individual participants' economic forecasts. Yet{{p}}other researchers such as Nunes (2013) have compared the Greenbook forecasts and FOMC participants' forecasts with each other,{{p}}with other forecasts, and with the actual outcomes.{{p}}Third, the current-outlook FMI draws on text about the Federal Reserve Board staff's views, whereas the future-outlook FMI ostensibly{{p}}reflects the views of the FOMC participants on both current conditions and future outlook. While these nuances may affect the{{p}}interpretation of the FMIs when compared with the Greenbook forecasts, the similarity of the FMIs and the Greenbook forecasts{{p}}suggests not.{{p}}Fourth, it may seem surprising that FOMC participants' views for the future outlook--as measured by the future-outlook FMI--are{{p}}well-captured by the one-quarter-ahead Greenbook forecast, since the policy-relevant horizon may be somewhat longer than just one{{p}}quarter ahead. The future-outlook FMI may thus be an even better proxy for Greenbook forecasts at longer horizons. Or, participants{{p}}may down-weight Greenbook forecasts at longer horizons if they view those forecasts as being uninformative; see Chang and Hanson{{p}}(2015).{{p}}Finally, indexes have yet to be constructed for 2011–2015 or for the period prior to 2006. Indexes over those periods may be more or{{p}}less accurate than those over 2006–2010. The indexes also could be generated algorithmically, drawing on Table 1.{{p}}Conclusions{{p}}Stekler and Symington propose and build innovative quantitative indexes (the "FMIs") that measure the extent of optimism or pessimism{{p}}expressed in the FOMC's minutes on the current and future outlook for the U.S. economy. Even though the text that Stekler and{{p}}Symington examine includes little or no quantitative information, Stekler and Symington's FMIs reveal much about the thinking of the{{p}}FOMC participants and about the Federal Reserve Board staff's input to the FOMC meetings.{{p}}The present note shows that these indexes can help infer the staff's Greenbook forecasts of the U.S. real GDP growth rate, years before{{p}}the public release of the Greenbook. The FOMC minutes are thus highly informative about a key input to monetary policymaking.{{p}}References{{p}}Banternghansa, C., and M. W. McCracken (2009) "Forecast Disagreement Among FOMC Members", Federal Reserve Bank of St. Louis{{p}}Working Paper No. 2009–059A, Research Division, Federal Reserve Bank of St. Louis, St. Louis, Missouri, December.{{p}}Banternghansa, C., and M. W. McCracken (2014) "The Effect of FOMC Forecast Disagreement on U.S. Treasuries", Presentation,{{p}}International Symposium on Forecasting, Rotterdam, The Netherlands, July 1.{{p}}Boukus, E., and J. V. Rosenberg (2006) "The Information Content of FOMC Minutes", mimeo, Federal Reserve Bank of New York, New{{p}}York, July.{{p}}Chang, A. C., and T. J. Hanson (2015) "The Accuracy of Forecasts Prepared for the Federal Open Market Committee", Finance and{{p}}Economics Discussion Series Paper No. 2015–062, Board of Governors of the Federal Reserve System, Washington, D.C., July.{{p}}Danker, D. J., and M. M. Luecke (2005) "Background on FOMC Meeting Minutes", Federal Reserve Bulletin, 2005, Spring, 175–179.{{p}}Doornik, J. A. (2009) "Autometrics", Chapter 4 in J. L. Castle and N. Shephard (eds.) The Methodology and Practice of Econometrics: A{{p}}Festschrift in Honour of David F. Hendry, Oxford University Press, Oxford, 88–121.{{p}}Doornik, J. A., and D. F. Hendry (2013) PcGive 14, Timberlake Consultants Press, London (3 volumes).{{p}} FRB: IFDP Note: Predicting Fed Forecasts http://www.federalreserve.gov/econresdata/notes/ifdp-notes/2016/predict...{{p}}4 of 5 2/16/2016 12:16 PM{{p}}Accessibility Contact Us Disclaimer Website Policies FOIA PDF Reader{{p}}Ericsson, N. R. (2015) "Eliciting GDP Forecasts from the FOMC's Minutes Around the Financial Crisis", International Finance Discussion{{p}}Paper No. 1152, Board of Governors of the Federal Reserve System, Washington, D.C., November, dx.doi.org/10.17016{{p}}/IFDP.2015.1152 ; and RPF Working Paper No. 2015–003, Research Program on Forecasting, Center of Economic Research,{{p}}Department of Economics, The George Washington University, Washington, D.C., November, www.gwu.edu/~forcpgm/2015-003.pdf ;{{p}}International Journal of Forecasting, in press.{{p}}Meade, E. E., N. A. Burk, and M. Josselyn (2015) "The FOMC Meeting Minutes: An Assessment of Counting Words and the Diversity of{{p}}Views", FEDS Note, Board of Governors of the Federal Reserve System, Washington, D.C., May 26.{{p}}Nunes, R. (2013) "Do Central Banks' Forecasts Take Into Account Public Opinion and Views?", International Finance Discussion Paper{{p}}No. 1080, Board of Governors of the Federal Reserve System, Washington, D.C., May.{{p}}Stekler, H. O., and H. Symington (2015) "Evaluating Qualitative Forecasts: The FOMC Minutes, 2006–2010", RPF Working Paper No.{{p}}2014–005 (September 2014; revised: February 2015), Research Program on Forecasting, Center of Economic Research, Department of{{p}}Economics, The George Washington University, Washington, D.C., cer.columbian.gwu.edu/files/2014-005-2.pdf ; International Journal{{p}}of Forecasting, dx.doi.org/10.1016/j.ijforecast.2015.02.003 , in press.{{p}}1. The author is a staff economist in the Division of International Finance, Board of Governors of the Federal Reserve System, Washington, DC 20551 USA{{p}}(ericsson@frb.gov), and a Research Professor of Economics, Department of Economics, The George Washington University, Washington, DC 20052 USA{{p}}(ericsson@gwu.edu). The views expressed in this paper are solely the responsibility of the author and should not be interpreted as necessarily representing or{{p}}reflecting the views of the Federal Open Market Committee, its principals, the Board of Governors of the Federal Reserve System, or of any other person{{p}}associated with the Federal Reserve System. This paper uses publicly available information, and only publicly available information. It does not use any{{p}}internal or confidential Federal Reserve Board information, either directly or indirectly. The author is grateful to Chris Erceg, Lowell Ericsson, Nancy Ericsson,{{p}}Joe Gruber, David Hendry, Lucas Husted, Matteo Iacoviello, Freja Ingelstam, Aaron Markiewitz, Jaime Marquez, Ellen Meade, J Seymour, Tara Sinclair,{{p}}Herman Stekler, and Joyce Zickler for helpful discussions and comments. All numerical results were obtained using PcGive Version 14.0B3, Autometrics{{p}}Version 1.5e, and Ox Professional Version 7.00 in 64-bit OxMetrics Version 7.00: see Doornik and Hendry (2013) and Doornik (2009). Return to text{{p}}2. See Stekler and Symington (2015) and Ericsson (2015) for further details. Return to text{{p}}3. See Danker and Luecke (2005) for a valuable perspective on the evolution of the FOMC's minutes. Return to text{{p}}4. In fact, Stekler and Symington's procedure itself involves two steps but, for expositional purposes, this description merges them into a single-step{{p}}quantification of the FOMC minutes. Return to text{{p}}5. Ericsson (2015, Sections 2–3) describes the econometric methodology for handling these three meetings during the extenuating circumstances of the{{p}}financial crisis, and it provides details of the online sources for the publicly available Greenbook forecasts and FOMC minutes. Return to text{{p}}6. Starting in 2010, the Greenbook forecasts appear in a Fed document called the Tealbook, which combines the previous Fed documents called the{{p}}Greenbook and the Bluebook. For simplicity, these more recent forecasts are still referred to as "Greenbook forecasts" herein. Return to text{{p}} Disclaimer: IFDP Notes are articles in which Board economists offer their own views and present analysis on a range of topics in{{p}}economics and finance. These articles are shorter and less technically oriented than IFDP Working Papers.{{p}}Last update: February 12, 2016{{p}}Home | Economic Research & Data{{p}} FRB: IFDP Note: Predicting Fed Forecasts http://www.federalreserve.gov/econresdata/notes/ifdp-notes/2016/predict...{{p}}5 of 5 2/16/2016 12:16 PM
    Date: 2016–02–12
    URL: http://d.repec.org/n?u=RePEc:fip:fedgin:2016-02-12&r=for
  2. By: Travis J. Berge; Nitish R. Sinha; Michael Smolyansky
    Abstract: FEDS Notes Print{{p}}August 2, 2016{{p}}Which market indicators best forecast recessions?{{p}}Travis Berge, Nitish Sinha, and Michael Smolyansky{{p}}Economists, staying true to their epithet as "dismal scientists," are seemingly in a perpetual state of worry over the risk that the economy{{p}}may enter a recession. This is especially so during periods of financial market stress, when spikes in volatility and declines in asset prices{{p}}lead many to wonder whether a recession is imminent. A central challenge, however, involves filtering the signal from the noise to{{p}}understand which economic and financial indicators are most informative for forecasting recessions. To complicate matters further, the{{p}}economic picture is often mixed--as has arguably been the case during the first half of 2016--which makes the task of discerning which{{p}}variables are informative predictors, and which are not, all the more important.{{p}}In this note, we use econometric methods to infer which economic and financial indicators reliably identify and predict recessions. We find{{p}}that, for forecasting the risk of recession 12 months from now, financial market indicators, such as the slope of the Treasury yield curve{{p}}and measures of corporate credit spreads, are particularly informative. In contrast, when attempting to identify whether the economy is{{p}}currently in recession, variables that describe real economic activity, especially the labor market, are the most reliable.{{p}}With these results in hand, we document the influence of the data flow during 2016 on the risk of the U.S. economy entering recession.{{p}}Although the first half of 2016 has seen bouts of financial market volatility and some tepid economic data, our model suggests that neither{{p}}have been severe enough to indicate a substantial increase in the risk of recession.{{p}}Methodology{{p}}Our objective is to predict a binary outcome: will the economy be expanding or contracting at a particular date in the future, given our{{p}}knowledge of the world today? Clearly, there is no single indicator, or even fixed set of indicators, that contains comprehensive{{p}}information about the state of the economy 3, 6, or 12 months from now. Therefore, we consider a set of 17 monthly variables chosen to{{p}}describe different aspects of the economy. Broadly speaking these indicators are measures of real economic activity such as labor market{{p}}indicators, and forward-looking financial variables such as equity returns, credit spreads, the Treasury yield curve and indicators of{{p}}financial market stress.1{{p}}Each of these 17 indicators, either alone or in combination with other variables could be used in a model to evaluate recession risks.{{p}}Therefore the solution to the econometric problem involves choosing between a very large number of potential models. We use a method{{p}}known as Bayesian Model Averaging (BMA) to elicit the best forecasting model. Recognizing that these indicators carry information about{{p}}different forecast horizons--financial markets tend to be forward-looking whereas variables that describe real economic activity are not--we{{p}}search for the best model at each forecast horizon.{{p}}Specifically, let if the National Bureau of Economic Research (NBER) has declared that month t falls in a recession, and if{{p}}the NBER has declared that month t falls in an expansion instead. Each model estimates the probability that month t will be declared a{{p}}recession by the NBER with following equation:{{p}}where is the cumulative standard normal probability distribution.2{{p}}We consider many regressions that take the form of equation (1), one model for every possible combination of the 17 indicators. BMA{{p}}estimates the probability that the NBER will declare a month to be a recession from each separate model, and then calculates a weighted{{p}}average of these estimates. Let denote the predicted probability of recession from model i, where . The Bayesian{{p}}model average forecast is the weighted sum:{{p}}where denotes each individual forecast. The weight assigned to any given model is determined by how well it explains movements in{{p}}the probability of recession (specifically, model i's posterior probability). In this way, combinations of variables that produce recession{{p}}probabilities that match the actual NBER dates at each forecast horizon receive larger weight in the average of the model forecasts, while{{p}}models that cannot explain or anticipate recessions receive little or zero weight in the averaged forecast.{{p}}Which indicators forecast best, and when?{{p}}Figure 1 shows how well the BMA model explains historical recessionary periods at two different forecast horizons, 0 and 12 months. A{{p}}forecast of whether the current month is in recession, known as nowcasting, is shown as the red line in the figure on the left.3 For{{p}}comparison, the blue dashed line shows the unconditional probability that a given month was declared recession by the NBER, about 15{{p}}percent. The estimated recession probabilities fit the historical data quite well, generally rising during recessions which are indicated as{{p}}Yt = 1 Yt = 0{{p}}Pr (Yt = 1 | xt−h) = Φ(α + x′ β) (1){{p}}t−h{{p}}Φ{{p}}p^it i = 1,…,217{{p}}p^ = (2) t Σi=1{{p}}2k{{p}}p^it w^{{p}}i{{p}}p^it{{p}} FRB: FEDS Notes: Which Market Indicators Best Forecast Recessions? Page 1 of 5{{p}} https://m-pubtest.frb.gov/econresdata/notes/feds-notes/2016/which-market-indicators-best-f... 8/2/2016{{p}}the shaded areas.4 The figure on the right asks a more ambitious question: given a particular month's data, what is the probability that 12{{p}}months later there will be a recession? Perhaps surprisingly, the fit of this model is also quite good, generally rising alongside the NBER{{p}}recessions. However, notably the model predicted only a 50 percent chance of recession immediately prior to the Great Recession of{{p}}2007-2009, and gave a similar signal in the late 1990's.{{p}} Note: Figures show , so that the date on the x-axis is the forecast that month t was declared recession given information t-h months prior. NBER{{p}}recession dates shaded.{{p}}Accessible version{{p}}Which variables matter for signaling recession risks? Table 1 shows variables for which the slope coefficient is non-zero at the 90-{{p}}percent confidence level. The reported coefficients and standard errors are themselves weighted averages from each individual{{p}}forecasting model, with weights equal to each model's posterior probability (i.e., the 's from equation (2)).5 The importance of each{{p}}variable can be summarized by its "posterior inclusion probability" -- i.e., the sum of the posterior probabilities of all models that include{{p}}that particular variable.{{p}}As seen in Panel A, the model that evaluates the risk of recession in the current month primarily relies on variables that describe the real{{p}}economy, such as the number of new jobs created. Interestingly, the model also includes the TED spread, a measure of financial stress{{p}}that gained prominence during 2007-2009. However, the model that forecasts the probability of recession 12 months hence ignores{{p}}variables describing real economic activity, relying instead on financial indictors. As seen in Panel B, the model that forecasts recession{{p}}12 months from now depends heavily on only two variables: the slope of the Treasury yield curve and the GZ credit spread index, a{{p}}measure of corporate credit market conditions that is described in detail by Favara, Gilchrist, Lewis, and Suarez (2016).{{p}}Figure 1: Estimated probabilities from the BMA model, zero- and twelve-month ahead forecasts{{p}}BMA, 0-month ahead Recession Probability Forecast BMA, 12-month ahead Recession Probability Forecast{{p}}Table 1: BMA relies on different indicators at each forecast horizon{{p}}Model forecasting the current month{{p}}Posterior inclusion probability (%) Coef. Std. Err.{{p}}Panel A:{{p}}Change in payroll employment 100 -5.1 1{{p}}TED spread 100 1.6 0.4{{p}}Change in initial claims 99 -0.2 0.1{{p}}Panel B:{{p}}Slope of yield curve 100 -1.4 0.2{{p}}GZ index 100 0.8 0.4{{p}}Pr (Yt | xt−h){{p}}β{{p}}w^{{p}}i{{p}} FRB: FEDS Notes: Which Market Indicators Best Forecast Recessions? Page 2 of 5{{p}} https://m-pubtest.frb.gov/econresdata/notes/feds-notes/2016/which-market-indicators-best-f... 8/2/2016{{p}} Note: Table shows only indicators with posterior inclusion probability greater than 90 percent.{{p}}Evolution of recession risk during 2016{{p}}The first half of 2016 has seen mixed economic and financial data. Financial market turmoil at the start of the year has been followed by{{p}}pockets of disappointing economic news. For example, payroll growth has slowed from its 2015 pace, the decline in commodity prices{{p}}continues to weigh on particular sectors, and real personal consumption appears to have been tepid in the first quarter of the year. On{{p}}net, while equity indices remain near their all-time highs, the yield curve has flattened in 2016. Moreover, there have been bouts of{{p}}financial volatility and periods of high credit spreads. Given this backdrop, we ask: how has the risk of recession, implied by the BMA{{p}}model, evolved during the course of 2016?{{p}}Figure 2 presents a snapshot of the forecasted recession probabilities--ranging from 0 to 12 months ahead--as implied by the model at the{{p}}three different junctures: November 2015, the orange line; February 2016, the blue dashed line; and May 2016, the black dotted line. As{{p}}shown, at each of these three junctures the model generally assigned recession probabilities that were below the unconditional probability{{p}}of the U.S. economy being in recession. A notable excpetion, however, occurred in back February 2016--at that point in time, the model{{p}}assigned a 17.3 percent probabilitity that October 2016 would be a recession. This spike in recession risk was driven by a deterioration in{{p}}financial market conditions during February. Since then, financial markets have recovered and the model-implied probabilities of recession{{p}}have correspondingly declined. In May, prior to the Brexit vote, the model assigned approximately a 10 percent chance that the economy{{p}}would be in recession 12 months hence.{{p}}Accessible version{{p}}An important caveat: Forecasting accuracy out-of-sample{{p}}While the BMA models were designed to fit the historical pattern of U.S. recessions, an additional issue is how well they perform out-of-sample.{{p}}A natural question to ask, therefore, is how strong a signal might the BMA methodology have sent ahead of the 2001 and 2008{{p}}recessions? To shed light on this issue, we perform a pseudo out-of-sample forecasting exercise by only using data ending prior to the{{p}}onset of each of these recessions to assess how well the model anticipated the subsequent downturn.6{{p}}As shown in Table 2, in September 2001, six months prior to the 2001 recession, the model forecast a 35 percent probability that the U.S.{{p}}economy would be in recession 12 months from that point in time, well above the unconditional average. The results also indicate that by{{p}}December, three months ahead of the NBER-dated recession, the model would have sent a fairly strong signal of the impending{{p}}downturn.{{p}} Note: NBER peak dated March, 2001; NBER trough is November, 2001.{{p}}Similarly, using data through June 2007, the model forecast a 25 percent probability that the U.S. economy would be in recession 12{{p}}months from that point in time. Three months later, with data through September 2007, the model forecast of a recession 12 months had{{p}}risen to 40 percent.{{p}}It is important to bear in mind that most forecasting models do not send very strong signals of recession far ahead of time. To that extent,{{p}}the fact that BMA sent somewhat elevated readings ahead of prior recessions is reassuring. On the other hand, the BMA recession{{p}}Figure 2: The evolution of recession risk{{p}}Table 2: Recession probabilities prior to 2001 recession{{p}}Forecast made using data through:{{p}}Sep-00 Dec-00{{p}}Current-month 6 4{{p}}Three-months hence 28 67{{p}}Six months hence 32 57{{p}}Twelve months hence 35 39{{p}} FRB: FEDS Notes: Which Market Indicators Best Forecast Recessions? Page 3 of 5{{p}} https://m-pubtest.frb.gov/econresdata/notes/feds-notes/2016/which-market-indicators-best-f... 8/2/2016{{p}}probabilities ahead of both the 2001 and 2008 recessions were not extraordinarily high, which underscores an important limitation of this{{p}}exercise, and of forecasting in general.7{{p}} Note: NBER peak dated December, 2007; NBER trough is June, 2009.{{p}} Note: Treasury yields from Gurkaynak, Swanson and Wright (2006).{{p}}References{{p}}Table 3: Recession probabilities prior to 2007 recession{{p}}Forecast made using data through:{{p}}Jun-07 Sep-07{{p}}Current-month 15 22{{p}}Three-months hence 26 22{{p}}Six months hence 26 41{{p}}Twelve months hence 25 41{{p}}Appendix Table 1: Variables included in forecasting models{{p}}Variable Definition/notes Transformation{{p}}Financial Variables{{p}}Slope of yield curve 10-year Treasury less 3-month yield{{p}}Curvature of yield curve 2 x 2-year minus 3-month and 10-year{{p}}GZ index Gilchrist and Zakrajsek (AER, 2012){{p}}TED spread 3-month ED less 3-month Treasury yield{{p}}BBB corporate spread BBB less 10-year Treasury yield{{p}}S 500, 1-month return 1-month log diff.{{p}}S 500, 3-month return 3-month log diff.{{p}}Trade-weighted dollar 3-month log diff.{{p}}VIX CBOE and extended following Bloom{{p}}Macroeconomic Indicators{{p}}Real personal consumption expend. 3-month log diff.{{p}}Real disposable personal income 3-month log diff.{{p}}Industrial production 3-month log diff.{{p}}Housing permits 3-month log diff.{{p}}Nonfarm payroll employment 3-month log diff.{{p}}Initial claims 4-week moving average 3-month log diff.{{p}}Weekly hours, manufacturing 3-month log diff.{{p}}Purchasing managers index 3-month log diff.{{p}}Berge, T.J. (2015) "Predicting Recessions with Leading Indicators: Model Averaging and Selection over the Business Cycle," Journal of{{p}}Forecasting 34(6): 455-471.{{p}}Bloom, N. (2009) "The impact of uncertainty shocks," Econometrica 77(3): 623-685.{{p}}Blue Chip Financial Forecasts, should be cited as follows: Wolters Klewer. Blue Chip Financial Forecasts.{{p}} https://lrus.wolterskluwer.com/product-family/blue-chip{{p}}Chauvet, M. and J. Piger (2008) "A comparison of the real-time performance of business cycle dating methods," Journal of Business and{{p}}Economic Statistics 26: 42-49.{{p}}Favara, G., S. Gilchrist, K.F. Lewis, E. Zakrajšek (2016) "Recession Risk and the Excess Bond Premium," FEDS Notes, April 8.{{p}}Gilchrist, S., and E. Zakrajšek (2012), "Credit Spreads and the Business Cycle Fluctuations," American Economic Review 102(4): 1692-{{p}} FRB: FEDS Notes: Which Market Indicators Best Forecast Recessions? Page 4 of 5{{p}} https://m-pubtest.frb.gov/econresdata/notes/feds-notes/2016/which-market-indicators-best-f... 8/2/2016{{p}}Last update: August 2, 2016{{p}}Home | Economic Research & Data{{p}}1720.{{p}}Gurkaynak, R., B. Sack and J.H. Wright (2006) "The U.S. Treasury Yield Curve: 1961 to the Present," FEDS Working Paper Series 2006-{{p}}28.{{p}}Hamilton J.D. (2011) "Calling recessions in real time," International Journal of Forecasting 27(4): 1006-1026.{{p}}Raftery, A.E. (1995) "Bayesian Model Selection in Social Research," Sociological Methodology 25: 111-163.{{p}}1. See the appendix for a complete list of indicators included in our data set. Our dataset begins in January 1973 and ends in May 2016. Some macroeconomic{{p}}indicators are released with a lag, so that values of very recent months are unavailable at the time of writing. For these indicators, we replace missing values{{p}}with consensus Blue Chip forecasts and Bloomberg sureys. Return to text{{p}}2. Specifically, given our 17 covariates, there are 217--more than 130,000--different potential recession models. We use the method of Raftery (1995) to perform{{p}}BMA, which approximates the posterior likelihood of each model with a maximum likelihood estimate of it's Bayesian Information Criterion. For further details,{{p}}see Berge (2015). Return to text{{p}}3. Note that nowcasts are not a purely academic exercise, because the NBER dates recessions well after the fact, typically 12 to 18 months after a given{{p}}recession event. Return to text{{p}}4. A more formal evaluation of the model, including an out-of-sample evaluation, is provided in Berge (2015). Return to text{{p}}5. For models that do not include a particular variable, BMA sets the coefficient to zero in these cases. Return to text{{p}}6. Importantly, owing to data constraints, we use current-vintage data for this exercise. Any judgement on the model's success based on forecasts using revised{{p}}data should be viewed as an upper-bound on the model capability. Return to text{{p}}7. See, for example, Chauvet and Piger (2008), Hamilton (2011), and Berge (2015) for discussions of the real-time performance of several different classes of{{p}}recession models. Return to text{{p}}Please cite as:{{p}}Berge, Travis, Nitish Sinha, and Michael Smolyansky (2016). "Which market indicators best forecast recessions?," FEDS Notes.{{p}} Washington: Board of Governors of the Federal Reserve System, August 2, 2016, http://dx.doi.org/10.17016/2380-7172.1805.{{p}} Disclaimer: FEDS Notes are articles in which Board economists offer their own views and present analysis on a range of topics in{{p}}economics and finance. These articles are shorter and less technically oriented than FEDS Working Papers.{{p}}Accessibility Contact Us Disclaimer Website Policies FOIA PDF Reader{{p}} FRB: FEDS Notes: Which Market Indicators Best Forecast Recessions? Page 5 of 5{{p}} https://m-pubtest.frb.gov/econresdata/notes/feds-notes/2016/which-market-indicators-best-f... 8/2/2016
    Date: 2016–08–02
    URL: http://d.repec.org/n?u=RePEc:fip:fedgfn:2016-08-02&r=for
  3. By: Stefano DellaVigna; Devin Pope
    Abstract: Academic experts frequently recommend policies and treatments. But how well do they anticipate the impact of different treatments? And how do their predictions compare to the predictions of non-experts? We analyze how 208 experts forecast the results of 15 treatments involving monetary and non-monetary motivators in a real-effort task. We compare these forecasts to those made by PhD students and non-experts: undergraduates, MBAs, and an online sample. We document seven main results. First, the average forecast of experts predicts quite well the experimental results. Second, there is a strong wisdom-of-crowds effect: the average forecast outperforms 96 percent of individual forecasts. Third, correlates of expertise---citations, academic rank, field, and contextual experience--do not improve forecasting accuracy. Fourth, experts as a group do better than non-experts, but not if accuracy is defined as rank ordering treatments. Fifth, measures of effort, confidence, and revealed ability are predictive of forecast accuracy to some extent, especially for non-experts. Sixth, using these measures we identify `superforecasters' among the non-experts who outperform the experts out of sample. Seventh, we document that these results on forecasting accuracy surprise the forecasters themselves. We present a simple model that organizes several of these results and we stress the implications for the collection of forecasts of future experimental results.
    JEL: C9 C91 C93 D03
    Date: 2016–08
    URL: http://d.repec.org/n?u=RePEc:nbr:nberwo:22566&r=for

This nep-for issue is ©2016 by Rob J Hyndman. It is provided as is without any express or implied warranty. It may be freely redistributed in whole or in part for any purpose. If distributed in part, please include this notice.
General information on the NEP project can be found at http://nep.repec.org. For comments please write to the director of NEP, Marco Novarese at <director@nep.repec.org>. Put “NEP” in the subject, otherwise your mail may be rejected.
NEP’s infrastructure is sponsored by the School of Economics and Finance of Massey University in New Zealand.
Use http://lists.repec.org/mailman/options/nep-for to sign off.