| Title: | Size-Power Tradeoff Visualization for Equal Predictive Ability of Two Forecasts |
|---|---|
| Description: | Offers a set of tools for visualizing and analyzing size and power properties of the test for equal predictive accuracy, the Diebold-Mariano test that is based on heteroskedasticity and autocorrelation-robust (HAR) inference. A typical HAR inference is involved with non-parametric estimation of the long-run variance, and one of its tuning parameters, the truncation parameter, trades off a size and power. Lazarus, Lewis, and Stock (2021)<doi:10.3982/ECTA15404> theoretically characterize the size-power frontier for the Gaussian multivariate location model. 'ForeComp' computes and visualizes the finite-sample size-power frontier of the Diebold-Mariano test based on fixed-b asymptotics together with the Bartlett kernel. To compute the finite-sample size and power, it works with the best approximating ARMA process to the given dataset. It informs the user how their choice of the truncation parameter performs and how robust the testing outcomes are. |
| Authors: | Nathan Schor [aut], Minchul Shin [aut, cre, cph] |
| Maintainer: | Minchul Shin <[email protected]> |
| License: | GPL (>= 3) |
| Version: | 0.9.0 |
| Built: | 2025-01-31 03:23:58 UTC |
| Source: | https://github.com/mcmcs/forecomp |
Error Statistics for the Survey of Professional Forecasters for GNP/GDP Price Index. All NAs are replaced with 0 as in Coroneo and Iacone (2020).
PGDPPGDP
'PGDP' A data frame with 219 rows and 16 columns
<https://www.philadelphiafed.org/-/media/frbp/assets/surveys-and-data/survey-of-professional-forecasters/data-files/pgdp/data_spf_error_statistics_pgdp_3_aic.xls?la=en&hash=148987D03D54DA5391A44F28CBC12729>
Coroneo, L., Iacone, F. (2020), Comparing Predictive Accuracy in Small Samples Using Fixed-Smoothing Asymptotics, Journal of Applied Econometrics, 35(4), 391-409.
'Plot_Tradeoff' creates a plot to show sensitivity of statistical significance to the choice of bandwidth and how size distortion and maximum power loss vary. It is designed for the Diebold-Mariano test for equal predictive accuracy (Diebold and Mariano, 2002). For a size-power tradeoff plot, see Lazarus, Lewis, Stock, and Watson (2018) and Lazarus, Lewis, and Stock (2021).
Plot_Tradeoff( data, f1 = NULL, f2 = NULL, y = NULL, loss_function = NULL, n_sim = 1000, m_set = NULL, verbose = TRUE, no_m_label = FALSE )Plot_Tradeoff( data, f1 = NULL, f2 = NULL, y = NULL, loss_function = NULL, n_sim = 1000, m_set = NULL, verbose = TRUE, no_m_label = FALSE )
data |
A data frame. |
f1 |
Column containing forecaster 1's predictions. Should be a string. |
f2 |
Column containing forecaster 2's predictions. Should be a string. |
y |
Column containing the realized value for the outcome variable. Should be a string. |
loss_function |
The transformation applied to the forecast error. Defaults to squared error loss. The user supplied function should take two inputs and a scalar output, loss = loss_function(f, y). For example, quadratic loss function would be defined as |
n_sim |
The number of simulations used to generate the ARIMA model. Defaults to 1,000. |
m_set |
The truncation parameter. Defaults to c(1:10, seq( 11, floor(nrow(data)/2), 10)). For a standard long-run variance calculation (for example, using Bartlett kernel), it controls the number of terms used in estimating the autocovariance matrix. It should be a vector of integers with the values of M you would like to plot. |
verbose |
TRUE to print out the progress to the console. Defaults to TRUE. |
no_m_label |
TRUE to plot without m labels. Defaults to FALSE. |
A list of length 2. The first element is a ggplot2 object of the size-power tradeoff. The second element is the underlying data used to construct the plot in element 1.
Nathan Schor and Minchul Shin
Diebold, F. X. & Mariano, R. S. (2002), Comparing Predictive Accuracy, Journal of Business & Economic Statistics, 20(1), 134-144.
Lazarus, E., Lewis, D. J., Stock, J. H. & Watson, M. W. (2018), HAR Inference: Recommendations for Practice, Journal of Business & Economic Statistics, 36(4), 541-559.
Lazarus, E., Lewis, D. J. & Stock, J. H. (2021), The Size-Power Tradeoff in HAR Inference, Econometrica, 89(5), 2497-2516.
# A typical example set.seed(1234) output = Plot_Tradeoff( data = TBILL, f1 = "SPFfor_Step1", f2 = "NCfor_Step1", y = "Realiz1", m_set = seq(from = 1, to = 70, by = 10) ) output[[1]] # The first element is a ggplot2 object of the size-power tradeoff. output[[2]] # The second element is the underlying data used to construct the plot in element 1. # An example with a user supplied loss function # To use the mean absolute error as a loss function rather than a quadratic loss function set.seed(1234) output = Plot_Tradeoff( data = TBILL, f1 = "SPFfor_Step1", f2 = "NCfor_Step1", y = "Realiz1", loss_function = function(f,y){ abs(f-y) }, m_set = seq(from = 1, to = 50, by = 10) ) # An example without (f1, f2, y). The function will take the first three columns and use them set.seed(1234) tmpdata = TBILL[, c("SPFfor_Step1", "NCfor_Step1", "Realiz1")] # data with [f1, f2, y] Plot_Tradeoff( data = tmpdata, m_set = seq(from = 1, to = 50, by = 10) )# A typical example set.seed(1234) output = Plot_Tradeoff( data = TBILL, f1 = "SPFfor_Step1", f2 = "NCfor_Step1", y = "Realiz1", m_set = seq(from = 1, to = 70, by = 10) ) output[[1]] # The first element is a ggplot2 object of the size-power tradeoff. output[[2]] # The second element is the underlying data used to construct the plot in element 1. # An example with a user supplied loss function # To use the mean absolute error as a loss function rather than a quadratic loss function set.seed(1234) output = Plot_Tradeoff( data = TBILL, f1 = "SPFfor_Step1", f2 = "NCfor_Step1", y = "Realiz1", loss_function = function(f,y){ abs(f-y) }, m_set = seq(from = 1, to = 50, by = 10) ) # An example without (f1, f2, y). The function will take the first three columns and use them set.seed(1234) tmpdata = TBILL[, c("SPFfor_Step1", "NCfor_Step1", "Realiz1")] # data with [f1, f2, y] Plot_Tradeoff( data = tmpdata, m_set = seq(from = 1, to = 50, by = 10) )
Error Statistics for the Survey of Professional Forecasters for Real GNP/GDP. All NAs are replaced with 0 as in Coroneo and Iacone (2020).
RGDPRGDP
'RDGP' A data frame with 219 rows and 16 columns
<https://www.philadelphiafed.org/-/media/frbp/assets/surveys-and-data/survey-of-professional-forecasters/data-files/rgdp/data_spf_error_statistics_rgdp_3_aic.xls?la=en&hash=3AC9E2D8A5299F93CA7E16CFAA974C22>
Coroneo, L., Iacone, F. (2020), Comparing Predictive Accuracy in Small Samples Using Fixed-Smoothing Asymptotics, Journal of Applied Econometrics, 35(4), 391-409.
Error Statistics for the Survey of Professional Forecasters for Treasury Bill Rate (Three Month). All NAs are replaced with 0 as in Coroneo and Iacone (2020).
TBILLTBILL
'TBILL' A data frame with 219 rows and 16 columns
<https://www.philadelphiafed.org/-/media/frbp/assets/surveys-and-data/survey-of-professional-forecasters/data-files/tbill/data_spf_error_statistics_tbill_1_aic.xls?la=en&hash=F432350F84B0E4CCE9A1E9D399447CA9>
Coroneo, L., Iacone, F. (2020), Comparing Predictive Accuracy in Small Samples Using Fixed-Smoothing Asymptotics, Journal of Applied Econometrics, 35(4), 391-409.
Error Statistics for the Survey of Professional Forecasters for Unemployment Rate. All NAs are replaced with 0 as in Coroneo and Iacone (2020).
UNEMPUNEMP
'UNEMP' A data frame with 219 rows and 16 columns
<https://www.philadelphiafed.org/-/media/frbp/assets/surveys-and-data/survey-of-professional-forecasters/data-files/unemp/data_spf_error_statistics_unemp_1_aic.xls?la=en&hash=4CAD0B11FEAB6C4D0F30C38965FE3354>
Coroneo, L., Iacone, F. (2020), Comparing Predictive Accuracy in Small Samples Using Fixed-Smoothing Asymptotics, Journal of Applied Econometrics, 35(4), 391-409.