Package 'ForeComp'

Diebold-Mariano Test (Bartlett kernel, normal approximation)

Description

Diebold-Mariano Test (Bartlett kernel, normal approximation). This is a two-sided test.

Usage

dm.test.bt(d, M = NA, Mopt = NA, cl = 0.05)

Arguments

d	loss differential
M	truncation parameter for the Bartlett kernel (if M = NA, then Mopt = 2 by default)
Mopt	option for automatic bandwidth, 1 if Lazarus et al. (2018), 2 if Newey and West (1994, default), 3 if Andrews textbook NW ($M=\lceil 0.75T^{1/3}\rceil$), 4 if CI baseline ($M=\lfloor T^{1/2}\rfloor$)
cl	confidence level (default = 0.05, i.e., 5%)

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

• pval is an associated p-value

Author(s)

Minchul Shin

---

Diebold-Mariano Test (Bartlett kernel, fixed-b approximation)

Description

Diebold-Mariano Test (Bartlett kernel, fixed-b approximation). This is a two-sided test.

Usage

dm.test.bt.fb(d, M = NA, Mopt = NA, cl = 0.05)

Arguments

d	loss differential
M	truncation parameter for the Bartlett kernel (if M = NA, then Mopt = 1 by default)
Mopt	option for automatic bandwidth, 1 if Lazarus et al. (2018, default), 2 if Newey and West (1994), 3 if Andrews textbook NW ($M=\lceil 0.75T^{1/3}\rceil$), 4 if CI baseline ($M=\lfloor T^{1/2}\rfloor$)
cl	confidence level (default = 0.05, i.e., 5%), Only 0.05 (5%) or 0.10 (10%) are allowed.

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

Author(s)

Minchul Shin

---

CNR (2017), |t|-test statitic

Description

Randomization test based on asymptotic symmetry, |t|-test statistic

Usage

dm.test.cnr.t(d, q = 2, cl = 0.05)

Arguments

d	loss differential
q	number of blocks
cl	confidence level (default = 0.05, i.e., 5%)

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

• pval is an associated p-value

Author(s)

Minchul Shin

---

CNR (2017), Wald-statistic

Description

Randomization test based on asymptotic symmetry, Wald-statistic

Usage

dm.test.cnr.w(d, q = 2, cl = 0.05)

Arguments

d	loss differential
q	number of blocks
cl	confidence level (default = 0.05, i.e., 5%)

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

• pval is an associated p-value

Author(s)

Minchul Shin

---

Diebold-Mariano Test (EWC, fixed-b approximation)

Description

Diebold-Mariano Test (EWC, fixed-b approximation). This is a two-sided test.

Usage

dm.test.ewc.fb(d, B = NA, Bopt = NA, cl = 0.05)

Arguments

d	loss differential
B	truncation parameter for the EWC long-run variance estimator (if B = NA, then Bopt = 1 by default)
Bopt	option for automatic bandwidth, 1 if Lazarus et al. (2018, default, $B=\lfloor 0.4T^{2/3}\rfloor$)
cl	confidence level (default = 0.05, i.e., 5%)

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

• pval is an associated p-value

Author(s)

Minchul Shin

---

Ibragimov and Muller (2010)

Description

t-Statistic based HAR-inference by Ibragimov and Muller (2010).

Usage

dm.test.im(d, q = 2, cl = 0.05)

Arguments

d	loss differential
q	number of blocks. When length(d) is not divisible by q, this implementation uses near-equal nonoverlapping blocks (block sizes differ by at most one) and issues a warning.
cl	confidence level (default = 0.05, i.e., 5%)

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

• pval is an associated p-value

Author(s)

Minchul Shin

---

Diebold-Mariano Test (with an original recommendation)

Description

This function performs the Diebold-Mariano test with their original recommendation. Let $d_{t}$ be a sequence of loss differential, $t=1,2,...,T$. Then, the function performs a statistical test for the following null hypothesis:

$E[d_{t}] = 0$

This function follows the original recommendation by Diebold and Mariano (1995), where the long-run variance is estimated using the rectangular kernel truncated at $(h-1)$. This function performs a two-sided test. Normal approximation is used to calculate critical values.

Usage

dm.test.r(d, h = 1, cl = 0.05)

Arguments

d	loss differential
h	h-step-ahead forecast (default = 1, i.e., comparing one-step-ahead forecasts)
cl	confidence level (default = 0.05, i.e., 5%)

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

• pval is an associated p-value

Author(s)

Minchul Shin

---

Diebold-Mariano Test (Modified-DM)

Description

Diebold-Mariano Test (Modified-DM). Finite-sample modification to the original DM's test. This is a two-sided test.

Usage

dm.test.r.m(d, h = 1, cl = 0.05)

Arguments

d	loss differential
h	h-step-ahead forecast (default = 1, i.e., one-step-ahead forecasts)
cl	confidence level (default = 0.05, i.e., 5%)

Details

This function is based on dm.test in the "forecast" package on CRAN.

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

• pval is an associated p-value

Author(s)

Minchul Shin

---

Diebold-Mariano Test (WPE, fixed-m approximation)

Description

Diebold-Mariano Test (WPE, fixed-m approximation). This is a two-sided test. See Coroneo and Iacone (2020)

Usage

dm.test.wpe.fb(d, M = NA, Mopt = NA, cl = 0.05)

Arguments

d	loss differential
M	truncation parameter for the WPE long-run variance estimator (if M = NA, then Mopt = 1 by default)
Mopt	option for automatic bandwidth, 1 if Coroneo and Iacone's default value (M = floor(T^(1/3)))
cl	confidence level (default = 0.05, i.e., 5%)

Value

This function returns a class with the following elements

• rej is a T/F value. TRUE (reject), FALSE (accept)

• stat is a test statistic

• pval is an associated p-value

Author(s)

Minchul Shin

---

Loss differential from two point forecasts

Description

This function computes the loss differential from two point forecasts. Let $f_{1,t}$ and $f_{2,t}$ be point forecasts generated by the forecaster 1 (e.g., forecasting model 1) and the forecaster 2 (e.g., forecasting model 2), respectively. $y_{t}$ is a forecasting target that both $f_{1,t}$ and $f_{2,t}$ try to forecast. For a given loss function the loss differential is defined and computed as

$d_{t} = L(y_{t}, f_{1,t}) - L(y_{t}, f_{2,t})$

for $t=1,2,...,T$.

Usage

loss.diff.p(y, f1, f2, type = "quad", ...)

Arguments

y	actual value, $y_{t}$, for $t=1,2,...,T$.
f1	point forecast 1, $f_{1,t}$, for $t=1,2,...,T$.
f2	point forecast 2, $f_{2,t}$, for $t=1,2,...,T$.
type	Type of loss functions to compare. "quad" : quadratic loss function (default) "abs" : absolute loss function "check" : check (quantile) loss function "linex" : linex loss function
...	Extra parameters to compute loss function. For type="quad" : no extra parameter is needed. For type="abs" : no extra parameter is needed. For type="check" : a scalar parameter (tau) is needed. tau $\in [0,1]$. Default value is tau=0.5 if unspecified. For type="linex" : a scalar parameter (c) is needed. c is a real number. Default value is c=1.

Details

Type of loss functions. Define $e_{t} = y_{t}-f_{t}$. Quadratic loss function is defined as

$L(y_{t}, f_{t}) = e_{t}^2$

Absolute loss function is defined as

$L(y_{t}, f_{t}) = |e_{t}|$

Check loss function is defined as

$L(y_{t}, f_{t}) = (\tau - 1(e_{t}<0))e_{t}$

Linex loss function is defined as

$L(y_{t}, f_{t}) = \exp(c e_{t}) - c e_{t} - 1$

If c is positive (negative) real number, then the linex function is exponential (linear) for e>0 and linear (exponential) for e<0.

A sample average of the returned sequence, $\bar{d}_{t} = \frac{1}{T}\sum_{t=1}^{T}d_{t}$, is the difference in average loss from forecaster 1 and forecaster 2. Positive $\bar{d}_{t}$ means that the forecaster 2 incurs lower expected loss compared to the forecaster 1. Negative $\bar{d}_{t}$ means that the forecaster 1 incurs lower expected loss compared to the forecaster 2. Statistical testing to test whether the loss differential is zero in expectation is possible. See, for example, dm.test.r.

Value

This function returns a sequence of the loss differential, $d_{t}$.

Author(s)

Minchul Shin

---

Simulated data 1

Description

A dataset containing y, f1, f2 based on one of McCracken (2019)'s data generating processes. The name of the specification is "Unconditional-Rolling". R = 175, Rbar = 175, h = 12, P = 75; Note that the DGP is constructed in a way that there is no difference between f1 and f2 in terms of expected quadratic loss.

Usage

mikedata

Format

A data frame with 75 rows and 3 variables:

y

Forecast target

f1

Point forecast based on forecasting model 1

f2

Point forecast based on forecasting model 2

Source

Simulated data.

---

Price Index for Gross National Product/Gross Domestic Product (PGDP)

Description

Error Statistics for the Survey of Professional Forecasters for GNP/GDP Price Index

Usage

PGDP

Format

'PGDP' A data frame with 219 rows and 31 columns

Source

<https://www.philadelphiafed.org/-/media/frbp/assets/surveys-and-data/survey-of-professional-forecasters/data-files/pgdp/data_spf_error_statistics_pgdp_3_aic.xls?la=en&hash=148987D03D54DA5391A44F28CBC12729>

---

Visualizes the size distortion maximum power loss tradeoff from the Diebold-Mariano test for equal predictive accuracy

Description

'Plot_Tradeoff' creates a plot to show sensitivity of statistical significance to the choice of bandwidth and how size distortion and maximum power loss vary. It is designed for the Diebold-Mariano test for equal predictive accuracy (Diebold and Mariano, 1995). For a size-power tradeoff plot, see Lazarus, Lewis, Stock, and Watson (2018) and Lazarus, Lewis, and Stock (2021).

Usage

Plot_Tradeoff(
  data,
  f1 = NULL,
  f2 = NULL,
  y = NULL,
  loss_function = NULL,
  n_sim = 1000,
  m_set = NULL,
  cl = 0.05,
  verbose = TRUE,
  no_m_label = FALSE,
  na_handling = c("zero", "drop", "error")
)

Arguments

data	A data frame.
f1	Column containing forecaster 1's predictions. Should be a string.
f2	Column containing forecaster 2's predictions. Should be a string.
y	Column containing the realized value for the outcome variable. Should be a string.
loss_function	The transformation applied to the forecast error. Defaults to squared error loss. The user supplied function should take two inputs and a scalar output, loss = loss_function(f, y). For example, quadratic loss function would be defined as loss_function=function(f,y){(f-y)^2}.
n_sim	The number of simulations used to generate the ARIMA model. Defaults to 1,000.
m_set	The truncation parameter. If NULL, the function constructs a default grid centered around the package default bandwidth for dm.test.bt.fb, $M_0 = \lceil 1.3\sqrt{T}\rceil$ (LLSW), where $T$ is the effective sample size after applying na_handling. It should be a vector of integers with the values of M you would like to plot.
cl	Significance level used in the hypothesis test. Defaults to 0.05. Only 0.05 and 0.10 are currently supported because the fixed-b approximation is implemented for these levels.
verbose	TRUE to print out the progress to the console. Defaults to TRUE.
no_m_label	TRUE to plot without m labels. Defaults to FALSE.
na_handling	How to handle missing/non-finite loss differentials. "zero" (default) replaces them with 0 (Application 2 convention), "drop" removes them (Application 1 convention), and "error" stops if any are present.

Value

A list of length 2. The first element is a ggplot2 object of the size-power tradeoff. The second element is the underlying data used to construct the plot in element 1.

Author(s)

Nathan Schor and Minchul Shin

References

Diebold, F. X. & Mariano, R. S. (1995), Comparing Predictive Accuracy, Journal of Business & Economic Statistics, 13(3), 253-263.

Lazarus, E., Lewis, D. J., Stock, J. H. & Watson, M. W. (2018), HAR Inference: Recommendations for Practice, Journal of Business & Economic Statistics, 36(4), 541-559.

Lazarus, E., Lewis, D. J. & Stock, J. H. (2021), The Size-Power Tradeoff in HAR Inference, Econometrica, 89(5), 2497-2516.

Examples

# A typical example
set.seed(1234)
output = Plot_Tradeoff(
  data = TBILL,
  f1   = "SPFfor_Step1",
  f2   = "NCfor_Step1",
  y    = "Realiz1",
  m_set = seq(from = 1, to = 70, by = 10)
)
output[[1]] # The first element is a ggplot2 object of the size-power tradeoff.
output[[2]] # The second element is the underlying data used to construct the plot in element 1.

# An example with a user supplied loss function
# To use the mean absolute error as a loss function rather than a quadratic loss function
set.seed(1234)
output = Plot_Tradeoff(
  data = TBILL,
  f1   = "SPFfor_Step1",
  f2   = "NCfor_Step1",
  y    = "Realiz1",
  loss_function = function(f,y){ abs(f-y) },
  m_set = seq(from = 1, to = 50, by = 10)
)

# An example without (f1, f2, y). The function will take the first three columns and use them
set.seed(1234)
tmpdata = TBILL[, c("SPFfor_Step1", "NCfor_Step1", "Realiz1")] # data with [f1, f2, y]
Plot_Tradeoff(
  data = tmpdata,
  m_set = seq(from = 1, to = 50, by = 10)
)

---

Real Gross National Product/Gross Domestic Product (RGDP)

Description

Error Statistics for the Survey of Professional Forecasters for Real GNP/GDP

Usage

RGDP

Format

'RDGP' A data frame with 219 rows and 31 columns

Source

<https://www.philadelphiafed.org/-/media/frbp/assets/surveys-and-data/survey-of-professional-forecasters/data-files/rgdp/data_spf_error_statistics_rgdp_3_aic.xls?la=en&hash=3AC9E2D8A5299F93CA7E16CFAA974C22>

---

3-Month Treasury Bill Rate (TBILL)

Description

Error Statistics for the Survey of Professional Forecasters for Treasury Bill Rate (Three Month)

Usage

TBILL

Format

'TBILL' A data frame with 219 rows and 31 columns

Source

<https://www.philadelphiafed.org/-/media/frbp/assets/surveys-and-data/survey-of-professional-forecasters/data-files/tbill/data_spf_error_statistics_tbill_1_aic.xls?la=en&hash=F432350F84B0E4CCE9A1E9D399447CA9>

---

Civilian Unemployment Rate (UNEMP)

Description

Error Statistics for the Survey of Professional Forecasters for Unemployment Rate

Usage

UNEMP

Format

'UNEMP' A data frame with 219 rows and 31 columns

Source

<https://www.philadelphiafed.org/-/media/frbp/assets/surveys-and-data/survey-of-professional-forecasters/data-files/unemp/data_spf_error_statistics_unemp_1_aic.xls?la=en&hash=4CAD0B11FEAB6C4D0F30C38965FE3354>

---