9-arima_b.qmd

---
title: "ETC3550/ETC5550 Applied forecasting"
author: "Ch9. ARIMA models"
institute: "OTexts.org/fpp3/"
pdf-engine: pdflatex
fig-width: 7.5
fig-height: 3.5
format:
  beamer:
    theme: monash
    aspectratio: 169
    fontsize: 14pt
    section-titles: false
    knitr:
      opts_chunk:
        dev: "cairo_pdf"
include-in-header: header.tex
execute:
  echo: false
  message: false
  warning: false
---


```{r setup, include=FALSE}
source("setup.R")
library(patchwork)
library(purrr)
```


## Backshift operator notation

* $B$ shifts the data back one period. $B y_{t} = y_{t - 1}$
* $B^2$ shifts the data back two periods: $B(By_{t}) = B^{2}y_{t} = y_{t-2}$
* A difference can be written as $(1 - B) y_{t}$
* A $d$th-order difference can be written as $(1 - B)^{d} y_{t}$
* A seasonal difference followed by a first difference can be written as
$(1-B)(1-B^m)y_t$

## AR(1) model

\begin{block}{}
  \centerline{$y_{t} = c + \phi_1 y_{t - 1} + \varepsilon_{t}$}
\end{block}

* When $\phi_1=0$, $y_t$ is **equivalent to WN**
* When $\phi_1=1$ and $c=0$, $y_t$ is **equivalent to a RW**
* When $\phi_1=1$ and $c\ne0$, $y_t$ is **equivalent to a RW with drift**
* When $\phi_1<0$, $y_t$ tends to **oscillate between positive and negative values**.

## Autoregressive models

A multiple regression with \textbf{lagged values} of $y_t$ as predictors.

\vspace*{-1.2cm}
\begin{align*}
y_t &= c + \phi_{1}y_{t - 1} + \phi_{2}y_{t - 2} + \cdots + \phi_{p}y_{t - p} + \varepsilon_{t} \\
&= c + (\phi_1 B + \phi_2 B^2 + \cdots + \phi_p B^p)y_t + \varepsilon_t
\end{align*}\pause\vspace*{-1.2cm}
\begin{align*}
(1 - \phi_1 B - \phi_2 B^2 - \cdots - \phi_p B^p)y_t &= c + \varepsilon_t \\
\phi(B) y_t &= c+\varepsilon_t
\end{align*}

* $\varepsilon_t$ is white noise.
* $\phi(B) = (1 - \phi_1 B - \phi_2 B^2 - \cdots - \phi_p B^p)$

## Stationarity conditions

We normally restrict autoregressive models to stationary data, and then some constraints on the values of the parameters are required.

\begin{block}{General condition for stationarity}
  Complex roots of $\phi(z) = 1-\phi_1 z - \phi_2 z^2 - \dots - \phi_pz^p$ lie outside the unit circle on the complex plane.
\end{block}\pause\vspace*{-0.3cm}

* For $p=1$: $-1<\phi_1<1$.
* For $p=2$: $-1<\phi_2<1\qquad \phi_2+\phi_1 < 1 \qquad \phi_2 -\phi_1 < 1$.
* More complicated conditions hold for $p\ge3$.
* fable takes care of this.

## Moving Average (MA) models
A multiple regression with \textbf{past \emph{errors}} as predictors.


\vspace*{-1.2cm}
\begin{align*}
  y_{t} &= c + \varepsilon_t + \theta_{1}\varepsilon_{t - 1} + \theta_{2}\varepsilon_{t - 2} + \cdots + \theta_{q}\varepsilon_{t - q}\\
&= c + (1 + \theta_1 B + \theta_2 B^2 + \cdots + \theta_q B^q)\varepsilon_t \\
&= c + \theta(B)\varepsilon_t
\end{align*}\pause\vspace*{-1.2cm}

* $\varepsilon_t$ is white noise.
* $\theta(B) = (1 + \theta_1 B + \theta_2 B^2 + \cdots + \theta_q B^q)$

## Invertibility

\begin{block}{General condition for invertibility}
  Complex roots of $\theta(z) = 1+\theta_1 z + \theta_2 z^2 + \dots + \theta_qz^q$ lie outside the unit circle on the complex plane.
\end{block}\pause

* For $q=1$: $-1<\theta_1<1$.
* For $q=2$: $-1<\theta_2<1\qquad \theta_2+\theta_1 >-1 \qquad \theta_1 -\theta_2 < 1$.
* More complicated conditions hold for $q\ge3$.
* fable takes care of this.

## ARIMA models

\fontsize{14}{14.5}\sf\vspace*{0.2cm}
\begin{block}{ARIMA($p, d, q$) model:\qquad $\phi(B)(1-B)^dy_{t} = c + \theta(B)\varepsilon_{t}$}
\begin{tabular}{rl}
AR:& $p =$ order of the autoregressive part\\
I: & $d =$ degree of first differencing involved\\
MA:& $q =$ order of the moving average part.
\end{tabular}
\end{block}\pause\vspace*{-0.2cm}

* Conditions on AR coefficients ensure stationarity.
* Conditions on MA coefficients ensure invertibility.
* White noise model: ARIMA(0,0,0)
* Random walk: ARIMA(0,1,0) with no constant
* Random walk with drift: ARIMA(0,1,0) with \rlap{const.}
* AR($p$): ARIMA($p$,0,0)
* MA($q$): ARIMA(0,0,$q$)


## R model
\vspace*{0.2cm}

\begin{block}{Intercept form}
  \centerline{$(1-\phi_1B - \cdots - \phi_p B^p) y_t' = c + (1 + \theta_1 B + \cdots + \theta_q B^q)\varepsilon_t$}
\end{block}

\begin{block}{Mean form}
  \centerline{$(1-\phi_1B - \cdots - \phi_p B^p)(y_t' - \mu) = (1 + \theta_1 B + \cdots + \theta_q B^q)\varepsilon_t$}
\end{block}

 * $y_t' = (1-B)^d y_t$
 * $\mu$ is the mean of $y_t'$.
 * $c = \mu(1-\phi_1 - \cdots - \phi_p )$.
 * fable uses intercept form

## Understanding ARIMA models
\fontsize{13}{14}\sf

* If $c=0$ and $d=0$, the long-term forecasts will go to zero.
* If $c=0$ and $d=1$, the long-term forecasts will go to a non-zero constant.
* If $c=0$ and $d=2$, the long-term forecasts will follow a straight line.
* If $c\ne0$ and $d=0$, the long-term forecasts will go to the mean of the data.
* If $c\ne0$ and $d=1$, the long-term forecasts will follow a straight line.
* If $c\ne0$ and $d=2$, the long-term forecasts will follow a quadratic trend.

## Understanding ARIMA models
\fontsize{14}{16}\sf\vspace*{0.2cm}

### Forecast variance and $d$
  * The higher the value of $d$, the more rapidly the prediction intervals increase in size.
  * For $d=0$, the long-term forecast standard deviation will go to the standard deviation of the historical data.

### Cyclic behaviour
  * For cyclic forecasts, $p\ge2$ and some restrictions on coefficients are required.
  * If $p=2$, we need $\phi_1^2+4\phi_2<0$. Then average cycle of length

\centerline{$(2\pi)/\left[\text{arc cos}(-\phi_1(1-\phi_2)/(4\phi_2))\right].$}