AR(1). Infinite MA

An autoregressive process of order 1, AR(1), may be written as an infinite moving average process in which the coefficient of $\varepsilon_{t-j}$ is $\phi_1^j$.

\begin{align*}
y_t &= \phi_1 y_{t-1} + \varepsilon_t \qquad t=1,\dots,T \tag{1}\\
y_{t-1} &= \phi_1 y_{t-2} + \varepsilon_{t-1} \tag{2}\\
\therefore y_t &= \phi_1(\phi_1 y_{t-2} + \varepsilon_{t-1}) + \varepsilon_t \\
&= \phi_1^2 y_{t-2} + \phi_1 \varepsilon_{t-1} + \varepsilon_t\
\end{align*}

Repeating the process $j-1$ times

\begin{align*}
y_t = \phi_1^j y_{t-j} + \phi_1^{j-1} \varepsilon_{t-(j-1)} + \phi_1^{j-2} \varepsilon_{t-(j-2)} + \dots + \phi_1 \varepsilon_{t-1} + \varepsilon_t \tag{3}
\end{align*}

Observe that if $j$ is sufficiently large, for $|\phi|<1$ the autoregressive terms become negligible. We may therefore say \begin{align*} y_t = \sum_{j=0}^\infty \phi_1^j \varepsilon_{t-j} \end{align*} i.e an infinite moving average.

Proof of Stationarity.

Mean (first moment)

\begin{align*}
E(y_t) = E\biggl(\sum_{j=0}^\infty \phi_1^j \varepsilon_{t-j} \biggr) = 0
\end{align*}

As observed above, this is only true when $|\phi|<1$. Additionally observe that when $|\phi|$ the autoregressive terms of (3) do not vanish and $E(y_t)$ now depends on $t$ and violates the stationarity condition.

Variance (second moment)

\begin{align*}
V(y_t) &= V\biggl(\sum_{j=0}^\infty \phi_1^j \varepsilon_{t-j} \biggr)\\
&= E\biggl(\sum_{j=0}^\infty \phi_1^j \varepsilon_{t-j} \biggr)^2
= E\biggl(\sum_{j=0}^\infty \phi_1^{2j} \varepsilon_{t-j}^2 \biggr)\\
&= \sum_{j=0}^\infty \phi_1^{2j}\underbrace{E ( \varepsilon_{t-j}^2 )}_{\sigma^2}
= \sigma^2 \sum_{j=0}^\infty \phi_1^{2j}
\end{align*}