Introduction to AR, MA Time Series, Autocorrelation, Partial Autocorrelation, Spectral Density

This foundational chapter provides a comprehensive introduction to the fundamental concepts of time series analysis. The chapter begins by establishing the mathematical framework of stochastic processes and introduces key definitions including stationarity (both strict and weak), Gaussian processes, and white noise. The autocorrelation function (ACF) and partial autocorrelation function (PACF) are presented as essential diagnostic tools for identifying the underlying structure of time series data. The spectral density function is introduced as a frequency-domain characterization of time series behavior. The chapter covers the four main components of time series—trends, cycles, seasonality, and irregular variations—and discusses how they combine in additive and multiplicative models. A detailed treatment of autoregressive (AR) and moving average (MA) processes is provided, including their mathematical formulations using the backshift operator, parameter estimation methods (method of moments, maximum likelihood, and ordinary least squares), and theoretical properties. Visual illustrations through R code examples demonstrate the ACF and PACF patterns characteristic of these processes. The chapter establishes the conceptual foundation necessary for understanding more advanced long memory time series models presented in subsequent chapters.