Forecasting

1. Forecasting is about predictability, not magic or wishful thinking

A forecasting model is intended to capture the way things move, not just where things are.

Predictability factors. Not everything is equally forecastable. The predictability of an event depends on how well we understand its drivers, how much historical data is available, and whether the forecast itself affects the outcome. For instance, electricity demand is highly predictable because its drivers (like temperature) are well understood, whereas currency exchange rates behave like a coin toss due to the efficient market hypothesis.

Forecasting vs. planning. It is vital to distinguish forecasting from goals and planning. Goals represent what you want to happen, planning is the operational response to align reality with those goals, and forecasting is the objective, data-driven prediction of what will happen under current conditions. Confusing these three leads to unrealistic targets and systemic business failures.

Time horizons. Organizations require a mix of short, medium, and long-term forecasts to function effectively:

• Short-term: Scheduling personnel, production, and transportation.
• Medium-term: Determining future resource requirements and raw material purchases.
• Long-term: Strategic planning, market opportunities, and capital investments.

2. Always visualize your data before modeling

The first thing to do in any data analysis task is to plot the data.

Visualizing patterns. Graphs are the most powerful tool in a forecaster's toolbox for identifying underlying structures. A simple time plot immediately reveals key features such as long-term trends, seasonal patterns, cyclic fluctuations, and sudden structural breaks. Without visualization, a modeler might apply a linear model to highly non-linear data, leading to disastrously inaccurate forecasts.

Key time series patterns. Understanding the distinct types of patterns in time series data is crucial for selecting the right mathematical model:

• Trend: A long-term increase or decrease in the data.
• Seasonal: Fluctuations of a fixed and known period linked to the calendar (e.g., weekly, monthly).
• Cyclic: Rises and falls of variable and unknown duration, often tied to economic business cycles.

Advanced plotting tools. Beyond standard time plots, specialized graphics help isolate complex behaviors. Seasonal plots overlap data from different years to highlight changes in seasonal patterns, while scatterplot matrices reveal relationships between multiple predictor variables. These visual diagnostics prevent the common mistake of treating random noise as a genuine, replicable pattern.

3. Establish simple benchmarks to evaluate complex models

Whatever forecasting methods we develop, they will be compared to these simple methods to ensure that the new method is better than these simple alternatives.

The benchmark philosophy. Before building complex statistical models, a forecaster must establish simple, intuitive benchmarks. If a sophisticated machine learning model cannot outperform a simple baseline, it is not worth the computational cost or complexity. These baseline methods require no parameter estimation and serve as a reality check for model performance.

Four essential baselines. The book outlines four simple benchmark methods that every forecaster should use:

• Average method: Forecasts of all future values are equal to the mean of the historical data.
• Naïve method: Forecasts are set to the value of the very last observation.
• Seasonal naïve method: Forecasts are set to the last observed value from the same season (e.g., last year's February sales).
• Drift method: Forecasts increase or decrease over time based on the average historical change.

Practical application. The naïve method works remarkably well for financial and economic time series, such as stock prices, where future movements are highly unpredictable. The seasonal naïve method is highly effective for strongly seasonal business data, such as monthly retail sales. Using these baselines ensures that any added model complexity delivers genuine, measurable value.

4. Structure judgmental forecasting to minimize human bias

Judgmental forecasts are subjective and therefore do not come free of bias or limitations.

Human cognitive limits. When historical data is unavailable—such as during a new product launch or a unique policy change—judgmental forecasting is the only option. However, human judgment is highly vulnerable to cognitive limitations, emotional biases, and political agendas. Forecasters often suffer from anchoring, where they over-rely on initial reference points, and wishful thinking, which inflates sales expectations.

The Delphi method. To harness collective intelligence while avoiding groupthink, the Delphi method uses a structured, anonymous approach:

• A panel of experts is assembled to answer specific forecasting questions.
• Experts submit forecasts and qualitative justifications anonymously to a facilitator.
• The facilitator compiles and shares anonymous feedback, allowing experts to revise their estimates.
• This iterative process continues until a consensus is reached, preventing dominant personalities from hijacking the group.

Key operational principles. To maximize the accuracy of judgmental forecasts, organizations must implement systematic processes. This includes segregating the forecasters from the users of the forecasts to prevent conflict of interest, documenting and justifying all assumptions, and systematically evaluating past judgmental forecasts against actual outcomes to facilitate continuous learning.

5. Evaluate forecast accuracy using out-of-sample test sets

A model which fits the data well does not necessarily forecast well.

The overfitting trap. A common mistake in forecasting is evaluating a model's accuracy based on how well it fits the historical data used to train it. A perfect fit can always be achieved by adding more parameters, but this leads to overfitting, where the model captures random noise instead of the true underlying pattern. Genuine forecast accuracy can only be determined by testing the model on new, unseen data.

Training vs. test sets. To properly evaluate a model, the available data should be split into two distinct parts:

• Training set: The historical data (typically 80%) used to estimate the model's parameters.
• Test set: The subsequent data (typically 20%) held back to evaluate the model's forecasting performance.
• The test set should ideally be at least as large as the maximum forecast horizon required.

Accuracy metrics. Forecasters use several scale-independent and scale-dependent metrics to measure errors. While Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) are popular for single series, the Mean Absolute Scaled Error (MASE) is highly recommended for comparing accuracy across different series because it scales errors relative to a simple naïve baseline.

6. Decompose time series to isolate trend, seasonality, and noise

Time series data can exhibit a huge variety of patterns and it is helpful to categorize some of the patterns and behaviours that can be seen in time series.

The decomposition framework. Time series decomposition splits an observed series into three distinct components: a trend-cycle component, a seasonal component, and a remainder (or noise) component. By isolating these elements, analysts can better understand the underlying forces driving the data. This can be done additively (when seasonal variation is constant) or multiplicatively (when seasonal variation scales with the level of the series).

Decomposition methods. The book highlights several techniques for extracting these components:

• Classical decomposition: A simple, historical method using moving averages, though it lacks estimates at the endpoints.
• X-12-ARIMA: A robust, iterative method widely used by government agencies that handles trading day and holiday adjustments.
• STL decomposition: A highly versatile and robust method based on Loess smoothing that handles any seasonal period and allows seasonality to change over time.

Forecasting with decomposition. Decomposed components can be forecast separately to improve overall accuracy. The seasonal component is typically forecast using a seasonal naïve method, while the seasonally adjusted component (trend-cycle plus remainder) is forecast using a non-seasonal method like Holt's linear trend. Recombining these individual forecasts often yields highly accurate results.

7. Master exponential smoothing (ETS) for trend and seasonal patterns

Forecasts produced using exponential smoothing methods are weighted averages of past observations, with the weights decaying exponentially as the observations get older.

The ETS framework. Exponential smoothing is one of the most successful and widely used forecasting families in business. Simple Exponential Smoothing (SES) is ideal for data with no trend or seasonality, using a single parameter ($\alpha$) to control how quickly weights decay. For trending data, Holt's linear method adds a trend component, while the Holt-Winters method incorporates a seasonal component to handle complex seasonal patterns.

Damped trends. A critical innovation in exponential smoothing is the damped trend method. Standard linear trends often over-forecast when projected far into the future; damping introduces a parameter ($\phi$) that gradually flattens the trend over time. This approach has proven to be exceptionally robust and is the default choice for automated, large-scale business forecasting.

State space models. The modern formulation of exponential smoothing uses Innovations State Space Models, designated as ETS (Error, Trend, Seasonal). This framework:

• Generates identical point forecasts to traditional heuristic methods.
• Provides a statistical basis for calculating exact prediction intervals.
• Allows for automated model selection by minimizing information criteria like the AICc.

8. Use ARIMA models to capture autocorrelation in stationary data

While exponential smoothing models were based on a description of trend and seasonality in the data, ARIMA models aim to describe the autocorrelations in the data.

Stationarity and differencing. Unlike exponential smoothing, ARIMA models require the time series to be stationary—meaning its statistical properties do not change over time. Non-stationary series with trends or seasonality must be made stationary through differencing. First-order differencing removes linear trends, while seasonal differencing removes seasonal patterns, stabilizing the mean of the series.

The ARIMA components. An ARIMA($p,d,q$) model combines three distinct elements to capture the autocorrelation structure of a stationary series:

• Autoregressive (AR, $p$): Forecasts the variable using a linear combination of its own past values.
• Integrated (I, $d$): The degree of differencing required to make the series stationary.
• Moving Average (MA, $q$): Forecasts the variable using past forecast errors.

Identification and selection. Identifying the correct orders of $p$ and $q$ is done by analyzing the Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots of the differenced data. In practice, the auto.arima() function in R automates this process by conducting unit root tests and searching for the model that minimizes the AICc, making ARIMA modeling highly accessible.

9. Combine regression with ARIMA to build dynamic regression models

The error series $\eta_t$ is assumed to follow an ARIMA model.

Merging regression and time series. While standard regression models assume that errors are independent and uncorrelated, real-world time series regression residuals almost always contain autocorrelation. Dynamic regression models solve this by combining the explanatory power of regression with the time-series tracking of ARIMA. The model uses predictor variables to explain the systematic variation and an ARIMA model to capture the remaining autocorrelation in the errors.

The danger of spurious regression. Ignoring autocorrelation in regression errors is highly dangerous. It leads to "spurious regression," where standard errors are severely underestimated, making irrelevant predictors appear highly statistically significant. To prevent this, all variables in a dynamic regression model must be differenced to achieve stationarity before the model is estimated.

Lagged effects. Dynamic regression also allows for lagged predictors, where the effect of an explanatory variable is distributed over several time periods:

• Useful for modeling advertising campaigns where the impact on sales persists for months.
• Allows the model to capture delayed economic policy impacts.
• The optimal lag length can be determined systematically by minimizing the AICc.

10. Reconcile hierarchical time series for aggregate consistency

The optimal combination approach optimally combines the independent base forecasts and generates a set of revised forecasts that are as close as possible to the univariate forecasts but also aggregate consistently with the hierarchical structure.

Hierarchical structures. Many business time series are naturally organized in a hierarchy, such as sales by country, state, region, and product category. Generating independent forecasts for each level of the hierarchy leads to "aggregate inconsistency," where the regional forecasts do not sum to the national forecast. Reconciling these forecasts is essential for coherent operational planning.

Traditional reconciliation methods. Historically, organizations used simple top-down or bottom-up approaches to force consistency:

• Bottom-up: Forecasts the lowest level and sums them upward; captures individual dynamics but can be highly noisy.
• Top-down: Forecasts the total and disaggregates it downward using historical proportions; simple but loses local information and seasonality.
• Middle-out: Combines both by forecasting a middle level, aggregating upward, and disaggregating downward.

The optimal combination approach. The modern, state-of-the-art solution is the optimal combination approach. This method generates independent "base" forecasts for every single node in the hierarchy and then uses a regression-based framework to adjust them. The resulting revised forecasts are unbiased, aggregate perfectly, and minimize the overall forecast variance by utilizing all available information across the entire hierarchy.