Volatility Based Kernels and Moving Average Means for Accurate Forecasting with Gaussian Processes

Authors	Gregory Benton, Wesley Maddox, Andrew Gordon Wilson
Title	Volatility Based Kernels and Moving Average Means for Accurate Forecasting with Gaussian Processes
Publication	Proceedings of the 39th International Conference on Machine Learning (ICML 2022)
Volume	162
Issue	x
Pages	1798-1816
Year	2022
DOI	x

Introduction

Background

• Gaussian processes (GP) have had significant success in time series modeling.
• GPs are strong candidates for modeling and forecasting time-dependent financial and climatological data.

Previous Research

• Building GPs has historically relied on selecting out-of-the-box kernels and mean functions that make assumptions that do not hold for all cases.
• Mat´ern and RBF kernels assume stationarity of the data, and constant or linear means assume that the underlying trends in the data are not time-varying.
• Spectral mixture and deep kernels have been primarily developed for general problems rather than constructing a kernel from prior knowledge about the task at hand.

Proposed Model

• In this work, the authors approach GP modeling by building on domain knowledge to construct a novel set of kernel and mean functions with inductive biases well suited for forecasting in domains such as finance and climatology.
• The authors develop a hierarchical GP model with specialized kernels, termed Volt, which uses forecasts of volatility to specify the covariance structure over future data observations.
• By considering not only a single volatility forecast but a distribution of volatility forecasts, the authors induce a distribution over covariance functions in the data domain.
• To capture trends in data, the authors jointly introduce Moving Average Gaussian Processes (Magpie), which replaces the standard parametric mean function in GPs with a moving average.

Significance

• The covariance structure described by Volt provides a faithful representation of the uncertainty in forecasts, while Magpie enables accurate trend capture in the probabilistic framework of GPs.
• The combination of Volt and Magpie solves challenging forecasting problems in time-evolving domains like stock prices or wind speeds, producing both accurate point estimates and calibrated uncertainties.
• The proposed method produces highly calibrated forecasts in financial and climatological domains and can be extended to multitask problems by accounting for correlations in both volatility and price across different financial assets and different spatial locations.

Proposed Model

• The proposed model assumes noisy observations of y(t) that follow a Gaussian process.
• f(t) is drawn from a GP with mean function v(t) and covariance function k(t, t0).
• The posterior predictive distribution, p(f(t) | D), over new data points t' is computed using p(f(t')|D, θ) = N(μ, Σ), where μ is the posterior mean and Σ is the posterior covariance.
• The model assumes that both the data S(t) and volatility V(t) have paths with log-normal marginal distributions.
• The model places a joint SDE structure over s(t) = log S(t) and v(t) = log V(t), with a drift term that arises from the log-transformation of the volatility.
• The structure allows for closed-form expressions for auto-covariance functions associated with both log-data and log-volatility, and enables the definition of the Volt model.
• The relationship between the log-price and log-volatility, which is mirrored by many stochastic volatility models, including GARCH and SABR.
• The approach moves from an SDE sampling approach to a proper forecasting system based on historical observations.
• The observations v = v(t) and s = s(t) each have a multivariate normal distribution, v and s correspond to Gaussian processes, and the model derives the mean and covariance functions of the two processes to form predictive distributions from GPs.
• To generate predictions using the log-volatility and log-price GPs, the model first infers both a volatility path from the observed time series, S = S(t), and the hyperparameters of both the data and volatility models.

Experiment

• Comparison of Volt and Magpie to baseline models of GPs with standard kernel and mean functions as well as probabilistic LSTMs in stock price forecasting.
• Volt is able to remedy a significant overconfidence that is present in alternative methods such as standard GP kernels or probabilistic LSTMs in stock price forecasting.
• Magpie mean function enables the distributions to be centered at the correct values in stock price forecasting.
• The Volt model outperforms competing methods such as LSTMs and GPs with Mat´ern and Spectral Mixture (SM) kernels in both stock price and foreign exchange data forecasting.
• Volt with a constant mean is slightly better than with a Magpie mean in terms of NLL, but the Magpie mean is key to achieving high calibration in stock price forecasting.
• Volt and Magpie applied to the problem of developing a stochastic weather model for wind speed.
• Volt models are key to producing accurate forecasts in wind speed forecasting.
• Constant means provide slightly better NLL values than Magpie means in wind speed forecasting.
• Forecast distributions must match the ground truth of the data in both stock price and wind speed forecasting.