Predicting Fatalities with Pre-trained Temporal Transformers: A Time Series Regression Approach

Abstract:

The ability to predict with precision events, such as the number of battle-related fatalities, is not only of academic interest, but it holds significant implications for policy-making and conflict prevention too. This broad interest has enlightened our research, which, specifically, delves into the use of temporal transformers as a new approach to predict
the number of battle-related deaths, at the country-level and over a forecast temporal horizon spanning from 3 to 14 months. Our Artificial Intelligence – Early Warning System (AI-EWS), proposed for the 2023/24 VIEWS prediction competition [Hegre et al., Forthcoming], leverages a multi-headed attention mechanism as outlined by Vaswani et al. [2017]. We chose the temporal transformers due to their proven efficacy in time series representation learning, as demonstrated in Zerveas et al. [2021]. The model incorporates residual connections from input to output, preserving linear activation, a method supported by empirical evidence for its effectiveness in time-series forecasting [Zeng et al., 2023]. The following section details the methodology used to harness this model for time series regression.

Authors:

Luca Macis, Marco Tagliapietra, Elena Siletti, and Paola Pisano