Journal cover Journal topic
Earth System Dynamics An interactive open-access journal of the European Geosciences Union

Journal metrics

  • IF value: 4.589 IF 4.589
  • IF 5-year<br/> value: 3.696 IF 5-year
    3.696
  • CiteScore<br/> value: 3.94 CiteScore
    3.94
  • SNIP value: 0.995 SNIP 0.995
  • SJR value: 2.742 SJR 2.742
  • IPP value: 3.679 IPP 3.679
  • h5-index value: 21 h5-index 21
doi:10.5194/esd-2016-51
© Author(s) 2016. This work is distributed
under the Creative Commons Attribution 3.0 License.
Research article
02 Nov 2016
Review status
A revision of this discussion paper is under review for the journal Earth System Dynamics (ESD).
Multivariate Anomaly Detection for Earth Observations: A Comparison of Algorithms and Feature Extraction Techniques
Milan Flach1, Fabian Gans1, Alexander Brenning2,4, Joachim Denzler3,4,5, Markus Reichstein1,4,5, Erik Rodner3,4, Sebastian Bathiany6, Paul Bodesheim1, Yanira Guanche3,4, Sebastian Sippel1, and Miguel D. Mahecha1,4,5 1Max Planck Institute for Biogeochemistry, Department Biogeochemical Integration, P. O. Box 10 01 64, D-07701 Jena, Germany
2Friedrich Schiller University Jena, Department of Geography, Jena, Germany
3Friedrich Schiller University of Jena, Department of Mathematics and Computer Sciences, Computer Vision Group, Jena, Germany
4Michael Stifel Center Jena for Data-driven and Simulation Science, Jena, Germany
5German Centre for Integrative Biodiversity Research (iDiv), Leipzig, Germany
6Wageningen University, Department of Environmental Sciences, Wageningen, Netherlands
Abstract. Today, many processes at the Earth's surface are constantly monitored by multiple data streams. These observations have become central to advance our understanding of e.g. vegetation dynamics in response to climate or land use change. Another set of important applications is monitoring effects of climatic extreme events, other disturbances such as fires, or abrupt land transitions. One important methodological question is how to reliably detect anomalies in an automated and generic way within multivariate data streams, which typically vary seasonally and are interconnected across variables. Although many algorithms have been proposed for detecting anomalies in multivariate data, only few have been investigated in the context of Earth system science applications. In this study, we systematically combine and compare feature extraction and anomaly detection algorithms for detecting anomalous events. Our aim is to identify suitable workflows for automatically detecting anomalous patterns in multivariate Earth system data streams. We rely on artificial data that mimic typical properties and anomalies in multivariate spatiotemporal Earth observations. This artificial experiment is needed as there is no 'gold standard' for the identification of anomalies in real Earth observations. Our results show that a well chosen feature extraction step (e.g. subtracting seasonal cycles, or dimensionality reduction) is more important than the choice of a particular anomaly detection algorithm. Nevertheless, we identify 3 detection algorithms (k-nearest neighbours mean distance, kernel density estimation, a recurrence approach) and their combinations (ensembles) that outperform other multivariate approaches as well as univariate extreme event detection methods. Our results therefore provide an effective workflow to automatically detect anomalies in Earth system science data.

Citation: Flach, M., Gans, F., Brenning, A., Denzler, J., Reichstein, M., Rodner, E., Bathiany, S., Bodesheim, P., Guanche, Y., Sippel, S., and Mahecha, M. D.: Multivariate Anomaly Detection for Earth Observations: A Comparison of Algorithms and Feature Extraction Techniques, Earth Syst. Dynam. Discuss., doi:10.5194/esd-2016-51, in review, 2016.
Milan Flach et al.
Milan Flach et al.
Milan Flach et al.

Viewed

Total article views: 433 (including HTML, PDF, and XML)

HTML PDF XML Total BibTeX EndNote
301 108 24 433 6 22

Views and downloads (calculated since 02 Nov 2016)

Cumulative views and downloads (calculated since 02 Nov 2016)

Viewed (geographical distribution)

Total article views: 433 (including HTML, PDF, and XML)

Thereof 433 with geography defined and 0 with unknown origin.

Country # Views %
  • 1

Saved

Discussed

Latest update: 27 Apr 2017
Publications Copernicus
Download
Short summary
Anomalies and extremes are often detected using univariate peak-over-threshold approaches in the Geoscience community. The Earth system is highly multivariate. We compare 8 multivariate anomaly detection algorithms and combinations of data preprocessing. We identify 3 anomaly detection algorithms which outperform univariate extreme event detection approaches. The workflows have the potential to reveal additional knowledge. Remarks on their application to "real" Earth observations are provided.
Anomalies and extremes are often detected using univariate peak-over-threshold approaches in the...
Share