ML4ITS - Machine Learning for Irregular Time Series

Repository with code and resources related to the current activity on time series analysis.

Interesting papers/resources

Refer to this document for a curated list of papers/articles.

Here some updated notes on articles/papers read

Time Series Classification

• Deep Learning for Time Series Classification (InceptionTime) (2020) [post]
• Deep learning for time series classification: a review (DMKD2019) [paper]

Time Series Forecasting

• Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting (NIPS2019) [paper]
• Multivariate Temporal Convolutional Network:A Deep Neural Networks Approach for Multivariate Time Series Forecasting (MDPI2019) [paper]
• Joint Modeling of Local and Global Temporal Dynamics for Multivariate Time Series Forecasting with Missing Values (IRREGULAR, forecasting with Missing values AAAI2020)
• Shape and Time Distortion Loss for Training Deep Time Series Forecasting Models (IRREGULAR, Non-stationary - NIPS2020)
• DIFFUSION CONVOLUTIONAL RECURRENT NEURAL NETWORK: DATA-DRIVEN TRAFFIC FORECASTING (ICLR2018) [paper]
• Time-series Extreme Event Forecasting with Neural Networks at Uber (2017) (Autoencoder and LSTM for rare event prediction in univariate TS)

Anomaly Detection / Failure Prediction

• Generative Adversarial Networks for Failure Prediction (ECML2019) [paper]
• A GAN-Based Anomaly Detection Approach for Imbalanced Industrial Time Series (IEEE2019) [paper]
• DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series (IEEE2019) [paper]
• Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network (KDD2019)

GAN for Data Imputation (Time Series Domain and not)

• [NoTS] GAIN: Missing Data Imputation using Generative Adversarial Nets (IJCAI2018) [paper] [code (tensorflow), code2] [mycode]
• [MTS] E²GAN: End-to-End Generative Adversarial Network for Multivariate Time Series Imputation (IJCAI2019) [paper] [code (tensorflow)] [mycode]
• [NoTS] MisGAN: Learning from Incomplete Data with Generative Adversarial Networks (ICLR2019) [paper] [code (tensorflow)] [mycode]
• GAN and Missing Data Imputation (medium post)
• Recurrent Neural Networks for Multivariate Time Series with Missing Values (Nature 2018) [paper]

Generative Models for (Multivariate) Time Series

• Quant GANs: Deep Generation of Financial Time Series (arXiv2019) [paper]
• Generating Financial Series with Generative Adversarial Networks (blog post) [part1][part2]
• Real-valued (Medical) Time Series Generation with Recurrent Conditional GANs (arXiv2017) [paper] [code (pytorch)]

Attention Mechanism for Time Series Analysis

• Forecasting stock prices with long-short term memory neural network based on attention mechanism (PlosONE 2020)
• DATA-GRU: Dual-Attention Time-Aware Gated Recurrent Unit for Irregular Multivariate Time Series (IRREGULAR, AAAI2020)
• DSANet: Dual Self-Attention Network for Multivariate Time Series Forecasting (CIKM2019 - Short)
• Multivariate Time Series Early Classification with Interpretability Using Deep Learning and Attention Mechanism (ECML-PKDD2019) [TSC,MV]
• Modeling Extreme Events in Time Series Prediction (KDD2019) [TSF, UV]
• Multi-Horizon Time Series Forecasting with Temporal Attention Learning (KDD2019)
• CAMP: Co-Attention Memory Networks for Diagnosis Prediction in Healthcare (ICDM2019)
• Temporal pattern attention for multivariate time series forecasting (ECML-PKDD2019) [TSF,MV]
• MuVAN: A Multi-view Attention Network for Multivariate Temporal Data (ICDM2018)
• Attend and Diagnose: Clinical Time Series Analysis Using Attention Models (AAAI2018)
• A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction (IJCAI2017) [TSF, UV]

Transfer Learning for Time Series

• ConvTimeNet: A Pre-trained Deep Convolutional Neural Network for Time Series Classification (IJCNN2019) [TSC, UV]
• Transfer Learning for Financial Time Series Forecasting (PRICAI2019) [paper]
• Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning (arXiv2019) [paper]
• Multi-source transfer learning of time series in cyclical manufacturing (Journal of Intelligent Manufacturing 2019) [paper]
• Transfer Learning Based Fault Diagnosis with Missing Data Due to Multi-Rate Sampling (MDPI2019) [paper]
• Transfer learning for time series classification (IEEE Conference on Big Data2018) [paper]
• Reconstruction and Regression Loss for Time-Series Transfer Learning (SIGKDD MiLeTS' 2018) [paper]
• Towards a universal neural network encoder for time series (CCIA2018) [TSC, UV]
• Transfer Learning with Deep Convolutional Neural Network for SAR Target Classification with Limited Labeled Data (MDPI2017) [paper]

Unsupervised Learning, Representation Learning and Self Supervised Learning for TS

• Self-Supervised Learning for Semi-Supervised Time Series Classification (PAKDD2020) [TSC, MV]
• Self-supervised representation learning from electroencephalography signals (IEEE MLSP 2019)
• Unsupervised Scalable Representation Learning for Multivariate Time Series (NIPS2019) [paper] [code]
• Deep Multivariate Time Series Embedding Clustering via Attentive-Gated Autoencoder (PAKDD2020) [TSClusering, MV]
• Learning Representations for Time Series Clustering [TSClustering, UV] (NIPS2019)
• Unsupervised Scalable Representation Learning for Multivariate Time Series [TSC, MV] (NIPS2019)
• Unsupervised pre-training of a Deep LStM-based Stacked Autoencoder for Multivariate time Series forecasting problems (NatureSR 2019) [TSF,MV]
• Adversarial Unsupervised Representation Learning for Activity Time-Series (AAAI2019)
• A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data (AAAI2019) [TSAD,MV]
• TimeNet: Pre-trained deep recurrent neural network for time series classification
• Unsupervised Pre-training of a Deep LSTM-based Stacked Autoencoder for Multivariate Time Series Forecasting Problems (Nature2019) [TSF,MV]

Few-Shot Learning for Time Series Classification in Low-Data Regime

• TapNet: Multivariate Time Series Classification with Attentional Prototypical Network [TSC, MV] (AAAI2020)
• Meta-Learning for Few-Shot Time Series Classification (2019) [paper] (ACM-IKDD2020) [UV,TSC]
• Deep Prototypical Networks for Imbalanced Time Series Classification under Data Scarcity (CIKM2019 - Short)
• Few-shot Time-series Classification with Dual Interpretability [TSC]

Reservoir Computing for TS Analysis

• Analysis of Wide and Deep Echo State Networks for Multiscale Spatiotemporal Time Series Forecasting (NICE2019) [paper]
• Network Traffic Prediction Using Variational Mode Decomposition and Multi- Reservoirs Echo State Network (IEEE2019) [paper]
• Spiking Echo State Convolutional Neural Network for Robust Time Series Classification (IEEE2018) [paper]

Time Series and Asynchronous data

• Modeling asynchronous event sequences with RNNs [paper]

Data Reconstruction

• Deep learning for irregularly and regularly missing data reconstruction [paper]

Latent Models and ODE models for Time Series Modeling

• Neural ODEs (2019) [paper], [code (pytorch)]
• Latent ODEs for Irregularly-Sampled Time Series [paper], [code (pytorch)]

Time Series as Images

• Multivariate Time Series as Images: Imputation Using Convolutional Denoising Autoencoder (2020) [paper]

Pytorch Packages

• Pytorch Forecasting Library blogpost
• pyts: a Python package for time series classification github

Dataset

• 25 Datasets for Deep Learning in IoT
• 41 Multivariate Time Series from UCI Repository
• UEA & UCR Time Series Classification Repository [paper]
• SmartMeter Energy Consumption Data in London Households
• NVE Hydrological API (HydAPI)
• Wikipedia Web Traffic Time Series
• Measuring Broadband America (MBA)
• Google Cluster Usage Traces (GCUT)
• Physionet MIMIC-III

Blog Post and other resources

• List of state of the art papers focus on deep learning and resources !!!!! [github repo]
• On the Automation of Time Series Forecasting Models: Technical and Organizational Considerations. [blogpost]
• PyTorch Dataset for multivariate time series [code]

Examples

• Example-1: CNN for univariate TS forecasting
• Example-2: Simple GAN for learning and generate a simple math function