Irregular and asynchronous sampled multivariate time series (MTS) data is often filled with missing values. Most existing methods embed features according to timestamp, requiring imputing missing values. However, imputed values can drastically differ from real values, resulting in inaccurate predictions made based on imputation. To address the issue, we propose a novel concept, “each value as a token (EVAT),” treating each feature value as an independent token, which allows for bypassing imputing missing values. To realize EVAT, we propose scalable numerical embedding, which learns to embed each feature value by automatically discovering the relationship among features. We integrate the proposed embedding method with the Transformer Encoder, yielding the Scalable nUMerical eMbeddIng Transformer (SUMMIT), which can produce accurate predictions given MTS with missing values. We induct experiments on three distinct electronic health record (EHR) datasets with high missing rates. The experimental results verify SUMMIT's efficacy, as it attains superior performance than other models that need imputation. (paper link)

Stroke is a leading cause of mortality and long-term disability worldwide. An accurate stroke risk prediction is crucial for its early detection and prevention. Using deep learning to exploit patients’ time-series electronic health records (EHRs) has been shown as a promising and efficient solution for such a prediction. Although time-series data could be more informative than a single cross-section in time, real-world time-series EHRs usually have a significantly high missing rate due to irregular patient visits. This could undermine sequential data’s benefits unless a proper deep-learning model design is adopted. Furthermore, deep models have long been challenged for their interpretability, which is especially crucial for medical applications. In this study, we propose an extreme design based on the concept of recurrent independent mechanisms (RIM), termed extreme RIM (X-RIM). With no need for imputation, X-RIM utilizes the information of each input feature’s temporal records through independent recurrent modules. Experiments on real-world data from the National Taiwan University Hospital showed that, in terms of the area under the precision-recall curve (AUPRC), the area under the receiver-operating characteristics curve (AUROC), and Youden Index, X-RIM (AUPRC: 0.210; AUROC: 0.764; Youden: 0.373) outperformed the classic risk score CHA2 DS2-VASc (AUPRC: 0.103; AUROC: 0.650; Youden: 0.223) and other benchmarks in stroke risk prediction. Additional experiments also indicate that individual feature contributions to a prediction could be evaluated intuitively under X-RIM’s independent structure to enhance interpretability. (paper link)