Supervised Multimodal Fission Learning

Learning from multimodal data sets can leverage complementary information and lead to improved performance for prediction tasks. A commonly used strategy to account for feature correlations in high-dimensional data sets is the latent variable approach. Several latent variable methods have been proposed for multimodal data sets; however, these methods either focus on extracting a shared component across all modalities or extracting a shared component and individual components specific to each modality, overlooking correlations within partial subsets of modalities. We propose multimodal fission learning (MMFL), the first supervised latent variable model that adopts a generalizable decomposition into globally joint, partially joint, and individual components from multimodal data sets. A key strength of MMFL is a natural extension to incorporate incomplete multimodal data in either training and test phases by leveraging the learned modality structure. Through simulation studies, we demonstrate that MMFL outperforms a variety of existing multimodal algorithms under both complete modality and incomplete modality settings. We applied MMFL to two real-world case studies: early prediction o