Abstract

In the era of big data, large-scale, multi-modal datasets are increasingly ubiquitous, offering
unprecedented opportunities for predictive modeling and scientific discovery. However,
these datasets often exhibit complex heterogeneity, such as covariate shift, posterior drift,
and missing modalities which can hinder the accuracy of existing prediction algorithms. To
address these challenges, we propose a novel Representation Retrieval (R2) framework,
which integrates a representation learning module (the representer) with a sparsity-induced
machine learning model (the learner). Moreover, we introduce the notion of "integrativeness"
for representers, characterized by the effective data sources used in learning representers,
and propose a Selective Integration Penalty (SIP) to explicitly improve the property.
Theoretically, we demonstrate that the R2 framework relaxes the conventional full-sharing
assumption in multi-task learning, allowing for partially shared structures, and that SIP can
improve the convergence rate of the excess risk bound. Extensive simulation studies
validate the empirical performance of our framework, and applications to two real-world
datasets further confirm its superiority over existing approaches.