A few months ago, I gave another Machine Learning subgroup presentation on ICYM²I: The Illusion of Multimodal Informativeness under Missingness by Choi et al. (2026) ¹. The paper asks a deceptively simple question: if a modality is missing for most patients, how can we know how useful it would be if it were collected for everyone?

This question is especially important in medicine. Chest X-rays (CXRs), magnetic resonance imaging (MRI), biopsies, echocardiograms, genomics, and laboratory tests are not collected randomly. They are ordered based on symptoms, previous test results, cost, clinical suspicion, and local workflow. Therefore, the presence of a modality, or the lack thereof, already contains information about the patient.

Complete-case analysis is not neutral

A common approach to multimodal research is to keep only patients who have every required modality. Researchers may begin with a dataset containing two modalities, \(X_1\) and \(X_2\), and an outcome \(Y\). After dropping every patient missing either modality, they train \(X_1\)-only, \(X_2\)-only, and \(X_1+X_2\) models on the remaining complete cases. The difference in performance is then interpreted as the value of adding a modality.

The issue is that the complete subset may not represent the target patient population. Let \(\Omega\) denote the full population and \(\Omega_{obs}\) the subset with all modalities observed. When test ordering depends on patient characteristics, \(\Omega_{obs} \neq \Omega\). For example, patients who receive both an ECG and a CXR may be older, more symptomatic, or more likely to have an abnormal initial test than patients who receive only an ECG. A comparison performed only on complete cases estimates the modality’s value for this selected group, not necessarily for every patient.

This is a distribution-shift problem rather than only an imputation problem. Filling in a missing modality may help a model produce a prediction, but it does not automatically correct the fact that the observed sample came from a systematically different population.

How can data be missing?

Missingness is commonly separated into three mechanisms ²:

• Missing completely at random (MCAR): whether a modality is missing is independent of both observed and unobserved patient information. One example might be a truly random sensor failure.
• Missing at random (MAR): missingness can be explained by observed covariates, \(C\). For example, CXR availability may depend on age, recorded symptoms, diagnoses, visit type, or ECG features.
• Missing not at random (MNAR): missingness still depends on information that is not observed. If an undocumented clinical judgment influences whether a test is ordered, the available data may not explain the missingness process.

ICYM²I operates under MAR and positivity. Positivity means that every relevant patient subgroup has at least some chance of appearing as a complete case:

Here, \(M_j=0\) indicates that a variable is observed. Without positivity, there is no complete example from which the method can learn about a particular subgroup.

ICYM²I: correcting the observed distribution

ICYM²I stands for In Case You Multimodal Missed It. Its goal is not to introduce a new fusion architecture or directly improve prediction. Instead, it tries to estimate predictive performance and modality informativeness under the full target distribution, even though only the complete cases contain every required input.

The framework can be summarized in four steps:

1. Estimate the probability of being a complete case. A propensity model uses observed covariates \(C\) to estimate

   \[\pi(C)=P(M_1=0,M_2=0,M_Y=0 \mid C).\]

2. Reweight the complete cases. Each complete patient receives an inverse probability weight

   \[w(C)=\frac{1}{\pi(C)}.\]

   A patient who had only a 5% chance of being complete receives more weight than a patient who had an 80% chance of being complete. In this way, uncommon complete cases stand in for similar patients who were excluded because a modality was missing.

3. Correct model training and evaluation. The unimodal and multimodal models are trained with a weighted loss:

   \[\hat{f}=\arg\min_f \sum_{i \in \Omega_{obs}} w_i\,\ell(f(X_i),Y_i).\]

   Weighted evaluation is also needed. Correcting only training or only evaluation would leave part of the source-to-target shift unaddressed.

4. Correct the estimated information from each modality. ICYM²I applies the same missingness correction to Partial Information Decomposition (PID), which describes how two modalities jointly contain information about an outcome.

What is Partial Information Decomposition?

Model performance alone does not fully explain how modalities contribute. A multimodal model can underperform because of overfitting or optimization difficulty even when both modalities contain useful information. PID instead divides the total information that \(X_1\) and \(X_2\) contain about \(Y\) into four parts ³:

where:

• Shared information is information about \(Y\) that is present in both modalities.
• Unique information is available from one modality but not the other.
• Complementary information appears only when the two modalities are considered together. This is also commonly described as synergy.

This decomposition is valuable for data-collection decisions. If an expensive second modality contributes substantial unique or complementary information, collecting it may be worthwhile. If nearly all of its information is already shared with a cheaper first modality, its additional value may be limited.

A synthetic example: XOR

The paper first tests ICYM²I using logical operators such as AND, OR, and XOR. XOR provides a particularly clear example. Neither input bit alone predicts the output, but the pair determines it perfectly. Therefore, its information should be almost entirely complementary.

After the authors introduce MAR missingness, naive complete-case analysis makes the first input appear to have substantial unique information. For XOR, the observed analysis estimates unique information of 0.34 and 0.07 for the two inputs and complementary information of only 0.62. ICYM²I returns the unique terms to approximately zero and recovers complementary information of 0.96, close to the oracle value of 0.99. In other words, missingness can create an illusion that one modality is independently informative even when the true signal exists only in the interaction between modalities.

Do chest X-rays add information beyond ECGs?

The paper then applies ICYM²I to model-based structural heart disease (SHD) detection. Its current clinical cohort contains 98,397 adults who received an ECG and a transthoracic echocardiogram (TTE) within one year, but only 12,587 patients (12.79%) also had a recorded CXR. The TTE provides the SHD label, while ECG and CXR are the two predictive modalities. Because clinicians do not order CXRs uniformly, the patients with both predictive modalities form a selected subset.

In the corrected analysis, the rounded AUROCs are 0.82 for ECG, 0.73 for CXR, and 0.83 for ECG+CXR. More importantly, naive PID attributes about 5% of the total information to the CXR alone, whereas ICYM²I reduces the CXR’s unique contribution to 1.8% and estimates very little complementary information. Most of the useful information is either captured by the ECG or shared between the modalities.

This result does not mean that CXRs are clinically unnecessary. It is specific to this cohort, outcome, representations, and model-based screening task. Rather, it shows that after accounting for why CXRs are missing, their incremental information for this particular prediction problem is much smaller than a naive complete-case analysis may suggest.

Limitations

ICYM²I still has several important limitations. First, the MAR assumption cannot be verified from the observed data alone, and the method has no theoretical guarantee under MNAR. The propensity model must include the variables that meaningfully explain why a modality is missing.

Second, positivity requires enough complete cases across the population. When a subgroup almost never has both modalities, its estimated completeness probability approaches zero and its inverse probability weight becomes very large. This produces high variance and unstable estimates.

Lastly, the PID formulation focuses on two paired modalities. Extending the decomposition to more than two modalities remains an open problem, and the results also depend on the quality and calibration of the learned representations. The authors therefore describe the clinical application as a proof of concept rather than a method ready for deployment.

Thoughts

This paper is compelling because it reframes missing data as not only a modeling challenge but also an evaluation challenge. In my previous post about FuseMoE, the central question was how a multimodal model can continue to make predictions when modalities are absent. ICYM²I asks a different question: before building a larger multimodal model, have we correctly estimated whether the additional modality contributes useful information at all?

This distinction is highly relevant to computational cardiology. Whether a patient receives an echocardiogram, CXR, cardiac MRI, or genetic test is influenced by symptoms, prior findings, access to care, and clinical judgment. A complete-case dataset can therefore concentrate the most complicated or suspicious patients. If we ignore this selection process, we risk confusing clinical workflow with biological signal.

ICYM²I does not eliminate the difficulty of missing clinical data, but it provides a useful framework for making the assumptions explicit. Before concluding that “more modalities are better,” we should ask why those modalities were collected, which patients are missing from the analysis, and whether the complete cases actually represent the population where the model will be used.