# SHAP-Weighted Cross-Modal Fusion Matches Early Fusion on Emotion Recognition, Exceeds Late Fusion

_Saturday, July 11, 2026 at 4:08 AM EDT · Science, AI · Latest · Tier 1 — Major_

A new arXiv study revisits XAI-guided adaptive fusion (XGAF), a tree-based mixture of unimodal and cross-modal experts whose sample-level weights come from TreeSHAP attribution magnitudes. The key finding: when experts have unequal feature dimensionalities, mean-abs and median-abs SHAP reductions suppress high-dimensional cross-modal experts, while sum-abs reduction preserves total attribution mass.

On MELD 7-class emotion recognition, sum-abs XGAF nearly matches early fusion across three face-sequence aggregators; the Transformer variant reaches 0.5983 weighted F1 versus 0.6018 for early fusion and 0.4598 for probability-average late fusion. McNemar testing shows no significant difference between sum-abs XGAF and early fusion on MELD (p=1.000), while XGAF remains significantly better than late fusion (p<0.0001). On CMU-MOSEI 3-class sentiment recognition, sum-abs XGAF reaches 0.6519 weighted F1, slightly exceeding early fusion (0.6485) and late fusion (0.5696).

Ablation studies show the main gain comes from adding cross-modal experts, especially the trimodal expert, rather than from complex per-sample routing. Diagnostics confirm mean-abs and median-abs weights are nearly uniform, while sum-abs weights concentrate on the trimodal expert.

## Sources

- [arXiv](https://arxiv.org/abs/2607.08573)

---
Canonical: https://techandbusiness.org/newswire/A9KxU337ELsycETkonCZVt
Retrieved: 2026-07-11T11:29:08.424Z
Publisher: Tech & Business (techandbusiness.org)