Supplementary Material for: Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments

This repository provides supplementary materials (source code, pre-trained models, and reproducibility scripts) for the paper "Confidence-Aware Gated Multimodal Fusion for Robust Temporal Action Localization in Occluded Environments". Content Source code: Official PyTorch implementation of the Gated Skeleton Refinement Module (Gated SRM) integrated into ActionFormer, along with all ablation variants and fusion baselines. Pre-trained models: Best-epoch model checkpoints for all configurations evaluated on the IKEA ASM and THUMOS14 datasets, including the proposed Gated SRM, Cross-Attention Fusion, and four ablation variants. Configuration and scripts: Complete YAML configuration files, training/evaluation scripts, and tools to reproduce all reported results, including mAP, Boundary-F1, per-class AP analysis, and statistical significance tests. New in v2 (R1 revision): Gate initialization ablation study (b_init ∈ {−2.0, 0.0, +2.0}; Section 5.6) Skeleton quality validation script: confidence vs. spatial jitter correlation analysis (Appendix A) TriDet integration code demonstrating the generalizability of the Gated SRM as an architecture-agnostic front-end module (Section 5.5) Cross-Attention Fusion and Late Fusion baseline implementations for comparison (Tables 3–7) End-to-end latency profiling including OpenPose inference (Section 6.3) Per-class AP visualization for all 32 IKEA ASM action classes (Appendix B) Abstract In industrial environments, robust Temporal Action Localization (TAL) is essential; how-ever, frequent occlusions often compromise the reliability of skeletal data, leading to nega-tive transfer in multimodal fusion. To address this challenge, we propose a Gated Skeleton Refinement Module (Gated SRM), a universal front-end preprocessing module, that ex-plicitly incorporates OpenPose confidence scores into the network architecture. By apply-ing these scores as a logarithmic bias within a self-attention mechanism, our method achieves soft suppression—dynamically attenuating the attention weights assigned to unreliable joints—before adaptively fusing the refined skeletal features with RGB repre-sentations through a learnable gating network. Extensive experiments on the heavily oc-cluded IKEA ASM dataset demonstrate that our approach effectively prevents the cata-strophic accuracy degradation typical of naive and established multimodal fusion strate-gies, improving the mean Average Precision (mAP) to 21.77%, maintaining parity with the RGB-only baseline while demonstrating superior robustness. Furthermore, the system maintains a practical end-to-end inference speed of approximately 9.2 frames per second (FPS), which is sufficient for monitoring macro-level industrial workflows. By prioritizing confidence-based data selection over data restoration, this sensor-metadata-driven archi-tecture offers a robust and principled approach acting as a critical fail-safe and safety-net for real-world action recognition under occlusion. Funding This research was supported by a grant from the Japan Society for the Promotion of Science (JSPS) Grants-in-Aid for Scientific Research (JP23K11724).