Are Joint-Embedding Predictive Objectives Effective for Longitudinal EHRs?

arXiv:2610.29528v1 Announce Type: cross Abstract: Masked language modelling (MLM) is the dominant pretraining objective for transformer encoders applied to longitudinal electronic health records (EHRs). However, reconstructing individual events may overemphasi