The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models.

arXiv:2610.26651v1 Announce Type: cross Abstract: Pretraining filtering is one of the most common safeguards for open-weight language models. Here we study its effectiveness, durability and scalability using the ESM-2 family of protein language models. We find