ReScraper: Unified Scraping and Cleaning of Web Data for Effective LLM Pretraining.

arXiv:2609.34287v1 Announce Type: cross Abstract: LLM pretraining corpora are normally cleaned by a stack of hand-written heuristics. A heuristic scraper extracts the main content from HTML, and dozens of rule-based filters then clean it, so corpus quality is