Small Language Model, Adaptive Fragmentation in Long-Text Summarization

Abstract

While Large Language Models (LLMs) excel at diverse NLP tasks, their computational demands limit deployment in specialized applications. Thus, Small Language Models (SLMs) come into play: this work investigates the use of an SLM for long-text summarization. As the SLM’s context window is small, fragmentation of a long document is mandatory. We systematically evaluate three preprocessing strategies for the fragmentation of long texts: naive token chunking, classical TextTiling, and neural sentence-embedding segmentation, all employed within a hierarchical summarization pipeline using DistilBART. The results of the experimental evaluation based on stratified sampling exhibit how suitable a fragmentation method is for a particular length of text. Actually, experiments on CNN/DailyMail articles reveal a length-dependent performance pattern: token chunking performs best for documents under 1,584 tokens, while neural segmentation excels for documents exceeding 2,250 tokens. Our results yield a practical, adaptive fragmentation policy that balances summary quality with computational efficiency.