Comparative analysis of legal text summarization using transformer-based models

Automatic summarization of legal documents is particularly challenging due to the detailed structure, legal specific terminology, and the necessity for same level of accuracy. This paper compares the application of transformer-based deep learning models to legal text summarization. We benchmark several architectures, Legal-BERT, Legal-BART, Legal-PEGASUS, Legal-LED, Legal-Longformer, and Hierarchical T5, on legal specific benchmark datasets, BillSum, CaseHOLD, and EURLEX. Results are evaluated on metrics of ROUGE, CaseHOLD accuracy, and processing time performance. We find a clear trade-off between accuracy and processing efficiency. Encoder-only models such as Legal-BERT are time efficient and entity extracting, encoder-decoder models such as Legal-BART generate a better overall quality, more coherent glance-descriptive summary. Long-context models (Legal-LED, Longformer) have a great deal of model cost to process long judgments; they perform excellently with lengthy documents. We found the best approach is highly dependent on the task and characterized by the document type being summarized, the required summary format (extractive vs. abstractive), and resources. In this study, we present a framework for determining when to use an appropriate transformer architecture with the goal of advancing the automation of legal document processing.