Análise da performance de modelos de LLM em sumarização de artigos científicos escritos em português brasileiro após processo de fine-tuning

Abstract

This paper evaluates large language models (LLMs) for summarizing Brazilian Portuguese scientific texts, focusing on fine-tuning effects. We compare PTT5 and Flan-T5 (encoder-decoder) and mBERT and RoBERTa (encoder-only) on 100 full papers (five fields, ≥ 2020). Generative models use chunking (512 tokens, overlap) with map-reduce aggregation, while encoder-only models perform extractive selection via cosine similarity to the authors' abstracts. Fine-tuning (LoRA) is assessed with ROUGE, BERTScore, and qualitative analysis. Results show Portuguese-aligned pretraining is crucial: PTT5 degrades after fine-tuning and Flan-T5 becomes unstable/consistent with language collapse; conversely, fine-tuning improves extractive models, especially mBERT, indicating more relevant and less redundant sentence selection.