Enhancing Text Summarization Consistency via Iterative Reset Strategies and Context Analysis in RAG-Based Summarization with Hybrid LLMs

Abstract

Introduction: This paper investigates the transformative impact of iterative reset strategies within a Retrieval-Augmented Generation (RAG) framework for text summarization. It examines how integrating resets into hybrid large language model (LLM) workflows can enhance summary coherence and reduce retrieval noise.

Objectives: The study aims to evaluate the effects of discrete reset methodologies on the performance of RAG-based summarization. It focuses on improving content overlap and stylistic consistency while measuring outcomes using standard metrics such as ROUGE, FactCC, and readability scores.

Methods: A comprehensive RAG pipeline is developed by combining text segmentation, semantic embedding, vector database indexing, keyword extraction, and stylistic analysis. Five reset strategies are implemented and tested on the eLife dataset, with iterative evaluations conducted to compare the resulting ROUGE and FactCC metrics alongside processing time and Flesch-Kincaid readability measures.

Results: The analysis reveals that iterative resets particularly the vector database reset (Method 3) and the full reset (Method 5) yield higher ROUGE scores (0.3716 and 0.3706, respectively) compared to baseline approaches. However, these methods also exhibit variable factual consistency, as evidenced by moderate FactCC scores.

Conclusions: The findings underscore that while iterative reset strategies significantly enhance content overlap and summary coherence in RAG frameworks, they also introduce challenges in maintaining factual accuracy. The study offers valuable insights into optimizing RAG workflows and suggests further exploration of adaptive reset mechanisms to achieve a balanced performance.