Local Research Assistant in a Resource-Constrained Environment

Abstract

Retrieval-Augmented Generation (RAG) effectively grounds Large Language Models in external knowledge, yet standard chunking methods frequently disrupt vital semantic connections required for accurate retrieval. While state-of-the-art solutions like Contextual Retrieval restore these links by prepending full-document context, they impose substantial computational and memory burdens that preclude deployment on consumer-grade hardware. To address this barrier, this study introduces a resource-optimized variant specifically engineered for privacy-centric applications running on constrained infrastructure. The proposed "Document Slice" methodology replaces the exhaustive full-document context with a sliding window of adjacent text chunks. This technique relies on the premise that semantic relevance is typically clustered locally, thereby drastically reducing memory overhead and mitigating the "Lost in the Middle" phenomenon. Empirical evaluation against an industry-standard baseline, utilizing a dataset of 250 multi-hop scientific questions, demonstrated a 60% reduction in preprocessing time. Crucially, this architectural efficiency operates comfortably within standard consumer memory limits while incurring a negligible performance penalty of less than 1% on Recall@20. Ultimately, these findings validate that highly accurate retrieval systems can be sustainably developed without industrial-scale compute, advancing the accessibility of "Green AI" research frameworks.