Semantic data processing systems optimize operator ordering and model selection to balance cost, latency, and quality, but treat each LLM invocation as independent. This overlooks prefix caching, a capability of modern inference engines that memoizes the KV-cache of shared input prefixes so that requests sharing a common prefix skip redundant prefill computation. We propose that when multiple queries are issued over a shared document corpus, the resulting redundancy across operators and queries constitutes a distinct and largely unexploited optimization space. We formalize this batch-query setting, in which operators across queries frequently share document context and can be co-scheduled to reuse a common prefix before any operator that modifies or restructures documents invalidates the cache. We characterize the tradeoffs between document caching and operator caching, along with complementary strategies including operator fusion, document batching, eager operator advancement, prefix-preserving predicate pushdown, and semantic filter consolidation, and show how the optimal strategy shifts as batch size and query overlap grow. We further extend this framework to a speculative-query setting, in which likely follow-up queries are predicted from query history and pre-executed when doing so is cost-justified. Building on these analyses, we develop cost models for cache-aware scheduling and implement a prototype execution system that demonstrates reduced prefill computation and per-query cost and latency, with implications for domains where large document corpora are queried repeatedly, such as legal contract review, scientific literature analysis, and enterprise data processing.

I’m participating in SuperUROP to build on the research I started during my UROP. Continuing this work over a full year will give me the space to explore more open questions that I didn’t have time to fully pursue during the UROP, and to grow in the area of evolving database research. I’m looking forward to taking a project through a more complete research arc, from formalizing the problem to building and evaluating a working system.