Query Transformation in Production RAG: Architecture, Latency Economics, and Retrieval Trade-Offs for HyDE, Multi-Query Expansion, and Step-Back Prompting
Standard retrieval-augmented generation (RAG) architectures operate on a naive assumption: that the raw user query is suitable for direct retrieval against a vector database or lexical search index. In production, this assumption fails across significant query distributions. Raw user queries are frequently short (averaging 4 to 8 words), structurally underspecified, conversational, or laden with unresolved pronoun bindings. Conversely, indexed document chunks typically contain 256 to 1024 tokens
1 min