Its Retrieval Augmented Generation. Basically a way to add context in a prompt sent to LLM, for example - adding info on how to escalate claims in a customer service agent on a query.
The step I don't trust is 'check if the evidence is sufficient'. Who decides that? The same model that wants to give an answer. My guess is most teams would get more from fixing retrieval quality first than from adding a loop on top of bad retrieval.
Agentic RAG is the loop that can switch tools when the first fetch is not enough, but the piece never says what sufficient is stored as. A second search can pass a looser check and the final answer still looks like one retrieval. I would keep each tool choice, the query, and the check that let the loop stop, and fail the answer when a later reader cannot tell which document the last hop used. One-shot RAG at least has a single retrieved context; the agentic path needs that context plus the hops that were rejected.
Good breakdown of when the extra loop is worth it. I would be curious how people measure the cost side, for example average retrieval calls per question, before deciding a use case really needs the agentic version.
Questions the knowledge base can't answer are where the loop hurts. The sufficiency check never passes, so the agent keeps rewriting the query until it hits whatever cap you set, and with no cap you pay for ten searches to get a guess. Log why each run stopped, enough evidence or out of budget, and answer the second case with "not found".
How do you decide when the agent should stop looping? The sufficiency check is the part I keep coming back to — every extra retrieval round adds latency and tokens, and if the corpus simply doesn't contain the answer, more loops won't create it. In practice I'd want a hard budget per query, a maximum number of rounds and a token ceiling, with a fallback that returns the best-effort answer and says what it couldn't verify, rather than chasing completeness. That seems more useful than treating agentic RAG as the default for every question.
The step that decides whether agentic RAG pays off is the check for whether the evidence is sufficient. If the same model that writes the answer also judges sufficiency, it tends to stop early on confident but wrong context, or keep looping on questions the corpus can't answer. A hard cap on retrieval rounds plus a separate check that every claim in the draft maps to a retrieved passage seem to help a lot. Have you seen teams send simple lookups through plain RAG and only escalate to the agent loop when the first retrieval scores low? That would keep latency and cost down for the common case.
What's a "RAG"? It'd be helpful if you would spell out your acronyms the first time you use them. thanks!
Its Retrieval Augmented Generation. Basically a way to add context in a prompt sent to LLM, for example - adding info on how to escalate claims in a customer service agent on a query.
The step I don't trust is 'check if the evidence is sufficient'. Who decides that? The same model that wants to give an answer. My guess is most teams would get more from fixing retrieval quality first than from adding a loop on top of bad retrieval.
Agentic RAG is the loop that can switch tools when the first fetch is not enough, but the piece never says what sufficient is stored as. A second search can pass a looser check and the final answer still looks like one retrieval. I would keep each tool choice, the query, and the check that let the loop stop, and fail the answer when a later reader cannot tell which document the last hop used. One-shot RAG at least has a single retrieved context; the agentic path needs that context plus the hops that were rejected.
Good breakdown of when the extra loop is worth it. I would be curious how people measure the cost side, for example average retrieval calls per question, before deciding a use case really needs the agentic version.
Questions the knowledge base can't answer are where the loop hurts. The sufficiency check never passes, so the agent keeps rewriting the query until it hits whatever cap you set, and with no cap you pay for ten searches to get a guess. Log why each run stopped, enough evidence or out of budget, and answer the second case with "not found".
How do you decide when the agent should stop looping? The sufficiency check is the part I keep coming back to — every extra retrieval round adds latency and tokens, and if the corpus simply doesn't contain the answer, more loops won't create it. In practice I'd want a hard budget per query, a maximum number of rounds and a token ceiling, with a fallback that returns the best-effort answer and says what it couldn't verify, rather than chasing completeness. That seems more useful than treating agentic RAG as the default for every question.
The step that decides whether agentic RAG pays off is the check for whether the evidence is sufficient. If the same model that writes the answer also judges sufficiency, it tends to stop early on confident but wrong context, or keep looping on questions the corpus can't answer. A hard cap on retrieval rounds plus a separate check that every claim in the draft maps to a retrieved passage seem to help a lot. Have you seen teams send simple lookups through plain RAG and only escalate to the agent loop when the first retrieval scores low? That would keep latency and cost down for the common case.