Skip to main content

​
Autonomous Retrieval

In an Agentic RAG architecture, the LLM is not forced to retrieve context for every query. Instead, it wields the retrieve_context tool as one of many capabilities, deciding dynamically whether external information is required.

​
Decision Logic

The decision process follows a strict “Tool Use” flow, typically powered by OpenAI function calling or ReAct prompting.

Decision Flowchart

​
Tool Definitions

To enable this behavior, the retrieve_context tool must be defined with a clear description that “sells” its utility to the LLM. Good Description:
“Use this tool to fetch specific details about Aris internal policies, recent project updates, or technical documentation. Do not guess. If the user asks about a specific entity, use this tool.”
Bad Description:
“Searches the database.”

​
Fallback Strategies

Retrieval is not guaranteed to succeed. The agent must handle empty or irrelevant results gracefully.
If retrieve_context returns [] (empty list):
  1. Reflect: The agent should analyze why the search failed (e.g., “The query was too specific” or “The term is a typo”).
  2. Rewrite: The agent generates a broader search query.
  3. Retry: Execute retrieve_context again with the new query.
  4. Give Up: If the second attempt fails, admit ignorance to the user.

​
Implementation (LangGraph / LangChain)

from typing import Annotated, Literal
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, END

class AgentState(TypedDict):
    messages: Annotated[list, add_messages]

def router(state: AgentState) -> Literal["retrieve", "end"]:
    last_message = state["messages"][-1]
    if last_message.tool_calls:
        return "retrieve"
    return "end"

# Define the graph
workflow = StateGraph(AgentState)
workflow.add_node("agent", agent_node)
workflow.add_node("retrieve", tool_node)

workflow.set_entry_point("agent")
workflow.add_conditional_edges("agent", router)
workflow.add_edge("retrieve", "agent")

app = workflow.compile()