RAG, State, and Multi-Agent Systems

An agent does not replace RAG. It turns retrieval into one tool among several and decides when, how, and whether to use it.

As the task grows, the agent also needs clear state. Only then should you consider splitting the work across specialised agents.

Intuition

In basic RAG, retrieval is fixed:

query → retrieve once → augment prompt → generate once

Inside an agent, retrieval is a choice:

goal → decide → retrieve or call another tool → observe → decide again

The agent may retrieve policy first, call a live fare API next, and return to the policy later for verification.

Retrieval becomes a tool

flowchart TB G[Travel goal] --> D{What is needed now?} D -->|Company rule| R[retrieve_policy] D -->|Current price| F[search_flights] D -->|Room availability| H[search_hotels] D -->|Final verification| C[check_policy] R --> O[Update state] F --> O H --> O C --> O O --> D

1. Rewrite the retrieval query

The user's complete request is a poor search query:

Plan my Paris trip for next week.

The agent can produce a focused query:

international cabin class and advance-booking policy

Focused retrieval finds the right clause more reliably.

2. Retrieve more than once

The agent may retrieve:

3. Skip retrieval when it cannot help

No policy document answers:

What is today's fare?

The agent should call the flight API instead of searching the vector index.

4. Verify against the source

After drafting the itinerary, the agent can re-retrieve the relevant clauses and check the final plan line by line.

Three things the agent must remember

1. Conversation state

Lifetime: one conversation thread.

It records what the user said and what the assistant replied:

Conversation state lets follow-up language such as "that hotel" make sense.

2. Task state

Lifetime: one active run.

It is the working record:

policy retrieved       yes
selected flight        ₹34,900
budget remaining       ₹45,100
hotel search           complete
compliance check       pending

Without task state, there is no reliable loop. The question "what is still missing?" can only be answered against a record of completed work.

3. Long-term memory

Lifetime: across sessions.

It stores durable facts worth keeping after this trip:

Long-term memory should be selective, permission-aware, and updateable. Today's live fare does not belong there.

State type Lifetime Example
Conversation One thread User prefers direct flights
Task One run Compliance re-check is pending
Long-term Across sessions Home airport is BOM

When one agent becomes crowded

The request grows:

Plan the entire trip — flights, hotel, three client meetings, and the expense estimate.

One agent now has:

Tool selection often gets worse as the list grows.

Split the goal, not the model

A multi-agent design assigns clear parts of the goal:

flowchart TB O[Orchestrator
decompose, route, assemble] O --> F[Flight agent
fares and cabin rules] O --> H[Hotel agent
tariff cap and location] O --> M[Meeting agent
calendars and time zones] O --> E[Expense agent
per-diem and totals] F --> S[Shared state and checkpoints] H --> S M --> S E --> S S --> A[Human approval]

The orchestrator breaks down the goal, gives each specialist a narrow tool set, and assembles typed results.

Why specialisation can help

Why it can hurt

Every handoff is a chance to lose:

Multi-agent systems also multiply model calls, traces, and failure paths.

Safe shared state

Agents should exchange structured artifacts, not vague chat messages:

{
  "flight_total_inr": 34900,
  "departure_date": "2026-08-22",
  "policy_checks": {
    "cabin": "pass",
    "advance_booking": "pass"
  },
  "source": "travel_policy_2026.pdf §4.2"
}

Typed fields preserve amounts, dates, status, and source references better than "I found a suitable flight."

What goes wrong

One-line summary

Inside an agent, retrieval is a chosen tool, task state keeps the loop coherent, and multiple agents are justified only when clear specialised subgoals outweigh handoff cost.

Key terms