Skip to content

Workflow · Research

Build an AI Research Assistant

A research agent that takes a question, searches the web, reads sources, and produces a cited briefing.

Intermediate2–3 hoursStack: Claude or GPT-5, Tavily / Brave Search API, Node.js / Python
ResearchAgentRAGSearch
Edited by The AIKnowHub team · Editorial team

Problem

Deep research takes hours: defining the questions, finding sources, reading them, cross-referencing, and synthesizing. Off-the-shelf chatbots either don't cite or don't go deep enough.

Final output

A 1–2 page Markdown briefing with footnote-style citations, distinguishing what's known from what's contested, plus a 'Limits of this briefing' section listing what wasn't covered.

Architecture

Question
  → Planner LLM (decomposes into 4–8 sub-queries)
  → Search API (per sub-query, ~5 results each)
  → Fetch & extract main content per URL
  → Synthesis LLM (writes briefing with [n] citations)
  → Markdown output + source URLs

Step-by-step

  1. 01

    Plan the research

    Ask the LLM to decompose the question into 4–8 specific sub-queries that together answer it. Return as JSON.

  2. 02

    Search per sub-query

    Hit Tavily / Brave / Exa for each sub-query. ~5 results per query. Deduplicate by URL.

  3. 03

    Fetch and clean

    Download each URL, extract main content with Readability. Truncate each source to ~2K tokens.

  4. 04

    Synthesize the briefing

    Feed the LLM the question + all labeled sources. Require [n] citations on every claim, a 'limits' section, and contested-claim handling.

  5. 05

    Render and save

    Output Markdown with footnote citations. Save to disk and optionally open in your reader.

What you'll build

A CLI (or web app) where you type a research question and get back a 1–2 page briefing with citations — like Perplexity Deep Research, but yours.

Architecture

Question
  → Planner LLM (breaks question into sub-queries)
  → Search API (per sub-query)
  → Fetch & extract top results
  → Synthesis LLM (writes briefing with citations)
  → Report

Step 1 — Plan

Ask the LLM to decompose the question:

Question: "What's the state of the art in long-context LLM evaluation in 2026?"

Output 5 web-search sub-queries that, together, would answer this:
1. ...
2. ...

Returning JSON with 4–8 sub-queries gives the agent enough surface area without exploding the budget.

Step 2 — Search

For each sub-query, call a search API:

  • Tavily — built for LLM agents, returns clean snippets.
  • Brave Search API — generous free tier.
  • Exa — semantic search, good for finding niche papers and blogs.

Aim for ~5 results per sub-query. Deduplicate by URL.

Step 3 — Fetch and clean

For each result URL, fetch the page and extract main content (Readability). Skip pages that fail extraction or are clearly nav-only.

Truncate each source to ~2K tokens to keep total context manageable.

Step 4 — Synthesize

Feed the model: original question + all sources, each labeled [1], [2], etc.

Prompt requirements:

  • Cite every factual claim with [n].
  • Distinguish "what we know" from "what's contested" from "what's unknown."
  • If sources disagree, surface it.
  • End with a "Limits of this briefing" section listing what's not covered.

Step 5 — Output

Render as Markdown with footnote-style citations linking back to source URLs.

Extending

  • Add Deep Research mode that iterates — first pass finds gaps, second pass fills them.
  • Add a PDF/local file input for "research these sources + the web."
  • Add a monitoring agent that re-runs the same query weekly and diffs the results.

Prompt examples

Copy any of these, replace the placeholders, run.

Planner prompt

You are a research planner. Given the user's question, output 4–8 specific web-search sub-queries that, together, would let you answer it well.

Rules:
- Each sub-query is concrete and search-engine-friendly (no quotes around full sentences).
- Cover different angles: definitions, state of the art, criticisms, alternatives, examples.
- Avoid duplicates and trivial variants.

Output JSON: { "subqueries": string[] }

Question: {{question}}

Synthesis prompt

You are a research synthesizer. Write a briefing answering the question below, using ONLY the provided sources.

Rules:
- Cite every factual claim with [n] referring to the source index.
- Distinguish: what we know, what's contested, what's unknown.
- If sources disagree, surface it explicitly.
- End with "Limits of this briefing" — what wasn't covered and why.
- 600–1200 words. Tight, no fluff.

Question: {{question}}

Sources:
{{sources}}

Cost estimate

Line itemApprox. cost
Search API (Tavily/Brave, ~30 queries)$0.10
Page fetches$0.00
Planner LLM call$0.05
Synthesis LLM call (Claude Sonnet, long context)$0.40
Total per run (approx.)~$0.55

Costs depend on model choice, content length, and how aggressively you cache.

Optimizations

  • Cache fetched pages by URL — researchers re-query similar topics often.
  • Use prompt caching on the synthesis system prompt.
  • Run searches in parallel — usually 10x faster.
  • Add a reranker between fetch and synthesis to pick the best 5–10 sources from the top 30.
  • For huge research jobs, run a second iteration: the synthesis LLM proposes follow-up queries, the agent runs them, then re-synthesizes.

Common mistakes

  • No citation enforcement — the LLM happily writes uncited claims if you don't require [n] markers.
  • Skipping the planner step — going straight from question to a single search query misses angles.
  • Letting the synthesis LLM 'fill in gaps' from its own training — explicitly forbid claims not in the sources.
  • Forgetting to handle paywalled or JS-heavy pages — Readability fails silently on some sites.
  • Not capping per-source token count — one huge page dominates the context.

Frequently asked questions

Perplexity is excellent but opaque — you can't tune the planner, choose your search provider, or change the synthesis prompt. Building your own gives you control over depth, source diversity, and cost.

Related on AIKnowHub