Technical case study

Research Paper Assistant

A research system for asking complex questions across a 500-paper corpus. It routes each question to the right combination of text, visual, vector, entity, and community evidence before producing a cited answer.

500: papers
27,381: evidence records
24,876: indexed points
9,970: entity vectors
406: communities
1,000: tested SSE clients

Next.js · FastAPI · LangGraph · PostgreSQL · PgBouncer · Qdrant · Neo4j · Valkey · Docker Compose

Overview

The problem is query planning, not only vector search

A figure request, a direct fact, a comparison across papers, and a question about a research theme should not run through the same retrieval path.

The project began as a multimodal retrieval system and grew into a complete research application. Its central responsibility is to decide what a question needs before it spends retrieval or model budget.

A stable evidence identifier connects extraction, vector search, graph construction, citation validation, and the web interface. That makes the final answer inspectable: a citation can be traced back to its paper, section, pages, modality, caption, and media payload.

Plan before retrieval: Classify scope, modality, reasoning type, conversational context, and whether the request can proceed.
Retrieve complementary evidence: Combine semantic similarity, lexical precision, entity matches, and Leiden community context.
Bound expensive work: Limit active workflows, retrieval expansion, repair, provider calls, and shared cache-miss leadership.
Preserve evidence identity: Carry canonical source metadata from the corpus to the cited answer and its rendered media.

Architecture

Separate connection handling from research execution

The web and API layers accept requests and stream state. PostgreSQL owns durable coordination. A bounded worker performs retrieval and model-backed research.

System architecture flowing from the Next.js research interface through Nginx, FastAPI edges, PgBouncer, PostgreSQL, and the research worker to Valkey, Qdrant, Neo4j, and the model provider — Runtime architectureAPI edges remain model-free. Expensive retrieval and generation are isolated in the worker, while shared state lives in PostgreSQL and Valkey.Open full size

Next.js application: Chat, history, source inspection, figure rendering, citations, and progress presentation.
FastAPI edges: Request validation, admission control, job creation, status reads, and server-sent event delivery.
PostgreSQL: Jobs, events, chat history, checkpoints, idempotency, and worker ownership.
Research worker: Query planning, hybrid retrieval, evidence assessment, synthesis, validation, and bounded repair.
Qdrant and Neo4j: Vector similarity for evidence and entities; graph topology for relationships and community context.
Valkey: Cross-process exact cache, token-owned leases, bounded followers, and shared quota counters.

Corpus pipeline

One canonical evidence model feeds every index

The corpus is built offline and promoted as a versioned release. Text, tables, figures, formulas, entities, and communities keep stable links to their source papers.

Pipeline from 500 research papers through parsing, canonical evidence, parallel vector graph and community builders, manifest validation, and runtime release — Corpus build and promotionA release reaches the runtime only after the manifest confirms its expected IDs, counts, and artifact versions.Open full size

PDF extraction produces canonical records with paper ID, section path, page range, modality, text or caption, and media availability. The downstream builders never invent a second evidence identity; they attach embeddings and graph relationships to those records.

The promoted corpus contains 27,381 canonical evidence records, 24,876 indexed evidence points, 9,970 entity vectors, 12,698 community memberships, 391 hierarchy links, and 94 generated community reports.

Hybrid retrieval

Run complementary searches in parallel, then normalize them

Dense, sparse, entity, and community retrieval solve different recall problems. Their results are merged into one evidence contract before reranking.

A planned query branches vertically into dense, sparse, entity, and community search before merging, deduplicating, reranking, and producing an evidence bundle — Hybrid retrieval flowQdrant serves evidence similarity while Neo4j supplies graph context for entity and community discovery.Open full size

Dense search: Finds semantically similar evidence when the wording differs from the question.
Sparse search: Preserves exact terminology, paper names, section labels, equations, and distinctive phrases.
Entity search: Finds evidence connected to extracted methods, datasets, metrics, and research concepts.
Community search: Finds broader topic context assembled from related entities and papers.
Rerank and diversify: Removes duplicate evidence, balances papers and modalities, and keeps the final context within budget.

Six logical evidence sources are served by three Qdrant collections: evidence, entity_nodes, and community_nodes. Neo4j stores the relationships needed for graph traversal.

Leiden communities

Turn a paper graph into focused topics and broader themes

Vector search finds nearby passages. Leiden exposes clusters of related research concepts and lets broad questions move from a theme back to specific supporting evidence.

Research papers produce entities and relations, a weighted graph, Leiden local moving, refinement, aggregation, stable communities, two hierarchy generations, and community reports — How Leiden communities are generatedLocal moving, refinement, and aggregation produce stable groups. Narrow topics can then be grouped into broader second-generation themes.Open full size

Suppose Paper A studies retrieval noise, Paper B studies evidence filtering, and Paper C studies factuality. Shared entities create weighted graph edges even when the papers do not use the same sentences. Leiden groups densely connected nodes, refines weak groups, and aggregates stable groups into the next level.

The runtime retrieves a community summary for orientation, follows its memberships to relevant entities and papers, and finally cites canonical evidence. Community summaries help discovery; they do not replace source evidence.

Query router

Five predictions run in parallel before a plan is built

The router does not answer the question. It describes the request along five independent dimensions, then a deterministic planner chooses stores, filters, graph depth, and evidence budgets.

A user question is normalized and sent to a query router whose five parallel heads predict scope, evidence, reasoning, context, and status before a deterministic planner creates the retrieval plan — Router and deterministic plannerThe five heads are parallel classifications, not a chain. Their combined output becomes an executable retrieval plan.Open full size

Scope: Single paper, multiple named papers, or the full corpus.
Evidence: Text, figure, table, formula, or a mixed evidence request.
Reasoning: Direct fact, explanation, comparison, or synthesis.
Context: Standalone question or a follow-up that must resolve earlier references.
Status: Proceed, ask for clarification, reject as out of scope, or stop because a dependency is blocked.

The measured hybrid-router query-status cascade reached approximately 0.9669 macro-F1. That number describes routing classification only; it is not an end-to-end answer-accuracy claim.

Query execution

Two questions, two different retrieval plans

Concrete traces show what the router changes and which data stores the workflow actually touches.

A figure from one paper

“Show me Figure 3 from the ReFilter paper and explain the Context Encoder.”

Router interpretation

Scope: Single paper
Evidence: Figure + adjacent text
Reasoning: Explain
Context: Standalone
Status: Proceed

Stores touched

• Qdrant evidence
• media artifacts

Execution path

1Resolve ReFilter to its canonical paper ID.
2Filter evidence to that paper and prefer the requested figure record.
3Retrieve adjacent section text needed to explain the component.
4Verify that the evidence ID has a loadable media payload.
5Return the figure, explanation, pages, caption, and source reference together.

Result

A narrow multimodal answer. No graph search or corpus-wide synthesis is needed.

A comparison across papers

“How do ReFilter and other gated RAG methods reduce irrelevant context?”

Router interpretation

Scope: Multiple papers
Evidence: Text + entity + community
Reasoning: Compare
Context: Standalone
Status: Proceed

Stores touched

• Qdrant evidence
• entity_nodes
• community_nodes
• Neo4j

Execution path

1Resolve the named paper and extract comparison concepts.
2Run evidence, entity, and community retrieval in parallel.
3Expand only the relevant retrieval-robustness neighborhood in Neo4j.
4Rerank with cross-paper diversity so one paper cannot dominate the context.
5Synthesize shared mechanisms and differences with paper-specific citations.

Result

A cross-paper comparison whose claims remain tied to distinct source evidence.

Research workflow

Every loop and terminal state is explicit

The LangGraph workflow behaves like a bounded state machine. Evidence can expand once, citation repair can run once, and unsuitable requests exit before expensive generation.

Four-lane LangGraph workflow covering request preparation, planning, an evidence sufficiency loop, answer synthesis, citation validation, one repair loop, and explicit blocked out-of-scope and clarify exits — Bounded LangGraph state machineThe workflow separates request preparation, planning, evidence acquisition, and answer quality. Decision diamonds show exactly where the only retrieval and repair loops occur.Open full size

Request preparation: Normalize the request, verify dependencies, and resolve follow-up references from bounded conversation history.
Planning: Run the router and convert five predictions into stores, filters, candidate budgets, and graph depth.
Evidence loop: Retrieve and score evidence. If coverage is weak, expand the plan once and retrieve again; otherwise continue.
Answer quality: Synthesize only from the evidence bundle, validate citations and media, repair once if needed, then finalize.

Normal

45 seconds · Focused requests

Research

180 seconds · Cross-paper synthesis

Deep

600 seconds · Wider bounded exploration

Evidence contract

The answer and its sources travel together

Generated prose is not the source of truth. The response carries structured evidence records that the interface can render and the validator can check.

Canonical identity: Evidence ID, paper ID, modality, section path, and page range remain stable through retrieval and generation.
Citation validation: Every citation must resolve to evidence included in the current answer bundle.
Visual evidence: A figure requires both source metadata and a loadable media payload; a caption alone does not imply that the image exists.
Complete display text: Search snippets may be shortened, but cited source cards render canonical content instead of a truncated index preview.

This distinction fixed the earlier partial-line problem. Search snippets were being displayed as complete passages, so lines ended mid-sentence. The display path now uses the canonical evidence payload while retrieval continues to use compact index text.

Valkey and scaling

Coordinate duplicate work, then scale the connection layer

Valkey prevents identical requests from starting duplicate research. API replicas scale status reads and event streams, while admission control keeps expensive generation bounded.

Identical incoming requests form a semantic cache key, pass through L1, SQLite, and Valkey, then use one token lease so one leader runs retrieval and generation while followers wait for the fenced cached answer — Safe cache reuse and request coalescingAn exact key includes resolved research context. On a shared miss, one token-owned leader computes the answer and followers reuse only its fenced publication.Open full size

The exact cache identity includes the normalized request, answer mode, corpus version, model, cache epoch, resolved paper scope, bounded context fingerprint, and response schema. Similar wording is not enough when those values differ.

In three separate 100-client bursts, coordinated mode produced one leader and one expensive generation per burst. The measured p95 latencies were 1.917, 1.706, and 1.885 seconds.

One thousand server-sent event clients connect through Nginx to four parallel API edges, PgBouncer, PostgreSQL jobs and events, and one bounded research worker with admission control — Horizontal connection scalingAPI edges distribute client connections and cursor replay; the research-worker count still controls expensive throughput.Open full size

Admission policy: Four active expensive requests, 32 FIFO waiters, and a maximum admission wait of five seconds.
Overload response: Return 503 with Retry-After when the queue is full; return 409 for overlapping work in the same conversation.
Connection test: 500 of 500 and 1,000 of 1,000 local SSE clients connected with no rejected or disconnected clients.
Reconnect test: Ten of ten cursor reconnects succeeded at both load levels without duplicate or stale event IDs.
Claim boundary: The 1,000-client run tested connection fan-out for 15 seconds with zero research workers and no model-provider attempts. It did not simulate 1,000 simultaneous generations.