Building RAG Applications as a Frontend Developer

Introduction#

Large language models are impressive by default, but they have a structural problem that becomes obvious the moment you try to build something real with them. Their knowledge is frozen at training time. They have no access to your company’s internal documentation, your product database, last month’s support tickets, or anything else that makes your application useful to the specific users it serves. When they do not know something, they do not say so clearly. They guess, and the guess often sounds convincing.

Retrieval Augmented Generation (RAG) is the most widely adopted pattern for solving this problem. Instead of relying solely on what the model learned during training, a RAG system retrieves relevant documents from a knowledge base at query time and provides them to the model as context. The model answers based on what it retrieved, not what it guessed. This dramatically reduces hallucinations, enables up to date answers, and makes it possible to build AI features over private data without any model fine tuning.

For frontend developers, RAG is no longer an infrastructure concern to hand off to a backend team. As AI features move closer to the product layer, understanding how to integrate, stream, and present RAG results in a React or Next.js application has become core work. This guide walks through the full picture: how RAG works, how to implement it in a Next.js API route, how to build a streaming React interface around it, and what to measure in production.

How RAG Works#

At its core, RAG consists of two distinct phases that happen in sequence for every user query.

The first phase is ingestion. Documents from your knowledge source, whether those are product docs, support articles, internal wikis, or PDFs, are processed, split into chunks, converted into vector embeddings (numerical representations of semantic meaning), and stored in a vector database. This happens once, or on a schedule when the source data changes, not at query time.

The second phase is retrieval and generation, which happens at query time. The user’s question is converted into an embedding using the same model used during ingestion. The vector database is searched for chunks whose embeddings are most semantically similar to the query. Those chunks are assembled into a context block and sent to the LLM alongside the original question. The model generates its response based on that retrieved context rather than guessing from memory.

The simplest possible version looks like this:

User query: "What is your refund policy?"
  ↓
Embed query → [0.23, -0.87, 0.14, ...]
  ↓
Search vector DB → Top 3 matching document chunks
  ↓
Build prompt:
  "Answer the question using only the provided context.
   Context: [retrieved chunks]
   Question: What is your refund policy?"
  ↓
LLM generates answer grounded in retrieved docs

This straightforward pipeline works well for simple, single document queries. As query complexity grows, the retrieval and assembly steps need to be more sophisticated. But even the basic version produces meaningfully better results than a model answering without any retrieved context.

Setting Up the Ingestion Pipeline#

Before building the query time interface, the ingestion pipeline needs to be in place. For a Next.js application, a simple ingestion script handles this:

// scripts/ingest-docs.ts
import { OpenAIEmbeddings } from "@langchain/openai";
import { SupabaseVectorStore } from "@langchain/community/vectorstores/supabase";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { createClient } from "@supabase/supabase-js";
import fs from "fs";

const supabase = createClient(
  process.env.SUPABASE_URL!,
  process.env.SUPABASE_SERVICE_KEY!
);

async function ingestDocuments() {
  // Load source documents
  const rawText = fs.readFileSync("./docs/knowledge-base.txt", "utf-8");

  // Split into overlapping chunks
  const splitter = new RecursiveCharacterTextSplitter({
    chunkSize: 500,
    chunkOverlap: 75,
  });
  const chunks = await splitter.createDocuments([rawText]);

  // Embed and store in Supabase vector store
  await SupabaseVectorStore.fromDocuments(
    chunks,
    new OpenAIEmbeddings({ model: "text-embedding-3-small" }),
    {
      client: supabase,
      tableName: "documents",
      queryName: "match_documents",
    }
  );

  console.log(`Ingested ${chunks.length} chunks`);
}

ingestDocuments();

Chunk size is one of the most consequential decisions in RAG implementation. Chunks that are too small lack enough context to be useful when retrieved. Chunks that are too large dilute the specific information the model needs and consume more of the context window. For most documentation use cases, chunks in the 400 to 600 token range with 50 to 100 token overlap work well as a starting point. Overlap ensures that content spanning a chunk boundary is not lost.

Building the Query API Route#

With documents ingested, the Next.js API route handles retrieval and generation at query time:

// app/api/chat/route.ts
import { OpenAIEmbeddings } from "@langchain/openai";
import { SupabaseVectorStore } from "@langchain/community/vectorstores/supabase";
import { streamText } from "ai";
import { openai } from "@ai-sdk/openai";
import { createClient } from "@supabase/supabase-js";

const supabase = createClient(
  process.env.SUPABASE_URL!,
  process.env.SUPABASE_SERVICE_KEY!
);

export async function POST(req: Request) {
  const { messages } = await req.json();
  const userQuery = messages[messages.length - 1].content;

  // Retrieve relevant chunks from the vector store
  const vectorStore = new SupabaseVectorStore(
    new OpenAIEmbeddings({ model: "text-embedding-3-small" }),
    { client: supabase, tableName: "documents", queryName: "match_documents" }
  );

  const relevantDocs = await vectorStore.similaritySearch(userQuery, 4);
  const context = relevantDocs.map((doc) => doc.pageContent).join("\n\n");

  // Stream the response with retrieved context injected
  const result = streamText({
    model: openai("gpt-4o-mini"),
    system: `You are a helpful assistant. Answer questions using only the provided context.
If the context does not contain enough information to answer, say so clearly.

Context:
${context}`,
    messages,
  });

  return result.toDataStreamResponse();
}

The key instruction in the system prompt is “use only the provided context.” Without this constraint, the model falls back on its training data when the retrieved content is thin, reintroducing the hallucination risk that RAG is meant to solve.

Building the Streaming React Interface#

The frontend component consumes the streaming response from the API route. Using the Vercel AI SDK’s useChat hook, this is minimal:

"use client";

import { useChat } from "ai/react";
import { useRef, useEffect } from "react";

export default function RagChatInterface() {
  const { messages, input, handleInputChange, handleSubmit, isLoading } = useChat({
    api: "/api/chat",
  });

  const bottomRef = useRef<HTMLDivElement>(null);

  useEffect(() => {
    bottomRef.current?.scrollIntoView({ behavior: "smooth" });
  }, [messages]);

  return (
    <div className="chat-container">
      <div className="messages">
        {messages.map((message) => (
          <div key={message.id} className={`message message--${message.role}`}>
            <p>{message.content}</p>
          </div>
        ))}
        {isLoading && (
          <div className="message message--assistant">
            <span className="typing-indicator">Searching knowledge base...</span>
          </div>
        )}
        <div ref={bottomRef} />
      </div>

      <form onSubmit={handleSubmit} className="input-form">
        <input
          value={input}
          onChange={handleInputChange}
          placeholder="Ask a question..."
          disabled={isLoading}
        />
        <button type="submit" disabled={isLoading || !input.trim()}>
          Send
        </button>
      </form>
    </div>
  );
}

The loading state message “Searching knowledge base…” is a deliberate UX decision. It sets accurate expectations: the system is retrieving information before answering, not simply generating. Users who understand this are more likely to trust accurate answers and interpret uncertainty statements correctly.

Showing Sources in the UI#

One of RAG’s most valuable properties for end users is the ability to show which documents supported each answer. This builds trust and allows users to verify claims. To surface sources in the UI, pass them through the response stream as structured data:

// In the API route, after retrieval
const sources = relevantDocs.map((doc) => ({
  content: doc.pageContent.slice(0, 150) + "...",
  source: doc.metadata.source,
  page: doc.metadata.page,
}));

const result = streamText({
  model: openai("gpt-4o-mini"),
  system: `...`,
  messages,
  onFinish: async (completion) => {
    // Sources can be appended to the response or stored against the message ID
  },
});

// Pass sources as a custom header or alongside the stream
return new Response(result.textStream, {
  headers: {
    "Content-Type": "text/plain",
    "X-Sources": JSON.stringify(sources),
  },
});

In the frontend component, a sources section below each assistant message shows where the information came from, with links to the original documents where applicable. This pattern is increasingly expected in enterprise AI interfaces, where audit trails and source verification are not optional.

Improving Retrieval Quality#

The single most important factor in RAG output quality is retrieval quality. A well written answer cannot compensate for irrelevant retrieved documents. The most effective improvements target the retrieval step directly.

Hybrid Search#

Pure semantic (vector) search matches meaning but misses exact phrases. Pure keyword search (BM25) matches terms but misses synonyms and paraphrases. Hybrid search combines both, using a ranking step to merge the results. Most production RAG systems use hybrid search because it handles both “what is the concept of X” (semantic) and “what does configuration option Y do” (keyword) queries well.

// Using Supabase with both vector and full-text search
const { data: vectorResults } = await supabase.rpc("match_documents", {
  query_embedding: queryEmbedding,
  match_count: 10,
});

const { data: keywordResults } = await supabase
  .from("documents")
  .select("*")
  .textSearch("content", userQuery)
  .limit(10);

// Combine and rerank using Reciprocal Rank Fusion
const mergedResults = reciprocalRankFusion([vectorResults, keywordResults]);
const topResults = mergedResults.slice(0, 4);

Query Expansion#

Short or ambiguous user queries often retrieve suboptimal results because the vocabulary in the query does not match the vocabulary in the documents. Query expansion generates alternative phrasings before retrieval:

const expansionResponse = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [
    {
      role: "user",
      content: `Generate 3 alternative phrasings of this question for document retrieval.
Return only the questions, one per line, no numbering.
Original: "${userQuery}"`,
    },
  ],
  max_tokens: 150,
});

const expandedQueries = expansionResponse.choices[0].message.content
  ?.split("\n")
  .filter(Boolean) ?? [];

// Retrieve for all queries and deduplicate
const allResults = await Promise.all(
  [userQuery, ...expandedQueries].map((q) => vectorStore.similaritySearch(q, 3))
);
const deduplicated = deduplicateByContent(allResults.flat());

What to Measure in Production#

RAG systems can silently degrade when the underlying knowledge base changes, when user query patterns shift, or when the embedding model is updated. Without evaluation, you will not know until users complain.

The four metrics worth tracking, as defined by the RAGAS evaluation framework, are as follows. Faithfulness measures whether the generated answer is grounded in the retrieved documents or introduces information the documents do not support. Answer Relevancy measures whether the answer actually addresses the user’s question. Context Precision measures how many of the retrieved chunks were genuinely relevant to the query. Context Recall measures whether all information needed to answer the question was captured in retrieval.

A simple starting point is a golden evaluation set: a collection of representative questions with known correct answers. Running the RAG pipeline against this set on a schedule, measuring the above metrics, and alerting when any metric drops below a defined threshold catches regressions before they affect production users.

Choosing a Vector Database#

For teams already using Supabase or Postgres, pgvector is the lowest friction starting point. It adds vector search as a Postgres extension, keeping the infrastructure simple and the data in a familiar place.

For applications requiring sub 100ms retrieval at large scale (millions of vectors), dedicated vector databases like Pinecone, Weaviate, or Qdrant are worth the added infrastructure. They are purpose built for high throughput similarity search, offer metadata filtering, and handle the operational concerns of large scale vector storage without manual index tuning.

For most frontend applications adding RAG to an existing Next.js app, pgvector with Supabase is the practical choice: fewer services to manage, no new vendor relationships, and retrieval performance that handles tens of thousands of documents without issue.

References#

DEV Community: RAG in 2026 A Practical Blueprint for Retrieval-Augmented Generation

Lushbinary: RAG Production Guide with Hybrid Search, Chunking, and RAGAS Evaluation

Starmorph: RAG Techniques Compared Including Agentic RAG and Adaptive Routing

Databricks: End-to-End RAG Workflow from Ingestion to Deployment

Vercel AI SDK Docs: Retrieval Augmented Generation

Supabase Docs: RAG with pgvector and Row Level Security