Integrating AI Chatbots into Web Apps: From Basic API to Streaming Responses in Next.js

Integrating artificial intelligence into web applications is one of the most requested features today. From automated customer support bots to smart personal assistants, chat interfaces are becoming standard. However, waiting for an LLM to generate a full response can take 10 to 30 seconds, causing a poor user experience. Streaming response generation solves this issue by displaying words as they are generated.

In this article, we will explore the differences between simple JSON responses and streaming, and learn how to build a real-time chatbot interface in Next.js using Server-Sent Events (SSE) and OpenAI SDK.

1. Standard JSON API Calls vs. Real-Time Streaming

When a user triggers an AI request, two main approaches can be used to deliver the response:

• •Standard JSON Fetch: The client sends a request and waits. The server communicates with OpenAI, waits for the entire sentence to complete, compiles it into a JSON block, and sends it back. This causes a long loading indicator and high bounce rates.
• •Streaming with Server-Sent Events (SSE): The server opens a persistent HTTP connection. As soon as the AI model generates a new token (a word or part of it), the server pushes it to the browser. The user sees text appearing immediately, reducing perceived loading time to milliseconds.

2. Backend Implementation in Next.js Route Handlers

Next.js App Router supports streaming responses natively using Web Streams. Here is how a simplified Route Handler for streaming look like:

First, we configure the OpenAI client and trigger the chat completion with the `stream: true` flag. Then, we transform the OpenAI readable stream into a standard HTTP Response stream. The browser reads the chunks and processes them on the fly.

3. Building the Chat UI and Processing Stream on Client

On the frontend, we use React state to hold the messages and manage the response stream. Standard `fetch` API is capable of reading streaming data using the `ReadableStream` reader:

• •Read the response body reader. Using a `while(true)` loop, read incoming chunks using `reader.read()`. Decode these binary chunks into text strings using `TextDecoder`.
• •Update the message state in real-time, appending the new tokens to the last message as they arrive, which causes React to re-render and dynamically display the word stream.
• •Auto-scroll to the bottom of the chat container as new content is printed so that the latest lines are always visible to the user.

4. Handling Edge Cases and Error Management

Building production-grade chat applications requires extra attention to detail:

• •Connection Interruptions: Implement reconnect logic and alert states if the HTTP stream breaks unexpectedly during generation.
• •Abort Requests: Provide a "Stop generating" button on the UI. Use `AbortController` in your Fetch requests to tell the server to cancel the LLM request and stop wasting API costs.
• •Markdown and Code Rendering: LLM answers often contain Markdown tables or formatted code blocks. Use components like `react-markdown` and syntax highlighters to render structured layouts.

Conclusion: Upgrade Your UX with Streaming AI

Real-time response streaming is a necessity for modern AI-driven web apps. By utilizing Server-Sent Events, Next.js route streaming, and responsive React chat containers, you can build interactive interfaces that feel fast, alive, and polished.

Looking to integrate OpenAI, Anthropic, or custom local LLM solutions with streaming features into your business app? Reach out using the contact form below and let's build it properly!