Next.js + Self-Hosted LLM: Full-Stack AI

You will build a full-stack AI application using Next.js connected to a self-hosted LLM on your GPU server. By the end, you will have a production-ready web app with streaming chat, server-side rendering, and API routes that talk directly to your own inference backend — no third-party AI APIs required.

Architecture

Next.js handles both the frontend and the API layer. The LLM runs on the same server via vLLM, and Next.js API routes proxy requests to it. This eliminates network latency between your application and the model.

Project Setup

npx create-next-app@latest ai-app --typescript --tailwind --app
cd ai-app
npm install openai

API Route with Streaming

Create a Route Handler that streams responses from the self-hosted model to the browser.

// app/api/chat/route.ts
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8000/v1",
  apiKey: "not-needed",
});

export async function POST(req: Request) {
  const { messages } = await req.json();

  const stream = await client.chat.completions.create({
    model: "meta-llama/Llama-3.1-8B-Instruct",
    messages,
    stream: true,
    max_tokens: 1024,
  });

  const encoder = new TextEncoder();
  const readable = new ReadableStream({
    async start(controller) {
      for await (const chunk of stream) {
        const content = chunk.choices[0]?.delta?.content;
        if (content) {
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({ content })}\n\n`));
        }
      }
      controller.enqueue(encoder.encode("data: [DONE]\n\n"));
      controller.close();
    },
  });

  return new Response(readable, {
    headers: { "Content-Type": "text/event-stream", "Cache-Control": "no-cache" },
  });
}

For the OpenAI SDK configuration details, see the Node.js SDK guide. For the API compatibility layer, check the vLLM compatibility guide.

Chat UI Component

Build a chat interface that consumes the streaming API route.

// app/components/Chat.tsx
"use client";
import { useState, useRef, useEffect } from "react";

interface Message {
  role: "user" | "assistant";
  content: string;
}

export default function Chat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState("");
  const [loading, setLoading] = useState(false);

  async function sendMessage() {
    if (!input.trim() || loading) return;
    const userMsg: Message = { role: "user", content: input };
    const updated = [...messages, userMsg];
    setMessages(updated);
    setInput("");
    setLoading(true);

    const res = await fetch("/api/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ messages: updated }),
    });

    const reader = res.body?.getReader();
    const decoder = new TextDecoder();
    let assistantContent = "";

    setMessages([...updated, { role: "assistant", content: "" }]);

    while (reader) {
      const { done, value } = await reader.read();
      if (done) break;
      const text = decoder.decode(value);
      for (const line of text.split("\n")) {
        if (line.startsWith("data: ") && line !== "data: [DONE]") {
          const parsed = JSON.parse(line.slice(6));
          assistantContent += parsed.content;
          setMessages([...updated, { role: "assistant", content: assistantContent }]);
        }
      }
    }
    setLoading(false);
  }

  return (
    <div className="max-w-2xl mx-auto p-4">
      {messages.map((m, i) => (
        <div key={i} className={m.role === "user" ? "text-right" : "text-left"}>
          <p className="inline-block p-3 rounded-lg bg-gray-100">{m.content}</p>
        </div>
      ))}
      <div className="flex gap-2 mt-4">
        <input value={input} onChange={(e) => setInput(e.target.value)}
          onKeyDown={(e) => e.key === "Enter" && sendMessage()}
          className="flex-1 p-2 border rounded" placeholder="Type a message..." />
        <button onClick={sendMessage} disabled={loading}
          className="px-4 py-2 bg-blue-600 text-white rounded">Send</button>
      </div>
    </div>
  );
}

For a dedicated React chat component, see the React chat UI guide.

Server Components for AI

Use React Server Components for AI-powered content that renders on the server, reducing client JavaScript.

// app/summary/page.tsx (Server Component)
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "http://localhost:8000/v1",
  apiKey: "not-needed",
});

export default async function SummaryPage() {
  const response = await client.chat.completions.create({
    model: "meta-llama/Llama-3.1-8B-Instruct",
    messages: [{ role: "user", content: "Summarise the benefits of self-hosted AI." }],
    max_tokens: 256,
  });

  return (
    <article className="max-w-2xl mx-auto p-8">
      <h1>AI-Generated Summary</h1>
      <p>{response.choices[0].message.content}</p>
    </article>
  );
}

Deployment on GPU Server

# Build and start
npm run build
npm start -- -p 3000

# Or with PM2 for process management
npm install -g pm2
pm2 start npm --name "ai-app" -- start -- -p 3000
pm2 save

Run vLLM and Next.js on the same server. vLLM uses the GPU for inference while Next.js uses CPU for rendering — the workloads do not compete. Place both behind Nginx for TLS termination.

For backend alternatives, compare vLLM vs Ollama. Add GPU monitoring to track inference performance alongside your application metrics. The self-hosting guide covers server planning, and our tutorials section has more full-stack patterns.