Connect Chrome Extension to Self-Hosted AI

What You’ll Connect

After this guide, your Chrome extension will call your own GPU-hosted LLM to provide AI assistance on any webpage — summarising articles, answering questions about page content, rewriting selected text, or translating passages. The extension’s service worker connects to your vLLM or Ollama endpoint on dedicated GPU hardware, and a sidebar panel displays streaming responses alongside the page being viewed.

The integration uses Chrome’s Manifest V3 architecture with a service worker that calls your OpenAI-compatible API. A content script extracts page context (selected text, article content) and passes it to the service worker, which sends it to your GPU server and streams the response back to the sidebar UI.

Prerequisites

• A GigaGPU server running a self-hosted LLM (setup guide)
• HTTPS access to your inference endpoint
• Chrome browser with developer mode enabled
• API key for your GPU inference server

Integration Steps

Create a Manifest V3 extension with a service worker, content script, and side panel. The service worker handles all API communication with your GPU server — Chrome’s Manifest V3 requires network requests to originate from the service worker or from pages with proper permissions. Add your GPU endpoint domain to the host_permissions in the manifest.

Build a content script that runs on every page and listens for user actions — text selection, right-click context menu, or keyboard shortcut. When triggered, the content script extracts the relevant page content (selected text, full article via Readability, or page metadata) and sends it to the service worker via chrome.runtime.sendMessage.

The service worker receives the page context, constructs a prompt, and calls your GPU endpoint with streaming enabled. It forwards token chunks back to the side panel via chrome.runtime.sendMessage. The side panel script accumulates tokens and renders the progressive response.

Code Example

Chrome extension components for connecting to your self-hosted LLM:

// manifest.json
{
  "manifest_version": 3,
  "name": "AI Assistant",
  "version": "1.0",
  "permissions": ["sidePanel", "contextMenus", "activeTab"],
  "host_permissions": ["https://your-gpu-server.gigagpu.com/*"],
  "background": { "service_worker": "background.js" },
  "content_scripts": [{
    "matches": ["<all_urls>"],
    "js": ["content.js"]
  }],
  "side_panel": { "default_path": "sidepanel.html" }
}

// background.js — Service worker
const API_URL = "https://your-gpu-server.gigagpu.com/v1/chat/completions";
const API_KEY = "your-api-key";

chrome.contextMenus.create({
  id: "ai-assist", title: "Ask AI about this",
  contexts: ["selection"]
});

chrome.contextMenus.onClicked.addListener(async (info, tab) => {
  chrome.sidePanel.open({ tabId: tab.id });

  const response = await fetch(API_URL, {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "Authorization": `Bearer ${API_KEY}`
    },
    body: JSON.stringify({
      model: "meta-llama/Llama-3-70b-chat-hf",
      messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: `Explain this: ${info.selectionText}` }
      ],
      stream: true, max_tokens: 1024
    })
  });

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    const chunk = decoder.decode(value);
    for (const line of chunk.split("\n")) {
      if (!line.startsWith("data: ") || line === "data: [DONE]") continue;
      const json = JSON.parse(line.slice(6));
      const token = json.choices[0]?.delta?.content || "";
      chrome.runtime.sendMessage({ type: "token", content: token });
    }
  }
  chrome.runtime.sendMessage({ type: "done" });
});

Testing Your Integration

Load the extension in Chrome via chrome://extensions with developer mode. Navigate to any webpage, select text, right-click and choose “Ask AI about this”. The side panel should open and display streaming tokens from your GPU server. Check the service worker console (via the extension’s “Inspect views” link) for any network errors.

Test with different content types: short selections, full articles, pages with complex formatting. Verify the content script correctly extracts text without HTML artifacts. Test the streaming on slow network connections to ensure tokens render smoothly without buffering delays.

Production Tips

Store the API key in Chrome’s chrome.storage.sync with an options page where users enter their own key, rather than hardcoding it. Add keyboard shortcuts (Ctrl+Shift+A) via the commands API for quick access. Implement conversation history in the side panel so users can ask follow-up questions about the same page content.

For distribution, package the extension and submit to the Chrome Web Store. Keep the GPU endpoint configurable so users can point to their own GigaGPU server or any OpenAI-compatible endpoint. Build a complete AI chatbot browser experience. Explore more tutorials or get started with GigaGPU to power your Chrome extension.