Connect VS Code to Self-Hosted Code Model on GPU

What You’ll Connect

After this guide, your VS Code editor will have AI code completion and chat powered by your own GPU server — no API costs, no rate limits. The Continue extension connects to your vLLM or Ollama endpoint on dedicated GPU hardware, delivering a Copilot-style experience with models you choose and control.

The integration gives every developer on your team inline autocomplete suggestions, AI chat for code explanation, and refactoring assistance — all routed through a private AI coding assistant endpoint. Source code never leaves your infrastructure.

Continue Extension –> HTTP Request –> GPU Server (vLLM) (typing code, reads editor context /v1/chat/ or Code model inference asking questions) builds prompt /v1/completions on dedicated GPU | | Inline suggestion <-- Extension renders <-- JSON response <-- Completion or chat response completion in editor from GPU server returned -->

Prerequisites

• A GigaGPU server with a code model (DeepSeek Coder, CodeLlama, or StarCoder) on an OpenAI-compatible API (vLLM guide)
• VS Code 1.80+ installed on your development machine
• The Continue extension installed from the VS Code Marketplace
• HTTPS or SSH tunnel access to your GPU server (Nginx proxy guide)
• Network connectivity between your workstation and the GPU endpoint

Integration Steps

Install the Continue extension from the VS Code Marketplace. Once installed, it creates a configuration file at ~/.continue/config.json (or opens a setup wizard). You will point this configuration at your self-hosted GPU endpoint.

Edit the Continue config to add your GPU server as a model provider. Set the provider type to openai (which works with any OpenAI-compatible API), configure the base URL to point at your GPU server, and specify the model name matching what your vLLM deployment serves.

For autocomplete (tab completion), configure a separate fast model entry — a smaller model like DeepSeek Coder 1.3B responds faster for inline suggestions while a larger model handles chat and refactoring tasks. This dual-model setup balances latency and quality.

Code Example

Edit ~/.continue/config.json to connect to your GPU inference server:

{
  "models": [
    {
      "title": "DeepSeek Coder V2 (Chat)",
      "provider": "openai",
      "model": "deepseek-ai/DeepSeek-Coder-V2-Instruct",
      "apiBase": "https://your-gpu-server.gigagpu.com/v1",
      "apiKey": "sk-your-gpu-api-key",
      "contextLength": 16384
    }
  ],
  "tabAutocompleteModel": {
    "title": "DeepSeek Coder 1.3B (Autocomplete)",
    "provider": "openai",
    "model": "deepseek-ai/deepseek-coder-1.3b-base",
    "apiBase": "https://your-gpu-server.gigagpu.com/v1",
    "apiKey": "sk-your-gpu-api-key"
  },
  "allowAnonymousTelemetry": false
}

Testing Your Integration

Open a source file in VS Code and start typing a function. After a brief pause, Continue should display an inline autocomplete suggestion from your GPU server. Press Tab to accept. Open the Continue chat panel (Ctrl+Shift+L) and ask a question about the current file — the response should come from your self-hosted model.

Test with different file types (Python, TypeScript, Rust) to verify the model handles diverse languages. Monitor the Continue output panel in VS Code for any connection errors. Check your GPU server’s request logs to confirm traffic originates from your workstation.

Production Tips

Autocomplete latency is critical for a good developer experience. Keep the autocomplete model small (1-7B parameters) so suggestions appear within 200-300ms. Use the larger chat model only for explicit requests where developers expect a longer wait.

For team rollouts, distribute the Continue config via your dotfiles repository or a shared configuration management tool. Every developer points at the same GPU endpoint, and you control which models are available. This standardises the AI coding experience across the team.

Continue also supports context providers that index your codebase, documentation, and git history. Combined with a self-hosted model on open-source code models, developers get context-aware assistance without exposing proprietary code externally. Explore more tutorials or get started with GigaGPU to build your private AI coding assistant.