Connect Slack to Self-Hosted LLM on GPU Server

What You’ll Connect

After this guide, your Slack workspace will have an AI assistant powered by your own GPU server — no API costs, no rate limits. Every message your team sends to the bot routes directly to a large language model running on dedicated GPU hardware, giving you complete ownership of your AI infrastructure.

The integration connects Slack’s Events API to a middleware service that forwards prompts to your vLLM or Ollama inference endpoint. Responses stream back into Slack channels or direct messages in real time. Your team gets a ChatGPT-style experience inside the tool they already use — without sending proprietary data to third-party APIs.

Slack Events API –> Middleware (Node.js/Python) –> GPU Server (vLLM/Ollama) | | | User sends Bolt SDK handles LLM processes prompt @bot message event routing Returns completion | | | Bot replies <-- Slack Web API <-- Middleware formats <-- GPU returns response -->

Prerequisites

Before starting, confirm you have the following ready:

• A GigaGPU dedicated server with your LLM running behind an API endpoint (see our self-host LLM guide)
• An OpenAI-compatible API served via vLLM or similar — follow our vLLM production guide if needed
• A Slack workspace where you have permission to install apps
• Node.js 18+ or Python 3.10+ on your middleware host
• HTTPS access to your GPU server (set up with our Nginx reverse proxy guide)

Integration Steps

Start at api.slack.com/apps and create a new Slack app from scratch. Under OAuth & Permissions, add the bot token scopes: chat:write, app_mentions:read, and im:history. Install the app to your workspace and copy the Bot User OAuth Token and Signing Secret.

Next, set up your middleware server. This sits between Slack and your GPU endpoint. Using Slack’s Bolt SDK simplifies event handling. Subscribe to app_mention and message.im events, and point the request URL to your middleware’s public endpoint.

The middleware receives each Slack event, extracts the user’s message, sends it to your self-hosted API, and posts the response back to the originating channel. For production deployments, add conversation threading so the bot maintains context across a message chain.

Code Example

This Node.js implementation uses Bolt and forwards requests to your GPU-hosted model using the OpenAI-compatible API format:

const { App } = require('@slack/bolt');
const OpenAI = require('openai');

const app = new App({
  token: process.env.SLACK_BOT_TOKEN,
  signingSecret: process.env.SLACK_SIGNING_SECRET,
  socketMode: true,
  appToken: process.env.SLACK_APP_TOKEN,
});

const client = new OpenAI({
  baseURL: 'https://your-gpu-server.gigagpu.com/v1',
  apiKey: process.env.GPU_API_KEY,
});

app.event('app_mention', async ({ event, say }) => {
  const prompt = event.text.replace(/<@[^>]+>/g, '').trim();
  const completion = await client.chat.completions.create({
    model: 'meta-llama/Llama-3-70b-chat-hf',
    messages: [
      { role: 'system', content: 'You are a helpful team assistant.' },
      { role: 'user', content: prompt }
    ],
    max_tokens: 1024,
  });
  await say({ text: completion.choices[0].message.content, thread_ts: event.ts });
});

(async () => { await app.start(); })();

Testing Your Integration

Mention your bot in any Slack channel: @YourBot summarise the quarterly report themes. The response should appear as a threaded reply within a few seconds, depending on model size and GPU tier.

Test edge cases including long prompts, concurrent requests from multiple users, and empty messages. Monitor your GPU server’s utilisation to confirm inference is running on-device. Check the Slack app dashboard for event delivery failures and retry patterns.

Production Tips

For a workspace-wide deployment, add rate limiting in your middleware to prevent a single user from saturating the GPU. Implement a request queue so bursts of mentions are handled sequentially rather than overwhelming inference capacity.

Store conversation history in a lightweight database keyed by Slack thread ID. This lets the bot maintain multi-turn context without re-sending the entire channel history. Cap context length based on your model’s token window.

Secure your pipeline end to end — follow our secure AI inference API guide for token authentication and TLS between your middleware and the GPU endpoint. Log all requests for compliance auditing, especially if the bot handles internal business data.

For teams exploring AI chatbot hosting at scale, dedicated GPU servers eliminate the unpredictable costs of per-token billing. Browse more integration tutorials or get started with GigaGPU dedicated GPU hosting to power your Slack AI assistant today.