Connect Flutter to Self-Hosted AI

What You’ll Connect

After this guide, your Flutter app will stream AI responses from your own GPU server on both iOS and Android — tokens appearing progressively as the model generates them. The Dart HTTP client connects to your vLLM or Ollama endpoint on dedicated GPU hardware using the OpenAI-compatible API, and Flutter’s widget system renders each token instantly.

The integration uses Dart’s http package with chunked response reading for streaming. A state management layer (Provider or Riverpod) holds conversation history and notifies widgets as new tokens arrive, keeping the UI responsive during long generations.

Prerequisites

• A GigaGPU server running a self-hosted LLM (setup guide)
• HTTPS access to your inference endpoint
• Flutter 3.10+ with Dart 3.0+
• API key for your GPU inference server

Integration Steps

Create a service class in Dart that handles HTTP streaming to your GPU endpoint. Use the http package’s Client.send() method with a streamed request to receive server-sent events. Parse each chunk for delta tokens and yield them through a Dart Stream that your UI layer consumes.

Build a state management layer using Provider or Riverpod. A ChatNotifier holds the conversation history, exposes a sendMessage method that calls the streaming service, and notifies listeners as each token arrives. Widgets rebuild efficiently because Flutter’s framework only updates the text widget showing the current assistant message.

Create the chat UI with a ListView for messages, a TextField for input, and a send button. Use AnimatedBuilder or Consumer to rebuild only the active message widget during streaming. Auto-scroll to the bottom as new tokens arrive using a ScrollController.

Code Example

Dart streaming service and chat state for your self-hosted LLM:

import 'dart:async';
import 'dart:convert';
import 'package:http/http.dart' as http;

class AiChatService {
  final String apiUrl;
  final String apiKey;

  AiChatService({required this.apiUrl, required this.apiKey});

  Stream<String> streamCompletion(List<Map<String, String>> messages) async* {
    final request = http.Request('POST',
        Uri.parse('$apiUrl/v1/chat/completions'));
    request.headers.addAll({
      'Content-Type': 'application/json',
      'Authorization': 'Bearer $apiKey',
    });
    request.body = jsonEncode({
      'model': 'meta-llama/Llama-3-70b-chat-hf',
      'messages': messages,
      'stream': true,
      'max_tokens': 1024,
    });

    final response = await http.Client().send(request);
    final stream = response.stream
        .transform(utf8.decoder)
        .transform(const LineSplitter());

    await for (final line in stream) {
      if (!line.startsWith('data: ') || line == 'data: [DONE]') continue;
      final json = jsonDecode(line.substring(6));
      final token = json['choices'][0]['delta']['content'] ?? '';
      if (token.isNotEmpty) yield token;
    }
  }
}

// Usage in a Riverpod notifier
class ChatNotifier extends ChangeNotifier {
  final AiChatService _service;
  final List<ChatMessage> messages = [];
  bool isStreaming = false;

  ChatNotifier(this._service);

  Future<void> send(String text) async {
    messages.add(ChatMessage(role: 'user', content: text));
    messages.add(ChatMessage(role: 'assistant', content: ''));
    isStreaming = true;
    notifyListeners();

    final apiMessages = messages.sublist(0, messages.length - 1)
        .map((m) => {'role': m.role, 'content': m.content}).toList();

    await for (final token in _service.streamCompletion(apiMessages)) {
      messages.last.content += token;
      notifyListeners();
    }
    isStreaming = false;
    notifyListeners();
  }
}

Testing Your Integration

Run the app on an emulator or device and send a test message. Verify tokens stream progressively into the message bubble. Test on both iOS and Android to confirm the HTTP streaming works cross-platform. Monitor the debug console for any SSE parsing errors. Test with long responses to ensure the ListView scrolls smoothly during streaming.

Test network error handling by toggling airplane mode mid-stream. The service should throw a catchable exception that the notifier handles gracefully, showing an error state in the UI with a retry option.

Production Tips

Route requests through your own backend API rather than calling the GPU server directly from the mobile app. This hides the API key, lets you add user authentication and rate limiting, and provides a stable endpoint that does not change when you scale your GPU infrastructure. Use API hosting patterns for the middleware layer.

Cache conversation history locally with Hive or SQLite for offline access to past conversations. Implement message pagination for long threads. Add haptic feedback on message send and receive for a polished experience. Build a complete AI chatbot mobile app with Flutter. Explore more tutorials or get started with GigaGPU to power your Flutter apps.