What You’ll Connect
After this guide, your Jupyter Notebook environment will run directly on your GPU server — no API costs, no rate limits. Notebook cells execute with full GPU access for model inference, fine-tuning, and AI-powered data analysis, all on dedicated GPU hardware you control.
The integration runs JupyterLab on your GigaGPU server and exposes it via SSH tunnel or HTTPS reverse proxy. You code interactively in your browser while GPU-accelerated libraries like PyTorch and transformers run on the remote hardware. Additionally, your notebooks can call a co-located vLLM endpoint for structured inference tasks.
SSH Tunnel / HTTPS –> JupyterLab on GPU Server (your laptop) secure connection runs on GPU hardware | | Interactive PyTorch, transformers notebook cells GPU-accelerated compute | | Results render <-- Jupyter kernel <-- GPU inference + training in browser executes on GPU direct hardware access -->Prerequisites
- A GigaGPU server with SSH access and GPU drivers installed
- Python 3.10+ with pip on the GPU server
- An SSH client on your local machine (or use the Nginx reverse proxy for HTTPS access)
- Optional: a running vLLM or Ollama endpoint on the same server for API-based inference
Integration Steps
SSH into your GPU server and install JupyterLab: pip install jupyterlab. Generate a Jupyter config and set a password hash for authentication. Start JupyterLab bound to localhost: jupyter lab --no-browser --port=8888 --ip=127.0.0.1.
On your local machine, create an SSH tunnel: ssh -L 8888:localhost:8888 [email protected]. Open http://localhost:8888 in your browser to access JupyterLab running on the GPU server. Every notebook cell executes on the remote GPU.
For team access, place JupyterHub behind an Nginx reverse proxy with TLS and token-based authentication. This lets multiple users share the GPU server through separate Jupyter environments while maintaining isolation between sessions.
Code Example
This notebook cell demonstrates calling a co-located inference endpoint and using the GPU directly for model operations:
# Cell 1: Direct GPU access for model loading
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "meta-llama/Llama-3-8b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name, torch_dtype=torch.float16, device_map="auto"
)
print(f"Model loaded on: {model.device}, GPU memory: {torch.cuda.memory_allocated()/1e9:.1f}GB")
# Cell 2: Call co-located vLLM endpoint for production inference
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="sk-local-key")
response = client.chat.completions.create(
model="meta-llama/Llama-3-70b-chat-hf",
messages=[{"role": "user", "content": "Explain gradient descent in 3 sentences."}],
max_tokens=200,
)
print(response.choices[0].message.content)
# Cell 3: GPU-accelerated data analysis
import pandas as pd
df = pd.read_csv("dataset.csv")
# Process each row through the model for classification
results = []
for _, row in df.iterrows():
resp = client.chat.completions.create(
model="meta-llama/Llama-3-70b-chat-hf",
messages=[{"role": "user", "content": f"Classify: {row['text']}"}],
max_tokens=20,
)
results.append(resp.choices[0].message.content)
df["ai_category"] = resultsTesting Your Integration
Run !nvidia-smi in a notebook cell to verify GPU access. The output should show your GPU model, driver version, and memory usage. Load a small model and run inference to confirm end-to-end functionality. If you have a co-located vLLM instance, test the API call from a notebook cell — since both run on the same server, requests to localhost have near-zero network latency.
Test with larger workloads: load a multi-gigabyte model, run a training loop, and confirm GPU utilisation stays high. Monitor memory usage to avoid OOM errors during interactive sessions.
Production Tips
For long-running experiments, use JupyterLab’s terminal to start tmux or screen sessions. This keeps training jobs alive even if your SSH tunnel drops. Jupyter’s built-in kernel restart feature can be used to free GPU memory between experiments without losing notebook state.
Set resource limits per Jupyter user if multiple people share the server. Use JupyterHub with configurable spawners that allocate specific GPUs to specific users, preventing one researcher from consuming all available memory.
Remote Jupyter on GPU hardware is the standard workflow for AI research and development. Combined with a co-located inference API running open-source models, your notebooks can both train and serve models from a single server. Browse more tutorials or get started with GigaGPU to launch your GPU-powered Jupyter environment.
