Table of Contents
For internal AI APIs consumed by multiple frontends, the API style choice matters for developer experience. GraphQL fits multi-resource fetches; REST / OpenAI-compatible fits standard LLM call patterns. The pragmatic answer: REST for the LLM tier; GraphQL or REST for application data layer.
For LLM tier: REST / OpenAI-compatible. Standard surface area; works with all SDKs and routers (LiteLLM); ecosystem maturity. GraphQL adds complexity for streaming + tool calling without commensurate benefit. For application data layer: GraphQL or REST is fine; pick by team preference. Don't mix LLM and data into one GraphQL endpoint — different concerns.
Comparison
- OpenAI-compatible REST: standard, works with every SDK, integrates with LiteLLM router, supports streaming SSE natively
- GraphQL: multi-resource queries, no over-fetching, strong typing — but streaming is awkward, tool-calling integration is custom, fewer LLM-specific tools
For LLM-specific operations (chat completion, embedding, structured output), the OpenAI-compatible REST surface area is decisively better:
- Drop-in OpenAI Python / Node / Go SDKs
- LiteLLM router compatibility
- Native streaming via SSE
- Structured output via
response_format - Tool / function calling standard shape
- Familiar to every AI engineer
Verdict
For LLM API tier in 2026, OpenAI-compatible REST is the right default. GraphQL is a fine layer for application data above the LLM tier; don't conflate the two. Most production teams: REST for AI; whatever for app data.
Bottom line
OpenAI-compatible REST for LLM tier. See AI DX.
