Vanguard AI
Enterprise LLM Gateway & Orchestration Platform
Overview
Built an enterprise-grade AI proxy that routes LLM calls across providers, logs token costs, and orchestrates cyclic multi-agent workflows with persistent memory.
The Challenge
Enterprise teams using multiple LLM providers face fragmented billing, zero observability into token costs, and no guardrails against prompt injection. Standard chatbot wrappers offer no persistent state or autonomous looping.
Our Solution
Vanguard AI acts as a drop-in OpenAI-compatible proxy. It routes requests via LiteLLM to the cheapest viable provider, logs every token to PostgreSQL for cost analytics, and provides a LangGraph orchestration engine for cyclic multi-agent workflows with database-backed persistent memory.
Technical Architecture
- Vanguard Shield — FastAPI + LiteLLM proxy gateway with API key auth
- Vanguard Sight — Token cost and latency logging to PostgreSQL
- Vanguard Core — LangGraph agent orchestration with JSONB state persistence
- Vanguard Intel — Reference application: Lead Qualification Agent
Results & Status
Phase 1 (Database Layer) completed. 4 SQLModel tables deployed. Phase 2 (FastAPI Proxy) in progress.