2025
AI/MLLLM Router
Built an LLM routing system, classifying prompt types, and directing them to cost-efficient models. Developed a web app for the model router with customizable metrics (cost, latency, and quality).
Overview
The LLM Router is a web-based application that automatically routes user prompts to the most cost-effective and performant language model based on the detected task type (e.g., summarization, code generation, question answering). The project mimics an internal feature of OpenRouter API, a platform that optimizes model selection for users and has been supported by NotDiamond.ai. The system allows users to test prompts, view routing decisions, and customize routing priorities such as cost, latency, and quality.
Challenge
Building an intelligent routing system that balances multiple competing factors while maintaining reliability required designing a reliable classification function to identify task types, balancing trade-offs among cost, speed, and model quality, collecting and maintaining accurate cost and latency data for multiple LLM providers, building a clear web interface that explains model selection, and evaluating routing performance with consistent benchmarks.
Solution
Developed a comprehensive routing system with prompt classification using lightweight NLP heuristics and keyword embeddings to detect intent, multi-model integration mapping each task type to the optimal model from a pool (OpenAI, Claude, Gemini), an interactive Next.js dashboard allowing users to adjust routing priorities through sliders, and an evaluation pipeline containing diverse test prompts to benchmark routing accuracy.
Impact
Built a functioning LLM routing system that directs prompts to cost-efficient, high-performance models. Developed a web app interface that visualizes routing metrics and enables fine-grained user control over decision rules, achieved measurable improvements in cost efficiency and response time through adaptive routing strategies, and created a comprehensive evaluation framework for continuous improvement.
Technologies
YEAR
2025
CATEGORY
AI/ML
