Deterministic Routing of Queries Between Local and Hosted LLM

Wayfinder Router

Open Source Project

Hacker News • 58 Points

02

The Challenge

Balancing cost, privacy, and performance in LLM deployments

The Dilemma: Local vs Hosted LLMs

01

Cost vs Capability Hosted LLMs offer powerful capabilities but at significant API costs
02

Privacy Concerns Sensitive data may require local processing to meet compliance requirements
03

Performance Trade-offs Local models are cheaper but may lack the sophistication for complex queries
04

Decision Fatigue Manually routing queries is inefficient and error-prone

04

The Solution

Intelligent, deterministic routing for optimal LLM selection

Introducing Wayfinder Router

Wayfinder Router is an open-source tool that automatically routes your LLM queries between local and hosted models based on deterministic rules. It analyzes query complexity, privacy requirements, and cost constraints to make optimal routing decisions in real-time.

Smart routing that saves costs without sacrificing quality

Core Capabilities

route

Deterministic Routing

Rule-based decisions ensure predictable, auditable query paths

security

Privacy-First

Sensitive data stays local; only appropriate queries reach hosted APIs

savings

Cost Optimization

Reduce API spend by routing simple queries to local models

speed

Low Latency

Instant routing decisions with minimal overhead

tune

Customizable Rules

Define your own routing logic based on query patterns

code

Easy Integration

Drop-in replacement for existing LLM API calls

How It Works

1

Query Analysis

Incoming query is analyzed for complexity, data sensitivity, and intent

→

2

Rule Evaluation

Deterministic rules determine the optimal model destination

→

3

Smart Routing

Query is routed to local or hosted LLM based on evaluation

→

4

Response Delivery

Response is returned with full transparency and logging

Impact: Before vs After

Without Wayfinder

Manual routing decisions
Unpredictable API costs
Privacy compliance risks
Inconsistent query quality

With Wayfinder Recommended
Automated intelligent routing
Up to 70% cost reduction
Privacy-by-design compliance
Optimal model selection every time

Thank You

Wayfinder Router: Making every query count

github.com/itsthelore/wayfinder-router