Edgee frames mid-session LLM switching as ski-rental problem
Start here
Edgee published a technical explainer describing how its Agent Gateway handles switching LLMs mid-conversation in coding sessions, framing the decision as a ski-rental online problem. The gateway treats routing as two coupled parts: a difficulty tier map assigning a model to each turn, and a switching policy deciding when a running session actually changes model. Because providers cache conversation history, each switch discards the warm cache and pays to rebuild it, which is why Edgee models switching as a cost trade-off rather than a free choice. Edgee says its Self-Learning Routing replays the past 30 days of an organization's sessions weekly to pick the cheapest tier map and switching settings that keep every turn on a capable model. Routing coding agents through Edgee requires a CLI install of about five minutes and no code changes, and nothing changes until the settings are applied; turning on classification in the console requires a Team or Enterprise plan.
Generated from reports · updated 4 h ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AISwitching LLMs mid-conversation is a ski-rental problem
Edgee's Agent Gateway treats LLM routing in coding sessions as two coupled problems: a difficulty tier map that assigns a model to each turn, and a switching policy that decides when a running session actually changes model. Because providers cache conversation history, every model switch discards the warm cache and pays to rebuild it, so the gateway frames switching as a ski-rental online problem. Self-Learning Routing replays the past 30 days of an organization's sessions weekly to pick the cheapest tier map and switching settings that keep every turn on a capable model.
Heat for this event
- Current, comparable range
- 9
- Peak, comparable range
- 10November 20 00:00
- Change in 24 hours
- –
The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.