Skip to content
Hot eventDeveloping

Edgee frames mid-session LLM switching as ski-rental problem

1 reports1 reporting sourcesUpdated 4 h ago

Start here

AI digest

Edgee published a technical explainer describing how its Agent Gateway handles switching LLMs mid-conversation in coding sessions, framing the decision as a ski-rental online problem. The gateway treats routing as two coupled parts: a difficulty tier map assigning a model to each turn, and a switching policy deciding when a running session actually changes model. Because providers cache conversation history, each switch discards the warm cache and pays to rebuild it, which is why Edgee models switching as a cost trade-off rather than a free choice. Edgee says its Self-Learning Routing replays the past 30 days of an organization's sessions weekly to pick the cheapest tier map and switching settings that keep every turn on a capable model. Routing coding agents through Edgee requires a CLI install of about five minutes and no code changes, and nothing changes until the settings are applied; turning on classification in the console requires a Team or Enterprise plan.

Generated from reports · updated 4 h ago

Report timeline

Follow the reports to see the event from different sides.

November 19
  1. Hacker News · AI
    Switching LLMs mid-conversation is a ski-rental problem

    Edgee's Agent Gateway treats LLM routing in coding sessions as two coupled problems: a difficulty tier map that assigns a model to each turn, and a switching policy that decides when a running session actually changes model. Because providers cache conversation history, every model switch discards the warm cache and pays to rebuild it, so the gateway frames switching as a ski-rental online problem. Self-Learning Routing replays the past 30 days of an organization's sessions weekly to pick the cheapest tier map and switching settings that keep every turn on a capable model.

Heat for this event

Current, comparable range
9
Peak, comparable range
10November 20 00:00
Change in 24 hours
–

The trend compares the same participants over hours observed continuously, so its range can be narrower than the current heat count. Hover or tap the chart for hourly heat; the arrow keys move through it.