Closed model providers change behavior between API versions with no real changelog. Building anything on top of them is a gamble.

This is one of the reasons I keep gravitating back to local models even when the closed API ones are technically stronger. I had a production pipeline running on a major closed API for about four months. Stable, tested, working. Then one day the outputs started drifting. Not breaking errors, just subtle behavioral changes. Format slightly different, refusals on things it used to handle fine, confidence on certain task types quietly degraded. No changelog. No notification. Support ticket response was essentially "models are updated periodically to improve quality." There is no way to pin to a specific checkpoint. You signed up for a service that reserves the right to change what the service does at any time. The thing that gets me is how normalized this is. If a database provider silently c