How AI Agents might practically work
Compound mistakes: an agent often needs to perform multiple steps to accomplish a task, and the overall accuracy decreases as the number of steps increases. If the model’s accuracy is 95% per step, over 10 steps, the accuracy will drop to 60%, and over 100 steps, the accuracy will be only 0.6%.
Herein lies the rub with agents. Once they tumble down a bad path, how can they recover? Reminds me of the rumor(?) that an AI Model, when instructed to last as long as possible in tetris merely paused the game.