we experience a "conversation" but the model is really re-reading the whole transcript cold on every single turn. Once that clicks, the growing cost per turn, the dropped details, and even "why did editing an old message change the answer?" all stop being mysterious.
Thank you! Very informative, and I love the diagrams.
Yep! This is why long chats get slow!
we experience a "conversation" but the model is really re-reading the whole transcript cold on every single turn. Once that clicks, the growing cost per turn, the dropped details, and even "why did editing an old message change the answer?" all stop being mysterious.