The same model, with and without the document
A fixed set of questions with answers written out by hand from the document before any model saw it. Every question is then run four ways. Nothing here is hand-picked: the whole set is below, including the questions we get wrong.
loading…
Check it yourself
Ask anything about that manual. The question goes down both paths at once — the model on its own, and the model with the document. Both answers appear side by side.