When an artificial intelligence system gives us a wrong answer, almost everyone reaches for the same explanation, which is that the system never learned the thing we asked about. The information was missing from its training, and so the remedy is to supply more of it, whether that means more data, a larger model, or another round of training on our own internal documents. This explanation has become so common that most organizations no longer treat it as an assumption, and they build their budgets and their improvement plans around it.

A new study from Google Research and the Technion tested that assumption directly, and in the most advanced systems available today it turns out to be wrong far more often than it is right.

The researchers separated two questions that an ordinary accuracy score combines into a single number. The first question is whether the model ever learned a particular fact at all, and the second question is whether the model can reach that fact at the moment someone asks for it. The authors describe the difference as empty shelves compared with lost keys, which is a useful image because the two situations look identical from the outside and require completely different responses. When they tested the newest systems, including GPT-5 and Gemini 3, the shelves were almost entirely full, because between 95 and 98 percent of the facts they tested for were already present inside the model. What those same systems could not do, in between a quarter and a third of cases, was reach a fact they held and state it.

The clearest evidence for this does not require any understanding of how these systems are built. When the researchers asked a question in open form and required the model to produce the answer by itself, it frequently failed, and this happened most often when the question approached the fact from an unfamiliar direction. When they presented that same question together with a list of possible answers and asked the model only to choose, it identified the correct one without difficulty. The knowledge was therefore clearly present, because the model could recognize it when it appeared, and yet the model could not produce that same knowledge on its own.

Three consequences follow from this, and none of them belong to the technical world.

The first consequence concerns money. If the knowledge an organization needs is already inside the system, then paying to add more knowledge will not improve anything, because the problem was never a shortage. This is the equivalent of responding to an empty warehouse floor by ordering more inventory, when the inventory you already own is sitting in a room that nobody has the key to.

The second consequence is less comfortable. The researchers found that a model’s ability to reach a fact depends on how closely the question resembles the situation in which the model originally encountered that fact, which means that phrasing, context, and the direction from which you approach a subject all affect what you receive. This is not a matter of clever wording or of tricks, and it is closer to the ordinary skill of asking a knowledgeable colleague a well framed question rather than a vague one. It also means that a meaningful part of the quality of any answer is contributed by the person asking, and almost no organization currently measures that contribution or trains anyone to improve it.

The third consequence concerns rare information. Facts that appear infrequently in the world, the specialized and unglamorous kind, were stored at rates close to those of famous facts, and the real gap between them appeared only when the model had to retrieve them. The depth that people assume these systems lack is largely present, and the difficulty lies in reaching it rather than in holding it.

There is a partial remedy, and it carries a price. When these systems are permitted to work through a problem before responding, rather than answering immediately, they recover between 40 and 65 percent of the facts that were present but not directly reachable, while recovering only 5 to 15 percent of the facts that were genuinely absent. In other words, this slower mode of working functions mainly as a way of unlocking what is already there rather than as a way of deriving something new. It also consumes time and computing cost, and the researchers state plainly that it is still unclear when a system should use it and when it should not. That question is not waiting for a technical answer, because it is a decision about when accuracy is worth paying for and when speed matters more, and a human being has to make it.

One limitation deserves attention before anyone applies this too widely. The facts in the study were drawn from Wikipedia, and they number 2,150, which means they are encyclopedic in character and appear repeatedly in the material these systems were trained on. Whether the same balance between storage and retrieval holds for your own internal documents, for the vocabulary of your industry, or for a contract that exists in exactly one place, is a question this research does not answer, and for most organizations it is the more important question.

What stays with me is that the shape of this problem is not unusual or foreign. Most organizations are not short of knowledge either, because the answer to almost any question already exists somewhere inside them, whether in a person’s memory, in a decision recorded two years ago, or in a document that nobody can find because they do not know the words it was written under. The usual response is to hire more people, to write more documentation, and to add more process, all of which are ways of buying more inventory. Meanwhile the thing that was needed was already in the building, and nobody could reach it.

An empty shelf and a locked cabinet look exactly the same when you are standing in the corridor. Only one of them is solved by buying more.

Sources

Nitay Calderon, Gal Yona and colleagues at Google Research and the Technion, “Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality.” Paper: arXiv:2602.14080. https://arxiv.org/abs/2602.14080

Accompanying article by Nitay Calderon and Gal Yona, Google Research blog, August 12, 2026. https://research.google/blog/empty-shelves-or-lost-keys-recall-is-the-bottleneck-for-parametric-factuality/

Benchmark: WikiProfile, 2,150 facts drawn from Wikipedia, each paired with ten probing tasks, evaluated across 13 models and around four million responses. https://huggingface.co/datasets/google/WikiProfile

Principal figures: GPT-5 and Gemini 3 encoded 95 to 98 percent of the tested facts, while failing to recall 26 to 34 percent of them directly, and still failing on 11 to 12 percent when permitted to think first. Extended reasoning recovered 40 to 65 percent of facts that were encoded but not directly recalled, compared with 5 to 15 percent of facts that were never encoded. Retrieval failures were concentrated on rare facts and on questions asked in reverse direction. The authors note that encoding appears near saturation on their own benchmark, which is built from Wikipedia.

I came across this research in a LinkedIn post by Avinatan Hassidim, Vice President at Google Research.