Optimization of Memory and Information Retrieval in Humans and Libraries

For many years I have been interested in the history of mnemonics, as reflected in the Memory /Mnemonics / Data Storage theme in HistoryofInformation. For centuries the primary goal of mnemonics was to enhance human memory and to optimize its utility through the teaching and practice of mnemonic techniques. These techniques, such as the "method of loci" or "memory palace," were described in the most famous ancient Roman treatise on rhetoric, persuasion and mnemonics entitled Rhetorica ad Herennium, a treatise that evolved when books (papyrus rolls) and written records were expensive and scarce to a degree that would be unimaginable today. Partly as a result of the scarcity of books, but also because of low rates of education an literacy in ancient Roman society, the universe of knowledge in the ancient world was vastly more limited than today. During the ancient world and the Middle Ages it was possible for certain scholars to memorize much more of the totality what they might be expected to know than we would expect. Besides memorization or regurgitation of random details, the purpose of a memory palace, or some other system of mnemonics, then as well as now, is to enable to practitioner to retrieve the specific information they are searching for in memory when they need it.

Memory can be viewed as an individual issue—what one person can memorize or record, or it can viewed as a collective concept, the records of a group or a society, or even possibly the world. When memory is thought of as a collective concept it is frequently in the context of libraries as preserving knowledge of a culture, or in a wider sense of world heritage. As early as 1784 the architect Etienne-Louis Boullée envisaged a library with a huge reading room that could contain the entire "Memory of the World." Prior to writing this essay I had compiled notes in HistoryofInformation about several other universal library schemes- more or less detailed - authored by Gabriel Naudé, Kurt Lasswitz, and Jorge Luis Borges. In 1627 Naudé in Paris expressed a goal of creating a public "universal library" in his book entitled Avis pour dresser une bibliothèque. In 1901 German scientist, philosopher and science fiction writer Kurt Lasswitz at Gotha, Germany published a story entitled Die Universalbibliothek (The Universal Library) describing a library which was universal in the sense that it not only contained all existing written works, but all possible written works. In 1939 Argentine writer and librarian Jorge Luis Borges in Buenos Aires published an essay entitled La bibliotheca total (The Total Library), describing his fantasy of an all-encompassing archive or universal library. In Borges' work this universal library was created, remarkably, by an abstract device that produced a random sequence of letters and symbols, ad infinitum. Those several fictional or aspirational schemes may have prepared the imagination of commentators who recognized the potential of the Internet to serve as a "universal library." 

Just as the value of memorizing or learning is in the ability to retrieve and apply appropriate knowledge in an appropriate circumstance, intellectual or practical, the usefulness of any library is entirely dependent upon how well it is organized and catalogued. In this respect a library catalogue enables a user of a library to conduct research analogous to an individual's referencing a personal memory palace, but on an enormously amplified scale, far beyond the capability of anyone, or even a whole society, to remember. Instead the library functions as an vast artificial memory, but a memory that must be searched, explored, sampled, absorbed, and shared. HistoryofInformation describes the history of library catalogues, beginning with the known catalogues of various medieval institutional libraries, the largest of which was the library of the Sorbonne, the earliest catalogue of which dates to the 14th century, and the first efforts in the 16th century, after printing enabled the expansion and distribution of the world of knowledge, by Conrad Gessner in his printed book entitled Bibliotheca universalis (1545-1555). The challenge of bringing order to libraries to make them accessible to users, resulted in many analog solutions over the centuries, culminating in vast multi-volume printed catalogues, the largest of which was the British Museum Catalogue, that ultimately expanded to 283 printed folio volumes. Huge printed catalogues like that, and even much larger stationary card catalogues that used to fill large halls in large libraries, were so time-consuming to consult that in 1949, before any electronic computer was available, since none had yet been manufactured for sale, Sanford Larkey began planning to automate information retrieval at the Army Medical Library, as the National Library of Medicine was then designated. This was the earliest project in electronic information retrieval anywhere, and the foundation, among many other things, of the enormous online databases produced decades later by the U.S. National Library of Medicine, including Medline, that are presently accessed by tens of millions of people around the world each day.

Whether or not they perceive the Internet as a universal library, Internet search engines, such as Google or Baidu, were originally modeled upon library information retrieval projects, such as those that originated at the Army Medical Library, but, of course, they base their searches on algorithms far more complex than an actual library searches, since they are searching a virtually infinite variety of information sources far more diverse than whatever media an actual library might contain. From my perspective the greatest advantages these search engines offer is their ability to pinpoint information in the most minutely specific way, searching through what we perceive to be a virtually infinite amount of data. In this respect they are incalculably superior in their information retrieval ability to any human memory, individual or collective. Nevertheless, at the present time to formulate a search through Google or Baidu we still need a search idea produced by our human brain, and sometimes it even takes human skill in designing the search to obtain appropriate results. In parallel with their evolving search engine algorithms, Google and Baidu sponsor extensive research in artificial intelligence. Will the artificial intelligence being developed by these search engines eventually anticipate our search requests before we think of them ourselves?

Jeremy M. Norman
December 2, 2020