Detail map of Pittsburgh, Pennsylvania, United States Overview map of Pittsburgh, Pennsylvania, United States

A: Pittsburgh, Pennsylvania, United States

"The Never-Ending Language Learning System"

1/2010

Supported by DARPA and Google, in January 2010 Tom M. Mitchell and his team at Carnegie Mellon University initiated the Never-Ending Language Learning System, or NELL, in an effort to develop a method for machines to teach themselves semantics, or the meaning of language.

"Few challenges in computing loom larger than unraveling semantics, understanding the meaning of language. One reason is that the meaning of words and phrases hinges not only on their context, but also on background knowledge that humans learn over years, day after day" (http://www.nytimes.com/2010/10/05/science/05compute.html?_r=1&hpw). 

"NELL has been in continuous operation since January 2010. For the first 6 months it was allowed to run without human supervision, learning to extract instances of a few hundred categories and relations, resulting in a knowledge base containing approximately a third of a million extracted instances of these categories and relations. At that point, it had improved substantially its ability to read three quarters of these categories and relations (with precision in the range 90% to 99%), but it had become inaccurate in extracting instances of the remaining fourth of the ontology (many had precisions in the range 25% to 60%).  

"The estimated precision of the beliefs it had added to its knowledge base at that point was 71%. We are still trying to understand what causes it to become increasingly competent at reading some types of information, but less accurate over time for others. Beginning in June, 2010, we began periodic review sessions every few weeks in which we would spend about 5 minutes scanning each category and relation. During this 5 minutes, we determined whether NELL was learning to read it fairly correctly, and in case not, we labeled the most blatant errors in the knowledge base. NELL now uses this human feedback in its ongoing training process, along with its own self-labeled examples. In July, a spot test showed the average precision of the knowledge base was approximately 87% over all categories and relations. We continue to add new categories and relations to the ontology over time, as NELL continues learning to populate its growing knowledge base" (http://rtw.ml.cmu.edu/rtw/overview, accessed 10-06-2010).

Timeline Themes