Top notch AI system about as smart as a four-year-old, lacks commonsense

Article from http://www.gizmag.com/ai-system-iq-four-year-old/28321/

Those who saw IBM’s Watson defeat former winners on Jeopardy! in 2011 might be forgiven for thinking that artificially intelligent computer systems are a lot brighter than they are. While Watson was able to cope with the highly stylized questions posed during the quiz, AI systems are still left wanting when it comes to commonsense. This was one of the factors that led researchers to find that one of the best available AI systems has the average IQ of a four-year-old.

To see just how intelligent AI systems are, a team of artificial and natural knowledge researchers at the University of Illinois as Chicago (UIC) subjected ConceptNet 4 to the verbal portions of the Weschsler Preschool and Primary Scale of Intelligence Test, which is a standard IQ test for young children. ConceptNet 4 is an AI system developed at MIT that relies on a commonsense knowledge base created from facts contributed by thousands of people across the Web.

While the UIC researchers found that ConceptNet 4 is on average about as smart as a four-year-old child, the system performed much better at some portions of the test than others. While it did well on vocabulary and in recognizing similarities, its overall score was brought down dramatically by a bad result in comprehension, or commonsense “why” questions.

“If a child had scores that varied this much, it might be a symptom that something was wrong,” said Robert Sloan, professor and head of computer science at UIC, and lead author on the study. “We’re still very far from programs with commonsense–AI that can answer comprehension questions with the skill of a child of eight.”

Sloan says AI systems struggle with commonsense because it relies not only on a large collection of facts, which computers can access easily through a database, but also on obvious things that we don’t even know we know – things that Sloan calls “implicit facts.” For example, a computer may know that water freezes at 32° F (0° C), but it won’t necessarily know that it is cold, which is something that even a four-year-old child will know.

“All of us know a huge number of things,” says Sloan. “As babies, we crawled around and yanked on things and learned that things fall. We yanked on other things and learned that dogs and cats don’t appreciate having their tails pulled.”

Sloan and his colleagues hope their study will hope identify areas for AI research to focus on to improve the intelligence of AI systems. They will present their study on July 17 at the US Artificial Intelligence Conference in Bellevue, Washington.

Article from http://www.gizmag.com/ai-system-iq-four-year-old/28321/