Every day in Seattle, Aristo takes a mostly multiple-choice fourth-grade science test. Aristo has been taking these tests since 2013, learning from mistakes and filling in knowledge gaps, but has yet to pass one. It’s the thinking part that’s hard.
That’s because Aristo is a computer program.
The Allen Institute for Artificial Intelligence on Lake Union’s north shore has been working for three years on the program. Each day, Aristo’s memory is wiped clean so it can’t cheat by remembering past right and wrong answers. Each week, the tests—but not their difficulty level—are changed.
On Aristo’s first attempt in 2013, it scored 27 out of 100—a solid F. Now its average score is 49—also an F, but the kind of F a teacher might scrawl “Shows improvement” beside. A passing score is 65, which Peter Clark, research manager for the 15-person project, thinks Aristo can hit sometime in 2017.
The slow process of getting Aristo up to speed on basic scientific concepts is due to the big difference between artificial intelligence applying logic and a brute-force sifting of facts until the right one is found. The Institute’s tests are designed to make Aristo think—that is, to search within its databases, find patterns among those facts, apply logic to those patterns, and answer questions that need logic and not solely memory.
Aristo once had a problem figuring out that a plant sitting next to a window would receive nurturing light. Another time, Aristo got stumped by the word “squeeze,” not knowing that it means the same thing as “compact” — a word it knew. If Aristo’s databases say iron, steel, and copper conduct electricity, can it figure out that all metals conduct electricity?
“We want to connect the dots between science and the common-sense world,” says Oren Etzioni, CEO of the Institute, founded in 2013 by Seattle billionaire Paul Allen.
Elsewhere at the Institute, other teams are working on artificial intelligence programs involving advanced searches through scientific literature, solving SAT-level math and geometry problems, and extracting information from images, videos, and diagrams. The institute makes Seattle something of a hub for the burgeoning science of AI, which has a way of capturing the public’s imagination with scenes of robots conquering the world. But a visit with Aristo shows how much grunt work is involved in developing a technology that is more fourth-grade flunky than menacing genius.
Teaching a computer program to think up to a child’s level intrigues Clark, 52, who has been tackling artificial intelligence since 1984. He has earned a Master’s degree from the University of Edinburgh and a Ph.D. at the University of Glasgow, both in artificial intelligence. One of his early projects involved a program that made oil-drilling recommendations based on geological factors. It was fed a bunch of geological information and matched that data with what was stored in its memory.
But that program simply crunched words and numbers, with no real thinking involved. Clark wants to create a computer system that can think, solve, and carry on some sort of conversation with people. “My dream is to have a coherent conversation with a computer about an advanced subject,” Clark says, believing this is possible in his lifetime.
Etzioni and Clark talk about artificial-intelligence systems eventually being able to help teach in schools and assist with medical diagnoses. Clark envisions a time when “kids can learn and have conversations [with a computer] as they do with a teacher.” To him, experimenting with artificial intelligence might provide a peek into how people think. How exactly does a brain biologically sort through millions of bits of information, recognize patterns, logically sort out what fits and what doesn’t, and come up with an answer? “It’s a bit opaque on how we do that,” Clark says.
If researchers can duplicate a little of that ability in a computer program, it might provide insight into how a human mind works, Clark says. Occasionally his team sees evidence of a tiny intuitive leap when Aristo solves a fourth-grade science problem: “It’s cool when a computer has those flashes of brilliance.”
Once Aristo passes the fourth-grade science tests, the team will begin trying to get it to pass eighth-grade science exams, and then 12th-grade. The teach-the-computer grunt work includes cramming in more raw information for it to sort through, increasing its computing power, and writing programs to boost its pattern-recognition and logic capabilities. Clark’s ultimate goal is to have a legitimate conversation with a computer on a sophisticated topic.
Etzioni and Clark know their evil-artificial-intelligences-taking-over-the-world films. Their eyes actually light up when talking about them. “They’re great movies. But they are very misleading about the state of the art of AI,” Clark says. “Skynet” from the Terminator movies is maybe centuries away. The helpful and conversational starship computers and the android Data from the Star Trek franchise are just as distant. Right now, existing artificial intelligences are stumbling idiot savants—good at one thing and worthless at everything else. The average elementary-school kid is more sophisticated, adaptable, and worldly.
In other words, the AI apocalypse won’t be a valid scenario for a very long time. Ditto with an artificial intelligence developing a personality—evil or benign, male or female—as in the movies and television.
To Clark, an artificial intelligence is still an experimental machine that needs countless improvements. “The reality of the moment is that AI systems are pretty much an ‘it,’ ” he says.