FROM HERE TO HUMAN-LEVEL AI
Computer Science Department
Stanford, CA 94305
It is not surprising that reaching
human-level AI has proved to be dif-
ﬁcult and progress has been slow—
though there has been important
progress. The slowness and the de-
mand to exploit what has been dis-
covered has led many to mistakenly
redeﬁne AI, sometimes in ways that
preclude human-level AI—by rele-
gating to humans parts of the task
that human-level computer programs
would have to do. In the terminology
of this paper, it amounts to settling
for a bounded informatic situation in-
stead of the more general common
sense informatic situation.
present AI systems and reaching
human-level AI requires programs
that deal with the common sense
informatic situation—in which the
phenomena to be taken into account
in achieving a goal are not ﬁxed in
We discuss reaching human-level AI,
emphasizing logical AI and especially
emphasizing representation problems
reasoning in the com-
mon sense informatic situation in-
information and of
clude nonmonotonic reasoning, ap-proximate concepts, formalized con-texts and introspection.
1 What is Human-Level AI?
The ﬁrst scientiﬁc discussion of human levelmachine intelligence was apparently by AlanTuring in the lecture [Turing, 1947]. The no-tion was ampliﬁed as a goal in [Turing, 1950],but at least the latter paper did not say whatwould have to be done to achieve the goal.Allen Newell and Herbert Simon in 1954 werethe ﬁrst people to make a start on program-ming computers for general intelligence. Theywere over-optimistic, because their idea ofwhat has to be done to achieve human-level in-telligence was inadequate. The General Prob-lem Solver (GPS) took general problem solv-ing to be the task of transforming one expres-sion into another using an allowed set of trans-formations.
Many tasks that humans can do, humans can-not yet make computers do. There are two ap-proaches to human-level AI, but each presentsdiﬃculties. It isn’t a question of deciding be-tween them, because each should eventuallysucceed; it is more a race.
- If we understood enough about how thehuman intellect works, we could simulateit. However, we don’t have have suﬃ-
cient ability to observe ourselves or others
to understand directly how our intellects
work. Understanding the human brain
well enough to imitate its function there-
fore requires theoretical and experimental
success in psychology and neurophysiol-
ogy. 1 See [Newell and Simon, 1972] for
the beginning of the information process-
ing approach to psychology.
- 2. To the extent that we understand the
problems achieving goals in the world
presents to intelligence we can write intel-
ligent programs. That’s what this article
- What problems does the world present to in-
- telligence? More narrowly, we consider the
- problems it would present to a human scale
- robot faced with the problems humans might
- be inclined to relegate to suﬃciently intelli-
- gent robots. The physical world of a robot
- contains middle sized objects about which its
- sensory apparatus can obtain only partial in-
- formation quite inadequate to fully determne
- the eﬀects of its future actions.
- world includes its interactions with people and
- also meta-information about the information
- it has or can obtain.
- Our approach is based on what we call the
- common sense informatic situation. In order
- to explain the common sense informatic situ-
- ation, we contrast it with the bounded infor-
- matic situation that characterizes both formal
- scientiﬁc theories and almost all (maybe all)
- experimental work in AI done so far.2
- 1Recent work with positron emission tomography has
- identiﬁed areas of the brain that consume more glucose
- when a person is doing mental arithmetic. This knowledge
- will help build AI systems only when it becomes possible
- to observe what is going on in these areas during mental
- 2The textbook [David Poole and Goebel, 1998] puts it
- this way. “To get human-level computational intelligence
- it must be the agent itself that decides how to divide up
- the world, and which relationships to reason about.
A formal theory in the physical sciences dealswith a bounded informatic situation. Scientistsdecide informally in advance what phenomenato take into account. For example, much ce-lestial mechanics is done within the Newtoniangravitational theory and does not take into ac-count possible additional eﬀects such as out-gassing from a comet or electromagnetic forcesexerted by the solar wind.
If more phenom-ena are to be considered, a person must makea new theory. Probabilistic and fuzzy uncer-tainties can still ﬁt into a bounded informaticsystem; it is only necessary that the set of pos-sibilities (sample space) be bounded.Most AI formalisms also work only in abounded informatic situation. What phenom-ena to take into account is decided by a personbefore the formal theory is constructed. Withsuch restrictions, much of the reasoning can bemonotonic, but such systems cannot reach hu-man level ability. For that, the machine willhave to decide for itself what information isrelevant. When a bounded informatic systemis appropriate, the system must construct orchoose a limited context containing a suitabletheory whose predicates and functions connectto the machine’s inputs and outputs in an ap-propriate way. The logical tool for this is non-monotonic reasoning.
2 The Common Sense InformaticSituation
The key to reachingContention:
human-level AI is making systems thatoperate successfully in the commonsense informatic situation.
In general a thinking human is in what we callthe common sense informatic situation ﬁrstdiscussed in3 [McCarthy, 1989].
It is moregeneral than any bounded informatic situation.The known facts are incomplete, and there isno a priori limitation on what facts are rel-3http://www-formal.stanford.edu/jmc/ailogic.html- evant.
It may not even be decided in ad-
- vance what phenomena are to be taken into
- account. The consequences of actions cannot
- be fully determined. The common sense in-
- formatic situation necessitates the use of ap-
- proximate concepts that cannot be fully de-
- ﬁned and the use of approximate theories in-
- volving them. It also requires nonmonotonic
- reasoning in reaching conclusions.
- The common sense informatic situation also
- includes some knowledge about the system’s
- mental state.
- A nice example of the common sense infor-
- matic situation is illustrated by an article in
- the American Journal of Physics some years
- ago. It discussed grading answers to a physics
- problem. The exam problem is to ﬁnd the
- height of a building using a barometer. The
- intended solution is to measure the air pres-
- sure at the top and bottom of the building
- and multiply the diﬀerence by the ratio of the
- density of mercury to the density of air.
- However, other answers may be oﬀered.
- drop the barometer from the top of the build-
- ing and measure the time before it hits the
- ground. (2) Measure the height and length of
- the shadow of the barometer and measure the
- length of the shadow of the building. (3) Rap-
- pel down the building using the barometer as
- a measuring rod. (4) Lower the barometer on
- a string till it reaches the ground and measure
- the string. (5) Oﬀer the barometer to the jani-
- tor of the building in exchange for information
- about the height. (6) Ignore the barometer,
- count the stories of the building and multiply
- by ten feet.
- Clearly it is not possible to bound in advance
- the common sense knowledge of the world
- that may be relevant to grading the prob-
- lem. Grading some of the solutions requires
- knowledge of the formalisms of physics and the
- physical facts about the earth, e.g.
- of falling bodies or the variation of air pres-
sure with altitude. However, in every case,the physics knowledge is embedded in com-mon sense knowledge. Thus before one canuse Galileo’s law of falling bodies s = 12 gt2, oneneeds common sense information about build-ings, their shapes and their roofs.
Bounded informatic situations are obtained bynonmonotonically inferring that only the phe-nomena that somehow appear to be relevantare relevant.
In the barometer example, thestudent was expected to infer that the barom-eter was only to be used in the conventionalway for measuring air pressure. For example,a reasoning system might do this by apply-ing circumscription to a predicate relevant in aformalism containing also metalinguistic infor-mation, e.g. that this was a problem assignedin a physics course. Formalizing relevance ina useful way promises to be diﬃcult.Common sense facts and common sense rea-soning are necessarily imprecise. The impreci-sion necessitated by the common sense infor-matic situation applies to computer programsas well as to people.
Some kinds of imprecision can be representednumerically and have been explored with theaid of Bayesian networks, fuzzy logic and simi-lar formalisms. This is in addition to the studyof approximation in numerical analysis and thephysical sciences.
3 The Use of Mathematical LogicWhat about mathematical logical languages?Mathematical
logic was devised to formal-ize precise facts and correct reasoning.Itsfounders, Leibniz, Boole and Frege, hoped touse it for common sense facts and reasoning,not realizing that the imprecision of conceptsused in common sense language was often anecessary feature and not always a bug. Thebiggest success of mathematical logic was informalizing mathematical theories. Since the- common sense informatic situation requires
- using imprecise facts and imprecise reason-
- ing, the use of mathematical logic for common
- sense has had limited success. This has caused
- many people to give up. Gradually, extended
- logical languages and even extended forms of
- mathematical logic are being invented and de-
- It is necessary to distinguish between mathe-
- matical logic and particular mathematical log-
- ical languages. Particular logical languages
- are determined by a particular choice of con-
- cepts and the predicate and function symbols
- to represent them. Failure to make the dis-
- tinction has often led to error. When a par-
- ticular logical language has been shown inad-
- equate for some purpose, some people have
- concluded that logic is inadequate. Diﬀerent
- concepts and diﬀerent predicate and function
- symbols might still succeed. In the words of
- the drive-in movie critic of Grapevine, Texas,
- “I’m surprised I have to explain this stuﬀ.”
- The pessimists about logic or some particular
- set of predicates might try to prove a theorem
- about its inadequacies for expressing common
- Since it seems clear that humans don’t use
- logic as a basic internal representation formal-
- ism, maybe something else will work better
- for AI. Researchers have been trying to ﬁnd
- this something else since the 1950s but still
- haven’t succeeded in getting anything that is
- ready to be applied to the common sense in-
- formatic situation. Maybe they will eventually
- succeed. However, I think the problems listed
- in the later sections of this article will apply
- to any approach to human-level AI.
- Mathematical logic has been concerned with
- how people ought to think rather than how
- people do think. We who use logic as a basic
- 4G¨odel’s theorem is not relevant to this, because the
- question is not one of decideability or of characterizing
AI formalism make programs reason logically.However, we have to extend logic and extendthe programs that use it in various ways.One important extension was the developmentof modal logic starting in the 1920s and usingit to treat modalities like knowledge, belief andobligation. Modalities can be treated eitherby using modal logic or by reifying conceptsand sentences within the standard logic. Myopinion is that reiﬁcation in standard logic ismore powerful and will work better.A second extension was the formalization ofnonmonotonic reasoning beginning in the late1970s—with circumscription and default logicand their variants as the major proposals.Nonmonotonic logic has been studied both aspure mathematics and in application to AIproblems, most prominently to the formaliza-tion of action and causality. Several variantsof the major formalisms have been devised.Success so far has been moderate, and it isn’tclear whether greater success can be obtainedby changing the the concepts and their rep-resentation by predicate and function symbolsor by varying the nonmonotonic formalism. 5We need to distinguish the actual use of logicfrom what Allen Newell,
[Newell, 1981] and[Newell, 1993], calls the logic level and whichwas also proposed in [McCarthy, 1979]6.4 Approximate Concepts andApproximate Theories
Other kinds of imprecision are more funda-mental for intelligence than numerical impre-cision. Many phenomena in the world are ap-propriately described in terms of approximateconcepts. Although the concepts are impre-cise, many statements using them have precisetruth values. We oﬀer two examples: the con-5One referee for KR96 foolishly and arrogantly pro-posed rejecting a paper on the grounds that the inadequacyof circumscription for representing action was known.6http://www-formal.stanford.edu/jmc/ascribing.html- cept of Mount Everest and the concept of the
- welfare of a chicken. The exact pieces of rock
- and ice that constitute Mount Everest are un-
- clear. For many rocks, there is no truth of the
- matter as to whether it is part of Mount Ever-
- est. Nevertheless, it is true without qualiﬁca-
- tion that Edmund Hillary and Tenzing Norgay
- climbed Mount Everest in 1953 and that John
- McCarthy never set foot on it.
- The point of this example is that it is possi-
- ble and even common to have a solid knowl-
- edge structure from which solid conclusions
- can be inferred based on a foundation built on
- the quicksand of approximate concepts with-
- out deﬁnite extensions.
- As for the chicken, it is clear that feeding it
- helps it and wringing its neck harms it, but
- it is unclear what its welfare consists of over
- the course of the decade from the time of its
- hatching. Is it better oﬀ leading a life of poul-
- try luxury and eventually being slaughtered
- or would it be better oﬀ escaping the chicken
- yard and taking its chances on starvation and
- foxes? There is no truth of the matter to be
- determined by careful investigation of chick-
- ens. When a concept is inherently ap-
- proximate, it is a waste of time to try to
- give it a precise deﬁnition. Indeed diﬀer-
- ent eﬀorts to deﬁne such a concept precisely
- will lead to diﬀerent results—if any.
- Most human common sense knowledge in-
- volves approximate concepts, and reaching
- human-level AI requires a satisfactory way
- of representing information involving approxi-
- mate concepts.
- 5 Nonmonotonic Reasoning
- Common sense reasoning is also imprecise in
- that it draws conclusions that might not be
- made if there were more information. Thus
- common sense reasoning is nonmonotonic.
- will not go into the details of any of the pro-
posals for handling nonmonotonic reasoning.In particular, getting from the common senseinformatic situation to a bounded informaticsituation needs nonmonotonic reasoning.6 Elaboration Tolerance
Human abilities in the common sense infor-matic situation also include what may becalled elaboration tolerance—the ability toelaborate a statement of some facts withouthaving to start all over. Thus when we beginto think about a problem, e.g. determiningthe height of a building, we form a boundedcontext and try to solve the problem within it.However, at any time more facts can be added,e.g. about the precision with which the timefor the barometer to fall can be estimated us-ing a stop watch and also the possibilities ofacquiring a stop watch.
Elaboration Tolerance7 discusses about 25elaborations of the Missionaries and Cannibalsproblem.
What I have so far said so far about ap-proximate concepts, nonmonotonic reasoningand elaboration tolerance is independent ofwhether mathematical logic, human languageor some other formalism is used.
In my opinion, the best AI results so far havebeen obtained using and extending mathemat-ical logic.
7 Formalization of Context
A third extension of mathematical logic in-volves formalizing the notion of context8[McCarthy, 1993]. Notice that when logicaltheories are used in human communicationand study, the theory is used in a contextwhich people can discuss from the outside. Ifcomputers are to have this facility and are to7http://www-formal.stanford.edu/jmc/elaboration.html8http://www-formal.stanford.edu/jmc/context.html- work within logic, then the “outer” logical lan-
- guage needs names for contexts and sentences
- giving their relations and a way of entering a
- context. Clearly human-level AI requires rea-
- soning about context.
- Human-level AI also requires the ability to
- transcend the outermost context the system
- has used so far. Besides in [McCarthy, 1993],
- this is also discussed in Making Robots
their Mental States9
- [McCarthy, 1996].
- Further work includes
- [Buvaˇc et al., 1995].
[Buvaˇc, 1996] and
- 8 Reasoning about
- Reasoning about actions has been a major AI
- activity, but this paper will not discuss my or
- other people’s current approaches, concentrat-
- ing instead on the long range problem of reach-
- ing human level capability. We regard actions
- as particular kinds of events and therefore pro-
- pose subsuming reasoning about actions under
- the heading of reasoning about events.
- Most reasoning about events has concerned
- determining the eﬀects of an explicitly given
- sequence of actions by a single actor. Within
- this framework various problems have been
- • The frame problem concerns not having
to state what does not change when an
- • The qualiﬁcation problem concerns not
having to state all the preconditions of an
action or other event. The point is both
to limit the set of preconditions and also
to jump to the conclusion that unstated
others will be fulﬁlled unless there is evi-
dence to the contrary. For example, wear-
ing clothes is a precondition for airline
travel, but the travel agent will not tellhis customer to be sure and wear clothes.• The ramiﬁcation problem concerns how totreat side-eﬀects of events other than theprincipal eﬀect mentioned in the event de-scription.
Each of these involves elaboration tolerance,e.g.
adding descriptions of the eﬀects ofadditional events without having to changethe descriptions of the events already de-scribed. When I wrote about applications ofcircumscription to formalizing commonsense10 [McCarthy, 1986], I hoped that a sim-ple abnormality theory would suﬃce for all ofthem. That didn’t work out when I tried it,but I still think a common nonmonotonic rea-soning mechanism will work. Tom Costello’sdraft “The Expressive Power of Circumscript-tion” 11 argues that simple abnormality theo-ries have the same expressive power as moreelaborate nonmonotonic formalisms that havebeen proposed.
intelligence requires reasoningabout strategies of action,
It also requires considering multipleactors and also concurrent events and contin-uous events. Clearly we have a long way togo.
Some of these points are discussed in a drafton narrative12 [McCarthy, 1995].
People have a limited ability to observe theirown mental processes. For many intellectualtasks introspection is irrelevant. However, itis at least relevant for evaluating how one isusing one’s own thinking time. Human-levelAI will require introspective ability.10http://www-formal.stanford.edu/jmc/applications.html11http://www-formal.stanford.edu/tjc/expressive.html12http://www-formal.stanford.edu/jmc/narrative.html- That robots also need introspection13
- is argued and how to do it is discussed in
- [McCarthy, 1996].
summarized as that of succeeding in the com-mon sense informatic situation.
The problems include:
- 10 Heuristics
- The largest qualitative gap between human
- performance and computer performance is in
- the area of heuristics, even though the gap is
- disguised in many applications by the millions-
- fold speed advantage of computers. The gen-
- eral purpose theorem proving programs run
- very slowly, and the special purpose programs
- are very specialized in their heuristics.
- I think the problem lies in our present in-
- ability to give programs domain and prob-
- lem dependent heuristic advice.
In my Ad-
- vice Taker paper14 [McCarthy, 1959] I adver-
- tised that the Advice Taker would express its
- heuristics declaratively. Maybe that will work,
- but neither I nor anyone else has been able to
- get a start on the problem in the ensuing al-
- most 40 years. Joseﬁna Sierra-Santibanez re-
- ports on some progress in a forthcoming arti-
- Another possibility is to express the advice in
- a procedure modiﬁcation language, i.e. to ex-
- tend elaboration tolerance to programs. Of
- course, every kind of modularity, e.g. object
- orientation, gives some elaboration tolerance,
- but these devices haven’t been good enough.
- Ideally, a general purpose reasoning system
- would be able to accept advice permitting it
- to run at a ﬁxed ratio speed of speeds to a
- special purpose program, e.g. at 1/20 th the
- 11 Summary
- Conclusion: Between us and human-level in-
- telligence lie many problems. They can be
common sense knowledge of the worldMany important aspects of what thisknowledge is in and how it can berepresented are still unsolved questions.This is particularly true of knowledge ofthe eﬀects of actions and other events.epistemologically adequate languagesare
expressingwhat a person or robot can ac-the world15tually learn about
[McCarthy and Hayes, 1969].
elaboration tolerance What
personknows can be elaborated without startingall over.
anonmonotonic reasoning Perhaps new sys-tems are needed.
contexts as objects This subject is just be-ginning. See the references of section 7.introspection AI systems will need to exam-ine their own internal states.
action The present puzzles of formalizing ac-tion should admit a uniform solution.I doubt that a human-level intelligent programwill have structures corresponding to all theseentities and to the others that might have beenlisted. A generally intelligent logical programprobably needs only its monotonic and non-monotonic reasoning mechanisms plus mecha-nisms for entering and leaving contexts. Therest are handled by particular functions andpredicates.
15http://www-formal.stanford.edu/jmc/mcchay69.html- 12 Remarks and Acknowledgements
- 1. To what extent will all these problems
have to be faced explicitly by people
working with neural nets and connection-
ist systems? The systems I know about
are too primitive for the problems even to
arise. However, more ambitious systems
will inhabit the common sense informatic
situation. They will have to be elabora-
tion tolerant and will require some kind
of mental model of the consequences of
- 3. I got useful suggestions from Eyal Amir,
Saˇsa Buvaˇc and Tom Costello.
- 4. Some additional relevant papers are in my
book [McCarthy, 1990] and on my Web
- 5. My understanding that I should prepare a
printable version of this invited talk came
rather late. I expect that both the spoken
version and the 1996 November Web ver-
sion will have better explanations of the
- 6. This work was partly supported by ARPA
(ONR) grant N00014-94-1-0775.
- 13 Conclusion
- Many will ﬁnd dismayingly large the list of
- tasks that must be accomplished in order to
- to reach human-level logical intelligence. Per-
- haps fewer but more powerful ideas would sim-
- plify the list. Others will claim that a sys-
- tem that evolves intelligence as life does will
- be more straightforward to build. Maybe, but
- the advocates of that approach have been at it
- as long as we have and still aren’t even close.
- So it’s a race.
It will be much more scientiﬁcally satisfying tounderstand human level artiﬁcial intelligencelogically than just achieve it by a computer-ized evolutionary process that produced an in-telligent but incomprehensible result. In fact,the logical approach would be worth pursuingeven if the intellectually lazy evolutionary ap-proach won the race.
[Buvaˇc, 1996] Buvaˇc, S. (1996). Quantiﬁca-In Proceedings oftional logic of context.
the Thirteenth National Conference on Ar-tiﬁcial Intelligence.
[Buvaˇc et al., 1995] Buvaˇc, S., Buvaˇc, V., andMason, I. A. (1995). Metamathematics ofcontexts. Fundamenta Informaticae, 23(3).[David Poole and Goebel, 1998] David Poole,A. M. and Goebel, R. (1998). Computa-tional Intelligence. Oxford.
[McCarthy, 1959] McCarthy, J. (1959). Pro-grams with Common Sense17.
In Mecha-nisation of Thought Processes, Proceedingsof the Symposium of the National PhysicsLaboratory, pages 77–84, London, U.K. HerMajesty’s Stationery Oﬃce. Reprinted inMcC90.
[McCarthy, 1979] McCarthy, J. (1979). As-cribing mental qualities to machines18.InRingle, M., editor, Philosophical Perspec-tives in Artiﬁcial Intelligence. HarvesterPress. Reprinted in [McCarthy, 1990].[McCarthy, 1986] McCarthy, J. (1986). Ap-plications of Circumscription to Formaliz-ing Common Sense Knowledge19. Artiﬁ-cial Intelligence, 28:89–116. Reprinted in[McCarthy, 1990].
17http://www-formal.stanford.edu/jmc/mcc59.html18http://www-formal.stanford.edu/jmc/ascribing.html19http://www-formal.stanford.edu/jmc/applications.html[Newell and Simon, 1972] Newell, A. and Si-mon, H. A. (1972). Human Problem Solving.Prentice–Hall, Englewood Cliﬀs, NJ.[Turing, 1950] Turing, A. (1950). Computingmachinery and intelligence. Mind.[Turing, 1947] Turing, A. M. (1947).Lec-ture to the london mathematical society.In The Collected Works of A. M. Tur-ing, volume Mechanical Intelligence. North-Holland. This was apparently the ﬁrst pub-lic introduction of AI, typescript in theKing’s College archive, the book is 1992.- [McCarthy, 1989] McCarthy, J. (1989). Ar-
- tiﬁcial Intelligence, Logic and Formalizing
- Common Sense20.
In Thomason, R., edi-
- tor, Philosophical Logic and Artiﬁcial Intel-
- ligence. Kl¨uver Academic.
- [McCarthy, 1990] McCarthy, J. (1990). For-
- malization of common sense, papers by John
- McCarthy edited by V. Lifschitz. Ablex.
- [McCarthy, 1993] McCarthy,
Available on http://www-
- Notes on Formalizing Context21.
- [McCarthy, 1995] McCarthy, J. (1995). Situa-
- tion Calculus with Concurrent Events and
- Narrative22. Contents subject to change.
- Reference will remain.
- [McCarthy, 1996] McCarthy, J. (1996). Mak-
- ing Robots Conscious of
- States23. In Muggleton, S., editor, Machine
- Intelligence 15. Oxford University Press.
- [McCarthy and Hayes, 1969] McCarthy,
- and Hayes, P. J. (1969). Some Philosophical
- Problems from the Standpoint of Artiﬁcial
In Meltzer, B. and Michie,
- D., editors, Machine Intelligence 4, pages
- 463–502. Edinburgh University Press.
- [Newell, 1981] Newell, A. (1981). The knowl-
- edge level. AI Magazine, 2(2):1–20. Origi-
- nally delivered as the Presidential Address,
- American Association for Artiﬁcial Intelli-
- gence, AAAI80, Stanford, CA, August 1980.
- [Newell, 1993] Newell, A. (1993). Reﬂections
- on the knowledge level. Artiﬁcial Intelli-
- gence, 59(1-2):31–38.