Combining Search an Analogical Reasoning in Pah Planning from Roa Maps Karen Haigh School of Compuer Science Carnegie Mellon Universiy Pisburgh PA 15213 fkhaigh,mmvg@cs.cmu.eu Manuela Veloso Absrac Pah planning from roa maps is a ask ha may involve muliple goal ineracions an muliple ways of achieving a goal. This problem is recognize as a icul problem solving ask. In his omain i is paricularly ineresing o explore learning echniques ha can improve he problem solver's eciency boh a plan generaion an plan execuion. We wan o suy he problem from wo paricular novel angles: ha of real execuion in an auonomous vehicle (insea of simulae execuion); an ha of inerspersing execuion an replanning as an aiional learning experience. This paper presens he iniial work owars his goal, namely he inegraion of analogical reasoning wih problem solving when applie o he omain of pah planning from large real maps. We show how he complexiy of pah planning is relae o muliple ways of achieving he goals. We review he case represenaion an escribe how hese cases are reuse in pah planning where we inerleave a breah-rs problem solving search echnique wih analogical case replay. Finally, we show empirical resuls using a real roa map. Inroucion The moivaion an long-erm goal of his work is o inegrae planning, real execuion, an learning by analogy in he omain of raversing roa maps o achieve This research was sponsore by he Avionics Laboraory, Wrigh Research an Developmen Cener, Aeronauical Sysems Division (AFSC), U. S. Air Force, Wrigh- Paerson AFB, OH 45433-6543 uner Conrac F33615-90-C-1465, Arpa Orer No. 7597. The rs auhor is also suppore in par by he Naural Sciences an Engineering Research Council of Canaa. The views an conclusions conaine in his ocumen are hose of he auhors an shoul no be inerpree as represening he ocial policies, eiher expresse or implie, of he U.S. Governmen or of he Canaian Governmen. muliple goals. Proigy, a planner an learner [Carbonell e al., 1990b], will be inegrae wih an auonomous navigaion vehicle which will execue he plans o achieve muliple goals while riving in a ciy. The planner will be given a roa map, a se of goals, an he iniial locaion of he vehicle. I will generae a near-opimal roue ha achieves all he goals. We inen o use NavLab [Thorpe, 1990], an auonomous vehicle riven by a se of neural neworks o execue he plan. NavLab will combine low-level percepion wih he high-level reasoning of he plan which will guie i in making more complex ecisions such as which way o urn a inersecions in orer o achieve he goals. Real execuion of he plan may lea o failures of planne seps, such as a blocke roa. The vehicle will ransmi his informaion o he planner for replanning an learning. Several researchers invesigae he problem of inerleaving planning an execuion [Hammon e al., 1990, Agre an Chapman, 1987, McDermo, 1978]. In his work we wan o suy he problem from wo paricular angles: ha of real execuion in an auonomous vehicle (insea of simulae execuion), an ha of inerspersing execuion an replanning as an aiional learning experience. We envision breaking he represenaion gap beween a high level reasoning planner an a vehicle in he real-worl execuing he plan. We repor on preliminary work owars reaching his moivaing scenario. This paper focuses on he evelopmen of a robus planning an learning sysem where we accumulae a library of cases as planning episoes o guie he iniial planning as well as any replanning neee a execuion ime. Pah planning for muliple goals involves a large search space wih a large se of alernaive ways o achieve each iniviual goal an many possible goal ineracions. We iniially invesigae he issues of esigning he omain using real roa maps. Then we iscuss how analogical reasoning applies o problem solving in his omain an show empirical resuls on he inegraion wih eph-rs an breah-rs search an iscuss our on going implemenaion of bes-rs search. We explore in paricular how case reuse aecs
he planning ime, since i is very imporan o reuce he search space, especially when replanning uring execuion. Pah Planning from Roa Maps Inroucion The problem examine in his paper is how o n a pah in a map when here are muliple goals. Goals may consis of moving o ieren locaions, geing orers, an/or elivering packages. Pah planning in graphs has been aresse by a variey of algorihms, such as Dijksra's shores pah algorihm [Aho e al., 1974]. However, since our goal is o implemen his in auonomous vehicles, we wan o be able o inerleave pah planning wih execuion. Pahs will have o be moie an alere uring riving (because of eours, for example) an, herefore, we nee a meho which is more exible han a shores pah algorihm, an where we can reuse previous experience as in [Goel e al., 1992]. Furhermore, our framework wih real roa maps iverges from he more general framework of pah planning in arbirary graphs. In fac, real roa maps are no saic since hey will have minor emporary variaions which will only be known a execuion ime. The pah planning problem is also characerize by an exremely large number of alernaive ways of reaching arge esinaions, many of which will be equivalen from a isance poin of view. The emphasis of his work is herefore on learning from experience in a real environmen raher han a simulae one. We use analogical reasoning o enable he planner o accumulae an reuse is planning experience. Finally, our pah planning process is no riven exclusively by ning he pah wih he minimum isance beween locaions. Our aim is raher o n accepable soluions o muliple goals which can be achieve in several alernaive ways, a problem reucible o he Hamilonian circui an herefore NPhar. Reusing previous experiences will reuce his complexiy. The Domain Represenaion When esigning a omain represenaion, we consiere how he represenaion woul aec he ulimae goal: using his planner in he auonomous vehicle. We nee a represenaion ha aequaely escribes one way srees, isances beween he iniial posiion an he esinaion, an he irecion of urns ha will nee o be mae. Unirece graphs o no suce since one way srees can no be represene, an alhough irece graphs woul be able o hanle his problem, here is no easy way o represen urning irecion. We are herefore using a sysem which explicily connecs one ciy block wih he nex one, hereby allowing us o sore all his require informaion. The srees are herefore ivie ino muliple segmens separae by inersecions. We recenly foun access o a eaile aabase of he complee Pisburgh sree map wih more han 20,000 sree segmens. The aabase inclues he spaial coorinaes of he inersecions which we will use o ene he urning irecion beween sree segmens [Bruegge e al., 1992]. As an example, in our curren simplie represenaion, he preicae (connece Sree1 Sree2 Disance Turning-irecion) encoes sae informaion abou he map. The omain is encoe as a series of operaors ha escribe possible acions. Currenly we have operaors ha move agens ou of an ino builings, an move agens beween srees. For example, he (goo-ajacen-sree <sree1> <sree2> <isance> ) operaor moves he agen beween wo ajacen srees, namely from <sree1> o <sree2>. The preconiions of his operaor require ha he wo srees be connece, an ha he agen is a <sree1>. I also calculaes he oal isance ravelle by aing he new isance o he curren oal. Sanar Means-En Analysis Search Because of he reasons escribe in he inroucion of previous secion, we use he proigy planning an learning sysem [Carbonell e al., 1990a]. Proigy's nonlinear planner uses means-ens analysis in is backwar-chaining search proceure which can reason abou muliple goals an muliple alernaive operaors relevan o he goals. This choice of operaors amouns o muliple ways of rying o achieve he same goal. The planning reasoning cycle involves several ecision poins, namely: which goal o selec from he se of pening goals an subgoals; which operaor o choose o achieve a paricular goal; which binings o choose o insaniae he chosen operaor; an wheher o apply an operaor whose preconiions are saise or o coninue subgoaling on a ye unachieve goal. Dynamic goal selecion from he se of pening goals enables he planner o inerleave plans, exploiing common subgoals an aressing issues of resource conenion. The planner reurns a parially orere plan as a resul of analyzing he epenencies among he seps in he oally orere soluion foun while planning. In a ypical roa map, each sree-secion is connece o a mos eigh oher sree-secions. This branching facor varies beween wo as he lowes value (i.e. a ea-en sree) an eigh as he highes (veway inersecions a boh ens). The branching facor in he map use for our experimens ranges beween wo an six, an averages abou 4.4. Therefore for a problem of ravelling n ciy blocks, search complexiy is loosely boune above by approximaely 8 n. This number is reuce by he fac ha he average branching facor is lower, an also by elim-
n P n i=0 4:4i P n i=0 8i 5 2134 3745 10 3:52 10 6 1:23 10 9 15 5:80 10 9 4:02 10 13 Table 1: Complexiy of search space (in number of noes expane) for ravelling n ciy blocks inaing goal loops, for example <goo-ajacen-sree x y > followe by <goo-ajacen-sree y x > [Carbonell e al., 1992]. For even relaively small values of n, however, reaching a soluion by sraigh-forwar breah-rs search is an exremely slow an eious process (see Table 1). Case Reuse Combine wih Search Given he complexiy escribe above, we feel ha applying analogy an case-base reasoning in he conex of map pah planning is highly appropriae an even necessary given he ime resrains require when inerleaving planning wih execuion. We can reuse cases in his conex because one soluion pah will ofen be a subpah of anoher problem. If he smaller problem has alreay been solve, we can hen reuse i an signicanly reuce he amoun of search necessary o n he new soluion. Case Represenaion In he original soluion pah, proigy ha o make various ecisions abou which pahs o follow. Of all he noes generae while solving a problem, only he ones on he soluion pah are sore o creae a case. Exraneous noes are iscare. Each relevan ecision from he original soluion (i.e. each noe in he soluion pah) an is jusicaion is sore in Lisp forma in orer o make reloaing he case in a proigyreaable forma very easy. Figure 1 conains an example of how a goal noe woul be sore. Essenially, i mainains all poiners o relae noes in he search ree (which operaors inrouce i, any oher applicable operaors lef, remaining goals). A complee case conains noes of a similar forma escribing ecisions an jusicaions for ecisions mae a operaor noes, bining noes an applie operaor noes. (sef (p4::nexus-chilren (n-noe 4)) (lis (p4::make-goal-noe :name 5 :paren (n-noe 4) :goal (p4::insaniae-conse-lieral '(AT BARTLETT-2 JANE )) :inroucing-operaors (lis (n-noe 4) )))) Figure 1: A skech of a goal noe in a case. Noe ha a case is no use as a simple \macrooperaor" [Fikes an Nilsson, 1971]. A case is selece base on a parial mach o a new problem solving siuaion. Hence, as oppose o a macro-operaor, a case guies an oes no icae he reconsrucion process. In aiion, inermeiae ecisions corresponing o choices inernal o each case can be bypasse or aape if heir jusicaions no longer hol. Case Reusage We follow he case reuse sraegy as evelope in [Veloso, 1992]. The replay echnique involves a closely couple ineracion beween planning using he omain heory, omain operaors, an similar cases which are erivaional races of boh successful an faile ecisions in pas planning episoes. The replay mechanism involves a reinerpreaion of he ecision jusicaions in he conex of he new problem, reusing pas ecisions when he jusicaions hol rue, an replanning using he omain heory when he ransfer fails. Once one (or more) case is is foun ha is similar o he new problem solving siuaion, i is reay o be reuse. The planner is calle an given he he se of operaors an he similar case as inpu. The replay algorihm is implemene by inerruping he planning algorihm a is ecision poins so ha i may make choices similar o he ones from he guiing case. Unil here is a mach beween a subgoal of he case an one of he caniae goals of he new problem, proigy oes breah-rs-search o maximize he chance ha a mach will be foun wih minimum eph. As soon as his mach has been foun, proigy immeiaely follows he case using eph-rs-search, an oes no expan he res of he noes a he same level as he mache noe. Once case noes for which similariy jusicaions hol have been exhause, proigy reurns o breah-rs-search unil is main goal sae has been achieve (see Figure 2). This meho allows us no only o minimize aiional searching, bu also o solve problems in which neiher he goal sae nor he iniial sae are he same as he original case. Noe ha, opimaliy of pahs is no necessarily preserve by analogical ransfer. The merging of opimal subplans uner a saiscing approach may resul in a non-opimal new plan. When here are muliple operaors o achieve goals, here is no known echnique ha boh ries o maximize he reuse of previous experience an also maximize he qualiy of he new similar soluion. We plan o invesigae an exploraion echnique ha allows unrie or unjusie seps in he new conex o be searche, iverging from he irec reuse of he pas experience. This exploraory search can be conuce when he planner is no oherwise occupie. I shoul be noe however, ha we are no explicily concerne wih always ning an opimal soluion, bu raher wih ning a reasonable soluion.
H \ HHHH \\ A A A A A A r r Noe no mache Noe mache in case Goal Noe Reusing a case of eph 4, a new problem of eph 6 is reuce o P 3 i=0 4:4i + 3 + 4:4 = 32 noes from abou 7250 noes Figure 2: Proigy's noe expansion ree when problem solving using a case Experimens using a Real Map The Map The map we use is ha of Pisburgh's Squirrel Hill isric, ajacen o Carnegie Mellon Universiy an ypical of a ciy environmen in which he auonomous vehicle migh be use. Besies he resienial housing, here is a small shopping srip in he area, a universiy an a golf course, giving he vehicle a reasonably large region in which o run errans. The region conains wo one-way srees (Darlingon-5 an 6, an Barle- 4 an 5) an several ea-en srees. This iversiy, combine wih he fory-eigh inersecions, causes he search space o be highly complex. Figure 3 shows a graphical version of he curren represenaion of he map. Resuls The experimens we ran were consruce by exening a base case involving wo operaors. There were eigh composable problems buil aroun each base case: exening he base case by one operaor (sree) a he iniial poin, exening i by wo operaors a he iniial poin, an hen each of hose hree cases were exene by one an wo operaors a he goal. Base cases were ranomly chosen from he map of Squirrel Hill, an he composable problems were manually selece for heir proximiy o he base case. We rs ran each problem wih a breah-rs search wih no analogy unil he soluion was foun, an hen ran he same problem using eph-rs search wih he minimal eph-boun necessary o n a soluion. Once hese wo runs were complee, we ran he same problem using analogy wih all he cases ha forme sub-pahs o he soluion. We expece ha he oal number P of noes expane woul be reuce from i=0 4:4iP where = S eph of he search, o a number Orer of i=0 4:4i + C + P E j=1 4:4j where S = he number of operaors ae a he sar, C = number of noes in he case's soluion pah, E = number of operaors by which he case was exene a he en, an S + C +(E?1) =. Breah-rs search an eph-rs search behave as expece. Breah-rs search for a soluion involving six operaors require beween 6,600 an 30,000 noes, averaging abou 15,000. The number of noes expane in a eph-rs search range from ning he goal in he minimum number of noes possible, o ning he goal wih nearly he same number of noes as he equivalen breah-rs search. In all he problems solve by reusing cases, he number of noes expane was reuce as rasically as expece. Figure 4 shows a graph of our resuls, where `1S' (2S) represen problems buil upon a case which require aing one (wo) <goo-ajacen-sree> operaors a he Sar an `1E' (2E) represen problems buil upon a case which require aing one (wo) operaors a he En in orer o reach a soluion. Number of noes expane 16384 4096 1024 256 64 16 BFS 2E plus 0S, 1S an 2S 1E plus 0S, 1S an 2S 2S 1S Minimum 4 1 2 3 4 5 6 Number of operaions in soluion pah Figure 4: Problem Solving using BFS as compare o case reusage. Noe ha he numbers are four imes greaer han he number of operaors because proigy creaes four noes per operaor. The way proigy applies operaors in is meansens analysis search accouns for he asymmery beween aing one (wo) sep a he beginning of a case an aing one (wo) a he en. A he beginning of he search, proigy will sop expaning noes as soon as i ns a mach. This siuaion explains why
Forbes 7 Shay 22 Shay 23 Shay 24 Shay 25 Murray 8 Darlingon 6 Barle 5 Beacon 6 Murray 9 Murray 10 Murray 11 Hobar 7 * Wenover 1 ** Wenover 2 Murock 3 Tech Frew 1 Frew 2 Forbes 8 Forbes 9 Wighman 9 Darlingon 4 Darlingon 5 Wighman 10 Barle 3 Barle 4 W 11 Raleigh W 12 B 2 B 3 B 4 B 5 Munhall Wighman 13 Hobar 6 * GK2 ** H 4 H 5 Wighman 14 Hempsea Covoe Murock 4 M 5 M 6 M 7 M 8 Murock 9 Schenley 6 Schenley 7 Schenley 8 Forbes 10 Darlingon C D 1 D 2 Eas Circui 1 Darlingon 3 EC2 EC3 Wes Circui Serpenine Barle 2 Beacon 1 Barle 1 Guarino-Kamin 1 Hobar 3 H 2 Hobar 1 Schenley 5 Schenley 4 Prospec Greenfiel R Panher Hollow Overlook Schenley 3 Panher Hollow Brige Figure 3: The Map: Pisburgh's Squirrel Hill
`1S' cases so closely approximae he minimum number of noes possible o n a soluion. Once he case has been exhause, proigy coninues wih breah- rs-search. Even if i can apply an operaor on one branch of he search, i may no ye have reache he nal goal. Meanwhile, he oher branches of he search ree will sill have subgoals o expan, an hereby creae more noes a he en of he problem han a he sar. The number of noes expane a he en of he case so ominae he number of noes expane a he beginning ha we combine all he `1E' cases an all he `2E' cases for he purposes of simplifying he graph. We also ran a few experimens in which he case use i no form a proper sub-pah of he opimal soluion of he new problem, generaing a soluion ha was non-opimal by one or wo operaors. The number of noes expane by proigy was approximaely ouble ha of he proper sub-pah problems, bu even he mos icul were solve in less han one hunre noes; several orers of magniue less han he equivalen breah-rs-search. Use of analogy insea of breah-rs-search resule in a reucion in compuaion ime from several hours o uner a minue for longer problems. This fac inicaes ha his sysem will be usable in he real-ime environmen of inerleaving planning an execuion. Noice ha he represenaion use an he experimens run are of reuce complexiy. In his iniial phase we focuse on eveloping an valiaing he basic framework. Renemens an exensions of he approach will resul from he inegraion wih he real auonomous vehicle. Discussion an Fuure Work The paper presens our iniial accomplishmens owars having a planner ecienly plan pahs in a real roa map. We implemene a real map of a consierably large par of Pisburgh's Squirrel Hill isric. Case reusage an analogical reasoning in pah planning wih roa maps is compaible wih human inuiion since no only is he roa map he same in each problem an planning siuaions similar, bu ning soluions requires a lo of compuaion an search. We have shown in his paper ha reusing cases in his conex is feasible an ecien. We are currenly eveloping a large case library an organizing i for ecien rerieval [Doorenbos an Veloso, 1993]. We are using spaial feaures of he maps for case inexaion. We are exening he cases in he library by planning wih more operaors, aing muliple goals, an generaing alernaive plans. Seconly, since plan qualiy migh no be preserve by analogical reasoning wih exension an aapaions of problems, we plan o evelop an exploraory moe wihin he sysem. This aiion will allow us o no only ensure ha opimal soluions are foun for problems solve using analogy, bu also o sore muliple opimal soluion pahs for one problem. In his omain we will also be invesigaing he merging of previous soluions so ha problems can be solve using more han one case, eiher by irecly linking several pahs, or by merging subgoals [Veloso, 1993]. Inroucing absracion planning o his omain is also in our research agena. We will exen he re-use an generalizaion of cases o ieren levels of absracion, for example highway movemen as oppose o major srees as oppose o minor srees. Anoher possible meho of absracing his kin of problem is by moving beween gri squares (i.e. A-3 o J-6). Since absracion has alreay been implemene wihin proigy [Knoblock, 1991], we envision a smooh inegraion. Finally, our immeiae focus is o connec he planner o he auonomous vehicle an se up he appropriae communicaion framework. Acknowlegemens The auhors woul like o hank Rober Driskill, Eugene Fink, an Bob Doorenbos for commens an suggesions on his paper as well as he whole proigy research group for helpful iscussions. References Agre, Phillip an Chapman, Davi 1987. Pengi: An implemenaion of a heory of aciviy. In Proceeings of he Sixh Naional Conference on Aricial Inelligence, San Maeo, CA. Morgan Kaufmann. 268{272. Aho, A. V.; Hopcrof, J. E.; an Ullman, J. D. 1974. The Design an Analysis of Compuer Algorihms. Aison-Wesley, Reaing, Massachuses. Bruegge, Bern; Blyhe, Jim; Jackson, Je; an Shufel, Je 1992. Objec-oriene sysem moeling wih om. In Proceeings of he OOPSLA '92 Conference. ACM Press. 359{376. Carbonell, Jaime G.; Gil, Yolana; Joseph, Rober; Knoblock, Craig A.; Minon, Seven; an Veloso, Manuela M. 1990a. Designing an inegrae archiecure: The proigy view. In Proceeings of he Twelfh Annual Conference of he Cogniive Science Sociey, MIT, MA. 997{1004. Carbonell, Jaime G.; Knoblock, Craig A.; an Minon, Seven 1990b. Proigy: An inegrae archiecure for planning an learning. In VanLehn, K., eior 1990b, Archiecures for Inelligence. Erlbaum, Hillsale, NJ. Also Technical Repor CMU-CS-89-189. Carbonell, Jaime G.; ; an PRODIGY Research Group, he 1992. PRODIGY4.0: The manual an uorial. Technical Repor CMU-CS-92-150, School of Compuer Science, Carnegie Mellon Universiy.
Doorenbos, Rober B. an Veloso, Manuela M. 1993. Knowlege organizaion an he uiliy problem. Technical Repor forhcoming, School of Compuer Science, Carnegie Mellon Universiy. Fikes, Richar E. an Nilsson, Nils J. 1971. Srips: A new approach o he applicaion of heorem proving o problem solving. Aricial Inelligence 2:189{208. Goel, Ashok; Donnelan, Michael; Vazquez, Nancy; an Callanine, To 1992. An inegrae experiencebase approach o navigaional pah planning for auonomous mobile robos. In Working noes of he AAAI Fall Symposium on Applicaions of Ari- cial Inelligence o Real-Worl Auonomous Mobile Robos, Cambrige, MA. 50{61. Hammon, Krisian; Converse, Timohy; an Marin, Charles 1990. Inegraing planning an acing in a case-base framework. In Proceeings of he Eighh Naional Conference on Aricial Inelligence, San Maeo, CA. Morgan Kaufmann. 292{297. Knoblock, Craig A. 1991. Auomaically Generaing Absracions for Problem Solving. Ph.D. Disseraion, School of Compuer Science, Carnegie Mellon Universiy, Pisburgh, PA. Available as echnical repor CMU-CS-91-120. McDermo, Drew 1978. Planning an acing. Cogniive Science 2. Thorpe, Charles E., eior 1990. The CMU Navlab. Kluwer Acaemic Publishers, Boson, MA. Veloso, Manuela M. 1992. Learning by Analogical Reasoning in General Problem Solving. Ph.D. Disseraion, School of Compuer Science, Carnegie Mellon Universiy, Pisburgh, PA. Available as echnical repor CMU-CS-92-174. Veloso, Manuela M. 1993. Planning for complex asks: Replay an merging of muliple simple plans. In Preprins of he AAAI 1993 Spring Symposium Series, Workshop on Founaions of Auomaic Planning: The Classical Approach an Beyon, Sanfor Universiy, CA.