Infotech@Aerospace 2011 29-31 March 2011, St. Louis, Missouri AIAA 2011-1510 Intelligent Algorithms for Maze Exploration and Exploitation Sydney Barker 1 and Dr. Kelly Cohen. 2 University of Cincinnati, Cincinnati, OH, 45221 Cody Lafountain 3 University of Cincinnati, Cincinnati, OH, 45221, United States FIS The purpose of the project is to develop maze exploration algorithms for a multi-agent system, using autonomous robots, that allows the agents to successfully navigate through an array of different mazes based on the game Theseus and the Minotaur. Theseus and the Minotaur is a maze game in which Theseus tries to get to the exit of each maze without being eaten by the Minotaur. For every one move Theseus makes, the Minotaur can make two. The mazes become progressively harder as each maze is completed. A single intelligence system is made up of algorithms for the robots to use in order successfully simulate the Theseus and the Minotaur game. One of the robots will represent Theseus and the other robot will represent the Minotaur. The Theseus robot will work to traverse the maze while avoiding the Minotaur. The Minotaur robot will work to navigate through the maze in order to catch the Theseus robot. The goal is to have the robots simulate the Theseus and the Minotaur game without any human interaction. The next step is to validate the developed algorithms in a laboratory experiment using programmable mobile robots. = Fuzzy Intelligent System Nomenclature I. Introduction AZES have been of interest to humans for many years and have been used throughout history for scientific M research in which the use of small laboratory animals for testing is possible. Recently mazes have started to be used to research the artificial intelligence of robots by examining the robots ability to traverse unfamiliar mazes. Maze exploration algorithms have been studied since the mid-nineteenth century and are related to the Graph Theory. Maze exploration algorithms and Graph Theory are used in mathematics and computer science and study mathematical structures to model pair-wise relations between objects from a particular collection. Maze exploration algorithms are usually applied to a single agent, most commonly a robot, traversing a maze, but recently the focus of research is applying Maze exploration algorithms to multi-agent systems. The idea is to have the systems be used autonomously in various situations to help humans, such as searching and working in hazardous or poisonous environments (example: nuclear plants or waste sites), navigating through burning buildings, monotonous and tedius tasks (example: carrying loads around a plant or warehouse), operations where the risk and cost of sending humans are too high (example: voyages to other planets), and national defense operations. This research will focus on maze algorithms for multi-agent systems, namely autonomous robots, using the Fuzzy Logic approach. Fuzzy Logic is reasoning with fuzzy sets and allows a system to simultaneously handle numeric data as well as linguistic knowledge. Fuzzy Logic deals with complex, nonlinear systems and uses decision making with estimated values with incomplete, uncertain information. Fuzzy Logic Systems include inputs, outputs, membership functions, and rules. Inputs and outputs are variables or concepts that are a part of a problem. Each input has a set of membership functions that describe the input and a degree of membership that weighs the importance of the input to the problem. The rules are set up into if-then statements. For example, if X is A then Y is B. X and Y are inputs 1 Student, Aerospace Engineering, 2705 Eden Avenue, Cincinnati, OH 45219, and AIAA Member Grade for third author. 2 Professor, Aerospace Engineering, 735 Rhodes, Cincinnati, OH 45221, and and AIAA Member Grade for third author. 3 Graduate Student, Aerospace Engineering, 221 E University Ave, Cincinnati, OH 45219, and AIAA Member Grade for third author. 1 Copyright 2011 by the, Inc. All rights reserved.
while A and B are membership functions. The rules can incorporate one of three possible operations: AND, OR, and NOT. Fuzzy Systems are asked a question or are given an input and the system uses all of its rules to some degree to get an array of outputs. The system takes the average of all the outputs using the centroid method in order to develop a single final output. The centroid method is a defuzzification method that finds the center of gravity of the area under the curve and outputs the value representing the center of gravity as the best answer. The objective of the research is to create a Fuzzy Intelligent System in MATLAB that can be implemented to an autonomous multi-agent system so that the multi-agent system (two robots) can autonomously traverse any maze. II. Methodology The research will begin by first playing the game Theseus and the Minotaur on the computer. A picture of the game simulation is shown below in Figure 1. Fig. 1 Theseus and the Minotaur Game Snap Shot The game is a series of 87 mazes and the objective of the game is to get Theseus safely to the exit of the maze in the least number of moves without being eaten by the Minotaur. Theseus can move one space at a time and for every one move Theseus makes, the Minotaur can make two moves. There are obstacles set up throughout the maze that Theseus can use to try and trap the Minotaur in order to proceed safely to the exit. The mazes get progressively harder as you complete each maze. Playing the game will reveal the Minotaur s tendencies, weaknesses, and predictability. While working through each maze, Theseus moves and the Minotaur s subsequent responses will be tracked. By tracking the Minotaur s responses, the tendencies and weaknesses of the Minotaur can be noted and strategies for traversing the maze can be developed. The expertise gained from playing the game will be used to formulate preliminary inputs, outputs, membership functions, and degrees of membership for the Theseus and the Minotaur mazes. From the inputs, outputs, and membership functions, a rule base for my fuzzy inference system can be formulated. To further refine the inputs, outputs, membership functions, and degrees of membership, an m- file in MATLAB will be written in order to create an interactive maze game resembling Theseus and the Minotaur. The interactive maze will prompt the user to choose where he or she wants Theseus to move next and the computer will calculate where the Minotaur s next two moves will be. At least ten to fifteen interactive mazes will be created in MATLAB. The interactive mazes will help to create an m-file in MATLAB to define the moves the Minotaur will make in response to Theseus moves and can be used for the FIS. Analyzing the interactive mazes will also help in refining the inputs, outputs, membership functions, and degrees of membership. Once all the inputs, outputs, and membership functions are completed, the rules can be completed. With the inputs, outputs, membership functions, degrees of membership, and rule base, the FIS will be created in MATLAB using the Fuzzy Toolbox function. A single m-file was created which contains the FIS created for Theseus and a reactionary heuristic for the Minotaur. The reactionary heuristic is the same as that used in the interactive games. Since the Minotaur s moves are predictable, a simple reactionary heuristic can be used rather than a Fuzzy Inference System. The m-file first prompts the user to choose a maze design in which he or she would like to work with. MATLAB presents the requested maze and plots the preliminary positions of Theseus and the Minotaur. The m-file runs the FIS to 2
determine where Theseus next move should be. Once Theseus move is determined and executed, the m-file then runs the reactionary heuristic to determine the Minotaur s two subsequent moves. The m-file is set-up in a WHILE loop and continues to run the same loop until either Theseus has reached the exit or the Minotaur has caught Theseus. When Theseus reaches the exit, MATLAB presents the message You Win. If the Minotaur catches Theseus, MATLAB shows the message You Lose. The FIS will first be tested in MATLAB using an interactive simulation. The purpose of testing the FIS is to make sure that the FIS is fully functional and there are no problems with the components of the FIS as well as to check if any information is missing in the FIS. Once the fuzzy based decision making algorithm is tested and validated, it will be applied to the laboratory mobile robots (either Khepera III robots or LEGO Mindstorms NXT 2.0). A representative maze environment will be built in the lab for the robots to navigate in. One of the robots will play Theseus and the other robot will play the Minotaur. The Theseus robot will utilize the above algorithm to navigate through the maze in order to reach the exit of the maze without being caught by the Minotaur. The Minotaur robot will use a simpler reactionary heuristic to calculate its moves in response to Theseus moves. III. Results From playing Theseus and the Minotaur game, I noticed three main weaknesses that the Minotaur has. 1. Minotaur s tendency and preference to move horizontally first, then vertically. If there are no obstacles blocking the Minotaur from moving horizontally and moving vertically does not get the Minotaur directly closed to Theseus, then the Minotaur will always move horizontally. 2. Minotaur s inability to move around obstacles. 3. Minotaur s inability to look ahead more than one move and play strategically. The Minotaur will move in immediate response to Theseus rather than strategically moving to prevent Theseus from getting to the exit. I used the expertise gained from playing the game to formulate inputs, outputs, membership functions, and rules for the Theseus and the Minotaur mazes. My first set of inputs, outputs, membership functions and rules were very broad: therefore, the fuzzy system was very large and too complex. Some of the rules incorporated the same inputs and membership functions as well as contradicted each other. Changes had to be made to my system. I found that it would benefit my system more if I could come up with more specific inputs and a single output. Many of my inputs were similar and could be combined into one input. I also eliminated inputs that I believed could be excluded from the system. By trimming the number of inputs and outputs, the repetitive and unnecessary rules could be eliminated. Having fewer rules would make my system more efficient and less complex. My final set of inputs, outputs, and membership functions are displayed below in Figure 2. Inputs Theseus Position Theseus Position 2 Number of Spaces Wall Position Wall Position 2 Wall Orientation Number of Walls Outputs Theseus Action Membership Functions below, above, left, right below above, left, right one, multiple above, below, left, right above, below, left, right vertical, horizontal, vertical and horizontal one, multiple Membership Functions up, down, left, right, stay still Fig 2. Final Set of Fuzzy System Components From the inputs, outputs, and membership functions in Figure 2, I was able to formulate a sufficient rule base for my fuzzy inference system. The last step to creating a Fuzzy Inference System for Theseus and the Minotaur game was to implement my inputs, outputs, membership functions, and rules to the Fuzzy MATLAB program. In order to create a functional fuzzy inference system in MATLAB, I had to determine the degrees of membership for each membership function corresponding to each input and output. Fifteen interactive mazes were created and tested in MATLAB. One of the interactive maze designs is shown below in Figure 3. 3
4.5 4 3.5 3 2.5 2 1.5 1 0.5-0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 Fig 3. Interactive Maze Design in MATLAB Creating and analyzing the interactive mazes helped to refine the current FIS and make a more functional system. A new rule base was created using the expertise gained from creating the interactive mazes in MATLAB. The new inputs, membership functions, and degrees of membership are shown below in Appendix A and the output is shown in Appendix B. Examples of the fuzzy toolbox are shown in Figures 4, 5, and 6. Figure 4 is a picture of the Membership Function editor, Figure 5 is a picture of the rule editor, and Figure 6 is a picture of the FIS editor. Fig 4. Membership Function Editor of the Fuzzy Toolbox 4
Fig 5. Rule Editor of Fuzzy Toolbox Fig 6. FIS Editor of Fuzzy Toolbox An m-file was created in MATLAB that would run the FIS for each interactive maze design created. This stage of the research project is currently being finished. The m-file prompts the user to choose which maze design he or she wants to work with. The m-file implements the FIS to the desired maze design. The m-file is set up to calculate the value given to each input for Theseus current position and then calculates the output. Once the output is determined, the m-file moves Theseus to the calculated new position. After Theseus move has been made, the m- file then calculates the two subsequent moves that the Minotaur will make. The m-file is made up of a series of FOR and WHILE loops so that the m-file continues this process for every time Theseus moves until Theseus has either reached the exit and won or has been caught by the Minotaur. The single m-file and FIS can be applied to any maze design created in MATLAB. Currently work is being done to refine the m-file. There are some problems with repetitive input calculations and coding. Testing the FIS in MATLAB presented many problems with my Fuzzy Inference System. The FIS had too many rules in the rule base and as a result some of the rules over lapped each 5
other or contradicted each other. Due to overlapping and contradicting rules, the FIS was forced to arbitrarily choose one of the rules when calculating the output and as a result the output was not correct. Overlapping rules also cause the FIS to get confused about which rule to use and would present an error reading in MATLAB. Another problem with the FIS was that it was too complex. There were too many inputs to consider when calculating the output. In order to minimize the complexity of a single FIS, cascade learning was used. Cascade learning takes a large system and splits the system up into different sets and sub-sets. In the case of the FIS in this research project, in order to get the desired output (Theseus Move) multiple small, simple fuzzy inference systems were created and then implemented together to get the final output (Theseus Move). The first cascade tree model is shown in Figure 7. Fig 7. Cascade Learning Implementation to the Fuzzy Inference The model had three sets to make up the entire system. The first cascade model proved to be too simple of a maze algorithm to complete a full maze. The m-file was debugged in MATLAB in order to observe what rules in the FIS were overlapping or missing in the rule base. Small changes and additions were made to the FIS sets as well as to the m-file. The small changes that were made included changing and adding rules, changing the m-file script, changing membership functions, adding inputs, and changing the order of my FIS sets. Apart from making small changes, strategies needed to be developed for Theseus to use when he is close to the maze exit and when he is far from the exit. The two strategies were named the End Game strategy and the Minotaur Trap strategy. The end game strategy is used by Theseus when Theseus is close to the exit and the Minotaur is far from Theseus and the exit. The end game strategy is comprised of a simple heuristic that does not require fuzzy thinking, but a straight forward path to get to the exit. The Minotaur Trap strategy is used when Theseus and the Minotaur are close to the exit or Theseus is far from the exit. In both cases Theseus is required to lure the Minotaur from the exit and trap him so that Theseus can safely reach the exit. The second version of the cascade tree model is shown in Figure 8. 6
Fig 8. Second Cascade Tree Model for the Fuzzy Inference The second cascade model successfully solved six of the fifteen mazes in MATLAB. The cascade model did not work for more complex mazes due to overlapping rules and missing information. In order to come up with another FIS set to add to the cascade, the m-file had to be debugged again. The problems that were found during the debugging process were analyzed in order to see what information was missing in the cascade model. From the analysis, a new FIS set was created as well as an additional strategy. Three strategies were defined for Theseus to use while in pursuit of the exit: the End Game strategy, Minotaur Lure strategy, and Minotaur Trap strategy. The end game strategy and Minotaur Trap strategy were used in the same way as the second cascade model. The Minotaur Lure strategy was used when the Minotaur is close to the exit and Theseus is far away from the exit. In this case, Theseus must move towards the exit in a manner that will lure the Minotaur from the exit and into the middle of the maze. Once the Minotaur is lured away from the exit, then Theseus and implements the Minotaur Trap strategy to trap the Minotaur and then move towards the exit. The third cascade model is shown in Figure 9. 7
Fig. 9 Final Cascade Tree Model for the Fuzzy Inference System The third cascade model allowed for more complex maze designs to be traversed and solved by Theseus. The entire cascade tree created for the Intelligent Algorithm contained four cascades. Once the m-file for the FIS is fully functional in MATLAB, it will be validated in simulation and robustness will be gauged by applying to logic to 10 additional untrained cases followed by the above described laboratory experiment. Once the fuzzy based decision making algorithm is tested and validated, it will be applied to the laboratory mobile robots (either Khepera III robots or LEGO Mindstorms NXT 2.0). A representative maze environment will be built in the lab for the robots to navigate in. One of the robots will play Theseus and the other robot will play the Minotaur. The Theseus robot will utilize the above algorithm to navigate through the maze in order to reach the exit of the maze without being caught by the Minotaur. The Minotaur robot will use a simpler reactionary heuristic to calculate its moves in response to Theseus moves. IV. Conclusion Creating maze exploration algorithms using fuzzy logic as an approach will be useful in traversing mazes where there is no certain path to completing the maze. The maze can be completed using many different paths. In problems where the completion of the maze can be done in many different ways, there is always an optimal path to completing the maze. By implementing fuzzy logic to the exploration algorithms, the optimal path can be found. The Fuzzy Inference System will be given the maze design and will use the inputs, outputs, membership functions, and rules to calculate the best solution and outputs that solution to the multi-agent system: Theseus robot and Minotaur robot. The best method to creating an algorithm using fuzzy logic is through a cascade learning concept. By splitting the fuzzy inference system into smaller sets, the entire system is simplified and the output can be calculated quicker. After the robots successfully implement the exploration algorithms and autonomously traverse multiple maze designs, the next step is to have the robots collaborate to solve the mazes rather than work against each other. A intelligent algorithm would be implemented to both robots and the robots would communicate with other in order to solve the maze. The robots would be placed in different location of the maze and they would share information about their surroundings and failed or successful attempts. By sharing information, each robot can use its observations as well as the other robots in order to traverse the maze. Communication between the two robots can be done using Bluetooth. The collaboration between the two agents can be expanded to other applications. One example is with Unmanned Air Vehicles (called UAVs). UAVs are autonomous systems. UAVs could collaborate with each other when surveying an area or flying in swarms. 8
Appendix Inputs Membership Functions Degrees of Membership Theseus Position Above Minotaur 0-180 Below Minotaur 180-360 Left of Minotaur 90-270 Right of Minotaur 0-90, 270-360 Distance from the Minotaur Close 0-1.4 Far 1.4-11 Wall 1 Position for Theseus Left of Theseus 90-270 Above Theseus 0-180 Below Theseus 180-360 Right of Theseus 0-90, 270-360 Wall Orientation Vertical 0 Horizontal 1 Wall 2 Position for Theseus Left of Theseus 90-270 Right of Theseus 0-90, 270-360 Above Theseus 0-180 Below Theseus 180-360 Wall Orientation Vertical 0 Horizontal 1 Wall Position 1 for Theseus Left of Theseus 90-270 Above Theseus 0-180 Below Theseus 180-360 Right of Theseus 0-90, 270-360 Wall Orientation Vertical 0 Horizontal 1 Wall Distance from Minotaur Close Less than 2 Far Greater than 2 Wall Position 2 from Minotaur Above Theseus 0-180 Right of Theseus 0-90, 270-360 Left of Theseus Below Theseus Wall Orientation Vertical 0 Horizontal 1 Wall Distance from Minotaur Close Less than 2 Far Greater than 2 APPENDIX A. Revised Inputs, Membership Functions, and Degrees of Membership 9
Output Membership Functions Degrees of Membership Theseus Move Up 0 Down 0.25 Left 0.5 Right 0.75 Stay 1 APPENDIX B. Revised Output, Membership Functions, and Degrees of Membership Acknowledgments The author of this paper would like to extend thanks to Dr. Kelly Cohen, Aerospace Professor at the University of Cincinnati and Project Advisor. Dr. Cohen provided his lab, needed materials, reference reading materials, and guidance for the current work and future work going into the research project. The author would also like to extend thanks to Chelsea Sabo, an Aerospace Graduate student at the University of Cincinnati. Chelsea served as the graduate assistant to the research project. Chelsea helped the Fuzzy Inference System get started and provided input and guidance throughout the research period. The author would like to thank Pablo Mora for help in writing the code for the interactive mazes. Thanks are extended to Cody Lafountain who helped correct the programming mistakes in MATLAB. Thanks are given to the McNair Scholars program and the Ohio Space Grant Consortium for grant that aided in developing the research project. References 1 Butler, Charles, and Caudill, Maureen, Naturally Intelligent Systems, The MIT Press, Cambridge, Mass, 2000, Chaps. 2-5. 2 Dixon, K. R., Khosla, P. R., and Malak, R. J., Incorporating Prior Knowledge and Previously Learned Information into Reinforcement Learning Agents, Institute for Complex Engineered Systems Technical Report Series, 31 Jan. 2000. 3 Huser, J., Peters, L., and Surmann, H., A Fuzzy System for Indoor Mobile Robot Navigation, Fourth IEEE International Conference on Fuzzy Systems, FUZZ-IEEE 95, IEEE, 1995, pp. 83-88. 4 Ishikawa, S., A Method of Autonomous Mobile Robot Navigation by Using Fuzzy Control, Advanced Robotics, 9 th ed., 1995, pp. 29-52. 5 Kosko, B., Fuzzy Thinking: The New Science of Fuzzy Logic, Hyperion, New York, 1993. 10