Scientific Theory in Informatics A1N

Scientific Theory in Informatics A1N Lecture 05 Automata Theory and Computability Theory Prof. David Vernon School of Informatics University of Skövde david.vernon@his.se Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 1 Lecture Overview Automata Theory and Languages Regular Languages» Finite Automata» Nondeterminism» Regular Expressions» Nonregular Languages Context-free Languages» Context-free Grammars» Pushdown Automata» Deterministic Context-Free Languages Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 2

Lecture Overview Computability Theory The Church-Turing Thesis» Turing Machines» Variants of Turing Machines» The Definition of Algorithm Decidability» Decidable Languages» Undecidability Reducibility Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 3 Motivation Theory of Computation is normally divided in three parts 1. Automata Theory and Languages 2. Computability Theory 3. Complexity Theory See: Introduction to the Theory of Computation, Michael Sipser, 3 rd edition, Cengage Learning, 2013 Chapters 1, 2 (Automata Theory) Chapters 3, 4, 5 (Computability Theory) Chapter 7, 9 (Complexity Theory) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 4

Motivation Why study the Theory of Computation? Determine what can and cannot be computed How quickly & with how much memory On what type of computational model Theory guides practice New application-specific programming language grammars String searching and pattern matching finite automata and regular expressions Programs take too long to run complexity analysis and algorithmic strategies Security and cryptography NP-completeness Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 5 Motivation Complexity Theory Easy problems (sort a million items in a few seconds) Hard problems (schedule a thousand classes in a hundred years) What makes some problems hard and others easy (computationally)? Complexity Theory addresses this question Surprise: we don t know the answer! Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 6

Motivation Computability Theory In the first half of the 20 th century, mathematicians such as Kurt Gödel, Alan Turing, and Alonzo Church discovered that certain basic problems cannot be solved by computers» e.g. determine whether a mathematical statement is true or false Complexity Theory: classify problems as easy ones and hard ones Computability Theory: classify problems are solvable and not solvable Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 7 Motivation Automata Theory Deals with the definitions and properties of mathematical models of computation Finite automaton (used in text processing, compilers, hardware design) Context-free grammar (used in programming languages and artificial intelligence) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 8

Automata Theory and Languages Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 9 Regular Languages Finite Automata Idealized (mathematical) model of a computer Simplest model:» Finite state machine» Finite automaton Extremely limited amount of memory Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 10

Regular Languages Finite Automata Example: controller for an automatic door» Front pad to detect presence of a person about to walk though» Rear pad to make sure it stays open long enough (and avoid hitting someone) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 11 Regular Languages Finite Automata Example: controller for an automatic door» Controller states: OPEN, CLOSED» Input conditions: FRONT, REAR, BOTH, NEITHER Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 12

Regular Languages Finite Automata Example: controller for an automatic door» Controller states: OPEN, CLOSED» Input conditions: FRONT, REAR, BOTH, NEITHER Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 13 Regular Languages Finite Automata Example: controller for an automatic door» Controller states: OPEN, CLOSED» Input conditions: FRONT, REAR, BOTH, NEITHER Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 14

Regular Languages Finite Automata Example: controller for an automatic door» Just one bit of memory (state: OPEN or CLOSED) Other examples» Elevator / lift controller (more states: one for each floor; inputs: lift buttons)» Dishwashers» Digital Thermostats» Washing machines Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 15 Regular Languages Finite Automata Finite automaton M 1 with three states State diagram Accept state q 2 Transitions: arrows Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 16

Regular Languages Finite Automata Automaton receives an input string, e.g. 1101 Output is either accept or reject» accept if in accept state at the end of the input string» reject otherwise Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 17 Regular Languages Finite Automata Automaton receives an input string, e.g. 1101 1. Start in state q 1 2. Read 1, follow transition from q 1 to q 2 3. Read 1, follow transition from q 2 to q 2 4. Read 0, follow transition from q 2 to q 3 5. Read 1, follow transition from q 3 to q 2 6. Accept because M 1 is in an accept state q 2 at the end of the input Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 18

Regular Languages Finite Automata Automaton receives an input string, e.g. 1101 M 1 accepts any string that ends in a 1, as it goes to its accept state whenever it reads the symbol 1 It accepts any string that ends in an even numbers of 0s following the last 1 It rejects all other strings Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 19 Regular Languages Finite Automata Formal definition of a finite automaton 1. The transition function specifies exactly one next state for each possible combination of a state and an input symbol Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 20

Regular Languages Finite Automata δ: Q Σ Q Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 21 Regular Languages Finite Automata If A is the set of all strings that a machine M accepts, we say that A is the language of machine M L(M) = A We say M recognizes A or M accepts A A machine may accept several strings but it always recognizes only one language L(M 1 ) = A A = {w w contains at least one 1 and an even number of 0s follow the last 1} Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 22

Regular Languages Finite Automata Finite automaton M 2 L(M 2 ) = A A = {w w ends in a 1} Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 23 Regular Languages Finite Automata Finite automaton M 3 L(M 3 ) = A A = {w w is the empty string ε or ends in a 0} Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 24

Regular Languages Finite Automata Finite automaton M 4 Two accept states q 1 and r 1 Operates over the alphabet Σ = {a, b} Accepts all strings that start and end with the same symbol Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 25 Regular Languages Finite Automata Formal definition of computation Let M = (Q, Σ, δ, q 0, F) be a finite automaton Let w = w 1 w 2 w n be a string where each w i is a member of the alphabet Σ M accepts w if a sequence of states r 0, r 1,, r n in Q exists with three conditions r 0 = q 0 machine starts in the start state δ (r i, w i+1 ) = r i+1, for i = 0,, n-1 goes from state to state r n F accepts input if end up in accept state Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 26

Regular Languages Finite Automata We say M recognizes language A if A = {w M accepts w} A language is called a regular language if some finite automaton recognizes it Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 27 Regular Languages Finite Automata Given a language, design a finite automaton to recognize it» Receive an input string, one symbol at a time» After each symbol, decide whether the string seen so far is in the language» Figure out what you need to remember about the string as you are reading it» Can t remember everything (even the whole string: FINITE automaton, FINITE number of states, FINITE memory)» Remember only crucial information (depends of the language) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 28

Regular Languages Finite Automata Given a language, design a finite automaton to recognize it Example: the language of strings with an odd number of 1s Alphabet Σ = {0, 1} Language L(M) = A, A = {w w contains an odd number of 1s} What do we need to remember? Entire string? no Number of 1s? no Whether number of 1s so far is even or odd? yes Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 29 Regular Languages Finite Automata Given a language, design a finite automaton to recognize it Example: the language of strings with an odd number of 1s State q o even so far State q 1 odd so far Finish this: Which is the accept state? What is the transition function δ? Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 30

Regular Languages Finite Automata Given a language, design a finite automaton to recognize it Example: the language of all strings that contain 001 as a substring. Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 31 Regular Languages Finite Automata The regular operations Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 32

Regular Languages Finite Automata The regular operations Union: Take all the strings in A and B and put them in one language Concatenation: Attach a string from A in front of a string from B in all possible ways to produce the strings in the new language Star: unary operator attach any number of strings in A together to get a string in the new language Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 33 Regular Languages Finite Automata The regular operations Let alphabet Σ = {a, b,, z} Let language A = {good, bad} and B = {boy, girl} A B = {good, bad, boy, girl} A B = {goodboy, goodgirl, badboy, badgirl} A * = {ε, good, bad, goodgood, goodbad, badgood, badbad, goodgoodgood, goodgoodbad, goodbadgood, goodbadbad, } The collection of regular languages is closed under all three of the regular operations Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 34

Regular Languages Nondeterminism Deterministic computation:» When a machine is in a give state and reads the next input symbol, we know what the next state will be Nondeterministic computation:» Several choices may exist for the next state DFA: deterministic finite automata NFA: nondeterministic finite automata Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 35 Regular Languages Nondeterminism DFA: deterministic finite automata» Exactly one exiting transition arrow for each symbol in the alphabet NFA: nondeterministic finite automata» Zero, one, or many exiting arrows for each alphabet symbol» Transition may also be labelled ε zero, one, or many arrows may exit from a state with the label ε Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 36

Regular Languages Nondeterminism How does an NFA compute?» If you read a symbol for which there is more than one transition (arrow)» The machine splits into multiple copies of itself and follows all of the possibilities in parallel» If there a subsequent choices, the machine splits again» If the next input symbol doesn t appear on any of the arrows exiting the state occupied by a copy of the machine, that copy dies» If ANY one of these copies of the machine is in an accept state at the end of the input, the NFA accepts the input string. Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 37 Regular Languages Nondeterminism How does an NFA compute?» If a state with an ε is encountered, without reading any input, the machine splits into multiple copies, one following the ε labelled arrows, and one staying at the current state Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 38

Regular Languages Nondeterminism How does an NFA compute?» Nondeterminism is a kind of parallel computation, with multiple independent processes/threads running concurrently» If at least one of the processes/ threads accepts, the entire computation accepts Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 39 Regular Languages Nondeterminism Computation for input 010110 Hint: where does q 3 come from? Split in q 2 because of ε labelled arrow Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 40

Regular Languages Nondeterminism Every NFA can be converted into an equivalent DFA Constructing NFAs is sometimes easier than constructing DFAs For example, DFA and NFA to recognize language A consisting of all strings over {0,1} containing a 1 in the third position from the end Accept 000100; reject 0011 Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 41 Regular Languages Nondeterminism Formal definition of an NFA Power set: set of all subsets of Q Σ ε = Σ {ε} Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 42

Regular Languages Nondeterminism δ: Q Σ ε P(Q) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 43 Regular Languages Regular Expressions In arithmetic, we can use operations to build expressions (5 + 3) x 4 Here we use regular operations to build regular expressions (0 1)0* [(0 1) 0* ] The value of a regular expression is a language (the set of all strings generated by that expression) In this example, the language consisting of all strings starting with a 0 or 1, followed by any number of 0s Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 44

Regular Languages Regular Expressions Another regular expression (0 1)*» The value of this expression is the language consisting of all possible strings of 0s and1s If Σ = {0, 1} we can write Σ as shorthand for the regular expression (0 1) More generally, if Σ is any alphabet,» The regular expression Σ describes the language consisting of all strings of length 1 over this alphabet» The regular expression Σ* describes the language consisting of all strings over that alphabet Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 45 Regular Languages Regular Expressions Important in computer science applications Searching for text patters that satisfy certain patterns Describe the patterns using regular expressions Tools such as awk and grep in UNIX, languages such as Perl, and most text editors all provide mechanisms for description of patterns by using regular expressions Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 46

Regular Languages Regular Expressions We distinguish between a regular expression R and the language L it describes: L (R) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 47 Regular Languages Regular Expressions Regular expressions and finite automata are equivalent in their descriptive power Any regular expression can be converted into a finite automaton that recognizes the language it describes (and vice versa) Recall: a regular language is one that is recognized by some finite automaton A language is regular if and only if some regular expression describes it Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 48

Regular Languages Regular Expressions Some languages cannot be recognized with a finite automaton (DFA or NFA) B = {0 n 1 n n 0} (We can t remember the number of 0s using a machine with finite memory, i.e. a finite number of states) There are nonregular languages Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 49 Regular Languages http://en.wikipedia.org/wiki/theory_of_computation Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 50

Context-free Languages Context-free Grammars Can describe situations which have a recursive structure Important application: programming languages» Specification» Compilation (Interpretation, Recognition) Parser: extracts (recognizes) the meaning of a program prior to generating the compiled code or performing the interpretation Given a grammar, it s straightforward to create a parser» Can even do it automatically with a software tool Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 51 Context-free Languages Context-free Grammars Any language that can be generated with a context-free grammars is called a context-free language (CFL)» They include all regular languages Push-down automata» Class of machines that recognize context-free languages Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 52

Context-free Languages Context-free Grammars Example: G 1 Substitution rules / productions Start variable A 0A1 A B B # Variable (upper case) Terminal (lower case) Sequence of substitutions is called a derivation A 0A1 00A11 000A111 000B111 000#111 Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 53 Context-free Languages Context-free Grammars Example: G 1 Parse tree for 000#111 in grammar G 1 Language of the grammar: L(G) 1 = {0 n #1 n n 0} Abbreviate multiple rules with the same left-hand variable A 0A1 B Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 54 or

Context-free Languages Context-free Grammars Formal definition Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 55 Context-free Languages Context-free Grammars If u, v, and w are strings of variables and terminals and A w is a production (i.e. rule of the grammar) we say uav yields uwv We say u derives v (written u * v) if u = v or if a sequence u 1, u 2,, u k exists for k 0 and u 1 u 2 u k v The language of the grammar is {w Σ* S * w} Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 56

Context-free Languages Context-free Grammars Example: Grammar G = (V, Σ, R, <EXPR>) V = {<EXPR>, <TERM>, <FACTOR>} Σ = {a, +,, (, ) } R: <EXPR> <EXPR> + <TERM> <TERM> <TERM> <TERM> <FACTOR> <FACTOR> <FACTOR> ( <EXPR> ) a This grammar can generate the strings a+a a and (a+a) a Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 57 Context-free Languages Context-free Grammars Example: Grammar G = (V, Σ, R, <EXPR>) Parse trees for a+a a and (a+a) a Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 58

Context-free Languages Push-down Automata (PDA) Another type of computational model Like nondeterministic finite automata but they have additional memory stack The stack allows push-down automata to recognize some non-regular languages Push-down automata is equivalent in power to context-free grammars Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 59 Context-free Languages Push-down Automata (PDA) Finite automaton Push-down automaton Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 60

Context-free Languages Push-down Automata (PDA) Stack operations» Push (down) a symbol onto the stack» Pop a symbol off the stack» Symbol popped is always the most recently pushed» Last in, first out (LIFO)» Has an unlimited amount of storage Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 61 Context-free Languages Push-down Automata (PDA) Recall that a finite automaton cannot recognize L(G) = {0n 1 n n 0} because it cannot store large numbers of elements in a finite memory But a PDA can do this: Read symbols from input Every time a 0 is read, push it on the stack Every time a 1 is read, pop a element off the stack If reading is finished and stack is empty, accept the input (expression) Otherwise, reject the input (expression): stack is empty while 1s remain; 1s are finished and stack is empty; 0s appear following 1s Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 62

Context-free Languages Push-down Automata (PDA) PDA can be deterministic or nondeterministic Noneterministic PDA are more powerful than deterministic PDA (different situation to DFA and NFA) Nondeterministic PDA can recognize languages that deterministic PDAs cannot Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 63 Context-free Languages Push-down Automata (PDA) Nondeterministic. Power set: set of all subsets of Q Γ ε Σ ε = Σ {ε} Transition function: current state, next input symbol read, top symbol on stack determine the next move of PDA (i.e. some combination of new state and stack operation) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 64

Context-free Languages Push-down Automata (PDA) L(M 1 ) = {0n 1 n n 0} Let M 1 be (Q, Σ, Γ, δ, q 1, F} Q = {q 1, q 2, q 3, q 4 } Σ = {0, 1} Γ = {0, $} F = (q 1, q 4 } δ is given by: Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 65 Context-free Languages Push-down Automata (PDA) a, b c When reading a from input, replace b on stack with c If a is ε, make the transition without reading any input If b is ε, make the transition without popping from the stack If c is ε, don t push any symbol on the stack Read 0, pop nothing, push 0 Read nothing, pop nothing, push $ Read 1, pop 0, push nothing Read1, pop 0, push nothing Read nothing, pop $, push nothing Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 66

Context-free Languages Push-down Automata (PDA) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 67 Context-free Languages Deterministic Context-Free Languages Deterministic push-down automata (DPDA) recognize deterministic context-free languages In contrast to DFA, we allow ε-moves in DPDA transition function» ε-input moves δ(q, ε, x)» ε-stack moves δ(q, a, ε) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 68

Context-free Languages Deterministic Context-Free Languages is not Ø Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 69 Context-free Languages Deterministic Context-Free Languages Major practical importance: DPDAs encompass a rich class of CFLs that include most programming languages DCFGs are sometimes inconvenient for expressing particular DCFL Alternative is to use a broader class of grammars LR(k) grammars Close enough to DCFGs to allow direct conversion to DPDAs Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 70

Automata and Computability Theory Computability Theory Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 71 The Church-Turing Thesis The story so far Finite automata: good models for devices with a small amount of memory Pushdown automata: good models for memory organized as a stack (last in, first out) Both are too restricted to serve as models of general purpose computers Enter Turing machines Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 72

The Church-Turing Thesis Turing machines Proposed by Alan Turing in 1936 Similar to finite automaton but has unlimited and unrestricted memory Can do anything a general purpose computer can do But cannot solve some problems (and, so, neither can computers) o Beyond the limits of theoretical computation Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 73 The Church-Turing Thesis Turing machines Infinite tape: unlimited memory Tape head (read and write symbols to the tape, move left and right) Tape only contains input string initially; blank everywhere else Computes until it decides to produce an output Output accept if it enters Accepting state; Output reject if it enters Rejecting state If it doesn t enter accepting or rejecting state, it loops forever, never halting Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 74

The Church-Turing Thesis Turing machines Turing machine M 1 for testing membership in the language B = {w#w w {0, 1}* } i.e. two identical strings separated by a # M 1 should accept if its input is a member of B, and reject otherwise M 1 = On input string w: o Move the tape head back and forth across the tape to corresponding squares on either side of the # to check if they contain the same symbol. o If they don t match or if there is no #, reject o Write an x every time a square is checked o When all symbols to the left of the # have been crossed off (with an x), check for any remaining symbols to the right of the # o If any symbols remain, reject; otherwise accept Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 75 The Church-Turing Thesis Turing machines Turing machine M 1 for testing membership in the language B = {w#w w {0, 1}* } two identical strings separated by a # Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 76

The Church-Turing Thesis Turing machines Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 77 The Church-Turing Thesis Turing machines Transition function δ: Q Γ Q Γ {L, R} When the Turing machine is in a certain state q The head is over the tape square containing the symbol a If δ(q, a) = (r, b, L) the machine writes the symbol b replacing the a goes to state r then head moves Left or Right one square, depending on L or R (after writing b) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 78

The Church-Turing Thesis Turing machines A Turing machine M = (Q, Σ, Γ, δ, q o, q accept, q reject ) computes as follows:» Initially M receives its input w = w 1 w 2 w n Σ* On the leftmost n squares on the tape The rest of the tape is filled with blank symbols The first blank marks the end of the input (why?)» The head starts at the leftmost square of the tape» Compute by following the rules specified by the transition function» If M tries to move its head to the left off the left-hand end of the tape, the head stays in the same place for that move» The computation continues until it enters either the accept or reject states, at which point it halts» Otherwise, M goes on forever (loops) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 79 The Church-Turing Thesis Turing machines Turing machine configuration: current state, current contents of memory (tape), current head location we write u q v to mean the tape contents comprises the two strings u and v and the current head location is the first symbol of v 1011q 7 01111 Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 80

The Church-Turing Thesis Turing machines A Turing machine M = (Q, Σ, Γ, δ, q o, q accept, q reject ) computes as follows:» Configuration C 1 yields configuration C 2 if the Turing machine can legally go from C 1 to C 2 in a single step» Suppose we have a, b, and c in Γ u and v in Γ* states q i and q j If in the transition function δ(q i, b) = (q j, c, L) LEFT We say that ua q i bv yields u q j acv Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 81 The Church-Turing Thesis Turing machines A Turing machine M = (Q, Σ, Γ, δ, q o, q accept, q reject ) computes as follows:» Configuration C 1 yields configuration C 2 if the Turing machine can legally go from C 1 to C 2 in a single step» Suppose we have a, b, and c in Γ u and v in Γ* states q i and q j If in the transition function δ(q i, b) = (q j, c, R) RIGHT We say that ua q i bv yields uac q j v Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 82

The Church-Turing Thesis Turing machines A Turing machine M = (Q, Σ, Γ, δ, q o, q accept, q reject ) computes as follows:» Start configuration of M on input w: q 0 w» Accepting configuration: q accept» Rejecting configuration: q reject Halting configuration» A Turing machine M accepts input w if a sequence of configurations C 1, C 2, C k exists, where C 1 is the start configuration of M on input w each C i yields C i+1 C k is an accepting configuration Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 83 The Church-Turing Thesis Turing machines The collection of strings that M accepts is the language of M or the language recognized by M A language is called Turing-recognizable if some Turing machine recognizes it (also called a recursively enumerable language) A Turing machine that halts on all inputs, i.e. a machine that never loops, is called a decider (because it always makes a decision to accept or reject) A language is called Turing-decidable or simply decidable if some Turing machine decides it (also called a recursive language) Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 84

The Church-Turing Thesis Turing machines A language is recognizable iff there is a Turing Machine which will halt and accept only the strings in that language; for strings not in the language, the TM either rejects, or does not halt at all. A language is decidable iff there is a Turing Machine which will accept strings in the language and reject strings not in the language. A recognizer of a language is a machine that recognizes that language A decider of a language is a machine that decides that language» Both halt in the accept state on strings that are in the language» A decider also halts (rejects) if the string is not in the language» A recognizer MAY or MAY NOT halt on strings that are not in the language Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 85 The Church-Turing Thesis Turing machines Turing machine M 1 = (Q, Σ, Γ, δ, q o, q accept, q reject ) for testing membership in the language B = {w#w w {0, 1}* } Q = {q 1 q 8, q accept, q reject } Σ = {0, 1, #} Γ = {0, 1, #, x, } The start, accept, and reject states are q 1, q accept, q reject δ is given by the state diagram Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 86

The Church-Turing Thesis Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 87 The Church-Turing Thesis Variants of Turing machines Alternative definitions nondeterminism & multi-tape machines but all have the same power (i.e. recognize the same class of languages)» Every multitape Turing machine has an equivalent single-tape Turing machine» Every nondeterministic Turing machine has an equivalent deterministic Turing machine Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 88

The Church-Turing Thesis The Definition of Algorithm Intuitive notion: collection of simple instructions for carrying out some task Not formally defined until 1936: Alonzo Church and Alan Turing» Church used the λ-calculus (Lambda calculus) to define algorithms» Turing used his machines» The two definitions are equivalent» Church-Turing thesis: the intuitive notion = Turing machine algorithms Turing machine serves as a precise model for the definition of an algorithm Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 89 The Church-Turing Thesis The Definition of Algorithm The input to a Turing machine is always a string If we want to process other type of objects, represent them as a string» The string representation of an object O is written O» The string representation of several objects O 1, O 2, O k is written O 1, O 2, O k Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 90

Decidability Decidable Languages We represent computational problems by languages For example, acceptance problem for DFAs» Test whether a particular DFA accepts a given string can be expressed as a language A DFA» A DFA contains the encodings of all DFAs together with the strings the DFA accepts A DFA = { B, w B is a DFA that accepts string w}» The problem of testing whether a DFA B accepts an input w is the same as testing whether B, w is a member of the language A DFA Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 91 Decidability Decidable Languages We formulate problems in terms of testing membership in a language Showing that a language is decidable is the same as showing that the computational problem is decidable A DFA is a decidable language A NFA is a decidable language A CFG is a decidable language Every context-free language is decidable Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 92

Decidability Undecidability Some problems are algorithmically unsolvable Turing machines (& thus all computers) are limited in a fundamental way What types of problems? ordinary problems e.g. Given a computer program and a precise specification of what that program should do, verify that it performs as specified (i.e. that it is correct) This general problem of software verification is not solvable by computer Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 93 Decidability Undecidability A TM = { M, w M is a Turing machine and M accepts w} A TM is undecidable i.e. can t determine whether or not a TM accepts a given input Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 94

Decidability Undecidability Note A TM is is Turing-recognizable The Turing machine U recognizes A TM U = On input B, w, where M is a TM and w is a string: 1. Simulate M on input w 2. If M ever enters its accept state, accept; If M ever enters its reject state, reject U: universal Turing machine Capable of simulating any other Turing machine based on a description of that machine This machine loops on B, w if M loops on w i.e. it does not decide A TM If the algorithm had some way to determine that M was not halting on w, it could reject. But there is no way to make this determination. Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 95 Decidability Undecidability: The Halting Problem HALT TM = { M, w M is a Turing machine and M halts on input w} HALT TM is undecidable i.e. can t determine whether or not a TM halts (by accepting or rejecting) a given input Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 96

Reducibility Reduction A reduction is a way of converting one problem to another problem so that the second problem can be used to solve the first problem If problem A reduces to problem B, we can use a solution of B to solve A» You can reduce the problem of finding your way around a city to the problem of obtaining a map of the city» You can reduce the problem of travelling from Skövde to Stockholm to the problem of buying a train ticket between the two cities. That problem in turn reduces to earning the money for the ticket. That problem reduces to the problem of finding a job Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 97 Reducibility Reduction If problem A reduces to problem B, we can use a solution of B to solve A» The problem of measuring the area of a rectangle reduces to the problem of measuring the length and width» The problem of solving a set of linear equations reduces to the problem of inverting a matrix Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 98

Reducibility Reduction When A is reducible to B, solving A cannot be harder than solving B, because a solution to B gives a solution to A In computability theory» If A is reducible to B, and B is decidable, A is also decidable» If A is undecidable, and A reduces to B, B is undecidable In complexity theory» all NP problems reduce to NP-complete problems» If an NP-complete problem is in P, all NP problems are» If an NP-complete problem is intractable, all NP problems are Scientific Theory in Informatics Lecture 05: Automata Theory and Computability Theory Slide 99