Context Free Grammars and Languages

Transcription

1 Chapter Nine Context Free Grammars and Languages 9.1 INTRODUCTION This chapter introduces the Context-Free Grammars (CFG) and the corresponding languages called Context-Free Languages (CFL). The context free languages and grammars are more powerful than the regular expressions and regular languages discussed previously. The CFGs have applications for description and recognition of high-level languages, parsing, language translation, generating and recognition of context-free languages, string processing, etc. Context-free grammars have been extensively used for describing the syntax of programming languages and natural languages. The parsing algorithms for context-free grammars play a large role in the implementation of compilers and interpreters for programming languages, and in programs that understand or translate natural language. Considering natural language, e.g., English, which is understood and spoken by human beings, we become the language processor for English. Given the sentences, black cat on the brown table, and table brown the on cat black, we are able to understand the meaning of first sentence, where as not of the second. That way we become the language recognizer of the English language. Because, depending on whether it is constructed as per the rules of English grammar or not, we accept or reject an English language sentence. Based on some situation, when we speak English language sentences that are built-up as per the rules of English grammar, we become the language generator. Therefore, a language, whether a natural language such as English or a programming language such as FORTRAN or C, in a abstract sense can be considered to be a set of sentences. One criteria for the syntactic specification of a language is that, invariably, a finite representation is required for an infinite class of sentences (here we consider sentence, word, phrase at the same level).

2 192 Fundamentals of Theory of Computation In one approach, a finite generative device, called grammar, is used to describe a syntactic structure of a language. Another approach is to specify a device or algorithm for recognizing well formed sentences, where language consist all the sentences recognized by the device or algorithm. A third possible method for the specification of a language is to specify a set of properties and then consider a language to be a set of words obeying these properties. In the case of regular languages discussed earlier, the regular expressions were generators of these languages, and the finite automata are recognizer of these languages. Consider a regular expression a * (b * +c * )b. The algorithm for construction of all the possible strings based on this regular expression can be specified in following steps: (1) Write character a, zero or any arbitrary number of times, (2) Write character b or c zero or arbitrary number of times, (3) Write b at the end. If this algorithm executes infinitely large number of times and the string produced every time is compared with all the previously constructed strings, if it matches any of them, the current string is dropped otherwise it is retained. The set of all these retained strings is language generated by this regular expression. To recognize any of such string, we feed this string into the corresponding finite automata. If it leads to a final state on reading last character b then the string has been recognized by the finite automata, else not. Definition 9.1 A recognizer is an algorithm, which takes an input string and either accepts or rejects it depending on whether or not the string is a sentence of the grammar. Definition 9.2 A parser is a recognizer, which also outputs the set of all legal derivation trees of the string sentence. The CFGs and CFLs are fundamental to computer science, because they help describe structure underlying programming languages. The basic signature of

3 Chapter 9. Context Free Grammars and Languages 193 a CFL is nested brackets. For example, nesting occurs in expressions and in many statically scoped structures in computer programs. In contrast, the basic structure of regular languages is iteration (looping) according to some ultimate periodicity. Some examples are as follows. EXAMPLE 9.1 Represent the C language productions for balanced braces. Solution The productions are: L-C main paren braces paren () braces {braces} 9.2 CONTEXT-FREE GRAMMARS AND LAN- GUAGES The context-free languages are powerful and more complex than the regular languages discussed earlier. The generators of context-free languages are called context-free grammars, and they are based on the more complete understanding of the structure of the strings accepted by the context free languages. To understand the above process, let us consider the regular expression a * (b * +c * )b again, and divide it into three parts : A for the start, which corresponds to a *, M for middle corresponding to b * +c *, and b is last part. Let the symbol S be start symbol for construction of any string in the language corresponding to a * (b * +c * )b, the symbol stands for can be replaced by. Then we write, (i) S AMb which means that S can be replaced by the sequence of symbols A, M and b. Symbol A consists alphabet a null or more number of times. Therefore, (ii) A

4 194 Fundamentals of Theory of Computation (iii) A aa In the rule (ii) above, the symbol stands for empty string, which shows that non-terminal symbol A can be substituted by empty string (or call it null string). The symbol M consists of an alphabet a any number of times, including null, or b any number of times including null. This can be specified in two stages, first for M, then its constituent parts. (iv) (v) M B M C Next, B and C can be specified in the similar way A was specified above. (vi) (vii) (viii) (ix) B B bb C C cc The equations (i) (ix) in the above are called the rules of the context-free grammar the generator of contest free language. These are also called substitution rules or production rules, because they guide a substitution process to arrive at the string, which belong to the context-free language. Making use of these rules, the language corresponding to regular expression a * (b * +c * )b can be defined by an alternate language generator specified in the form of following algorithm. Algorithm for generating regular language for the regular expression a * (b * +c * )b. (1) Start with symbol S as current string (2) Search a variable symbol in the current string (left to right) matching with some left hand side symbol of a production (say A). Replace A by sequence of symbols in the right hand side of the production rule.

5 Chapter 9. Context Free Grammars and Languages 195 (3) Repeat step (ii) until there is no symbol in the current string matching with any of the left hand side symbols of production rules. In the above, the current string is progressively constructed starting with S and it is modified through the process of substitutions. EXAMPLE 9.2 Find out whether the string aacccb is recognized as an element of the language corresponding to the regular expression a * (b * +c * )b. Solution. We generate aacccb using following steps: S AMb, by rule (i) aamb, by rule (iii) aaamb, by rule (iii) aamb, by rule (ii) aacb, by rule (v) aaccb, by rule (ix) aacccb, by rule (ix) aaccccb, by rule (ix) aacccb, by rule (viii) Since, we are able to generate the string of the language of regular expression a * (b * +c * )b, it confirms our claim of using context-free grammar as a language generator for a language called context free language. It must be noted that, first we designated the language corresponding to a * (b * +c * )b as regular language, and when it was shown to be generated by the CFG, it has been designated as CFL. This is true. It will be proved later that all the regular languages are subset of context free languages also. But, reverse is not true. During the process of generating string aacccb using CFG, the symbols S, M, A, B, C, a, b, and c have been used. The symbols a, b, c the final symbols in the generated string, are called terminal symbols or terminals of the CFG. The symbols S, M, A, B, C are encountered in the process of generation but

6 196 Fundamentals of Theory of Computation they are not the final symbols, are called Non-terminal symbols. Since the generation process started with the symbol S, it is called start symbol of the CFG, and rules (i) to (ix) are called production rules of the CFG. Let us consider an intermediate stage in the substitution process, and look at the current strings, say aacb and aacccb. The letter C present in these strings is in two different contexts. In the first case the substrings aa and b present before and after C respectively, represent context of C. In the second case the substrings aacc and b represent a different context of C. In both of these cases, C has been substituted by cc as per substitution rule (ix) C cc without any consideration for the contexts, i.e., substrings before and after C. Therefore, the substitution process in CFG is independent of the context. Hence the term context free grammar, and corresponding language as context free language. It should be noted that this feature is due to allowing only a single variable and no other symbol on the left side of the productions. The following part gives formal description of context free grammar. Definition 9.3. A context free grammar G is a quadruple G = (V,, S, P) (9.1) where, V is a finite set of variables called Non terminal symbols, is finite set of terminal symbols; V and are disjoint, i.e., V =, S is a special variable symbol, called the start symbol, P is finite set of production rules, where each production is of the form A, where A V and is a string of symbols defined as (V ) *. The language L(G), generated by G, is called CFL if G generates each string in L(G). Every regular grammar is context free and hence all the regular languages are context-free languages. It is already shown that regular language corresponding to the regular expression a * (b * +c * )b can be generated by context-free grammar. Later on it will be shown that language {a n b n n 0} is context

7 Chapter 9. Context Free Grammars and Languages 197 free language, however we know that the same is not regular. Therefore, we see that family of regular languages is proper subset of a family of context-free languages (figure 9.1). Context-free languages Regular languages Figure 9.1: Relation between Regular and Context-Free Languages. It can be easily verified from the above definition that following grammars are context free: G 1 = ({S},{a, b}, S, {S asa, S bsb, S }) G 2 = ({S, A, B}, {a, b}, {S abb, S aabb, B bbaa, A }) Apart from the symbols used in the definition of CFG, following other symbols are also used during the derivation process of language strings, discussed in the remaining part of this chapter. The uppercase English alphabets A, B, C, D, E, S represent variable symbols; these are members of V. The symbol S implicitly represents start symbol. The lowercase letters a, b, c, d, e and digits (i.e., 0 9) represent terminals. The uppercase letters X, Y, Z represent variables as well as terminals. The lowercase letters u, v, w, x, y, z represent the strings of terminals. The Greek alphabets,, represent strings as members of (V ) *.

8 198 Fundamentals of Theory of Computation 9.3 DERIVATIONS OF CONTEXT-FREE LAN- GUAGES For the CFG defined earlier, we now give a formal definition of corresponding context free language. This needs a notation and formal procedure for derivation or generation of language strings. Let A,, are symbols, where A V, and, (V ) *, and (A ) P. Using this rule, string can be substituted for A, or we say that A derives in grammar G. This can be expressed by A G (9.2) This is called direct derivation. Let us consider the strings, 1, 2,, n (V ) *, and n 1, such that 1 derives 2, 2 derives 3,, n-1 derive n, expressed by 1 G 2, 2 G 3,., n-1 G n The above process can also be expressed by 1 * G n (9.3) This states that 1 derives n in CFG G in zero or more number of steps. It should be noted that * G is a reflexive and transitive closure of G. Using above deductions, we can define L(G), the CFL generated by CFG G, as follows: L(G) = {w * S * G w} (9.4) If it is implicit that grammar is G, then 1 * G n is written as 1 * n. Also, in this derivation 1 derives n by application of (n 1) number of production rules from P. If derives by application of productions from P, exactly i number of times then it is expressed by i

9 Chapter 9. Context Free Grammars and Languages 199 If S drives in zero or more steps, i.e., S *, then the intermediate forms from S to are called sentential forms. Thus, in (9.3), 1, 2,, n-1 are sentential forms, provided that 1 = s, and n *. Given any two grammars G 1, G 2, they are treated equal if the languages generated by them are equal, i.e., L(G 1 ) = L(G 2 ). EXAMPLE 9.3 Let G be a context-free grammar defined by G = (V,, P, S), where, V = {S} = {a, b} and P = {S asb, S }. This grammar has only one variable symbol S, there are two terminal symbols a and b, and two production, S asb and S. A possible derivation can be S asb aasbb aaasbbb aaabbb. The above derivation can also be written as S * aaabbb. In the above, the sentential form aasbb has been obtained by applying the production rule S asb, and aaabbb has been obtained by applying the production rule S on the sentential form aaasbbb. If we continue the other derivations, it can be easily seen that the language for this CFG is L = L(G) = {a n b n n 0} (9.5) It is clear that L is not regular (see chapter 7), hence, it can be concluded that some CFL are not regular. Definition 9.4. A context-free grammar specified as G = (V,, S, P) is

10 200 Fundamentals of Theory of Computation regular grammar if and only if every production is out of the following forms only. A a A ab A where A, B V and a. EXAMPLE 9.4 Let there is a CFG G = (V,, S, P), where, V = {S, NP, VP, N}, NP stand for Noun Phrase, VP is Verb Phrase, and N is Noun, = {man, dog, bites, likes}, P = {S NP VP, NP N, VP V NP, N man, N dog, V likes, V bites}. Find out various derivations using this grammar. Solution The above CFG is description of a limited subset of natural language. In fact, the original motivation for context free grammars were description of natural languages. The format in this example is structuring of language by linguists, and it is close to the format followed by computer scientists, called BNF (Backus Naur Form) notation. Let us see generating of some sentences through G. S NP VP N VP man VP man V NP man likes NP man likes N man likes dog

11 Chapter 9. Context Free Grammars and Languages 201 Hence, S * man likes dog Similarly, some other sentences that can be generated through G are: dog likes man man bites dog dog bites man man likes man man bites man dog likes dog dog bites dog Though some of the sentences in the above are not accepted in a civilized world, but as per context-free grammar G defined above, they are valid. The CFG defined above can generate still more number of sentences. However, it is left an exercise for the reader to find out the remaining possible sentences. When computers are used to perform arithmetic computations, they must follow rigid mechanical procedures so that computation can be automated. These rigid criteria are expressed in the syntax of programming languages through the context-free grammar of the corresponding language. The property of being context-free helps in parsing the statements of high-level languages unambiguously. The parsing represents structure of the statements through which various computations are to be performed. For example, following rules can specify syntax of various arithmetic expressions: <expr> <expr> + <expr> <expr> <expr> - <expr> <expr> <expr> * <expr> <expr> <expr> / <expr> <expr> (<expr>) <expr> id This shows that <expr> in the left-hand side of each rule can be substituted by

12 202 Fundamentals of Theory of Computation the expression shown on the right hand side of the sign. This way we are able to generate the original expression governed by these rules. In addition, the relation between every sub-expression generated and its parent symbol indicates the order of computation and association of various sub-expressions using operators. EXAMPLE 9.5 Let G be a context-free grammar defined for generating and recognition of arithmetic expressions, specified as under. G = (V,, S, P) where V = {E}, (E stands for expression) = {+,, *, /, (, ), id} S = E, and P is as given below, (i) to (vi). (i) E E + E (ii) E E E (iii) E E / E (iv) E E * E (v) E (E) (vi) E id Find out various derivations using this grammar. Solution A string of an expression id * (id + id) / id id can be generated by using CFG as follows: Steps 1 E E E by (ii) 2 E * E E by (iv) 3 id * E E by (vi) 4 id * E / E E by (iii) 5 id * (E) / E E by (v) Parsing Computing 6 id * (E + E) / E E by (i) 7 id * (id + E) / E E by (vi) 8 id * (id + id) / E E by (vi) 9 id * (id + id) / id E by (vi) 10 id * (id + id) / id id by (vi)

13 Chapter 9. Context Free Grammars and Languages 203 The above derivation tells us many things. First, it shows that original expression id * (id + id) / id id is a syntactically correct expression, because we have been able to generate it using the production rules given in the grammar. Second, it shows association between various symbols and the order of their computation. As per those in step1 the value of first E (in E E) is computed as multiplication of E * E. This indicates the precedence of multiplication over the subtraction. In the move from step 3 to 4 it shows that the first E in step 3 is computed as E / E. After derivation is completed, computation follows in reverse direction. For example, (E) is step 5 is obtained by (E + E) operation in step 6. EXAMPLE 9.6 Show that the language L = {a m b n m n, and m, n 0} is context-free. As per example 9.2 for m = n, productions P can be specified as follows: S asb S Let us first assume that m > n. For this, a grammar similar to a n b n can be constructed and the character a can be added one or more times on the left of the sentential forms of the corresponding language L. Thus, productions for m > n can be given as S AE A aa a E aeb Similarly, for m < n, character b can be added one or more times to the right of the sentential form of L. Thus, productions for m < n are specified as S EB E aeb B bb b Combining the above two set of productions, we get the productions for the grammar of {a m b n m n, and m, n 0}.

14 204 Fundamentals of Theory of Computation S AE EB E aeb A aa a B bb b Since, all the productions are of the form A, where A V, and (V )* (definition 9.1), the language based on expression a m b n for m n is a CFL. 9.4 PARSING AND SYNTAX TREES The process of derivation of strings in CFG (Context-Free Grammar) can be represented using tree structures. These trees called derivation trees or parsetrees, and the process is called parsing. Compilers use parsing in translation of high-level language into machine or low-level language. The derivation process can be described using an algorithm. Following is informal description of the algorithm as a recognizer. It scans an input string X 1, X 2,, X n from left to right looking ahead some fixed number k of symbols. As each symbol X i is scanned, a set of states S i is constructed, which represents the condition of the recognition process at that point in the scan. Each state in this set represents: (1) a production, showing that we are currently scanning a portion of the input string, which is derived from its right, (2) a point in that production, which shows how much of the production s right side we have recognized so far, (3) a pointer back to the position in the input string at which we begin to look for that instance of the production, and (4) a k- symbol string, which is a syntactically allowed successor to that instance of the productions. A derivation tree represents a clear view of parsing process. In addition, it shows the structure of the strings generated in the form of various components of the string. A tree also shows the association among the various components parts of the strings generated.

15 Chapter 9. Context Free Grammars and Languages 205 Starting node of a tree represents the symbol S in CFG, internal nodes represent the Non-terminal symbols or variable symbols in V, and terminal nodes represent the alphabet of CFG or just the null string symbol. When a node is labeled with, this node is the only child of its parent. If an interior node A is root of a parse-tree and A X 1 X 2... X n is a production in CFG then X 1, X 2,..., X n will be child nodes of A. E E * E id ( E ) E + E id id Figure 9.2: Derivation tree for id * (id + id). If E E + E E E (E) id are productions in the CFG, where E stands for expression and id stands for identifier, then derivation tree for id (id + id) can be constructed as shown in figure 9.2. Scanning the leaf nodes of this tree from left to right produces the expression id (id + id). It should be noted that derivation process in the parse-tree is similar to the conventional derivation of strings in CFG. In addition, derivation through a parse-tree is a pictorial representation of parsing the process. EXAMPLE 9.7 Construct a derivation tree for the CFG

16 206 Fundamentals of Theory of Computation S asa a A bb Solution Using these productions we try to derive the string aaabbbb through steps 1 5 shown below. Step Sentential Form 1 S asa 2 aasaa 3 aaaaa 4 aaabba 5 aaabbbb Different steps for construction of derivation-tree for aaabbbb are shown in figure 9.3. STEP 1: S S asa a S A S STEP 2: a S A S * aasaa a S A

17 Chapter 9. Context Free Grammars and Languages 207 S STEP 3: a S A S * aaaaa a S A a STEP 4: S S * aaabba a S A a S A a b b STEP 5: S a S A S * aaabbbb a S A b b a b b Figure 9.3: Parsing of string aaabbbb.

18 208 Fundamentals of Theory of Computation We note that the derivation tree is a natural description of a particular sentential form of the grammar G. The current sentential form at each stage of construction of the derivation tree is obtained by reading the labels of leaf nodes of this tree from left to right. The sentential form at each stage is called the yield of the tree constructed so far. For example, in step 4 above, yield of this tree is aaabba and S * aaabba. We note that every application of production rule installs a sub-tree at one of the existing leaf (with variable label) of the parse-tree constructed so far. For instance, at step 4 above, a sub-tree has been installed under the existing leaf node A, with its root node A and child nodes as bb. A sub-tree with its root A is called A-tree, and therefore the entire derivation tree is called S-tree. Like derivations, a derivation tree in CFG is called leftmost derivation tree if every time, production rule is applied on the left most non-terminal node. Alternatively, it is rightmost derivation tree if every time production rule is applied on the right most non-terminal node of the tree constructed so far. However, the final tree construction in both the cases will be same. Definition 9.5. Let Q = (V,, S, P) be a CFG. Assume that its two derivations named D and D are as follows: D = 1 2 i n D = 1 2 i n where, 1 V, n * and i, i (V ) *, for i = 1... n. We note that D and D are derivations from single non-terminal symbol. We say a derivation D precedes D, written as D < D, if during the process of derivation the left most non-terminal in D is replaced before to that in D. EXAMPLE 9.8 Consider, that productions in grammar G are given as S SS (S)

19 Chapter 9. Context Free Grammars and Languages 209 The following three derivations generate the same string of balanced parenthesis. D 1 = S SS (S)S ((S))S (( ))S (( ))(S) (( ))( ) D 2 = S SS (S)S ((S))S ((S))(S) (( ))(S) (( ))( ) D 3 = S SS (S)S ((S))S ((S))(S) ((S))( ) (( ))( ) We note that that D 1 < D 2 and D 2 < D 3. S S S ( S ) ( S ) ( S ) Figure 9.4: Derivation tree for balanced brackets. The parse-tree for all the above derivations D 1, D 2 and D 3 is same as shown in figure 9.4. Definition 9.6. Any two derivations D and D are similar, if they are related by a relation of reflexive closure, symmetric closure and transitive closure on the relation of <. Therefore, D and D have equivalence relation between them. 9.5 LEFTMOST AND RIGHTMOST DERIVA- TIONS

20 210 Fundamentals of Theory of Computation Definition 9.7. For each generation tree T of * w, V, there is a derivation w 1 w 2 w r = w of w, whose generation tree is T, satisfying the following: for 0 r 1, if w i = u i i v i, w i+1 = u i y i v i, and i y i, then y i ( V) +, u i *, where * is free semigroup with identity, generated by. Such a derivation is called left-most derivation. In the above definition, if position of u i and v i are exchanged, and other part remains the same, derivation is called rightmost derivation. The result of both the derivations, however, are same. EXAMPLE 9.9 Let us consider the CFG G = {V,, S, P}, where V = {A, B, S}, = {a, b}, S is start symbol, and the production rules P are (i) S AB (ii) A (iii) A aa (iv) B (v) B bbb Find out various derivations. Solution Let us consider the derivation of string abb. The leftmost derivation is given as S AB by (i) aab by (iii) ab by (ii) abbb by (v) abb by (iv) Same string is derived using the rightmost derivation. S AB by (i)

21 Chapter 9. Context Free Grammars and Languages 211 AbbB Abb aabb abb by (v) by (iv) by (iii) by (ii) We note that result of derivation comes same in both the cases. In addition, various production rules applied are same, only the order of their use is different. 9.6 SYSNTAX DIRECTED TRANSLATION One of the major areas in which the theory of languages has an opportunity for practical impact is the formal specification of semantics. Ever since the syntax directed ALGOL was specified in BNF (Backus-Naur Form), it has been accepted that CFG is a reasonable and concise formalism for specification of most of the structure of programming language. One common technique in the construction of compilers is to specify this structure of the source language by means of a context-free grammar and then define its translation into the object language in terms of this grammar. Compilers utilizing this principle are called syntax directed. Certain constraint on the source language, such as proper declaration of identifiers or proper use of goto statements, cannot be included in the context-free specifications of the language. Therefore, many systems allow the use of symbol tables in the specification of the source language. Once the structure of a source program has been understood by a compiler, in terms of the context-free grammar and symbol tables (i.e., program has been parsed and tables constructed), the object code program can be constructed. The specifications of the object program is often made in terms of the parse-tree of the source program. One common formalism for describing translations on a context-free grammar is the Syntax Directed Translation (SDT). Here, associated with each production of the underline CFG is a rule for permuting the order of the nonterminals on the right side of the production and iteroducing output symbols on the right side.

22 212 Fundamentals of Theory of Computation 9.7 CONTEXT-FREE V/S REGULAR GRAMMARS It was shown in section 9.1 that regular expressions can be represented by context-free grammars. Also, it should be noted that the approach of representation of regular expression or regular language by CFG is a far simplistic approach, compared to FA. Let us consider the regular expression a * ba * b(a+b) * and its FA shown in figure 9.5, M = ({q 0, q 1, q 2 }, {a, b}, q 0,, {q 2 }). a a a,b q 0 b q 1 b q 2 Figure 9.5: FA for regular expression a * ba * b(a+b) * The transition funciton for this FA can be given as (q 0, a) = q 0, (q 1, b) = q 2 (q 0, b) = q 1, (q 2, a) = q 2 (q 1, a) = q 1, (q 2, b) = q 2. a a a,b S b A b B Figure 9.6: States in FA expressed by non-terminal of CFG. It can be easily verified that string aababab L(M).

23 Chapter 9. Context Free Grammars and Languages 213 Let us represent the states in the above FA by variables of CFG: S, A, B. The modified FA of figure 9.5 is shown in figure 9.6. Let us try to generate the string aababab using this FA. For this, we traverse through the states S, A, B and list the string progressively, as it gets constructed. S as aas aaba aabaa aababb aababab aabababb. Since S generates the final string aabababb, we can write S * aabababb. Let us terminate this process using the production B. Hence, S * aababab. Looking at this generation process, we find that the productions rules have been used in derivation from S to aababab. For example, to derive from S to as, production rule S as has been used. Similarly, to derive from aas to aaba, the production rule S ba has been used. All these productions are listed below. S as S ba A aa A bb B ab B bb B Each variable in the production rules of CFG correspond to a state in the FA. It may be noted that, these productions are precisely as per the definition of the regular grammar (definition 9.4). A correspondence exists between various transitions in the states and production rules. For example, the state transition, (S, a) = S corresponds to production rule S as (S, b) = A corresponds to production rule S ba, and (B, a) = B corresponds to production B ab In general we say, (p, a) = q corresponds to p aq

24 214 Fundamentals of Theory of Computation Also, for each final state f F, there is a production f. Every FA leads to CFG exactly in a similar way, and therefore a language generated by CFG for a regular expression is exactly the language recognized by the FA. Theorem 9.1. For any language L *, L is regular if and only if L is generated using some regular grammar G, that is, L = L(G). Proof. The proof involves two parts, the first is L is regular if L = L(G) for G as regular, and the second is only if part, for which it is to be proved that, If language L = L(G) is regular, then G be a regular grammar. (i) First let us suppose that G is regular. Given this we want to prove that L = L(G) is regular. For this let, M = (Q,, q 0,, F) be a FA which recognizes L. Let us define CFG G as follows: G = (V,, S, P) V = Q, S = q 0, P = {A aa (A, a) = A, and A A F} a 1 a 2 a m-2 a m-1 a m q 0 q 1 q 2 q m-2 q m-1 q m Figure 9.7: Sequence of transions for string a 1 a 2 a m to FA. Suppose a string x = a 1 a 2 a m is recognized by FA M, then there are sequence of transitions of the form shown in figure 9.7, and q m F. When states in FA represent variables in G, then as per the definition of G, we have the derivation: S = q 0 a 1 q 1 a 1 a 2 q 2. a 1 a 2 a m-1 a m q m a 1 a 2 a m-1 a m = x, for q m =.

25 Chapter 9. Context Free Grammars and Languages 215 Similarly, if every x is generated like this, then x L(G). And, since L is generated by FA, L is regular. (ii) In the second part of the proof, it is to be proved that if L = L(G) is regular, then G is regular. Let G = (V,, S, P). It is possible to reverse the construction just discussed above in (i). This means, given the variables set V, define the state set Q for every production A ab such that there is a transition, (A, a) = B The problem remains to be solved is how to handle the productions like A a and A. The solutions approach should be such that these productions can be made to lead to final state f F. Hence, production A a can be represented by (A, a) = f and A by (A, ) = f. Assume that there is only one final state f F. Looking to these transitions the FA M = (Q,, q 0,, F) will be an NFA with -transitions, which can be defined as follows: Q = V {f} q 0 = S F = {f} For any q V, and a, the element of (q, a) includes all those states q for which, (q aq ) P, where P is set of productions in G. Also, if (q a) P, then (q, a) = f F. For any q V, (q, ) = f if (q ) P, otherwise for q V, (q, ) =, if (q ) P. If x L, and L = L(G), the derivation for x can be as follows: S a 1 q 1 a 1 a 2 q 2 a 1 a 2... a m-1 q m-1 If there is sequence in the FA like one shown in figure 9.8(a), the last step will be, a 1 a 2 a m-1 q m-1 a 1 a 2 a m q m.

26 216 Fundamentals of Theory of Computation Alternatively, if there is sequence in FA like one shown in figure 9.8(b), the last step will be, a 1 a 2 a m-1 q m-1 a 1 a 2... a m-1 a 1 a 2 a m-1 a m f q 0 q 1 q 2 q m-1 Figure 9.8(a): FA to accept a 1 a 2 a m. a 1 a 2 a 3 a m-1 f q 0 q 1 q 2 q m-1 q m-1 Figure 9.8(a): FA to accept a 1 a 2 a m-1. In every case it follows that x L(M). On the other hand, if x = q 1 q 2 q m is accepted by the FA M, there is a sequence of transitions as specified by figure 9.5(a) or the one given in 9.5(b). Here each transition (q i-1, a i ) = q i comes from a production q i-1 a i q i P, where P is set of production in the grammar G. The last step is either a production q m-1 q m, where q m F, or q m F or q m. We conclude that if L = L(G), then FA exists which accepts L, and therefore, L is regular. This proves the theorem. Theorem 9.2. If L 1 and L 2 are context-free languages then L 1 L 2, L 1 L 2 and L 1 * are also context free languages. Proof. This can be proved, almost on the similar lines of regular and finite automata discussed earlier. To prove that L 1 L 2, L 1 L 2, and L * exists in the form of CFL, it is required to find out corresponding CFG for each of these cases. The proof is found out in three steps discussed below.

27 Chapter 9. Context Free Grammars and Languages 217 Assume the CFG corresponding to L 1, L 2 are G 1 = (V 1,, S 1, P 1 ) G 2 = (V 2,, S 2, P 2 ) (i) For L 1 L 2 Let G = (V,, S, P) be union CFG that generates L 1 L 2. Let V and P be defined as V = V 1 V 2 {S}, P = P 1 P 2 {S S 1 S 2 } If there is a string x L(G 1 ) then S 1 generates x, i.e., S 1 G1 * x. Similarly, for y L(G 2 ), S 2 G2 * y. In the case of x, union CFG first uses the production S S 1, then continues with the derivation of x in G 1. While in the case of y, the union CFG first uses the production S S 2 first, then continues with the derivation of y in G 2. Therefore, L(G 1 ) L(G), L(G 2 ) L(G), which can be combined to give L(G 1 ) L(G 2 ) L(G). On the other hand, if x is derivable from a production in G, the derivation must start with S S 1 or S S 2. For S S 1, all subsequent productions will be in G 1. Since V 1 V 2 =, therefore, x L(G 1 ). Similar is case with y L(G 2 ). Hence, x y L(G 1 ) L(G 2 ), which is L(G) L(G 1 ) L(G 2 ). (ii) For L 1 L 2 Let G = (V,, S, P) be a CF Grammar, which generates the CF language L 1 L 2. Let G 1, G 2 be the CFGs corresponding to L 1, L 2 respectively. Let V 1 V 2 =. Various components of G can be specified as follows: V = V 1 V 2 {S} P = P 1 P 2 {S S 1 S 2 }

28 218 Fundamentals of Theory of Computation Let x 1 L(G 1 ), x 2 = L(G 2 ), and x = x 1 x 2 L(G 1 ) L(G 2 ). Then x L(G) can be derived as follows: S S 1 S 2 G1 * x 1 S 2 G2 * x 1 x 2 = x where, S 1 S 2 x 1 S 2 is derivation in G 1, and x 1 S 2 * x 1 x 2 is a derivation in G 2. On the other hand if x can be derived form S 1 S 2 and therefore x = x 1 x 2, where x 1 can be derived from S 1 in G and x 2 can be derived from S 2 in G. Since V 1 V 2 =, the above S 1 x 1 derivable from G implies it derivable from G 1. Similarly x 2 derivable from G implies that it is derivable from G 2. Hence x = x 1 x 2 L(G 1 )L(G 2 ). (iii) For L 1 * For this it is required to construct G = (V,, S, P), which generates L 1 *. Let V = V 1 {S} and S V 1. The language L * 1 will contain strings of the form x = x 1, x 2,,..., x n, where x i L 1. Since x i can be derived form S 1, for G it is only required to get these strings derived form S. Therefore, a production in G in terms of production of G 1 can be specified as S S 1 S Hence, the production in G can be expressed as P = P 1 {S S 1 S } For x to be member of L(G), x = or x can be derived from some string x i * L(G 1 ). In the later case, the only productions in G are those that exists in G 1. Therefore, x L (G 1 ) *. This proves the theorem. EXAMPLE 9.10 Indicate, which of the following statements pertaining to closure properties are True/False. Give examples for justification of True/False.

29 Chapter 9. Context Free Grammars and Languages 219 a) The union of CFL and Regular language is CFL. b) The intersection of any two CFL is always CFL. Solution a). This is true, because every regular language is CFL also. For example, we can write the top-level productions of new CFL as S A B, where A may generate the given CFL, and B may generate the given regular Language. b). This is False. Consider {a m b m c n m, n 0} {a m b n c n m, n 0}. This is {a n b n c n n 0}. Because {a m b m c m } {a m b m c n } and {a n b n c n } {a m b n c n }. We know that {a n b n c n } is not CFL. 9.8 AMBIGUOUS GRAMMARS AND LAN- GUAGES One special desirable feature in a grammar for a programming language is freedom from ambiguity. There is extensive interest in question of ambiguity for natural languages as well, and context-free languages. Some of the known results are: (1) it is recursively unsolvable whether an arbitrary grammar for a language is ambiguous; (2) there exists inherently ambiguous languages, for example, {a i b j c i d k i, j, k 1} {a i b j c k d j i, j, k 1}, and, (3) no regular language is inherently ambiguous. These considerations motivate the study of ambiguity in context-free languages. Given a grammar G = (, V, S, P), a word w is said to be ambiguously derivable if there exists at least two derivations of w from S whose associated

30 220 Fundamentals of Theory of Computation generation trees are different. A grammar G is said to be ambiguous if there exists an ambiguously derivable word in L(G), and is otherwise unambiguous. Definition 9.8. A CFG is unambiguous if for every w L(G) there is a unique parse-tree in G with yield w. A language L is said to be inherently ambiguous if there is no unambiguous grammar for L. Following are additional terms related to ambiguity. The degree of ambiguity of a sentence is the number of its distinct derivation trees. A sentence is unambiguous if its degree of ambiguity is unity. A grammar is unambiguous if each of its sentences is unambiguous. A grammar has bounded ambiguity if there is a bound b on the degree of ambiguity of any sentence of the grammar. A grammar is reduced if every non-terminal appears in derivation of some sentence. If T 1 and T 2 are two distinct generation trees, then their associated leftmost derivations are distinct; the converse also holds. Thus a word is ambiguously derivable if and only if it has two leftmost derivations. It is well established that there is no decision procedure for determining whether an arbitrary grammar is ambiguous. Also there is no decision procedure (i.e., it is unsolvable) for determining if an arbitrary language is inherently ambiguous. In other words, there is no algorithm for determining of an arbitrary language whether it is generated by some unambiguous grammar. Since parse tree is a semantics expression of generated language string, the two different parse trees represent two different meanings of the generated string. Given a grammar G = (V,, P,S), a word w is said to be unambiguously derivable if there exists two derivations of w from S, whose associated generation trees are different. A grammar G is said to be ambiguous if there exists an ambiguously derivable word in L(G), and is otherwise unambiguous. A language L is said to be inherently ambiguous if there is no unambiguous gram-

31 Chapter 9. Context Free Grammars and Languages 221 mar for L. EXAMPLE 9.11 Let us consider the grammar for arithmetic expressions with productions given below. E E + E E * E (E) id E Case 1: E E + E id + E id + E E id + id E id + id id E id + E E E id id Figure 9.9: (a) Derivation and parse-tree for id + id id. For the sake of simplicity only two operations: + and have been considered for this purpose as these are sufficient to express the ambiguity presence in the generated arithmetic expressions. Let us generate the string id + id id. The corresponding derivations and derivations trees are shown in figure 9.9(a) and (b). The case 1 shows that 2nd and 3rd term in the expression are to be multiplied and then the result is to be added into the first term. Whereas the derivation tree in case 2 shows first and second id terms of the expression are to be added together first and then the result of this is to be multiplied with the third term in the expression.

32 222 Fundamentals of Theory of Computation Case 2: E E E E + E E id + E E id + id E id + id id E E E E + E id id id Obviously the two derivations represent different meanings of the expression id + id id. Therefore, this grammar is ambiguous and the corresponding language is ambiguous too. An ambiguity in ALGOL 60, for example, is Figure 9.9: (b) Derivation and parse-tree for id + id id. if 1 then if 2 then 1 else 2 where represents any Boolean expression, a for clause, and an unconditional statement. Ambiguous grammars represent different parse-trees for the same expression. The different parse-trees always represent different meanings. Therefore, it is necessary to remove the ambiguity in the CFGs. One way to represent the ambiguity in a language expression is to consider the precedence of operators. Thus, in id + id * id, the part id * id needs to be processed before the + operator. Hence, in the parse-tree answer id * id is computed first. However, when

33 Chapter 9. Context Free Grammars and Languages 223 there are identical operators they are grouped from left to right. For example, id + id + id may be grouped as ((id + id) + id) or (id + (id + id)). In two cases the parse-tree are different, but they will evaluate the same result. Naturally, we are interested to have some algorithm to remove all kinds of ambiguities in a CFG. However, it is recursively unsolvable whether an arbitrary grammar for a language is ambiguous. In other words, there is no algorithm to determine if an arbitrary language is generated by some ambiguous grammar or not. In fact, there are CFLs that have ambiguous CFGs, but removal of ambiguity for such grammar is impossible. It is not always true that ambiguity can be removed from a grammar. However, an ambiguous grammar can always be converted into unambiguous grammar by some changes in it. Sometimes, ambiguous grammars can be made unambiguous by adding some non-terminal symbols through process called disambiguation so that for a given expression there is one parse-tree and one derivation only. EXAMPLE 9.12 Disambiguate the CFG G = (V,, S, P), where V = {E} = {(, ), +, *, id} S = {E} P = {E E + E, E E E, E (E), E id} Solution This grammar is ambiguous as it generates two difference parse trees for the arithmetic expression id + id id. Following modifications can be made in G to disambiguate it. V = {E, T, F} P = {E T, T F, F id, E E + T, T T * F, F (E)} The and S remain unchanged. The derivation and parse-tree for id + id id are shown in figure We note that no other derivation and parse-tree are possible for this string. Hence, the modified grammar is unambiguous. E E + T T + T F + T id + T

34 224 Fundamentals of Theory of Computation id + T F id + F F id + id F id + id id E E + T T T F F id id id Figure 9.10: Parse-Tree for an unambiguous Grammar. Similarly, if id * id + id is expression, the derivation is as follows: E E + T T + T T * F + T F * F + T id * F + T id * F + T id * id + T id * id + id.

35 Chapter 9. Context Free Grammars and Languages CHAPTER SUMMARY 9.10 REVIEW QUESTIONS 9.11 BIBLIOGRAPHIC NOTES Panini is earliest known grammarian. Since his time, the grammars of the languages are described in terms of their block structure, which describes how the sentences are recursively built up from smaller phrases, and eventually individual words or word elements. The context-free grammar formalism developed by Noam Chomsky, has turned the grammatical structures into rigorous mathematics. A CFG provides a simple and precise mechanism for describing the methods by which phrases in some natural language are built from smaller blocks, capturing the block structure of sentences in a natural way. Its simplicity makes the formalism amenable to rigorous mathematical study. However, the important features of natural language syntax such as agreement and reference cannot be expressed in a natural way. Block structure was first introduced in computer programming language Algol by John Backus and Peter Naur and after them it is now called Backus- Naur Form (BNF). Context-free grammars are simple enough to allow the construction of efficient parsing algorithms, which, for a given string, determine whether and how it can be generated from the grammar. The topic of context-free languages and their ambiguities remained of prime importance ever since the implementation of machine interpretation and processing of computer languages. In the recent past due to wide spread adoption of Internet, the volume of natural language text handling and its processing by machines has overshot the high level languages.

36 226 Fundamentals of Theory of Computation Among the earliest works, David G. Cantor in reference [4] proves that there does not exist an algorithm to determine whether a Backus system (Backus Naur Format BNF) is ambiguous. The article proves a theorem Ambiguity problem is unsolvable. Seymour and Ginsburg in [1] prove four principle results about ambiguity in CF languages. It shows that problem of determining whether an arbitrary language which is inherently ambiguous is recursively solvable. It also presents a decision procedure for determining whether an arbitrary bounded grammar is ambiguous. Also proves that no language contained in w 1 * w 2 *, each w i is word, is inherently ambiguous. In an application to the area of CFG and CFL, [5] presents review of efforts to automate the writing of translators of programming languages. It gives a formal study of syntax and its application to translator writing system. In addition, an interesting analysis has been presented of top-down and bottom-up parsing. The article, edited by Nicklaus-Wirth, is a first approach to formal method for automating the process of compiler writing. The article [3] presents schemes for translation on CFG, namely generalized syntax directed translation, and another for the specification of the translation is in terms of tree automata a finite automaton with output, operating on the derivation trees of context-free grammars. The paper shows that tree-automata provide an exact characterization for those generalized syntax-directed systems. A lucid introduction has been given for a new type of grammar Indexed Grammar in [6] for generating formal languages. The class of indexed languages includes all context-free languages and some context-sensitive languages, but yet is proper subset of context sensitive languages. In addition, the closure properties and decidability features of languages generated by this grammar are similar to those of context-free languages. EXERCISES 1. Prove, whether the following are contest-free languages? (a) {a i b i c i i 1 } (b) {a i b i c i i = 2 n, n 1 }

37 Chapter 9. Context Free Grammars and Languages 227 (c) {a i b j i j, i, j are integers } (d) {w {a, b} * w = w R } (e) Set of all strings of balanced parenthesis, i.e. each left parenthesis has a corresponding balanced right parenthesis. (e) {a i b j i j} (f) {w {a, b} * w has number of a s twice the number of b s } (g) {a i b j c k for i, j, k 1} (h) {a i b j c k 2i = j + k, for i, j, k 1} (i) {w {a, b} * w is a palindrome} (j) {{a} * {b} * } (k) {baa + abb } * } (l) {a i a j a k a l i + j = k + l, i, j, k, l are positive integers} 2. Construct context-free grammars to accept the following language over {0, 1}. Explain your grammar. (a) {w w starts and ends with the same symbol}. (b) {w w is odd} (c) {w w is odd and its middle symbol is zero} (d) {0 n 1 n n > 0} {0 n 1 2n n > 0} (e) {0 i 1 j 2 k i j or j k} (f) Binary string with twice as many 1 s as 0 s} 3. Given the context-free grammar G, describe the language and find out the strings set for language. G = (V,, S, P), V = {S, A}, = {a, b}, S = {s AA, A AAA, A a, A ba, A Ab}. 4. For each of the following cases, find out language generated by each CFG. (a) S asa bsb a b (b) S SS as Sa b (c) S as bs b (d) S asa B B bb a (e) S AA BB

38 228 Fundamentals of Theory of Computation A aa B bb 5. If G = (V,, A, B} V = {S, A, B} = {a, b} P = {S ab ba, A a as baa, B b bs abb} Derive the string abbbaaab L(G), using (a) Left most derivation (b) Right most derivation 6. Given the productions for the CFG G, S asb ab SS show that if x L(G), and x = uv, then u abb. [Hint: Use mathematical induction]. 7. Find out the CFG s over the alphabet = {a, b}, for the following language description: - (a) All words in which letter b is not doubled. (b) All the words have equal number of a s and b s. 8. Given the production for a CFG as S bba B aaa A Bb Show that the word bbaabaab is not in the language generated by this grammar. 9. Using the mathematical induction, prove that the language strings, corresponding to the CFG given below, have number of a s more than number of b s. A a Aa baa Aab AbA 10. Corresponding to the CF-grammars given below describe the language generated by each. Also, categorize the language as regular or non-regular. (a) S AB A aaa bab a b B ab bb

39 Chapter 9. Context Free Grammars and Languages 229 (b) S bs aa a A aa bb B bs b (c) S AAS ab aa A ab ba 11. Given the Productions for generating arithmetic expressions as E E + E E * E E - E E E (E) id, Generate the following expressions: (a) (id + id) / id id (b) id / id + (id * id) / id 12. Are the following grammars ambiguous? If yes, suggest an equivalent unambiguous grammar. (a) S SS (s) (b) G = (V,, S, P), V = {E, F}, = {a, b, -}, S = E, and P = {E F E, F a, E E F, F b, E F}. 13. Explain, why the grammar shown below, which generates strings with equal number of 0 s and 1 s is ambiguous? S OA 1B A 0AA 1S 1 B 1BB 0S Give the left most and right most derivations for the balanced parenthesis string ((( ))( )) in the case of problem 11 above. 15. Show that following CFGs are ambiguous. (a) S abb bb Sa a (b) S SaSaS b (c) S as abb A (d) A Aa a 16. Show that CFG S asa bsb a b generates palindrome language and it is unambiguous. 17. Give an algorithm for deciding whether an arbitrary context-free grammar generates an empty set. 18. Given a context-free grammar