Normal Forms and Parsing: Testing Membership and Parse Tree Reconstruction, Slides of Theory of Automata

Download Normal Forms and Parsing: Testing Membership and Parse Tree Reconstruction and more Slides Theory of Automata in PDF only on Docsity!

Normal forms and parsing

Testing membership and parsing

• Given a grammar

• How can we know if a string x is in its

language?

• If so, can we reconstruct a parse tree for x?

S → 0S1 | 1S0S1 | T

T → S | e

Problems

• How do we know when to stop?

S → 0S1 | 1S0S1 | T

T → S | ε

x = 00111

S 0S

1S0S

00S

01S0S

0T

10S10S

when do we stop?

Problems

• Idea: Stop derivation when length exceeds | x |

• Not right because of ε-productions

• We might want to eliminate ε-productions too

S → 0S1 | 1S0S1 | T

T → S | ε

x = 01011

S ⇒ 0S1 ⇒ 01S0S11 ⇒ 01S011 ⇒ 01011

1 3 7 6 5

Unit productions

• A unit production is a production of the form

where A 1 and A 2 are both variables

• Example

A 1 → A 2

S → 0S1 | 1S0S1 | T

T → S | R | ε

R → 0SR

grammar: unit productions:

S T

R

Removal of unit productions

• If there is a cycle of unit productions

delete it and replace everything with A 1

• Example

A 1 → A 2 → ... → A k → A 1

S → 0S1 | 1S0S1 | T

T → S | R | ε R → 0SR

S T

R

S → 0S1 | 1S0S

S → R | ε R → 0SR

T is replaced by S in the {S, T} cycle

Removal of ε-productions

• A variable N is nullable if there is a derivation

• How to remove ε-productions (except from S)

Find all nullable variables N 1 , ..., N k For i = 1 to k For every production of the form A → αN i β, add another production A → αβ If N i → ε is a production, remove it If S is nullable, add the special production S → ε

N ⇒* ε

Example

• Find the nullable variables

S → ACD

A→ a B → ε C → ED | ε D → BC | b E → b

B C D

grammar nullable variables

^ Find all nullable variables^ N 1 , ..., N k

Eliminating ε-productions

S → ACD

A→ a B → ε C → ED | ε D → BC | b E → b

nullable variables: B, C, D

For i = 1 to k For every production of the form A → αN i β, add another production A → αβ If N i → ε is a production, remove it

D → C

S → AD

D → B

D → ε S → AC S → A C → E

Recap

• After eliminating ε-productions and unit

productions, we know that every derivation

doesn’t shrink in length and doesn’t go into

cycles

• Exception: S → ε

• We will not use this rule at all, except to check if ε

∈ L

S ⇒* a 1 …a k where a 1 , …, a k are terminals

Algorithm 1 for testing

membership

• We can now use the following algorithm to

check if a string x is in the language of G

Eliminate all ε-productions and unit productions If x = ε and S → ε, accept; else delete S → ε Let X := S While some new production P can be applied to X Apply P to X If X = x , accept If | X | > | x |, backtrack If no more productions can be applied to X , reject

Practical limitations of Algorithm I

• Previous algorithm can be very slow if x is long

• There is a faster algorithm, but it requires that

we do some more transformations on the

grammar

G = CFG of the java programming language x = code for a 200-line java program

algorithm might take about 10200 steps!

Exercise

• Convert this CFG into Chomsky Normal Form:

S → ε |ADDA A → a C → c D → bCb

Algorithm 2 for testing

membership

S → AB | BC

A → BA | a B → CC | b C → AB | a

x = baaba

Idea: We generate each substring of x bottom up

b a a b a

B AC AC B AC

SA B SC SA

– B B

– SAC

SAC