Cs133 lec10

Turing Machines and Algorithms

Decidability

CS 133 : Automata Theory and Computability Turing Machines Henry N. Adorna Nestine Hope S. Hernandez Algorithms and Complexity Laboratory Department of Computer Science University of the Philippines, Diliman {hnadorna,nshernandez}@dcs.upd.edu.ph

LEC 10

Turing Machines and Algorithms

Decidability

Turing Machines

Turing Machines and Algorithms Terminology for Describing TMs Turing Machines and Computers

Decidability Decidable Problems Concerning Regular Languages Decidable Problems Concerning CFLs

Turing Machines and Algorithms

Decidability

What is algorithm?

Turing Machines and Algorithms

Decidability

What is algorithm?

Informally, an algorithm is a collection of simple instructions for carrying out some task.

• have had a long history in mathematics • BUT was not defined precisely until the twentieth century • before, mathematicians relied upon an intuitive notion of what

algorithms were when using and describing them • was insufficient for gaining a deeper understanding of algorithms

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Given a Diophantine equation with any number of unknown quantities and with rational integral numerical coefficients: To devise a process according to which it can be determined in a finite number of operations whether the equation is solvable in rational integers.

A Diophantine equation is an equation of the form p(x1 , x2 , . . . , xn ) = 0, where p is a polynomial with integer coefficients.

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Given a Diophantine equation with any number of unknown quantities and with rational integral numerical coefficients: To devise a process according to which it can be determined in a finite number of operations whether the equation is solvable in rational integers.

A Diophantine equation is an equation of the form p(x1 , x2 , . . . , xn ) = 0, where p is a polynomial with integer coefficients.

The intuitive concept may have been adequate for giving algorithms for certain tasks, but it was useless for showing that no algorithm exists for a particular task.

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Given a Diophantine equation with any number of unknown quantities and with rational integral numerical coefficients: To devise a process according to which it can be determined in a finite number of operations whether the equation is solvable in rational integers.

A Diophantine equation is an equation of the form p(x1 , x2 , . . . , xn ) = 0, where p is a polynomial with integer coefficients.

The intuitive concept may have been adequate for giving algorithms for certain tasks, but it was useless for showing that no algorithm exists for a particular task.

Proving that an algorithm does not exist requires having a clear definition of algorithm.

Turing Machines and Algorithms

Decidability

Church-Turing Thesis A clear definition of algorithms came in the 1936 papers of Alonzo Church and Alan Turing.

Church used a notational system called the λ-calculus while Turing did it with his “machines”.

But these two definitions were shown to be equivalent. And the connection between the informal notion of algorithm and the precise definition has come to be called the Church-Turing thesis.

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Is D decidable?

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Is D decidable? No.

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Is D decidable? No. (but we can show it is Turing-recognizable)

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Is D decidable? No. (but we can show it is Turing-recognizable) Let D1 = {p|p is a polynomial over x with an integral root}.

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Is D decidable? No. (but we can show it is Turing-recognizable) Let D1 = {p|p is a polynomial over x with an integral root}.

M1 = “The input is a polynomial p over the variable x.

1. Evaluate p with x set successively to the values 0, 1, −1, 2, −2, . . ..

If at any point the polynomial evaluates to 0, accept.”

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Is D decidable? No. (but we can show it is Turing-recognizable) Let D1 = {p|p is a polynomial over x with an integral root}.

M1 = “The input is a polynomial p over the variable x.

1. Evaluate p with x set successively to the values 0, 1, −1, 2, −2, . . ..

If at any point the polynomial evaluates to 0, accept.”

For the multivariable case, we can present a similar TM that recognizes D. Both M1 and M are recognizers but not deciders.

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Is D decidable? No. (but we can show it is Turing-recognizable) Let D1 = {p|p is a polynomial over x with an integral root}.

M1 = “The input is a polynomial p over the variable x.

1. Evaluate p with x set successively to the values 0, 1, −1, 2, −2, . . ..

If at any point the polynomial evaluates to 0, accept.”

For the multivariable case, we can present a similar TM that recognizes D. Both M1 and M are recognizers but not deciders.

We can convert M1 to be a decider for D1 by restricting the search to certain bounds (±k cmax , where k is the number of terms, cmax is the coefficient with c1 largest absolute value, and c1 is the coefficient of the highest order term.

Turing Machines and Algorithms

Decidability

Hilbert’s 10th Problem Let D = {p|p is a polynomial with an integral root}.

Is D decidable? No. (but we can show it is Turing-recognizable) Let D1 = {p|p is a polynomial over x with an integral root}.

M1 = “The input is a polynomial p over the variable x.

1. Evaluate p with x set successively to the values 0, 1, −1, 2, −2, . . ..

If at any point the polynomial evaluates to 0, accept.”

For the multivariable case, we can present a similar TM that recognizes D. Both M1 and M are recognizers but not deciders.

We can convert M1 to be a decider for D1 by restricting the search to certain bounds (±k cmax , where k is the number of terms, cmax is the coefficient with c1 largest absolute value, and c1 is the coefficient of the highest order term.

Matijasevi˘c’s theorem shows that calculating such bounds for multivariable polynomials is impossible.

Turing Machines and Algorithms

Decidability

1. formal description -spells out in full the TM’s states, transition function, etc.

2. implementation description - uses English prose to describe the way that the TM moves its head and the way it stores data on its tape.

3. high-level description -uses English prose to describe an algorithm, ignoring the implementation details.

Turing Machines and Algorithms

Decidability

Example Let A be the language consisting of all strings representing undirected graphs that are connected.

A = {< G > |G is a connected undirected graph}.

Turing Machines and Algorithms

Decidability

Example Let A be the language consisting of all strings representing undirected graphs that are connected.

A = {< G > |G is a connected undirected graph}.

M = â&#x20AC;&#x153;On input < G >, the encoding of a graph G: 1. Select the first node of G and mark it.

2. Repeat the ff until no new nodes are marked: â&#x20AC;˘ For each node in G, mark it if is attached by an edge to a node

that is already marked.

3. Scan all nodes of G to determine whether they all are marked. If they are, accept; otherwise, reject.

Turing Machines and Algorithms

Decidability

Turing Machines and Algorithms

Decidability

• M first checks to determine whether the input is the proper encoding of some graph. M scans the tape to be sure that there are 2 lists and that they are in proper form.

• For stage 1, M marks the first node with a dot on the leftmost digit.

• For stage 2, M scans the list of nodes to find an undotted node n1 and flags it by marking it differently(underline). Then M scans the list again to find a dotted n2 and underlines it too. Then M scans the list of edges and tests whether n1 and n2 appears in that edge. If yes, M dots n1 , removes the underline, and goes back to the beginning of stage 2. If not, M checks the next edge.

• If there are no more edges, {n1 , n2 } is not an edge, M moves the underline on n2 to the next dotted node and now calls this node n2 .

Repeat ...

If there are no more dotted nodes, n1 is not attached to any dotted nodes. Then M sets the next undotted node as n1 . Repeat ...

If there are no more undotted nodes, move to stage 3.

• For stage 3, M scans the list of nodes to determine whether all are dotted. If yes, accept. If not, reject.

Turing Machines and Algorithms

Decidability

Think about this ...

Compare the Turing machines and the common sort of computer that we use daily. These models appear rather different, but they accept exactly the same languages.

Explain (1) how a computer can simulate a Turing machine and (2) how a Turing machine can simulate a computer.

Turing Machines and Algorithms

Decidability

â&#x20AC;˘ we ignored the real limitations that every implementation of any

programming language has on any real computer â&#x20AC;˘ these limitations, such as the size of the address space, are not

fundamental limits â&#x20AC;˘ as the years progress we expect computers will grow indefinitely in measures such as address-space size, main-memory size, etc

Turing Machines and Algorithms

Decidability

• we ignored the real limitations that every implementation of any

programming language has on any real computer • these limitations, such as the size of the address space, are not

fundamental limits • as the years progress we expect computers will grow indefinitely in measures such as address-space size, main-memory size, etc • by focusing on Turing machines, where these limitations do not

exist, we are able to capture the idea of what some computing device will be capable of doing, if not today, then at some time in the future.

Turing Machines and Algorithms

Decidability

• we ignored the real limitations that every implementation of any

programming language has on any real computer • these limitations, such as the size of the address space, are not

fundamental limits • as the years progress we expect computers will grow indefinitely in measures such as address-space size, main-memory size, etc • by focusing on Turing machines, where these limitations do not

exist, we are able to capture the idea of what some computing device will be capable of doing, if not today, then at some time in the future.

• We divide problems that can be solved by a TM into 2 classes: • those that have an algorithm (a TM that halts whether or not it

accepts its input) • those that are only solved by TM that may run forever on input

they do not accept

Turing Machines and Algorithms

Decidability

Turing Machines and Algorithms

Decidability

ADF A = {< B, w > | B is a DFA that accepts input string w}

Turing Machines and Algorithms

Decidability

ADF A = {< B, w > | B is a DFA that accepts input string w}

Theorem ADF A is a decidable language.

Turing Machines and Algorithms

Decidability

ADF A = {< B, w > | B is a DFA that accepts input string w}

Theorem ADF A is a decidable language.

M = â&#x20AC;&#x153;On input < B, w >, where B is a DFA and w is a string: 1. Simulate B on input w.

2. If the simulation ends in an accept state, accept. If it ends in a nonaccepting state, reject.â&#x20AC;?

Turing Machines and Algorithms

Decidability

AN F A = {< B, w > | B is a NFA that accepts input string w}

Turing Machines and Algorithms

Decidability

AN F A = {< B, w > | B is a NFA that accepts input string w}

Theorem AN F A is a decidable language.

Turing Machines and Algorithms

Decidability

AN F A = {< B, w > | B is a NFA that accepts input string w}

Theorem AN F A is a decidable language.

N = â&#x20AC;&#x153;On input < B, w >, where B is an NFA and w is a string: 1. Convert NFA B to an equivalent DFA C. 2. Run TM M on input < C, w >.

3. If M accepts, accept; otherwise, reject.â&#x20AC;?

Turing Machines and Algorithms

Decidability

AN F A = {< B, w > | B is a NFA that accepts input string w}

Theorem AN F A is a decidable language.

N = â&#x20AC;&#x153;On input < B, w >, where B is an NFA and w is a string: 1. Convert NFA B to an equivalent DFA C. 2. Run TM M on input < C, w >.

3. If M accepts, accept; otherwise, reject.â&#x20AC;?

Running TM M in stage 2 means incorporating M into the design of N as a subprocedure.

Turing Machines and Algorithms

Decidability

AREX = {< R, w > | R is a regular expression that generates string w}

Turing Machines and Algorithms

Decidability

AREX = {< R, w > | R is a regular expression that generates string w}

Theorem AREX is a decidable language.

Turing Machines and Algorithms

Decidability

AREX = {< R, w > | R is a regular expression that generates string w}

Theorem AREX is a decidable language.

M = â&#x20AC;&#x153;On input < B, w >, where R is a regular expression and w is a string: 1. Convert regular expression R to an equivalent NFA A. 2. Run TM N on input < A, w >.

3. If N accepts, accept; otherwise, reject.â&#x20AC;?

Turing Machines and Algorithms

Decidability

AREX = {< R, w > | R is a regular expression that generates string w}

Theorem AREX is a decidable language.

M = â&#x20AC;&#x153;On input < B, w >, where R is a regular expression and w is a string: 1. Convert regular expression R to an equivalent NFA A. 2. Run TM N on input < A, w >.

3. If N accepts, accept; otherwise, reject.â&#x20AC;?

B shows DFA, NFA or regular expression are equivalent because the machine is able to convert one form of encoding to another.

Turing Machines and Algorithms

Decidability

Turing Machines and Algorithms

EDF A = {< A > | A is a DFA and L(A) = â&#x2C6;&#x2026;}

Decidability

Turing Machines and Algorithms

EDF A = {< A > | A is a DFA and L(A) = â&#x2C6;&#x2026;}

Theorem EDF A is a decidable language.

Decidability

Turing Machines and Algorithms

Decidability

EDF A = {< A > | A is a DFA and L(A) = ∅}

Theorem EDF A is a decidable language.

T = “On input < A >, where A is a DFA: 1. Mark the start state of A. 2. Repeat until no new states get marked: B Mark any state that has a transition coming into it from any state that is already marked.

3. If no accept state is marked, accept; otherwise, reject.”

Turing Machines and Algorithms

Decidability

Turing Machines and Algorithms

EQDF A = {< A, B > | A and B are DFAs and L(A) = L(B)}

Decidability

Turing Machines and Algorithms

EQDF A = {< A, B > | A and B are DFAs and L(A) = L(B)}

Theorem EQDF A is a decidable language.

Decidability

Turing Machines and Algorithms

EQDF A = {< A, B > | A and B are DFAs and L(A) = L(B)}

Theorem EQDF A is a decidable language.

We construct a new DFA C from A and B, where C accepts only those strings that are accepted by either A or B but not by both.

F = â&#x20AC;&#x153;On input < A, B >, where A and B are DFAs : 1. Construct DFA C as described.

2. Run TM T on input < C > 3. If T accepts, accept; otherwise, reject.â&#x20AC;?

Decidability

Turing Machines and Algorithms

Decidability

Turing Machines and Algorithms

ACF G = {< G, w > | G is a CFG that generates string w}

Decidability

Turing Machines and Algorithms

ACF G = {< G, w > | G is a CFG that generates string w}

Theorem ACF G is a decidable language.

Decidability

Turing Machines and Algorithms

Decidability

ACF G = {< G, w > | G is a CFG that generates string w}

Theorem ACF G is a decidable language.

Note : If G were in Chomsky normal form, any derivation of w has 2n â&#x2C6;&#x2019; 1 steps, where n is the length of w.

Turing Machines and Algorithms

Decidability

ACF G = {< G, w > | G is a CFG that generates string w}

Theorem ACF G is a decidable language.

Note : If G were in Chomsky normal form, any derivation of w has 2n − 1 steps, where n is the length of w.

S = “On input < G, w >, where G is a CFG and w is a string: 1. Convert G to an equivalent grammar in Chomsly normal form.

2. List all derivations with 2n − 1 steps, where n is the length of w, except if n = 0, then instead list all derivations with 1 step.

3. If any of these derivations generate w, accept; if not, reject.”

Turing Machines and Algorithms

Decidability

Turing Machines and Algorithms

ECF G = {< G > | G is a CFG and L(G) = â&#x2C6;&#x2026;}

Decidability

Turing Machines and Algorithms

ECF G = {< G > | G is a CFG and L(G) = â&#x2C6;&#x2026;}

Theorem ECF G is a decidable language.

Decidability

Turing Machines and Algorithms

Decidability

ECF G = {< G > | G is a CFG and L(G) = ∅}

Theorem ECF G is a decidable language.

R = “On input < G >, where G is a CFG: 1. Mark all terminal symbols in G. 2. Repeat until no new variables get marked: B Mark any variable A where G has a rule A → U1 U2 · · · Uk and each symbol U1 , · · · , Uk has already been marked.

3. If the start variable is not marked, accept; otherwise, reject.”

Turing Machines and Algorithms

Decidability

Turing Machines and Algorithms

Decidability

EQCF G = {< G, H > | G and H are CFGs and L(G) = L(H)}

Turing Machines and Algorithms

Decidability

EQCF G = {< G, H > | G and H are CFGs and L(G) = L(H)} EQCF G is not a decidable language.

Turing Machines and Algorithms

Theorem Every context-free language is decidable.

Decidability

Turing Machines and Algorithms

Theorem Every context-free language is decidable.

Let A be a CFL. We want to show that A is decidable.

Decidability

Turing Machines and Algorithms

Theorem Every context-free language is decidable.

Let A be a CFL. We want to show that A is decidable.

Let G be a CFG for A and design a TM MG that decides A. MG = â&#x20AC;&#x153;On input w: 1. Run TM S on input < G, w >.

2. If this machine accepts, accept; if it rejects, reject.â&#x20AC;?

Decidability

Turing Machines and Algorithms

Decidability

Questions?

See you next meeting!