Cs133 lec13

Time Complexity

NP-completeness

CS 133 : Automata Theory and Computability Intractability Henry N. Adorna Nestine Hope S. Hernandez Algorithms and Complexity Laboratory Department of Computer Science University of the Philippines, Diliman {hnadorna,nshernandez}@dcs.upd.edu.ph

LEC 13

Time Complexity

NP-completeness

The number of steps that an algorithm uses on a particular input may depend on several parameters.

Time Complexity

NP-completeness

The number of steps that an algorithm uses on a particular input may depend on several parameters. For simplicity, we compute the running time of an algorithm purely as a function of the length of the string representing the input and don’t consider any other parameters.

Time Complexity

NP-completeness

The number of steps that an algorithm uses on a particular input may depend on several parameters. For simplicity, we compute the running time of an algorithm purely as a function of the length of the string representing the input and don’t consider any other parameters.

Definition Let M be a deterministic Turing machine that halts on all inputs.

The running time or time complexity of M is the function f : N → N, where f (n) is the maximum number of steps that M uses on any input of length n. If f (n) is the running time of M , we say that M runs in time f (n) and that M is an f (n) time Turing machine.

Customarily we use n to represent the length of the input.

Time Complexity

NP-completeness

Big-O Notation Because the exact running time of an algorithm often is a complex expression, we usually just estimate it.

Time Complexity

NP-completeness

Big-O Notation Because the exact running time of an algorithm often is a complex expression, we usually just estimate it. We consider only the highest order term of the expression for the running time of the algorithm, disregarding both the coefficient of that term and any lower order terms, because the highest order term dominates the other terms on large inputs.

Time Complexity

NP-completeness

Big-O Notation Because the exact running time of an algorithm often is a complex expression, we usually just estimate it. We consider only the highest order term of the expression for the running time of the algorithm, disregarding both the coefficient of that term and any lower order terms, because the highest order term dominates the other terms on large inputs.

Example: f1 (n) = 6n3 + 2n2 + 20n + 45

Time Complexity

NP-completeness

Big-O Notation Because the exact running time of an algorithm often is a complex expression, we usually just estimate it. We consider only the highest order term of the expression for the running time of the algorithm, disregarding both the coefficient of that term and any lower order terms, because the highest order term dominates the other terms on large inputs.

Example: f1 (n) = 6n3 + 2n2 + 20n + 45 ⇒ f1 (n) = O(n3 ), the asymptotic notation or big-O notation

Time Complexity

NP-completeness

Big-O Notation Because the exact running time of an algorithm often is a complex expression, we usually just estimate it. We consider only the highest order term of the expression for the running time of the algorithm, disregarding both the coefficient of that term and any lower order terms, because the highest order term dominates the other terms on large inputs.

Example: f1 (n) = 6n3 + 2n2 + 20n + 45 ⇒ f1 (n) = O(n3 ), the asymptotic notation or big-O notation Let f and g be functions f, g : N → R+ .

Say that f (n) = O(g(n)) if positive integers c and n0 exist such that for every integer n ≥ n0 , f (n) ≤ c g(n).

Time Complexity

NP-completeness

Big-O Notation Because the exact running time of an algorithm often is a complex expression, we usually just estimate it. We consider only the highest order term of the expression for the running time of the algorithm, disregarding both the coefficient of that term and any lower order terms, because the highest order term dominates the other terms on large inputs.

Example: f1 (n) = 6n3 + 2n2 + 20n + 45 ⇒ f1 (n) = O(n3 ), the asymptotic notation or big-O notation Let f and g be functions f, g : N → R+ .

Say that f (n) = O(g(n)) if positive integers c and n0 exist such that for every integer n ≥ n0 , f (n) ≤ c g(n). f2 (n) = 3n log2 n + 5nlog2 log2 n + 2

Time Complexity

NP-completeness

Big-O Notation Because the exact running time of an algorithm often is a complex expression, we usually just estimate it. We consider only the highest order term of the expression for the running time of the algorithm, disregarding both the coefficient of that term and any lower order terms, because the highest order term dominates the other terms on large inputs.

Example: f1 (n) = 6n3 + 2n2 + 20n + 45 ⇒ f1 (n) = O(n3 ), the asymptotic notation or big-O notation Let f and g be functions f, g : N → R+ .

Say that f (n) = O(g(n)) if positive integers c and n0 exist such that for every integer n ≥ n0 , f (n) ≤ c g(n). f2 (n) = 3n log2 n + 5nlog2 log2 n + 2 ⇒ f2 (n) = O(n log n) f3 (n) = O(n2 ) + O(n)

Time Complexity

NP-completeness

Big-O Notation Because the exact running time of an algorithm often is a complex expression, we usually just estimate it. We consider only the highest order term of the expression for the running time of the algorithm, disregarding both the coefficient of that term and any lower order terms, because the highest order term dominates the other terms on large inputs.

Example: f1 (n) = 6n3 + 2n2 + 20n + 45 ⇒ f1 (n) = O(n3 ), the asymptotic notation or big-O notation Let f and g be functions f, g : N → R+ .

Say that f (n) = O(g(n)) if positive integers c and n0 exist such that for every integer n ≥ n0 , f (n) ≤ c g(n). f2 (n) = 3n log2 n + 5nlog2 log2 n + 2 ⇒ f2 (n) = O(n log n) f3 (n) = O(n2 ) + O(n) ⇒ f3 (n) = O(n2 )

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≼ 0}

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≥ 0} M1 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat if both 0s and 1s remain on the tape: B Scan across the tape, crossing off a single 0 and a single 1.

3. If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject.

Otherwise, if neither 0s nor 1s remain on the tape, accept. ”

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≥ 0} M1 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat if both 0s and 1s remain on the tape: B Scan across the tape, crossing off a single 0 and a single 1.

3. If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject.

Otherwise, if neither 0s nor 1s remain on the tape, accept. ” stage 1:

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≥ 0} M1 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat if both 0s and 1s remain on the tape: B Scan across the tape, crossing off a single 0 and a single 1.

3. If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject.

Otherwise, if neither 0s nor 1s remain on the tape, accept. ” stage 1: 2n = O(n)

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≥ 0} M1 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat if both 0s and 1s remain on the tape: B Scan across the tape, crossing off a single 0 and a single 1.

3. If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject.

Otherwise, if neither 0s nor 1s remain on the tape, accept. ” stage 1: 2n = O(n) stage 2:

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≥ 0} M1 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat if both 0s and 1s remain on the tape: B Scan across the tape, crossing off a single 0 and a single 1.

3. If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject.

Otherwise, if neither 0s nor 1s remain on the tape, accept. ” stage 1: 2n = O(n) stage 2: n2 O(n) = O(n2 )

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≥ 0} M1 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat if both 0s and 1s remain on the tape: B Scan across the tape, crossing off a single 0 and a single 1.

3. If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject.

Otherwise, if neither 0s nor 1s remain on the tape, accept. ” stage 1: 2n = O(n) stage 2: n2 O(n) = O(n2 ) stage 3:

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≥ 0} M1 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat if both 0s and 1s remain on the tape: B Scan across the tape, crossing off a single 0 and a single 1.

3. If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject.

Otherwise, if neither 0s nor 1s remain on the tape, accept. ” stage 1: 2n = O(n) stage 2: n2 O(n) = O(n2 ) stage 3: O(n)

Time Complexity

NP-completeness

Analyzing Algorithms k k

A = {0 1 | k ≥ 0} M1 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat if both 0s and 1s remain on the tape: B Scan across the tape, crossing off a single 0 and a single 1.

3. If 0s still remain after all the 1s have been crossed off, or if 1s still remain after all the 0s have been crossed off, reject.

Otherwise, if neither 0s nor 1s remain on the tape, accept. ” stage 1: 2n = O(n) stage 2: n2 O(n) = O(n2 ) stage 3: O(n) Thus the total time of M1 on an input of length n is O(n) + O(n2 ) + O(n) = O(n2 ).

Time Complexity

NP-completeness

Definition Let t : N → R+ be a function.

Define the time complexity class, T IM E(t(n)), to be the collection of all languages that are decidable by an O(t(n)) time Turing machine.

Time Complexity

NP-completeness

Definition Let t : N → R+ be a function.

Define the time complexity class, T IM E(t(n)), to be the collection of all languages that are decidable by an O(t(n)) time Turing machine.

Hence, in the preceding slide, A ∈ T IM E(n2 ) because M1 decides A in time O(n2 ) and T IM E(n2 ) contains all languages that can be decided on O(n2 ).

Time Complexity

NP-completeness

Definition Let t : N → R+ be a function.

Define the time complexity class, T IM E(t(n)), to be the collection of all languages that are decidable by an O(t(n)) time Turing machine.

Hence, in the preceding slide, A ∈ T IM E(n2 ) because M1 decides A in time O(n2 ) and T IM E(n2 ) contains all languages that can be decided on O(n2 ).

Is there a machine that decides A asymptotically more quickly?

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.�

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.� stage 1:

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.� stage 1: O(n)

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.� stage 1: O(n) stage 2:

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.� stage 1: O(n) stage 2: (1 + log2 n)O(n) = O(n log n)

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.� stage 1: O(n) stage 2: (1 + log2 n)O(n) = O(n log n) stage 3:

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.� stage 1: O(n) stage 2: (1 + log2 n)O(n) = O(n log n) stage 3: O(n)

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.� stage 1: O(n) stage 2: (1 + log2 n)O(n) = O(n log n) stage 3: O(n) Thus the total time of M1 on an input of length n is O(n) + O(n log n) + O(n) = O(n log n).

Time Complexity

NP-completeness

M2 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Repeat as long as some 0s and some 1s remain on the tape: B Scan across the tape, checking whether the total number of 0s and 1s remaining is even or odd. If it is odd, reject.

B Scan again across the tape, crossing off every other 0 starting with the first 0, and then crossing off every other 1 starting with the first 1.

3. If no 0s and no 1s remain on the tape, accept. Otherwise, reject.� stage 1: O(n) stage 2: (1 + log2 n)O(n) = O(n log n) stage 3: O(n) Thus the total time of M1 on an input of length n is O(n) + O(n log n) + O(n) = O(n log n).

Hence, A ∈ T IM E(n log n)

Time Complexity

NP-completeness

M3 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Scan across the 0s on tape 1 until the first 1. At the same time, copy the 0s onto tape 2.

3. Scan across the 1s on tape 1 until the end of the input. For each 1 read on tape 1, cross off a 0 on tape 2. If all 0s are crossed off before all the 1s are read, reject.

4. If all the 0s have now been crossed off, accept. if any 0s remain, reject.�

Time Complexity

NP-completeness

M3 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Scan across the 0s on tape 1 until the first 1. At the same time, copy the 0s onto tape 2.

3. Scan across the 1s on tape 1 until the end of the input. For each 1 read on tape 1, cross off a 0 on tape 2. If all 0s are crossed off before all the 1s are read, reject.

4. If all the 0s have now been crossed off, accept. if any 0s remain, reject.�

Hence, the time complexity of A on a single-tape TM is O(n log n) and on a 2-tape TM it is O(n).

Time Complexity

NP-completeness

M3 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Scan across the 0s on tape 1 until the first 1. At the same time, copy the 0s onto tape 2.

3. Scan across the 1s on tape 1 until the end of the input. For each 1 read on tape 1, cross off a 0 on tape 2. If all 0s are crossed off before all the 1s are read, reject.

4. If all the 0s have now been crossed off, accept. if any 0s remain, reject.�

Hence, the time complexity of A on a single-tape TM is O(n log n) and on a 2-tape TM it is O(n).

In computability theory, the Church-Turing thesis implies that all reasonable models of computation are equivalent, that is, they all decide the same class of languages.

Time Complexity

NP-completeness

M3 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Scan across the 0s on tape 1 until the first 1. At the same time, copy the 0s onto tape 2.

3. Scan across the 1s on tape 1 until the end of the input. For each 1 read on tape 1, cross off a 0 on tape 2. If all 0s are crossed off before all the 1s are read, reject.

4. If all the 0s have now been crossed off, accept. if any 0s remain, reject.�

Hence, the time complexity of A on a single-tape TM is O(n log n) and on a 2-tape TM it is O(n).

In computability theory, the Church-Turing thesis implies that all reasonable models of computation are equivalent, that is, they all decide the same class of languages.

In complexity theory, the choice of model affects the time complexity of languages.

Time Complexity

NP-completeness

M3 = “On input string w: 1. Scan across the tape and reject if a 0 is found to the right of a 1.

2. Scan across the 0s on tape 1 until the first 1. At the same time, copy the 0s onto tape 2.

3. Scan across the 1s on tape 1 until the end of the input. For each 1 read on tape 1, cross off a 0 on tape 2. If all 0s are crossed off before all the 1s are read, reject.

4. If all the 0s have now been crossed off, accept. if any 0s remain, reject.�

Hence, the time complexity of A on a single-tape TM is O(n log n) and on a 2-tape TM it is O(n).

In computability theory, the Church-Turing thesis implies that all reasonable models of computation are equivalent, that is, they all decide the same class of languages.

In complexity theory, the choice of model affects the time complexity of languages.

In complexity theory, we classify computational problems according to their time complexity. But with which model do we measure time?

Time Complexity

NP-completeness

Complexity Relationships among models

Theorem Let t(n) be a function, where t(n) ≼ n. Then every t(n) time multitape Turing machine has an equivalent O(t2 (n)) time single-tape TM.

Time Complexity

NP-completeness

Complexity Relationships among models

Theorem Let t(n) be a function, where t(n) ≼ n. Then every t(n) time multitape Turing machine has an equivalent O(t2 (n)) time single-tape TM.

Theorem Let t(n) be a function, where t(n) ≼ n. Then every t(n) time nondeterministic single-tape Turing machine has an equivalent 2O(t(n)) time deterministic single-tape TM.

Time Complexity

NP-completeness

Complexity Relationships among models

Theorem Let t(n) be a function, where t(n) ≼ n. Then every t(n) time multitape Turing machine has an equivalent O(t2 (n)) time single-tape TM.

Theorem Let t(n) be a function, where t(n) ≼ n. Then every t(n) time nondeterministic single-tape Turing machine has an equivalent 2O(t(n)) time deterministic single-tape TM. The theorems above demonstrated at most a square or polynomial difference between the time complexity measures on deterministic single-tape and multitape TMs

Time Complexity

NP-completeness

Complexity Relationships among models

Theorem Let t(n) be a function, where t(n) ≼ n. Then every t(n) time multitape Turing machine has an equivalent O(t2 (n)) time single-tape TM.

Theorem Let t(n) be a function, where t(n) ≼ n. Then every t(n) time nondeterministic single-tape Turing machine has an equivalent 2O(t(n)) time deterministic single-tape TM. The theorems above demonstrated at most a square or polynomial difference between the time complexity measures on deterministic single-tape and multitape TMs and at most an exponential difference between the time complexity of problems on deterministic and nondeterministic TMs.

Time Complexity

NP-completeness

From here on we focus on aspects of time complexity theory that are unaffected by polynomial differences in running time. We consider such differences to be insignificant and ignore them. The decision to disregard polynomial differences doesn’t imply that we consider such differences unimportant. On the contrary, we certainly do consider the difference between time n and time n3 to be an important one.

But some questions that do not depend on polynomial differences are important too.

Time Complexity

NP-completeness

Definition P is the class of language that are decidable in polynomial time on a deterministic single-tape TM. [ P= T IM E(nk ) k

Time Complexity

NP-completeness

Definition P is the class of language that are decidable in polynomial time on a deterministic single-tape TM. [ P= T IM E(nk ) k

1. P is invariant for all models of computation that are polynomially equivalent to the deterministic single-tape TM, and 2. P roughly corresponds to the class of problems that are realistically solvable on a computer.

Time Complexity

NP-completeness

Examples of Problems in P

Time Complexity

NP-completeness

Examples of Problems in P P AT H = {< G, s, t > | G is a directed graph that has a directed path from s to t.}

Time Complexity

NP-completeness

Examples of Problems in P P AT H = {< G, s, t > | G is a directed graph that has a directed path from s to t.} P AT H â&#x2C6;&#x2C6; P

Time Complexity

NP-completeness

Examples of Problems in P P AT H = {< G, s, t > | G is a directed graph that has a directed path from s to t.} P AT H ∈ P M = “On input < G, s, t > where G is a directed graph with nodes s and t: 1. Place a mark on node s.

2. Repeat the ff until no addtl nodes are marked: B Scan all the edges of G. If an edge (a, b) is found going from a marked node a to an unmarked node b, mark node b.

3. If t is marked, accept. Otherwise, reject.”

Time Complexity

NP-completeness

Examples of Problems in P

Time Complexity

NP-completeness

Examples of Problems in P RELP RIM E = {< x, y > | x and y are relatively prime.}

Time Complexity

NP-completeness

Examples of Problems in P RELP RIM E = {< x, y > | x and y are relatively prime.} RELP RIM E â&#x2C6;&#x2C6; P

Time Complexity

NP-completeness

Examples of Problems in P RELP RIM E = {< x, y > | x and y are relatively prime.} RELP RIM E ∈ P E = “On input < x, y > where x and y are natural numbers in binary: 1. Repeat until y = 0: B Assign x ← x mod y.

B Exchange x and y.

2. Output x.”

Time Complexity

NP-completeness

Examples of Problems in P RELP RIM E = {< x, y > | x and y are relatively prime.} RELP RIM E ∈ P E = “On input < x, y > where x and y are natural numbers in binary: 1. Repeat until y = 0: B Assign x ← x mod y.

B Exchange x and y.

2. Output x.”

R = “On input < x, y > where x and y are natural numbers in binary: 1. Run E on < x, y >.

2. If the result is 1, accept. Otherwise, reject.”

Time Complexity

NP-completeness

There are problems, including many interesting and useful ones, wherein polynomial time algorithms that solve them arenâ&#x20AC;&#x2122;t known to exist.

Time Complexity

NP-completeness

There are problems, including many interesting and useful ones, wherein polynomial time algorithms that solve them arenâ&#x20AC;&#x2122;t known to exist. Why have we been unsuccessful in finding polynomial time algorithms for these problems?

Time Complexity

NP-completeness

There are problems, including many interesting and useful ones, wherein polynomial time algorithms that solve them arenâ&#x20AC;&#x2122;t known to exist. Why have we been unsuccessful in finding polynomial time algorithms for these problems? We donâ&#x20AC;&#x2122;t know.

Time Complexity

NP-completeness

There are problems, including many interesting and useful ones, wherein polynomial time algorithms that solve them arenâ&#x20AC;&#x2122;t known to exist. Why have we been unsuccessful in finding polynomial time algorithms for these problems? We donâ&#x20AC;&#x2122;t know. Perhaps these problems have, as yet undiscovered, polynomial time algorithms that rest on unknown principles.

Time Complexity

NP-completeness

There are problems, including many interesting and useful ones, wherein polynomial time algorithms that solve them arenâ&#x20AC;&#x2122;t known to exist. Why have we been unsuccessful in finding polynomial time algorithms for these problems? We donâ&#x20AC;&#x2122;t know. Perhaps these problems have, as yet undiscovered, polynomial time algorithms that rest on unknown principles. Or possibly some of these problems simply cannot be solved in polynomial time.

Time Complexity

NP-completeness

Time Complexity

HAM P AT H = {< G, s, t > | G is a directed graph with a Hamiltonian path from s to t.}

NP-completeness

Time Complexity

NP-completeness

HAM P AT H = {< G, s, t > | G is a directed graph with a Hamiltonian path from s to t.} We can easily obtain an exponential time algorithm for this problem by modifying the algorithm for P AT H. We need only add a check to verify that the potential path is Hamiltonian.

Time Complexity

NP-completeness

HAM P AT H = {< G, s, t > | G is a directed graph with a Hamiltonian path from s to t.} We can easily obtain an exponential time algorithm for this problem by modifying the algorithm for P AT H. We need only add a check to verify that the potential path is Hamiltonian.

No one knows whether HAM P AT H is solvable in polynomial time.

Time Complexity

NP-completeness

HAM P AT H = {< G, s, t > | G is a directed graph with a Hamiltonian path from s to t.} We can easily obtain an exponential time algorithm for this problem by modifying the algorithm for P AT H. We need only add a check to verify that the potential path is Hamiltonian.

No one knows whether HAM P AT H is solvable in polynomial time.

But the HAMPATH problem is polynomial verifiable.

Time Complexity

NP-completeness

HAM P AT H = {< G, s, t > | G is a directed graph with a Hamiltonian path from s to t.} We can easily obtain an exponential time algorithm for this problem by modifying the algorithm for P AT H. We need only add a check to verify that the potential path is Hamiltonian.

No one knows whether HAM P AT H is solvable in polynomial time.

But the HAMPATH problem is polynomial verifiable.

That is, even though we donâ&#x20AC;&#x2122;t know of a fast way to determine whether a graph contains a Hamiltonian path, if such a path were discovered somehow, we could easily convince someone else of its existence, simply by presenting it.

Time Complexity

NP-completeness

Definition A verifier for a language A is an algorithm V, where A = {w | V accepts < w, c > for some string c}.

We measure the time of a verifier only in terms of the length of w, so a polynomial time verifier runs in polynomial time in the length of w.

A language A is polynomially verifiable if it has a polynomial time verifier.

Time Complexity

NP-completeness

Definition A verifier for a language A is an algorithm V, where A = {w | V accepts < w, c > for some string c}.

We measure the time of a verifier only in terms of the length of w, so a polynomial time verifier runs in polynomial time in the length of w.

A language A is polynomially verifiable if it has a polynomial time verifier.

Definition NP is the class of languages that have polynomial time verifiers.

Time Complexity

Theorem A language is in NP iff it is decided by some nondeterministic polynomial time TM.

NP-completeness

Time Complexity

NP-completeness

Theorem A language is in NP iff it is decided by some nondeterministic polynomial time TM. The following nondeterministic TM decides the HAM P AT H problem in nondeterministic polynomial time.

N1 = “On input < G, s, t > where G is a directed graph with nodes s and t: 1. Write a list of m numbers p1 , · · · , pm , where m is the number of nodes in G. Each number in the list is nondeterministically selected to be between 1 and m.

2. Check for repetitions in the list. If any are found, reject.

3. Check whether s = p1 and t = pm . if either fail, reject.

4. For each i between 1 and m − 1, check whether (pi , pi+1 ) is an edge of G. If any are not, reject. Otherwise, all tests have been passed, so accept.”

Time Complexity

NP-completeness

Definition N T IM E(t(n)) = {L|L is a language decided by a O(t(n)) time nondeterministic TM}.

Corollary NP =

S

k

N T IM E(nk )

Time Complexity

NP-completeness

Examples of Problems in NP CLIQU E = {< G, k > |G is an undirected graph with a k-clique}

Time Complexity

NP-completeness

Examples of Problems in NP CLIQU E = {< G, k > |G is an undirected graph with a k-clique}

CLIQUE is in NP.

Time Complexity

NP-completeness

Examples of Problems in NP CLIQU E = {< G, k > |G is an undirected graph with a k-clique}

CLIQUE is in NP.

Time Complexity

NP-completeness

Examples of Problems in NP CLIQU E = {< G, k > |G is an undirected graph with a k-clique}

CLIQUE is in NP.

Time Complexity

NP-completeness

Examples of Problems in NP SU BSET SU M = {< S, t > |S = {x1P , · · · , xk } and for some {y1 , · · · , yl } ⊆ {x1 , · · · , xk }, we have yi = t}.

SU BSET SU M is in NP.

Time Complexity

NP-completeness

Examples of Problems in NP SU BSET SU M = {< S, t > |S = {x1P , · · · , xk } and for some {y1 , · · · , yl } ⊆ {x1 , · · · , xk }, we have yi = t}.

SU BSET SU M is in NP.

Time Complexity

NP-completeness

Examples of Problems in NP SU BSET SU M = {< S, t > |S = {x1P , · · · , xk } and for some {y1 , · · · , yl } ⊆ {x1 , · · · , xk }, we have yi = t}.

SU BSET SU M is in NP.

Time Complexity

NP-completeness

P = NP ?

Time Complexity

NP-completeness

â&#x20AC;˘ There are problems that are in NP but that are not known to be

in P.

Time Complexity

NP-completeness

â&#x20AC;˘ There are problems that are in NP but that are not known to be

in P. â&#x20AC;˘ Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal.

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal. No one has yet

proven the existence of a single language in NP that is not in P.

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal. No one has yet

proven the existence of a single language in NP that is not in P. • The question of whether P = N P is one of the greatest unsolved

problems in theoretical computer science!

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal. No one has yet

proven the existence of a single language in NP that is not in P. • The question of whether P = N P is one of the greatest unsolved

problems in theoretical computer science!

• The best method for solving languages in NP deterministically

uses exponential time.

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal. No one has yet

proven the existence of a single language in NP that is not in P. • The question of whether P = N P is one of the greatest unsolved

problems in theoretical computer science!

• The best method for solving languages in NP deterministically

uses exponential time.

N P ⊆ EXP T IM E =

[ k

k T IM E 2n

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal. No one has yet

proven the existence of a single language in NP that is not in P. • The question of whether P = N P is one of the greatest unsolved

problems in theoretical computer science!

• The best method for solving languages in NP deterministically

uses exponential time.

N P ⊆ EXP T IM E =

[

k T IM E 2n

k

but it is not yet known whether NP is contained in a smaller deterministic time complexity class.

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal. No one has yet

proven the existence of a single language in NP that is not in P. • The question of whether P = N P is one of the greatest unsolved

problems in theoretical computer science!

• The best method for solving languages in NP deterministically

uses exponential time.

N P ⊆ EXP T IM E =

[

k T IM E 2n

k

but it is not yet known whether NP is contained in a smaller deterministic time complexity class.

• Stephen Cook and Leovin Levin: They discovered certain problems in NP whose individual complexity is related to that of the entire class.

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal. No one has yet

proven the existence of a single language in NP that is not in P. • The question of whether P = N P is one of the greatest unsolved

problems in theoretical computer science!

• The best method for solving languages in NP deterministically

uses exponential time.

N P ⊆ EXP T IM E =

[

k T IM E 2n

k

but it is not yet known whether NP is contained in a smaller deterministic time complexity class.

• Stephen Cook and Leovin Levin: They discovered certain problems in NP whose individual complexity is related to that of the entire class.

If a polynomial time algorithm exists for any of these problems, all problems in NP would be polynomial time solvable.

Time Complexity

NP-completeness

• There are problems that are in NP but that are not known to be

in P. • Does that mean that polynomial verifiability is more powerful

than polynomial decidability?

• It is quite possible that P and NP could be equal. No one has yet

proven the existence of a single language in NP that is not in P. • The question of whether P = N P is one of the greatest unsolved

problems in theoretical computer science!

• The best method for solving languages in NP deterministically

uses exponential time.

N P ⊆ EXP T IM E =

[

k T IM E 2n

k

but it is not yet known whether NP is contained in a smaller deterministic time complexity class.

• Stephen Cook and Leovin Levin: They discovered certain problems in NP whose individual complexity is related to that of the entire class.

If a polynomial time algorithm exists for any of these problems, all problems in NP would be polynomial time solvable.

Time Complexity

NP-completeness

NP-completeness The phenomenon of NP-completeness is important for both theoretical and practical reasons. â&#x20AC;˘ theoretical: A researcher attempting to prove that P = N P only

needs to find a polynomial time algorithm for an NP-complete problem to achieve this goal. â&#x20AC;˘ practical: May prevent wasting time searching for a nonexistent

polynomial time algorithm to solve a particular problem.

Time Complexity

NP-completeness

NP-completeness The phenomenon of NP-completeness is important for both theoretical and practical reasons. â&#x20AC;˘ theoretical: A researcher attempting to prove that P = N P only

needs to find a polynomial time algorithm for an NP-complete problem to achieve this goal. â&#x20AC;˘ practical: May prevent wasting time searching for a nonexistent

polynomial time algorithm to solve a particular problem.

Note: The author (Michael Sipser, Introduction to the Theory of Computation) believes that P 6= N P , so proving that a problem is NP-complete is evidence of its nonpolynomiality.

Time Complexity

NP-completeness

Satisfiability Problem A Boolean formula is satisfiable if some assignment of 0s and 1s to the variables makes the formula evaluate to 1.

Time Complexity

NP-completeness

Satisfiability Problem A Boolean formula is satisfiable if some assignment of 0s and 1s to the variables makes the formula evaluate to 1.

Example φ = (x ∧ y) ∨ (x ∧ z)

Time Complexity

NP-completeness

Satisfiability Problem A Boolean formula is satisfiable if some assignment of 0s and 1s to the variables makes the formula evaluate to 1.

Example φ = (x ∧ y) ∨ (x ∧ z) The satisfiability problem is to test whether a Boolean formula is satisfiable.

SAT = {< φ > | φ is a satisfiable Boolean formula.}

Time Complexity

NP-completeness

Satisfiability Problem A Boolean formula is satisfiable if some assignment of 0s and 1s to the variables makes the formula evaluate to 1.

Example φ = (x ∧ y) ∨ (x ∧ z) The satisfiability problem is to test whether a Boolean formula is satisfiable.

SAT = {< φ > | φ is a satisfiable Boolean formula.}

Theorem (Cook-Levin Theorem) SAT ∈ P iff P = N P

Time Complexity

NP-completeness

Polynomial Time Reducibility When problem A is efficiently reducible to problem B, an efficient solution to B can be used to solve A efficiently.

Time Complexity

NP-completeness

Polynomial Time Reducibility When problem A is efficiently reducible to problem B, an efficient solution to B can be used to solve A efficiently.

Definition A function f : Σ∗ → Σ∗ is a polynomial time computable function if some polynomial time Turing machine M exists that halts with just f (w) on its tape, when started on any input w.

Time Complexity

NP-completeness

Polynomial Time Reducibility When problem A is efficiently reducible to problem B, an efficient solution to B can be used to solve A efficiently.

Definition A function f : Σ∗ → Σ∗ is a polynomial time computable function if some polynomial time Turing machine M exists that halts with just f (w) on its tape, when started on any input w.

Definition Language A is polynomial time mapping reducible, or simply polynomial time reducible, to language B, written A ≤P B, if a polynomial time computable function f : Σ∗ → Σ∗ exists, where for every w, w ∈ A ⇐⇒ f (w) ∈ B.

Time Complexity

NP-completeness

Polynomial Time Reducibility When problem A is efficiently reducible to problem B, an efficient solution to B can be used to solve A efficiently.

Definition A function f : Σ∗ → Σ∗ is a polynomial time computable function if some polynomial time Turing machine M exists that halts with just f (w) on its tape, when started on any input w.

Definition Language A is polynomial time mapping reducible, or simply polynomial time reducible, to language B, written A ≤P B, if a polynomial time computable function f : Σ∗ → Σ∗ exists, where for every w, w ∈ A ⇐⇒ f (w) ∈ B. The function f is called the polynomial time reduction of A to B.

Time Complexity

NP-completeness

Polynomial Time Reducibility When problem A is efficiently reducible to problem B, an efficient solution to B can be used to solve A efficiently.

Definition A function f : Σ∗ → Σ∗ is a polynomial time computable function if some polynomial time Turing machine M exists that halts with just f (w) on its tape, when started on any input w.

Definition Language A is polynomial time mapping reducible, or simply polynomial time reducible, to language B, written A ≤P B, if a polynomial time computable function f : Σ∗ → Σ∗ exists, where for every w, w ∈ A ⇐⇒ f (w) ∈ B. The function f is called the polynomial time reduction of A to B. If one language is polynomial time reducible to a language already known to have a polynomial time solution, we obtain a polynomial time solution to the original language.

Time Complexity

Theorem If A ≤P B and B ∈ P , then A ∈ P .

NP-completeness

Time Complexity

NP-completeness

Theorem If A ≤P B and B ∈ P , then A ∈ P .

Proof Let M be the polynomial time algorithm deciding B and f be the polynomial time reduction from A to B. We describe a polynomial time algorithm N deciding A as follows:

Time Complexity

NP-completeness

Theorem If A ≤P B and B ∈ P , then A ∈ P .

Proof Let M be the polynomial time algorithm deciding B and f be the polynomial time reduction from A to B. We describe a polynomial time algorithm N deciding A as follows: N = “On input w: 1. Compute f (w) 2. Run M on input f (w) and output whatever M outputs.”

Time Complexity

NP-completeness

Theorem If A ≤P B and B ∈ P , then A ∈ P .

Proof Let M be the polynomial time algorithm deciding B and f be the polynomial time reduction from A to B. We describe a polynomial time algorithm N deciding A as follows: N = “On input w: 1. Compute f (w) 2. Run M on input f (w) and output whatever M outputs.”

• w ∈ A whenever f (w) ∈ B because f is a reduction from A to B

Time Complexity

NP-completeness

Theorem If A ≤P B and B ∈ P , then A ∈ P .

Proof Let M be the polynomial time algorithm deciding B and f be the polynomial time reduction from A to B. We describe a polynomial time algorithm N deciding A as follows: N = “On input w: 1. Compute f (w) 2. Run M on input f (w) and output whatever M outputs.”

• w ∈ A whenever f (w) ∈ B because f is a reduction from A to B • M accepts f (w) whenever w ∈ A

Time Complexity

NP-completeness

Theorem If A ≤P B and B ∈ P , then A ∈ P .

Proof Let M be the polynomial time algorithm deciding B and f be the polynomial time reduction from A to B. We describe a polynomial time algorithm N deciding A as follows: N = “On input w: 1. Compute f (w) 2. Run M on input f (w) and output whatever M outputs.”

• w ∈ A whenever f (w) ∈ B because f is a reduction from A to B • M accepts f (w) whenever w ∈ A • N runs in polynomial time because each of its two stages runs in

polynomial time.

Time Complexity

NP-completeness

3SAT problem A literal is a Boolean variable or a negated Boolean variable, as in x or x.

Time Complexity

NP-completeness

3SAT problem A literal is a Boolean variable or a negated Boolean variable, as in x or x.

A clause is several literals connected with ∨s, as in (x1 ∨ x2 ∨ x3 ).

Time Complexity

NP-completeness

3SAT problem A literal is a Boolean variable or a negated Boolean variable, as in x or x.

A clause is several literals connected with ∨s, as in (x1 ∨ x2 ∨ x3 ).

A formula is in conjunctive normal form, called a cnf − f ormula, if it comprises several clauses connected with ∧s,

Time Complexity

NP-completeness

3SAT problem A literal is a Boolean variable or a negated Boolean variable, as in x or x.

A clause is several literals connected with ∨s, as in (x1 ∨ x2 ∨ x3 ).

A formula is in conjunctive normal form, called a cnf − f ormula, if it comprises several clauses connected with ∧s, as in (x1 ∨ x2 ∨ x3 ∨ x4 ) ∧ (x3 ∨ x5 ∨ x6 ) ∧ (x3 ∨ x6 )

Time Complexity

NP-completeness

3SAT problem A literal is a Boolean variable or a negated Boolean variable, as in x or x.

A clause is several literals connected with ∨s, as in (x1 ∨ x2 ∨ x3 ).

A formula is in conjunctive normal form, called a cnf − f ormula, if it comprises several clauses connected with ∧s, as in (x1 ∨ x2 ∨ x3 ∨ x4 ) ∧ (x3 ∨ x5 ∨ x6 ) ∧ (x3 ∨ x6 ) It is a 3cnf − f ormula, if all the clauses have three literals.

Time Complexity

NP-completeness

3SAT problem A literal is a Boolean variable or a negated Boolean variable, as in x or x.

A clause is several literals connected with ∨s, as in (x1 ∨ x2 ∨ x3 ).

A formula is in conjunctive normal form, called a cnf − f ormula, if it comprises several clauses connected with ∧s, as in (x1 ∨ x2 ∨ x3 ∨ x4 ) ∧ (x3 ∨ x5 ∨ x6 ) ∧ (x3 ∨ x6 ) It is a 3cnf − f ormula, if all the clauses have three literals.

Let 3SAT = {< φ > | φ is a satisfiable 3cnf-formula}.

Time Complexity

NP-completeness

3SAT problem A literal is a Boolean variable or a negated Boolean variable, as in x or x.

A clause is several literals connected with ∨s, as in (x1 ∨ x2 ∨ x3 ).

A formula is in conjunctive normal form, called a cnf − f ormula, if it comprises several clauses connected with ∧s, as in (x1 ∨ x2 ∨ x3 ∨ x4 ) ∧ (x3 ∨ x5 ∨ x6 ) ∧ (x3 ∨ x6 ) It is a 3cnf − f ormula, if all the clauses have three literals.

Let 3SAT = {< φ > | φ is a satisfiable 3cnf-formula}.

Theorem 3SAT is polynomial time reducible to CLIQUE.

Time Complexity

Proof: Let φ = (a1 ∨ b1 ∨ c1 ) ∧ (a2 ∨ b2 ∨ c2 ) ∧ · · · ∧ (ak ∨ bk ∨ ck ).

NP-completeness

Time Complexity

Proof: Let φ = (a1 ∨ b1 ∨ c1 ) ∧ (a2 ∨ b2 ∨ c2 ) ∧ · · · ∧ (ak ∨ bk ∨ ck ).

The reduction f generates the string < G, k >, where G is an undirected graph defined as:

NP-completeness

Time Complexity

NP-completeness

Proof: Let φ = (a1 ∨ b1 ∨ c1 ) ∧ (a2 ∨ b2 ∨ c2 ) ∧ · · · ∧ (ak ∨ bk ∨ ck ).

The reduction f generates the string < G, k >, where G is an undirected graph defined as: • The nodes in G are organized into k groups of three nodes each called the triples, t1 , . . . , tk .

• Each triple corresponds to one of the clauses in φ, and • each node in a triple corresponds to a literal in the associated

clause. Label each node of G with its corresponding literal in φ.

• The edges of G connect all but two types of pairs of nodes in G. • No edge is present between nodes in the same triple and • no edge is present between two nodes with contradictory labels, as in x2 and x2 .

Time Complexity

NP-completeness

Proof: Let φ = (a1 ∨ b1 ∨ c1 ) ∧ (a2 ∨ b2 ∨ c2 ) ∧ · · · ∧ (ak ∨ bk ∨ ck ).

The reduction f generates the string < G, k >, where G is an undirected graph defined as: • The nodes in G are organized into k groups of three nodes each called the triples, t1 , . . . , tk .

• Each triple corresponds to one of the clauses in φ, and • each node in a triple corresponds to a literal in the associated

clause. Label each node of G with its corresponding literal in φ.

• The edges of G connect all but two types of pairs of nodes in G. • No edge is present between nodes in the same triple and • no edge is present between two nodes with contradictory labels, as in x2 and x2 .

We show that φ has a satisfiable iff G has k-clique.

Time Complexity

Suppose that Ď&#x2020; has a satisfying assignment.

NP-completeness

Time Complexity

NP-completeness

Suppose that Ď&#x2020; has a satisfying assignment. â&#x20AC;˘ In that satisfying assignment, at least one literal is true in every

clause.

Time Complexity

NP-completeness

Suppose that φ has a satisfying assignment.

• In that satisfying assignment, at least one literal is true in every

clause.

• In each triple of G, we select one node corresponding to a true

literal in the satisfying assignment. If more than one literal is true in a particular clause, we choose one of the true literals arbitrarily.

Time Complexity

NP-completeness

Suppose that φ has a satisfying assignment.

• In that satisfying assignment, at least one literal is true in every

clause.

• In each triple of G, we select one node corresponding to a true

literal in the satisfying assignment. If more than one literal is true in a particular clause, we choose one of the true literals arbitrarily.

• The nodes just selected form a k-clique.

Time Complexity

NP-completeness

Suppose that φ has a satisfying assignment.

• In that satisfying assignment, at least one literal is true in every

clause.

• In each triple of G, we select one node corresponding to a true

literal in the satisfying assignment. If more than one literal is true in a particular clause, we choose one of the true literals arbitrarily.

• The nodes just selected form a k-clique.

Therefore G contains a k-clique.

Time Complexity

Suppose that G has a k-clique.

NP-completeness

Time Complexity

NP-completeness

Suppose that G has a k-clique.

• No two of the clique’s nodes occur in the same triple because

nodes in the same triple aren’t connected by edges. Therefore each of the k triples contains exactly one of the k clique nodes.

Time Complexity

NP-completeness

Suppose that G has a k-clique.

• No two of the clique’s nodes occur in the same triple because

nodes in the same triple aren’t connected by edges. Therefore each of the k triples contains exactly one of the k clique nodes.

• We assign truth values to the variables of φ so that each literal

labeling a clique node is made true. Doing so is always possible because two nodes labeled in a contradictory way are not connected by an edge and hence both can’t be in the clique.

Time Complexity

NP-completeness

Suppose that G has a k-clique.

• No two of the clique’s nodes occur in the same triple because

nodes in the same triple aren’t connected by edges. Therefore each of the k triples contains exactly one of the k clique nodes.

• We assign truth values to the variables of φ so that each literal

labeling a clique node is made true. Doing so is always possible because two nodes labeled in a contradictory way are not connected by an edge and hence both can’t be in the clique.

• Each triple contains a clique node and hence each clause contains

a literal that is assigned TRUE.

Time Complexity

NP-completeness

Suppose that G has a k-clique.

• No two of the clique’s nodes occur in the same triple because

nodes in the same triple aren’t connected by edges. Therefore each of the k triples contains exactly one of the k clique nodes.

• We assign truth values to the variables of φ so that each literal

labeling a clique node is made true. Doing so is always possible because two nodes labeled in a contradictory way are not connected by an edge and hence both can’t be in the clique.

• Each triple contains a clique node and hence each clause contains

a literal that is assigned TRUE. Therefore φ is satisfiable.

Time Complexity

NP-completeness

Suppose that G has a k-clique.

• No two of the clique’s nodes occur in the same triple because

nodes in the same triple aren’t connected by edges. Therefore each of the k triples contains exactly one of the k clique nodes.

• We assign truth values to the variables of φ so that each literal

labeling a clique node is made true. Doing so is always possible because two nodes labeled in a contradictory way are not connected by an edge and hence both can’t be in the clique.

• Each triple contains a clique node and hence each clause contains

a literal that is assigned TRUE. Therefore φ is satisfiable.

If CLIQUE is solvable in polynomial time, so is 3SAT.

Time Complexity

NP-completeness

Suppose that G has a k-clique.

• No two of the clique’s nodes occur in the same triple because

nodes in the same triple aren’t connected by edges. Therefore each of the k triples contains exactly one of the k clique nodes.

• We assign truth values to the variables of φ so that each literal

labeling a clique node is made true. Doing so is always possible because two nodes labeled in a contradictory way are not connected by an edge and hence both can’t be in the clique.

• Each triple contains a clique node and hence each clause contains

a literal that is assigned TRUE. Therefore φ is satisfiable.

If CLIQUE is solvable in polynomial time, so is 3SAT. Polynomial time reducibility allows us to link their complexities.

Time Complexity

Definition A language B is NP-complete if it satisfies two conditions: 1. B is in NP, and 2. every A in NP is polynomial time reducible to B.

NP-completeness

Time Complexity

Definition A language B is NP-complete if it satisfies two conditions: 1. B is in NP, and 2. every A in NP is polynomial time reducible to B.

Theorem If B is NP-complete and B â&#x2C6;&#x2C6; P , then P = N P .

NP-completeness

Time Complexity

NP-completeness

Definition A language B is NP-complete if it satisfies two conditions: 1. B is in NP, and 2. every A in NP is polynomial time reducible to B.

Theorem If B is NP-complete and B â&#x2C6;&#x2C6; P , then P = N P .

Theorem If B is NP-complete and B â&#x2030;¤P C for C in NP, then C is NP-complete.

Time Complexity

NP-completeness

Definition A language B is NP-complete if it satisfies two conditions: 1. B is in NP, and 2. every A in NP is polynomial time reducible to B.

Theorem If B is NP-complete and B â&#x2C6;&#x2C6; P , then P = N P .

Theorem If B is NP-complete and B â&#x2030;¤P C for C in NP, then C is NP-complete.

Once we have one NP-complete problem, we may obtain others by polynomial reduction from it.

Time Complexity

NP-completeness

3SAT problem A literal is a Boolean variable or a negated Boolean variable, as in x or x.

A clause is several literals connected with ∨s, as in (x1 ∨ x2 ∨ x3 ).

A formula is in conjunctive normal form, called a cnf − f ormula, if it comprises several clauses connected with ∧s, as in (x1 ∨ x2 ∨ x3 ∨ x4 ) ∧ (x3 ∨ x5 ∨ x6 ) ∧ (x3 ∨ x6 ) It is a 3cnf − f ormula, if all the clauses have three literals.

Let 3SAT = {< φ > | φ is a satisfiable 3cnf-formula}.

Theorem 3SAT is polynomial time reducible to CLIQUE.

Time Complexity

NP-completeness

Proof: Let φ = (a1 ∨ b1 ∨ c1 ) ∧ (a2 ∨ b2 ∨ c2 ) ∧ · · · ∧ (ak ∨ bk ∨ ck ).

The reduction f generates the string < G, k >, where G is an undirected graph defined as: • The nodes in G are organized into k groups of three nodes each called the triples, t1 , . . . , tk .

• Each triple corresponds to one of the clauses in φ, and • each node in a triple corresponds to a literal in the associated

clause. Label each node of G with its corresponding literal in φ.

• The edges of G connect all but two types of pairs of nodes in G. • No edge is present between nodes in the same triple and • no edge is present between two nodes with contradictory labels, as in x2 and x2 .

We show that φ has a satisfiable iff G has k-clique.

Time Complexity

NP-completeness

Suppose that φ has a satisfying assignment.

• In that satisfying assignment, at least one literal is true in every

clause.

• In each triple of G, we select one node corresponding to a true

literal in the satisfying assignment. If more than one literal is true in a particular clause, we choose one of the true literals arbitrarily.

• The nodes just selected form a k-clique.

Therefore G contains a k-clique.

Time Complexity

NP-completeness

Suppose that G has a k-clique.

• No two of the clique’s nodes occur in the same triple because

nodes in the same triple aren’t connected by edges. Therefore each of the k triples contains exactly one of the k clique nodes.

• We assign truth values to the variables of φ so that each literal

labeling a clique node is made true. Doing so is always possible because two nodes labeled in a contradictory way are not connected by an edge and hence both can’t be in the clique.

• Each triple contains a clique node and hence each clause contains

a literal that is assigned TRUE. Therefore φ is satisfiable.

If CLIQUE is solvable in polynomial time, so is 3SAT. Polynomial time reducibility allows us to link their complexities.

Time Complexity

Definition A language B is NP-complete if it satisfies two conditions: 1. B is in NP, and 2. every A in NP is polynomial time reducible to B.

NP-completeness

Time Complexity

Definition A language B is NP-complete if it satisfies two conditions: 1. B is in NP, and 2. every A in NP is polynomial time reducible to B.

Theorem If B is NP-complete and B â&#x2C6;&#x2C6; P , then P = N P .

NP-completeness

Time Complexity

NP-completeness

Definition A language B is NP-complete if it satisfies two conditions: 1. B is in NP, and 2. every A in NP is polynomial time reducible to B.

Theorem If B is NP-complete and B â&#x2C6;&#x2C6; P , then P = N P .

Theorem If B is NP-complete and B â&#x2030;¤P C for C in NP, then C is NP-complete.

Time Complexity

NP-completeness

Definition A language B is NP-complete if it satisfies two conditions: 1. B is in NP, and 2. every A in NP is polynomial time reducible to B.

Theorem If B is NP-complete and B â&#x2C6;&#x2C6; P , then P = N P .

Theorem If B is NP-complete and B â&#x2030;¤P C for C in NP, then C is NP-complete.

Once we have one NP-complete problem, we may obtain others by polynomial reduction from it.

Time Complexity

NP-completeness

Cook-Levin Theorem

Theorem SAT is NP-complete.

Time Complexity

NP-completeness

Cook-Levin Theorem

Theorem SAT is NP-complete.

Proof: B Show that SAT is in NP.

Time Complexity

NP-completeness

Cook-Levin Theorem

Theorem SAT is NP-complete.

Proof: B Show that SAT is in NP. A nondeterministic polynomial TM can guess an assignment to a given formula Ď&#x2020; and accept if the assignment satisfies Ď&#x2020;.

Time Complexity

NP-completeness

Cook-Levin Theorem

Theorem SAT is NP-complete.

Proof: B Show that SAT is in NP. A nondeterministic polynomial TM can guess an assignment to a given formula Ď&#x2020; and accept if the assignment satisfies Ď&#x2020;.

B Show that every language A in NP is polynomial time reducible to SAT.

Time Complexity

NP-completeness

Cook-Levin Theorem

Theorem SAT is NP-complete.

Proof: B Show that SAT is in NP. A nondeterministic polynomial TM can guess an assignment to a given formula Ď&#x2020; and accept if the assignment satisfies Ď&#x2020;.

B Show that every language A in NP is polynomial time reducible to SAT. Take any language A in NP. Let N be a nondeterministic TM that decides A in nk time for some constant k.

Time Complexity

NP-completeness

A tableau for N on w is an nk Ă&#x2014; nk table whose rows are the configurations of a branch of the computation of N on input w.

Time Complexity

NP-completeness

A tableau for N on w is an nk Ă&#x2014; nk table whose rows are the configurations of a branch of the computation of N on input w.

The problem of determining whether N accepts w is equivalent to the problem of determining whether an accepting tableau for N on w exists.

Time Complexity

NP-completeness

We need a polynomial time reduction f from A to SAT. On input w, the reduction produces a formula Ď&#x2020;.

Time Complexity

NP-completeness

We need a polynomial time reduction f from A to SAT. On input w, the reduction produces a formula φ.

• Say that Q and Γ are the state set and tape alphabet of N .

Let C = Q ∪ Γ ∪ {#}.

• For each i and j between 1 and nk and for each s in C we have a

variable xi,j,s . xi,j,s takes on the value 1 if cell[i, j] of the tableau contains an s.

• The formula φ is the AND of four parts

φcell ∧ φstart ∧ φmove ∧ φaccept .

Time Complexity

NP-completeness

Time Complexity

NP-completeness

Time Complexity

NP-completeness

Time Complexity

NP-completeness

Time Complexity

NP-completeness

• Estimate the number of variables: • An nk × nk table contains n2k cells.

• Each cell has l = |C| variables associated with it.

• Because l depends only on TM N and not on the length of the input, the total number of variables is O(n2k ).

Time Complexity

NP-completeness

• Estimate the number of variables: • An nk × nk table contains n2k cells.

• Each cell has l = |C| variables associated with it.

• Because l depends only on TM N and not on the length of the input, the total number of variables is O(n2k ).

• Estimate the size of each of the parts of φ: • φcell contains a fixed-size fragment of the formula for each cell, so its size is O(n2k ).

• φstart has a fragment for each cell in the top row, so its size is O(nk ).

• φmove and φaccept each contains a fixed-size fragment of the formula for each cell, so its size is O(n2k ).

• Thus φ’s total size is O(n2k ).

• This bound is sufficient for our purposes because it shows that the size of φ is polynomial in n. If it were more than polynomial, the reduction wouldn’t have any chance of generating it in polynomial time.

Time Complexity

NP-completeness

• Estimate the number of variables: • An nk × nk table contains n2k cells.

• Each cell has l = |C| variables associated with it.

• Because l depends only on TM N and not on the length of the input, the total number of variables is O(n2k ).

• Estimate the size of each of the parts of φ: • φcell contains a fixed-size fragment of the formula for each cell, so its size is O(n2k ).

• φstart has a fragment for each cell in the top row, so its size is O(nk ).

• φmove and φaccept each contains a fixed-size fragment of the formula for each cell, so its size is O(n2k ).

• Thus φ’s total size is O(n2k ).

• This bound is sufficient for our purposes because it shows that the size of φ is polynomial in n. If it were more than polynomial, the reduction wouldn’t have any chance of generating it in polynomial time.

• Each component of the formula is composed of many nearly identical fragments, which differ only at the indices in a simple way. Therefore we may easily construct a reduction that produces φ in polynomial time from the input w.

Time Complexity

NP-completeness

• Estimate the number of variables: • An nk × nk table contains n2k cells.

• Each cell has l = |C| variables associated with it.

• Because l depends only on TM N and not on the length of the input, the total number of variables is O(n2k ).

• Estimate the size of each of the parts of φ: • φcell contains a fixed-size fragment of the formula for each cell, so its size is O(n2k ).

• φstart has a fragment for each cell in the top row, so its size is O(nk ).

• φmove and φaccept each contains a fixed-size fragment of the formula for each cell, so its size is O(n2k ).

• Thus φ’s total size is O(n2k ).

• This bound is sufficient for our purposes because it shows that the size of φ is polynomial in n. If it were more than polynomial, the reduction wouldn’t have any chance of generating it in polynomial time.

• Each component of the formula is composed of many nearly identical fragments, which differ only at the indices in a simple way. Therefore we may easily construct a reduction that produces φ in polynomial time from the input w.

Therefore, SAT is NP-complete.

Time Complexity

Corollary 3SAT is NP-complete

NP-completeness

Time Complexity

NP-completeness

Corollary 3SAT is NP-complete B Obviously, 3SAT is in NP. B It remains to be shown that all languages in NP reduce to 3SAT in polynomial time.

Time Complexity

NP-completeness

Corollary 3SAT is NP-complete B Obviously, 3SAT is in NP. B It remains to be shown that all languages in NP reduce to 3SAT in polynomial time. â&#x20AC;˘ We modify the proof of SAT so that it directly produces a 3-cnf formula.

Time Complexity

NP-completeness

Corollary 3SAT is NP-complete B Obviously, 3SAT is in NP. B It remains to be shown that all languages in NP reduce to 3SAT in polynomial time.

• We modify the proof of SAT so that it directly produces a 3-cnf formula.

• Note that φcell , φaccept , φstart are already in cnf.

Time Complexity

NP-completeness

Corollary 3SAT is NP-complete B Obviously, 3SAT is in NP. B It remains to be shown that all languages in NP reduce to 3SAT in polynomial time.

• We modify the proof of SAT so that it directly produces a 3-cnf formula.

• Note that φcell , φaccept , φstart are already in cnf.

• φmove is a big AND of subformulas, each of which is an OR of ANDs.

Time Complexity

NP-completeness

Corollary 3SAT is NP-complete B Obviously, 3SAT is in NP. B It remains to be shown that all languages in NP reduce to 3SAT in polynomial time.

• We modify the proof of SAT so that it directly produces a 3-cnf formula.

• Note that φcell , φaccept , φstart are already in cnf.

• φmove is a big AND of subformulas, each of which is an OR of ANDs. The distributive laws state that we can replace an OR of ANDs with an equivalent AND of ORs.

Time Complexity

NP-completeness

Corollary 3SAT is NP-complete B Obviously, 3SAT is in NP. B It remains to be shown that all languages in NP reduce to 3SAT in polynomial time.

• We modify the proof of SAT so that it directly produces a 3-cnf formula.

• Note that φcell , φaccept , φstart are already in cnf.

• φmove is a big AND of subformulas, each of which is an OR of ANDs. The distributive laws state that we can replace an OR of ANDs with an equivalent AND of ORs. • If the clause contains l literals, (a1 ∨ a2 ∨ · · · ∨ al ), we can replace it with the l − 2 clauses (a1 ∨ a2 ∨ z1 ) ∧ (z1 ∨ a3 ∨ z2 ) ∧ (z2 ∨ a4 ∨ z3 ) ∧ · · · ∧ (zl−3 ∨ al−1 ∨ al ).

Time Complexity

Corollary CLIQUE is NP-complete.

NP-completeness

Time Complexity

NP-completeness

reduction from 3SAT When constructing a polynomial time reduction from 3SAT to a language, we look for structures in that language that can simulate the variables and clauses in Boolean formulas.

Let us call this structures as gadgets.

Time Complexity

NP-completeness

If G is an undirected graph, a vertex cover of G is a subset of the nodes where every edge of G touches one of those nodes.

Time Complexity

NP-completeness

If G is an undirected graph, a vertex cover of G is a subset of the nodes where every edge of G touches one of those nodes. The vertex cover problem asks whether a graph contains a vertex cover a specified size: V ERT EX â&#x2C6;&#x2019; COV ER = {< G, k > where G is an undirected graph that has a k-node vertex cover}.

Time Complexity

NP-completeness

If G is an undirected graph, a vertex cover of G is a subset of the nodes where every edge of G touches one of those nodes. The vertex cover problem asks whether a graph contains a vertex cover a specified size: V ERT EX â&#x2C6;&#x2019; COV ER = {< G, k > where G is an undirected graph that has a k-node vertex cover}.

Theorem VERTEX-COVER is NP-complete.

Time Complexity

NP-completeness

If G is an undirected graph, a vertex cover of G is a subset of the nodes where every edge of G touches one of those nodes. The vertex cover problem asks whether a graph contains a vertex cover a specified size: V ERT EX − COV ER = {< G, k > where G is an undirected graph that has a k-node vertex cover}.

Theorem VERTEX-COVER is NP-complete.

We need to show that 3SAT is polynomial time reducible to V ERT EX − COV ER. The reduction maps a 3cnf φ into a graph G and a number k, so that φ is satisfiable whenever G has a vertex cover with k nodes.

Time Complexity

NP-completeness

â&#x20AC;˘ For each variable X in Ď&#x2020;, we produce an edge connecting two

nodes.

We label the 2 nodes in this gadget x and x.

Time Complexity

NP-completeness

â&#x20AC;˘ For each variable X in Ď&#x2020;, we produce an edge connecting two

nodes.

We label the 2 nodes in this gadget x and x. â&#x20AC;˘ Each clause is a triple of three nodes that are labeled with the 3 literals of the clause.

These nodes are connected to each other and to the nodes in the variables gadgets that have the identical labels.

Time Complexity

NP-completeness

• For each variable X in φ, we produce an edge connecting two

nodes.

We label the 2 nodes in this gadget x and x.

• Each clause is a triple of three nodes that are labeled with the 3 literals of the clause.

These nodes are connected to each other and to the nodes in the variables gadgets that have the identical labels.

• Thus the total number of nodes that appear in G is 2m + 3l where φ has m variables and l clauses. Let k = m + 2l.

Time Complexity

NP-completeness

• For each variable X in φ, we produce an edge connecting two

nodes.

We label the 2 nodes in this gadget x and x.

• Each clause is a triple of three nodes that are labeled with the 3 literals of the clause.

These nodes are connected to each other and to the nodes in the variables gadgets that have the identical labels.

• Thus the total number of nodes that appear in G is 2m + 3l where φ has m variables and l clauses. Let k = m + 2l.

Time Complexity

NP-completeness

We need to show that Ď&#x2020; is satisfiable if and only if G has a vertex cover with k nodes.

Time Complexity

NP-completeness

We need to show that Ď&#x2020; is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose Ď&#x2020; is satisfiable.

Time Complexity

NP-completeness

We need to show that φ is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose φ is satisfiable.

• Put the nodes of the variable gadgets that correspond to the

TRUE literals in the assignment into the vertex cover.

Time Complexity

NP-completeness

We need to show that φ is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose φ is satisfiable.

• Put the nodes of the variable gadgets that correspond to the

TRUE literals in the assignment into the vertex cover.

• Select one TRUE literal in every clause and put the remaining

two nodes from every clause gadget into the vertex cover.

Time Complexity

NP-completeness

We need to show that φ is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose φ is satisfiable.

• Put the nodes of the variable gadgets that correspond to the

TRUE literals in the assignment into the vertex cover.

• Select one TRUE literal in every clause and put the remaining

two nodes from every clause gadget into the vertex cover.

• These k = m + 2l nodes cover all edges. (Why?)

Time Complexity

NP-completeness

We need to show that φ is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose φ is satisfiable.

• Put the nodes of the variable gadgets that correspond to the

TRUE literals in the assignment into the vertex cover.

• Select one TRUE literal in every clause and put the remaining

two nodes from every clause gadget into the vertex cover.

• These k = m + 2l nodes cover all edges. (Why?) • Hence, G has a vertex cover with k nodes.

Time Complexity

NP-completeness

We need to show that Ď&#x2020; is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose G has a vertex cover with k nodes.

Time Complexity

NP-completeness

We need to show that Ď&#x2020; is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose G has a vertex cover with k nodes. â&#x20AC;˘ The vertex must contain one node in each variable gadget and

two in every clausegadget in order to cover the edges of the variable and the edges within the clause gadgets.

Time Complexity

NP-completeness

We need to show that Ď&#x2020; is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose G has a vertex cover with k nodes. â&#x20AC;˘ The vertex must contain one node in each variable gadget and

two in every clausegadget in order to cover the edges of the variable and the edges within the clause gadgets. That accounts for all the k nodes, so none are left over.

Time Complexity

NP-completeness

We need to show that Ď&#x2020; is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose G has a vertex cover with k nodes. â&#x20AC;˘ The vertex must contain one node in each variable gadget and

two in every clausegadget in order to cover the edges of the variable and the edges within the clause gadgets. That accounts for all the k nodes, so none are left over. â&#x20AC;˘ We take the nodes of the variable gadgets that are in the vertex

cover and assign the corresponding literals TRUE.

Time Complexity

NP-completeness

We need to show that φ is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose G has a vertex cover with k nodes.

• The vertex must contain one node in each variable gadget and

two in every clausegadget in order to cover the edges of the variable and the edges within the clause gadgets. That accounts for all the k nodes, so none are left over.

• We take the nodes of the variable gadgets that are in the vertex

cover and assign the corresponding literals TRUE. This assignment satisfies φ. (Why?)

Time Complexity

NP-completeness

We need to show that φ is satisfiable if and only if G has a vertex cover with k nodes.

B Suppose G has a vertex cover with k nodes.

• The vertex must contain one node in each variable gadget and

two in every clausegadget in order to cover the edges of the variable and the edges within the clause gadgets. That accounts for all the k nodes, so none are left over.

• We take the nodes of the variable gadgets that are in the vertex

cover and assign the corresponding literals TRUE. This assignment satisfies φ. (Why?) • Hence, φ has a satisfying assignment.

Time Complexity

Theorem SUBSETSUM is NP-complete.

NP-completeness

Time Complexity

Theorem SUBSETSUM is NP-complete.

We need to show that 3SAT is polynomial time reducible to SU BSET SU M .

NP-completeness

Time Complexity

NP-completeness

Theorem SUBSETSUM is NP-complete.

We need to show that 3SAT is polynomial time reducible to SU BSET SU M .

The reduction maps a 3cnf φ with variables x1 , · · · , xn and clauses c1 , · · · ck into an instance of the SU BSET SU M problem < S, t >wherein the elements of S and the number t are the rows in the table on the next slide.

Time Complexity

NP-completeness

Time Complexity

NP-completeness

The reduction maps a 3cnf φ with variables x1 , · · · , xn and clauses c1 , · · · ck into an instance of the SU BSET SU M problem < S, t >wherein the elements of S and the number t are the rows in the table on the next slide expresses in ordinary decimal notation.

• S contains one pair of numbers, yi , zi for each variable xi in φ.

• The decimal representation is in two parts.

The left-hand part comprises a 1 followed by l − i 0s.

The right-hand part contains one digit for each clause.

• the j th digit of yi is 1 if clause cj contains literal xi • the j th digit of zi is 1 if clause cj contains literal xi

• S also contains one pair of numbers gj , hj for each clause cj .

These numbers are equal and consist of a 1 followed by k − j 0s.

• The target number t, consists of l 1s followed by k 3s.

Time Complexity

Suppose Ď&#x2020; is satisfiable.

NP-completeness

Time Complexity

NP-completeness

Suppose φ is satisfiable.

We construct a subset of S as follows: • Select yi if xi is assigned TRUE. Select zi if otherwise.

• so far, the sum contains a 1 in each of the first l digits and a

number between 1 and 3 in each of the last k digits • Select enough of the g and h numbers to bring each of the last k

digits up to 3.

• Sum would then be equal to t.

Time Complexity

NP-completeness

Time Complexity

Suppose that a subset of S sums to t.

NP-completeness

Time Complexity

NP-completeness

Suppose that a subset of S sums to t.

First, note that a â&#x20AC;&#x153;carryâ&#x20AC;? into the next column in the table never occurs when a subset of S is added.

Time Complexity

NP-completeness

Suppose that a subset of S sums to t.

First, note that a “carry” into the next column in the table never occurs when a subset of S is added.

We make the satisfying assignment as follows: • To get a 1 in each of the first l columns, the subset must have either yi or zi for each i but not both.

• if the subset contains yi , assign xi as TRUE; otherwise, assign it

FALSE • For each of the final k columns, at most 2 can come from g and

h. To get a 3, at least 1 (for each these columns) must come from some yi or zi in the subset.

• if it is yi , then xi appears in cj and is assigned TRUE, so cj is

satisfied.

• if it is zi , then xi appears in cj and xi is assigned FALSE, so cj is

satisfied.

• Therefore, φ is satisfied.

Time Complexity

NP-completeness

Questions?

See you next meeting!