‣ master theorem ‣ integer multiplication ‣ matrix multiplication ‣ convolution and FFT D C II

(1)

Lecture slides by Kevin Wayne 

Copyright © 2005 Pearson-Addison Wesley 

http://www.cs.princeton.edu/~wayne/kleinberg-tardos

D ÎVIDE ÂND C ÔNQUER II

‣ master theorem

‣ integer multiplication

‣ matrix multiplication

‣ convolution and FFT

(2)

D ÎVIDE ÂND C ÔNQUER II

‣ master theorem

‣ integer multiplication

‣ matrix multiplication

‣ convolution and FFT

S ^ECTIONS 4.4–4.6

(3)

Divide-and-conquer recurrences

Goal. Recipe for solving common divide-and-conquer recurrences:

with T(0) = 0 and T(1) = Θ(1).

Terms.

・ a ≥ 1 is the number of subproblems.

・ b ≥ 2 is the factor by which the subproblem size decreases.

・ ^f (n) ≥ 0 is the work to divide and combine subproblems.

Recursion tree. [ assuming n is a power of b ]

・ a = branching factor.

・ ^a ⁱ = number of subproblems at level i.

・ ^{1 + log} ^b n levels.

・ ^{n / b} ⁱ = size of subproblem at level i.

T (n) = a T n

b + f (n)

T (n)

T (n / b) T (n / b) ... T (n / b)

(4)

Divide-and-conquer recurrences: recursion tree

Suppose T (n) satisfies T (n) = a T (n / b) + n ^c with T (1) = 1, for n a power of b.

1 + log _b n

n ^c

a (n / b) ^c

a ⁱ (n / b ⁱ ) ^c

⋮

T (n)

a ² (n / b ² ) ^c

⋮

T (1) T (1)

T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) ... T (1) T (1) T (1) T (n / b)

T (n / b ² ) T (n / b ² ) T (n / b ² )

T (n / b)

T (n / b ² ) T (n / b ² ) T (n / b ² )

T (n / b)

T (n / b ² ) T (n / b ² ) T (n / b ² )

...

... ... ...

n ^log ^b ^a

a ^log ^b ⁿ = n ^log ^b ^a

log n

(5)

Divide-and-conquer recurrences: recursion tree analysis

Suppose T (n) satisfies T (n) = a T (n / b) + n ^c with T (1) = 1, for n a power of b.

Let r = a / b ^c . Note that r < 1 iff c > log b a.

Geometric series.

・ If 0 < r < 1, then 1 + r + r ² + r ³ + … + r ^k ≤ 1 / (1 − r).

・ If r = 1, then 1 + r + r ² + r ³ + … + r ^k = k + 1.

・ If r > 1, then 1 + r + r ² + r ³ + … + r ^k = (r ^k+1 − 1) / (r − 1).

c < log b a cost dominated by cost of leaves

c = log b a cost evenly

distributed in tree

c > log b a cost dominated by cost of root

T (n) = n ^c

log _b n

i=0

r ⁱ =

(n ^c ) r < 1 (n ^c log n) r = 1 (n ^log ^b ^a ) r > 1

<latexit sha1_base64="NEShmpjKNsOBbDerTDYmROKsym4=">AAAC/nicbVFNj9MwEHXCxy7lq7scuYyoQN1LlSAkdlWKVuLCcZFadqU6jRx32nrrOJHtoFZRDvwaTogrf4QD/wanm5VotyPZenrz3sx4nORSGBsEfz3/3v0HDw8OH7UeP3n67Hn76PiryQrNccQzmemrhBmUQuHICivxKtfI0kTiZbL8VOcvv6E2IlNDu84xStlciZngzDoqbv8ZdtUJ0D4M6ktNOFBTpHEpBkE1KanM5nECqgI9EbeqFk1wLlTJXVtTtWifDhdoWdeZT+ANUIsrW4pZVes1fIAQKB2HmEZbUqhrg9rjGOx33A7Dqj2ejxC2KKppM1Tc7gS9YBNwF4QN6JAmLuIj75hOM16kqCyXzJhxGOQ2Kpm2gkt0rywM5owv2RzHDiqWoonKzf4reO2YKcwy7Y6ysGH/d5QsNWadJk6ZMrswu7ma3JcbF3Z2GpVC5YVFxW8azQoJNoP6M2EqNHIr1w4wroWbFfiCacat+/KtLpvaOfKtl5SrQgmeTXGHlXZlNau3GO7u7C4Yve2d9cIv7zrnp806D8lL8op0SUjek3PymVyQEeFe32Petbf0v/s//J/+rxup7zWeF2Qr/N//AAuD6zQ=</latexit><latexit sha1_base64="NEShmpjKNsOBbDerTDYmROKsym4=">AAAC/nicbVFNj9MwEHXCxy7lq7scuYyoQN1LlSAkdlWKVuLCcZFadqU6jRx32nrrOJHtoFZRDvwaTogrf4QD/wanm5VotyPZenrz3sx4nORSGBsEfz3/3v0HDw8OH7UeP3n67Hn76PiryQrNccQzmemrhBmUQuHICivxKtfI0kTiZbL8VOcvv6E2IlNDu84xStlciZngzDoqbv8ZdtUJ0D4M6ktNOFBTpHEpBkE1KanM5nECqgI9EbeqFk1wLlTJXVtTtWifDhdoWdeZT+ANUIsrW4pZVes1fIAQKB2HmEZbUqhrg9rjGOx33A7Dqj2ejxC2KKppM1Tc7gS9YBNwF4QN6JAmLuIj75hOM16kqCyXzJhxGOQ2Kpm2gkt0rywM5owv2RzHDiqWoonKzf4reO2YKcwy7Y6ysGH/d5QsNWadJk6ZMrswu7ma3JcbF3Z2GpVC5YVFxW8azQoJNoP6M2EqNHIr1w4wroWbFfiCacat+/KtLpvaOfKtl5SrQgmeTXGHlXZlNau3GO7u7C4Yve2d9cIv7zrnp806D8lL8op0SUjek3PymVyQEeFe32Petbf0v/s//J/+rxup7zWeF2Qr/N//AAuD6zQ=</latexit><latexit sha1_base64="NEShmpjKNsOBbDerTDYmROKsym4=">AAAC/nicbVFNj9MwEHXCxy7lq7scuYyoQN1LlSAkdlWKVuLCcZFadqU6jRx32nrrOJHtoFZRDvwaTogrf4QD/wanm5VotyPZenrz3sx4nORSGBsEfz3/3v0HDw8OH7UeP3n67Hn76PiryQrNccQzmemrhBmUQuHICivxKtfI0kTiZbL8VOcvv6E2IlNDu84xStlciZngzDoqbv8ZdtUJ0D4M6ktNOFBTpHEpBkE1KanM5nECqgI9EbeqFk1wLlTJXVtTtWifDhdoWdeZT+ANUIsrW4pZVes1fIAQKB2HmEZbUqhrg9rjGOx33A7Dqj2ejxC2KKppM1Tc7gS9YBNwF4QN6JAmLuIj75hOM16kqCyXzJhxGOQ2Kpm2gkt0rywM5owv2RzHDiqWoonKzf4reO2YKcwy7Y6ysGH/d5QsNWadJk6ZMrswu7ma3JcbF3Z2GpVC5YVFxW8azQoJNoP6M2EqNHIr1w4wroWbFfiCacat+/KtLpvaOfKtl5SrQgmeTXGHlXZlNau3GO7u7C4Yve2d9cIv7zrnp806D8lL8op0SUjek3PymVyQEeFe32Petbf0v/s//J/+rxup7zWeF2Qr/N//AAuD6zQ=</latexit><latexit sha1_base64="NEShmpjKNsOBbDerTDYmROKsym4=">AAAC/nicbVFNj9MwEHXCxy7lq7scuYyoQN1LlSAkdlWKVuLCcZFadqU6jRx32nrrOJHtoFZRDvwaTogrf4QD/wanm5VotyPZenrz3sx4nORSGBsEfz3/3v0HDw8OH7UeP3n67Hn76PiryQrNccQzmemrhBmUQuHICivxKtfI0kTiZbL8VOcvv6E2IlNDu84xStlciZngzDoqbv8ZdtUJ0D4M6ktNOFBTpHEpBkE1KanM5nECqgI9EbeqFk1wLlTJXVtTtWifDhdoWdeZT+ANUIsrW4pZVes1fIAQKB2HmEZbUqhrg9rjGOx33A7Dqj2ejxC2KKppM1Tc7gS9YBNwF4QN6JAmLuIj75hOM16kqCyXzJhxGOQ2Kpm2gkt0rywM5owv2RzHDiqWoonKzf4reO2YKcwy7Y6ysGH/d5QsNWadJk6ZMrswu7ma3JcbF3Z2GpVC5YVFxW8azQoJNoP6M2EqNHIr1w4wroWbFfiCacat+/KtLpvaOfKtl5SrQgmeTXGHlXZlNau3GO7u7C4Yve2d9cIv7zrnp806D8lL8op0SUjek3PymVyQEeFe32Petbf0v/s//J/+rxup7zWeF2Qr/N//AAuD6zQ=</latexit>

(6)

Divide-and-conquer recurrences: master theorem

Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence

with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then,

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

Pf sketch.

・ Prove when b is an integer and n is an exact power of b.

・ Extend domain of recurrences to reals (or rationals).

・ Deal with floors and ceilings.

T (n) = a T n

b + (n ^c )

<latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit><latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit><latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit><latexit sha1_base64="RRWvKmBD1IL0biczVisiLGro+Bk=">AAACb3icbVBtSxtBEN6c2lq1NeoXQZClQUiohLtSqEUEQRA/KiRVyMUwt5lLFvf2jt25YjjuH/lr/Cb6K/wF3bx8MNGB3X145pmdmSfKlLTk+08Vb2l55dPn1S9r6xtfv21Wt7b/2jQ3AtsiVam5icCikhrbJEnhTWYQkkjhdXR3Ns5f/0NjZapbNMqwm8BAy1gKIEf1quetum7w8JifjC/g4SFvhQpj4vUwNiAKXRZRyUMjB0PiTnnoFD+mT9gaIkFd34pGr1rzm/4k+HsQzECNzeKyt1XZDvupyBPUJBRY2wn8jLoFGJJCYbkW5hYzEHcwwI6DGhK03WKycMkPHNPncWrc0cQn7NuKAhJrR4kb/CABGtrF3Jj8KNfJKT7qFlJnOaEW00ZxrjilfOwe70uDgtTIARBGulm5GILziZzHc10mf2co5jYp7nMtRdrHBVbRPRkonYvBomfvQftn808zuPpVOz2a2bnK9th3VmcB+81O2QW7ZG0m2AN7ZM/spfLq7Xr7Hp9KvcqsZofNhdf4D8cluuQ=</latexit>

at most 2 extra levels in recursion tree

3 2

(7)

Divide-and-conquer recurrences: master theorem

Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence

with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then,

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

Extensions.

・ Can replace Θ with O everywhere.

・ Can replace Θ with Ω everywhere.

・ Can replace initial conditions with T(n) = Θ(1) for all n ≤ n 0 and require recurrence to hold only for all n > n 0 .

T (n) = a T n

b + (n ^c )

(8)

Divide-and-conquer recurrences: master theorem

Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence

with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then,

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

Ex. [Case 1] T (n) = 3 T(⎣n / 2⎦) + 5 n.

・ a = 3, b = 2, c = 1 < log _b a = 1.5849....

・ ^{T(n) = Θ(n} ^log ² ³ ^{) = O(n} ^1.58 ^).

T (n) = a T n

b + (n ^c )

(9)

Divide-and-conquer recurrences: master theorem

Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence

with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then,

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

Ex. [Case 2] T (n) = T(⎣n / 2⎦) + T(⎡n / 2⎤) + 17 n.

・ a = 2, b = 2, c = 1 = log _b a.

・ ^T (n) = Θ(n log n).

T (n) = a T n

b + (n ^c )

ok to intermix floor and ceiling

(10)

Divide-and-conquer recurrences: master theorem

Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence

with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then,

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

Ex. [Case 3] T (n) = 48 T(⎣n / 4⎦) + n ³ .

・ a = 48, b = 4, c = 3 > log _b a = 2.7924....

・ ^T ^{(n) = Θ(n} ³ ^).

T (n) = a T n

b + (n ^c )

(11)

Master theorem need not apply

Gaps in master theorem.

・ Number of subproblems is not a constant.

・ Number of subproblems is less than 1.

・ Work to divide and combine subproblems is not Θ(n ^c ).

T (n) = n T (n/2) + n ²

T (n) = 1

2 T (n/2) + n ²

T (n) = 2 T (n/2) + n log n

<latexit sha1_base64="NFzJ+YRkSM1S9XB/ZQOOUWaF4bk=">AAACT3icbZDNSgMxEMez9bt+VT16CRZFUepuEayIIHjxqGBV6JaSTac1mE2WZFZalr6BT+NVn8Kbb+JJTOsebHUg8M9vZjKZf5RIYdH3P7zC1PTM7Nz8QnFxaXlltbS2fmt1ajjUuZba3EfMghQK6ihQwn1igMWRhLvo8WKYv3sCY4VWN9hPoBmzrhIdwRk61Crt3OyqPRqenoWntErDA+ruh9Uh2XdE0VDqLlXFVqnsV/xR0L8iyEWZ5HHVWvPWw7bmaQwKuWTWNgI/wWbGDAouYVAMUwsJ44+sCw0nFYvBNrPRQgO67UibdrRxRyEd0d8dGYut7ceRq4wZPtjJ3BD+l2uk2Kk1M6GSFEHxn0GdVFLUdOgObQsDHGXfCcaNcH+l/IEZxtF5ODZl9HYCfGyTrJcqwXUbJqjEHho2cC4Gk579FfVq5aQSXB+Vz2u5nfNkk2yRXRKQY3JOLskVqRNOnskLeSVv3rv36X0V8tKCl4sNMhaFhW/957Ap</latexit><latexit sha1_base64="NFzJ+YRkSM1S9XB/ZQOOUWaF4bk=">AAACT3icbZDNSgMxEMez9bt+VT16CRZFUepuEayIIHjxqGBV6JaSTac1mE2WZFZalr6BT+NVn8Kbb+JJTOsebHUg8M9vZjKZf5RIYdH3P7zC1PTM7Nz8QnFxaXlltbS2fmt1ajjUuZba3EfMghQK6ihQwn1igMWRhLvo8WKYv3sCY4VWN9hPoBmzrhIdwRk61Crt3OyqPRqenoWntErDA+ruh9Uh2XdE0VDqLlXFVqnsV/xR0L8iyEWZ5HHVWvPWw7bmaQwKuWTWNgI/wWbGDAouYVAMUwsJ44+sCw0nFYvBNrPRQgO67UibdrRxRyEd0d8dGYut7ceRq4wZPtjJ3BD+l2uk2Kk1M6GSFEHxn0GdVFLUdOgObQsDHGXfCcaNcH+l/IEZxtF5ODZl9HYCfGyTrJcqwXUbJqjEHho2cC4Gk579FfVq5aQSXB+Vz2u5nfNkk2yRXRKQY3JOLskVqRNOnskLeSVv3rv36X0V8tKCl4sNMhaFhW/957Ap</latexit><latexit sha1_base64="NFzJ+YRkSM1S9XB/ZQOOUWaF4bk=">AAACT3icbZDNSgMxEMez9bt+VT16CRZFUepuEayIIHjxqGBV6JaSTac1mE2WZFZalr6BT+NVn8Kbb+JJTOsebHUg8M9vZjKZf5RIYdH3P7zC1PTM7Nz8QnFxaXlltbS2fmt1ajjUuZba3EfMghQK6ihQwn1igMWRhLvo8WKYv3sCY4VWN9hPoBmzrhIdwRk61Crt3OyqPRqenoWntErDA+ruh9Uh2XdE0VDqLlXFVqnsV/xR0L8iyEWZ5HHVWvPWw7bmaQwKuWTWNgI/wWbGDAouYVAMUwsJ44+sCw0nFYvBNrPRQgO67UibdrRxRyEd0d8dGYut7ceRq4wZPtjJ3BD+l2uk2Kk1M6GSFEHxn0GdVFLUdOgObQsDHGXfCcaNcH+l/IEZxtF5ODZl9HYCfGyTrJcqwXUbJqjEHho2cC4Gk579FfVq5aQSXB+Vz2u5nfNkk2yRXRKQY3JOLskVqRNOnskLeSVv3rv36X0V8tKCl4sNMhaFhW/957Ap</latexit><latexit sha1_base64="NFzJ+YRkSM1S9XB/ZQOOUWaF4bk=">AAACT3icbZDNSgMxEMez9bt+VT16CRZFUepuEayIIHjxqGBV6JaSTac1mE2WZFZalr6BT+NVn8Kbb+JJTOsebHUg8M9vZjKZf5RIYdH3P7zC1PTM7Nz8QnFxaXlltbS2fmt1ajjUuZba3EfMghQK6ihQwn1igMWRhLvo8WKYv3sCY4VWN9hPoBmzrhIdwRk61Crt3OyqPRqenoWntErDA+ruh9Uh2XdE0VDqLlXFVqnsV/xR0L8iyEWZ5HHVWvPWw7bmaQwKuWTWNgI/wWbGDAouYVAMUwsJ44+sCw0nFYvBNrPRQgO67UibdrRxRyEd0d8dGYut7ceRq4wZPtjJ3BD+l2uk2Kk1M6GSFEHxn0GdVFLUdOgObQsDHGXfCcaNcH+l/IEZxtF5ODZl9HYCfGyTrJcqwXUbJqjEHho2cC4Gk579FfVq5aQSXB+Vz2u5nfNkk2yRXRKQY3JOLskVqRNOnskLeSVv3rv36X0V8tKCl4sNMhaFhW/957Ap</latexit>

(12)

Consider the following recurrence. Which case of the master theorem? 

       

A. Case 3: T(n) = Θ(n).

B. Case 2: T(n) = Θ(n log n).

C. Case 1: T(n) = Θ(n ^log ² ³ ) = O(n ^1.585 ).

D. Master theorem not applicable.

Divide-and-conquer II: quiz 1

T (n) =

(1) n = 1

3T ( n/2 ) + (n) n > 1

<latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit>

(13)

Consider the following recurrence. Which case of the master theorem? 

       

A. Case 1: T(n) = Θ(n).

B. Case 2: T(n) = Θ(n log n).

C. Case 3: T(n) = Θ(n).

D. Master theorem not applicable.

Divide-and-conquer II: quiz 2

T (n) =

0 n 1

T ( n/5 ) + T (n 3 n/10 ) + ¹¹ ₅ n n > 1

<latexit sha1_base64="7VmkhoT1OeXEp4XqE2EINKSyhHk=">AAAC03icbVHLbtNAFB2bVwmPpmXJ5ooIlAoRPEBEpYhSiQ3LIiVtpYwVjSfX6ajjsTUzRomsbBBbPoRP4m8Yu5ZoEu7q6Jz7PDcplLQuiv4E4Z279+4/2HvYefT4ydP97sHhuc1LI3AicpWby4RbVFLjxEmn8LIwyLNE4UVy/aXWL76jsTLXY7cqMM74QstUCu48Nev+Hvf1EbDRJzbqsAQXUlfCt7PrDhtF8AqYw6WDCmQKa9DAFAIFxqYUs9injPtMpSrPDei3Q2CmwUfwGnxbeAPv4Z9Mo9s6Sw0XFaXralj33Rl00ozpMNTzdqFZtxcNoiZgF9AW9EgbZ7OD4JDNc1FmqJ1Q3NopjQoXV9w4KRT6C0uLBRfXfIFTDzXP0MZV4+kaXnpmDqlfPc21g4a9XVHxzNpVlvjMjLsru63V5P+0aenS47iSuigdanEzKC0VuBzqB8FcGhROrTzgwki/K4gr7t1y/o0bU5reBYqNS6plqaXI57jFKrd0htcu0m3PdsH5uwGNBvTbh97pcevnHnlOXpA+oeQjOSVfyRmZEBHsB8PgJPgcTsIq/BH+vEkNg7bmGdmI8Ndf5PTZpg==</latexit><latexit sha1_base64="7VmkhoT1OeXEp4XqE2EINKSyhHk=">AAAC03icbVHLbtNAFB2bVwmPpmXJ5ooIlAoRPEBEpYhSiQ3LIiVtpYwVjSfX6ajjsTUzRomsbBBbPoRP4m8Yu5ZoEu7q6Jz7PDcplLQuiv4E4Z279+4/2HvYefT4ydP97sHhuc1LI3AicpWby4RbVFLjxEmn8LIwyLNE4UVy/aXWL76jsTLXY7cqMM74QstUCu48Nev+Hvf1EbDRJzbqsAQXUlfCt7PrDhtF8AqYw6WDCmQKa9DAFAIFxqYUs9injPtMpSrPDei3Q2CmwUfwGnxbeAPv4Z9Mo9s6Sw0XFaXralj33Rl00ozpMNTzdqFZtxcNoiZgF9AW9EgbZ7OD4JDNc1FmqJ1Q3NopjQoXV9w4KRT6C0uLBRfXfIFTDzXP0MZV4+kaXnpmDqlfPc21g4a9XVHxzNpVlvjMjLsru63V5P+0aenS47iSuigdanEzKC0VuBzqB8FcGhROrTzgwki/K4gr7t1y/o0bU5reBYqNS6plqaXI57jFKrd0htcu0m3PdsH5uwGNBvTbh97pcevnHnlOXpA+oeQjOSVfyRmZEBHsB8PgJPgcTsIq/BH+vEkNg7bmGdmI8Ndf5PTZpg==</latexit><latexit sha1_base64="7VmkhoT1OeXEp4XqE2EINKSyhHk=">AAAC03icbVHLbtNAFB2bVwmPpmXJ5ooIlAoRPEBEpYhSiQ3LIiVtpYwVjSfX6ajjsTUzRomsbBBbPoRP4m8Yu5ZoEu7q6Jz7PDcplLQuiv4E4Z279+4/2HvYefT4ydP97sHhuc1LI3AicpWby4RbVFLjxEmn8LIwyLNE4UVy/aXWL76jsTLXY7cqMM74QstUCu48Nev+Hvf1EbDRJzbqsAQXUlfCt7PrDhtF8AqYw6WDCmQKa9DAFAIFxqYUs9injPtMpSrPDei3Q2CmwUfwGnxbeAPv4Z9Mo9s6Sw0XFaXralj33Rl00ozpMNTzdqFZtxcNoiZgF9AW9EgbZ7OD4JDNc1FmqJ1Q3NopjQoXV9w4KRT6C0uLBRfXfIFTDzXP0MZV4+kaXnpmDqlfPc21g4a9XVHxzNpVlvjMjLsru63V5P+0aenS47iSuigdanEzKC0VuBzqB8FcGhROrTzgwki/K4gr7t1y/o0bU5reBYqNS6plqaXI57jFKrd0htcu0m3PdsH5uwGNBvTbh97pcevnHnlOXpA+oeQjOSVfyRmZEBHsB8PgJPgcTsIq/BH+vEkNg7bmGdmI8Ndf5PTZpg==</latexit><latexit sha1_base64="7VmkhoT1OeXEp4XqE2EINKSyhHk=">AAAC03icbVHLbtNAFB2bVwmPpmXJ5ooIlAoRPEBEpYhSiQ3LIiVtpYwVjSfX6ajjsTUzRomsbBBbPoRP4m8Yu5ZoEu7q6Jz7PDcplLQuiv4E4Z279+4/2HvYefT4ydP97sHhuc1LI3AicpWby4RbVFLjxEmn8LIwyLNE4UVy/aXWL76jsTLXY7cqMM74QstUCu48Nev+Hvf1EbDRJzbqsAQXUlfCt7PrDhtF8AqYw6WDCmQKa9DAFAIFxqYUs9injPtMpSrPDei3Q2CmwUfwGnxbeAPv4Z9Mo9s6Sw0XFaXralj33Rl00ozpMNTzdqFZtxcNoiZgF9AW9EgbZ7OD4JDNc1FmqJ1Q3NopjQoXV9w4KRT6C0uLBRfXfIFTDzXP0MZV4+kaXnpmDqlfPc21g4a9XVHxzNpVlvjMjLsru63V5P+0aenS47iSuigdanEzKC0VuBzqB8FcGhROrTzgwki/K4gr7t1y/o0bU5reBYqNS6plqaXI57jFKrd0htcu0m3PdsH5uwGNBvTbh97pcevnHnlOXpA+oeQjOSVfyRmZEBHsB8PgJPgcTsIq/BH+vEkNg7bmGdmI8Ndf5PTZpg==</latexit>

(14)

Akra–Bazzi theorem

Theorem. [Akra–Bazzi 1998] Given constants a i > 0 and 0 < b i < 1,

functions ^⎜ h i (n) ^⎜ = O(n / log ² n) and g(n) = O(n ^c ). If T(n) satisfies the recurrence:

then, T(n) = , where p satisfies .

Ex . T(n) = T(⎣n / 5⎦) + T(n – 3⎣n / 10⎦) + 11/5 n, with T(0) = 0 and T(1) = 0.

・ ^a ¹ ^{= 1, b} ¹ ^{= 1/5, a} ² ^{= 1, b} ² = 7/10 ⇒ p = 0.83978… < 1.

・ ^h ¹ (n) = ⎣n / 5⎦ – n / 5, h ² (n) = 3/10 n – 3⎣n / 10⎦.

・ g(n) = 11/5 n ⇒ T(n) = Θ(n).

T (n) =

k

i=1

a _i T (b _i n + h _i (n)) + g(n)

k

i=1

a _i b ^p _i = 1 n ^p 1 +

n 1

g(u) u ^p+1 du

a i subproblems of size b i n

small perturbation to handle

floors and ceilings

(15)

D ÎVIDE ÂND C ÔNQUER II

‣ master theorem

‣ integer multiplication

‣ matrix multiplication

‣ convolution and FFT

S ^ECTION 5.5

(16)

Integer addition and subtraction

Addition. Given two n-bit integers a and b, compute a + b.

Subtraction. Given two n-bit integers a and b, compute a – b.

Grade-school algorithm. Θ(n) bit operations.

Remark. Grade-school addition and subtraction algorithms are optimal.

1 1 1 1 1 1 0 1

1 1 0 1 0 1 0 1

+ ⁰ ¹ ¹ ¹ ¹ ¹ ⁰ ¹

1 0 1 0 1 0 0 1 0

“bit complexity”

(instead of word RAM)

(17)

Integer multiplication

Multiplication. Given two n-bit integers a and b, compute a × b.

Grade-school algorithm (long multiplication). Θ(n ² ) bit operations.

Conjecture. [Kolmogorov 1956] Grade-school algorithm is optimal.

1 1 0 1 0 1 0 1

× ⁰ ¹ ¹ ¹ ¹ ¹ ⁰ ¹

1 1 0 1 0 1 0 1

0 0 0 0 0 0 0 0

1 1 0 1 0 1 0 1

0 0 0 0 0 0 0 0

0 1 1 0 1 0 0 0 0 0 0 0 0 0 0 1

(18)

Divide-and-conquer multiplication

To multiply two n-bit integers x and y:

・ Divide x and y into low- and high-order bits.

・ Multiply four ½n-bit integers, recursively.

・ Add and shift to obtain result.

1 x y = (2 ^m a + b) (2 ^m c + d) = 2 ^2m ac + 2 ^m (bc + ad) + bd

2 3 4

c = ⎣ y / 2 ^m ⎦ d = y mod 2 ^m m = ⎡ n / 2 ⎤

Ex. x = 1 0 0 0 1 1 0 1 y = 1 1 1 0 0 0 0 1

use bit shifting to compute 4 terms

a = ⎣ x / 2 ^m ⎦ b = x mod 2 ^m

(19)

Divide-and-conquer multiplication

M ^ULTIPLY (x, y, n)

_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

I ^F (n = 1)

R ^ETURN x 𐄂 y.

E ^LSE

m ← ⎡ n / 2 ⎤.

a ← ⎣ x / 2 ^m ⎦; b ← x mod 2 ^m . c ← ⎣ y / 2 ^m ⎦; d ← y mod 2 ^m . e ← M ^ULTIPLY (a, c, m).

f ← M ULTIPLY (b, d, m).

g ← M ULTIPLY (b, c, m).

h ← M ULTIPLY (a, d, m).

R ETURN 2 ^2m e + 2 ^m (g + h) + f.

_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Θ(n) 4 T(⎡n / 2⎤)

Θ(n)

(20)

How many bit operations to multiply two n-bit integers using the divide-and-conquer multiplication algorithm? 

       

A. T(n) = Θ(n ^1/2 ).

B. T(n) = Θ(n log n).

C. T(n) = Θ(n ^log ² ³ ) = O(n ^1.585 ).

D. T(n) = Θ(n ² ).

Divide-and-conquer II: quiz 3

T (n) =

(1) n = 1

4T ( n/2 ) + (n) n > 1

<latexit sha1_base64="C59SKrult62qTBWLhiGWTGCUjp8=">AAACvnicbVFda9swFJW9ry77aNo+7kVbupGukNml0Jay0bGXPXaQrIXIBFm+TkQl2UjXI8HkZ+zH7bfsZXLqwZLsgsTh3HvP1T1KSyUdRtGvIHzw8NHjJztPO8+ev3i5293b/+6KygoYiUIV9jblDpQ0MEKJCm5LC1ynCm7Suy9N/uYHWCcLM8RFCYnmUyNzKTh6atL9OeybI8ou6cfm6rAUptLUwiu6ZYddsuEMkPfjI/qOMoQ51lTm9ND48viQLilj42hwBjrxtad02GdKgFTUfDihzDaw0T72yq2Q2RL69Feow8Bk7eRJtxcNolXQbRC3oEfauJ7sBfssK0SlwaBQ3LlxHJWY1NyiFAr8KpWDkos7PoWxh4ZrcEm98m9J33omo3lh/TFIV+y/HTXXzi106is1x5nbzDXk/3LjCvPzpJamrBCMuB+UV4piQZvPoJm0IFAtPODCSv9WKmbccoH+y9amrLRLEGub1PPKSFFksMEqnKPljYvxpmfbYHQyuBjE3057V+etnTvkFXlD+iQmZ+SKfCXXZEQE+R28Dt4Hx+HncBrqsLgvDYO254CsRTj/A38N0Zw=</latexit><latexit sha1_base64="C59SKrult62qTBWLhiGWTGCUjp8=">AAACvnicbVFda9swFJW9ry77aNo+7kVbupGukNml0Jay0bGXPXaQrIXIBFm+TkQl2UjXI8HkZ+zH7bfsZXLqwZLsgsTh3HvP1T1KSyUdRtGvIHzw8NHjJztPO8+ev3i5293b/+6KygoYiUIV9jblDpQ0MEKJCm5LC1ynCm7Suy9N/uYHWCcLM8RFCYnmUyNzKTh6atL9OeybI8ou6cfm6rAUptLUwiu6ZYddsuEMkPfjI/qOMoQ51lTm9ND48viQLilj42hwBjrxtad02GdKgFTUfDihzDaw0T72yq2Q2RL69Feow8Bk7eRJtxcNolXQbRC3oEfauJ7sBfssK0SlwaBQ3LlxHJWY1NyiFAr8KpWDkos7PoWxh4ZrcEm98m9J33omo3lh/TFIV+y/HTXXzi106is1x5nbzDXk/3LjCvPzpJamrBCMuB+UV4piQZvPoJm0IFAtPODCSv9WKmbccoH+y9amrLRLEGub1PPKSFFksMEqnKPljYvxpmfbYHQyuBjE3057V+etnTvkFXlD+iQmZ+SKfCXXZEQE+R28Dt4Hx+HncBrqsLgvDYO254CsRTj/A38N0Zw=</latexit><latexit sha1_base64="C59SKrult62qTBWLhiGWTGCUjp8=">AAACvnicbVFda9swFJW9ry77aNo+7kVbupGukNml0Jay0bGXPXaQrIXIBFm+TkQl2UjXI8HkZ+zH7bfsZXLqwZLsgsTh3HvP1T1KSyUdRtGvIHzw8NHjJztPO8+ev3i5293b/+6KygoYiUIV9jblDpQ0MEKJCm5LC1ynCm7Suy9N/uYHWCcLM8RFCYnmUyNzKTh6atL9OeybI8ou6cfm6rAUptLUwiu6ZYddsuEMkPfjI/qOMoQ51lTm9ND48viQLilj42hwBjrxtad02GdKgFTUfDihzDaw0T72yq2Q2RL69Feow8Bk7eRJtxcNolXQbRC3oEfauJ7sBfssK0SlwaBQ3LlxHJWY1NyiFAr8KpWDkos7PoWxh4ZrcEm98m9J33omo3lh/TFIV+y/HTXXzi106is1x5nbzDXk/3LjCvPzpJamrBCMuB+UV4piQZvPoJm0IFAtPODCSv9WKmbccoH+y9amrLRLEGub1PPKSFFksMEqnKPljYvxpmfbYHQyuBjE3057V+etnTvkFXlD+iQmZ+SKfCXXZEQE+R28Dt4Hx+HncBrqsLgvDYO254CsRTj/A38N0Zw=</latexit><latexit sha1_base64="C59SKrult62qTBWLhiGWTGCUjp8=">AAACvnicbVFda9swFJW9ry77aNo+7kVbupGukNml0Jay0bGXPXaQrIXIBFm+TkQl2UjXI8HkZ+zH7bfsZXLqwZLsgsTh3HvP1T1KSyUdRtGvIHzw8NHjJztPO8+ev3i5293b/+6KygoYiUIV9jblDpQ0MEKJCm5LC1ynCm7Suy9N/uYHWCcLM8RFCYnmUyNzKTh6atL9OeybI8ou6cfm6rAUptLUwiu6ZYddsuEMkPfjI/qOMoQ51lTm9ND48viQLilj42hwBjrxtad02GdKgFTUfDihzDaw0T72yq2Q2RL69Feow8Bk7eRJtxcNolXQbRC3oEfauJ7sBfssK0SlwaBQ3LlxHJWY1NyiFAr8KpWDkos7PoWxh4ZrcEm98m9J33omo3lh/TFIV+y/HTXXzi106is1x5nbzDXk/3LjCvPzpJamrBCMuB+UV4piQZvPoJm0IFAtPODCSv9WKmbccoH+y9amrLRLEGub1PPKSFFksMEqnKPljYvxpmfbYHQyuBjE3057V+etnTvkFXlD+iQmZ+SKfCXXZEQE+R28Dt4Hx+HncBrqsLgvDYO254CsRTj/A38N0Zw=</latexit>

Case 3 of master theorem

(21)

Karatsuba trick

To multiply two n-bit integers x and y:

・ Divide x and y into low- and high-order bits.

・ To compute middle term bc + ad, use identity:

・ Multiply only three ½n-bit integers, recursively.

bc + ad = ac + bd – (a – b) (c – d)

a = ⎣ x / 2 ^m ⎦ b = x mod 2 ^m c = ⎣ y / 2 ^m ⎦ d = y mod 2 ^m m = ⎡ n / 2 ⎤

1 1 3 2 3

x y = (2 ^m a + b) (2 ^m c + d) = 2 ^2m ac + 2 ^m (bc + ad ) + bd

x = 1 0 0 0 1 1 0 1

a b

y = 1 1 1 0 0 0 0 1

c d

middle term

= 2 ^2m ac + 2 ^m (ac + bd – (a – b)(c – d)) + bd

(22)

Karatsuba multiplication

K ^ARATSUBA -M ^ULTIPLY (x, y, n)

_______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

I ^F (n = 1)

R ^ETURN x 𐄂 y.

E ^LSE

m ← ⎡ n / 2 ⎤.

a ← ⎣ x / 2 ^m ⎦; b ← x mod 2 ^m . c ← ⎣ y / 2 ^m ⎦; d ← y mod 2 ^m .

e ← K ^ARATSUBA -M ^ULTIPLY (a, c, m).

f ← K ^ARATSUBA -M ^ULTIPLY (b, d, m).

g ← K ^ARATSUBA -M ^ULTIPLY ( ⎢ a – b ⎢ , ⎢ c – d ⎢ , m).

Flip sign of g if needed.

R ^ETURN 2 ^2m e + 2 ^m (e + f – g) + f.

_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Θ(n)

3 T(⎡n / 2⎤)

Θ(n)

(23)

Karatsuba analysis

Proposition. Karatsuba’s algorithm requires O(n ^1.585 ) bit operations to multiply two n-bit integers.

Pf. Apply Case 3 of the master theorem to the recurrence:

Practice.

・ Use base 32 or 64 (instead of base 2).

・ Faster than grade-school algorithm for about 320–640 bits.

T (n) = 3T (n/2) + (n) = T (n) = (n ^log ² ³ ) = O(n ^1.585 )

<latexit sha1_base64="fPBfL+MXRKQlG3xpj3ADkn6kpJA=">AAACmHicbVHfT9swEHYy2Bj7VdgjLxbVJNCkLilDK5omMfEAPI1N7UBquspxrq2FYwf7MlFF+TP3sL9lLzghAlp20knffd+dz/4cZ1JYDIK/nv9kZfXps7Xn6y9evnr9prWx+dPq3HAYcC21uYiZBSkUDFCghIvMAEtjCefx5VGln/8GY4VWfZxnMErZVImJ4AwdNW7p/o7apdFn+sXlnis+dKvyvaui/gyQ1fJVzhIaidTdCGxT3g3SB72/ikjq6bhL98p78VvFh5393n65O261g05QB30Mwga0SRNn4w1vM0o0z1NQyCWzdhgGGY4KZlBwCeV6lFvIGL9kUxg6qFgKdlTUzpT0nWMSOtHGpUJasw8nCpZaO09j15kynNllrSL/pw1znPRGhVBZjqD47aJJLilqWtlME2GAo5w7wLgR7q6Uz5hhHN1nLGypz86AL7ykuM6V4DqBJVbiNRpWOhfDZc8eg0G3c9AJv39sH/YaO9fIFtkmOyQkn8ghOSFnZEA4+UP+eSveqr/lf/WP/dPbVt9rZt6ShfB/3ABdlsTC</latexit><latexit sha1_base64="fPBfL+MXRKQlG3xpj3ADkn6kpJA=">AAACmHicbVHfT9swEHYy2Bj7VdgjLxbVJNCkLilDK5omMfEAPI1N7UBquspxrq2FYwf7MlFF+TP3sL9lLzghAlp20knffd+dz/4cZ1JYDIK/nv9kZfXps7Xn6y9evnr9prWx+dPq3HAYcC21uYiZBSkUDFCghIvMAEtjCefx5VGln/8GY4VWfZxnMErZVImJ4AwdNW7p/o7apdFn+sXlnis+dKvyvaui/gyQ1fJVzhIaidTdCGxT3g3SB72/ikjq6bhL98p78VvFh5393n65O261g05QB30Mwga0SRNn4w1vM0o0z1NQyCWzdhgGGY4KZlBwCeV6lFvIGL9kUxg6qFgKdlTUzpT0nWMSOtHGpUJasw8nCpZaO09j15kynNllrSL/pw1znPRGhVBZjqD47aJJLilqWtlME2GAo5w7wLgR7q6Uz5hhHN1nLGypz86AL7ykuM6V4DqBJVbiNRpWOhfDZc8eg0G3c9AJv39sH/YaO9fIFtkmOyQkn8ghOSFnZEA4+UP+eSveqr/lf/WP/dPbVt9rZt6ShfB/3ABdlsTC</latexit><latexit sha1_base64="fPBfL+MXRKQlG3xpj3ADkn6kpJA=">AAACmHicbVHfT9swEHYy2Bj7VdgjLxbVJNCkLilDK5omMfEAPI1N7UBquspxrq2FYwf7MlFF+TP3sL9lLzghAlp20knffd+dz/4cZ1JYDIK/nv9kZfXps7Xn6y9evnr9prWx+dPq3HAYcC21uYiZBSkUDFCghIvMAEtjCefx5VGln/8GY4VWfZxnMErZVImJ4AwdNW7p/o7apdFn+sXlnis+dKvyvaui/gyQ1fJVzhIaidTdCGxT3g3SB72/ikjq6bhL98p78VvFh5393n65O261g05QB30Mwga0SRNn4w1vM0o0z1NQyCWzdhgGGY4KZlBwCeV6lFvIGL9kUxg6qFgKdlTUzpT0nWMSOtHGpUJasw8nCpZaO09j15kynNllrSL/pw1znPRGhVBZjqD47aJJLilqWtlME2GAo5w7wLgR7q6Uz5hhHN1nLGypz86AL7ykuM6V4DqBJVbiNRpWOhfDZc8eg0G3c9AJv39sH/YaO9fIFtkmOyQkn8ghOSFnZEA4+UP+eSveqr/lf/WP/dPbVt9rZt6ShfB/3ABdlsTC</latexit><latexit sha1_base64="fPBfL+MXRKQlG3xpj3ADkn6kpJA=">AAACmHicbVHfT9swEHYy2Bj7VdgjLxbVJNCkLilDK5omMfEAPI1N7UBquspxrq2FYwf7MlFF+TP3sL9lLzghAlp20knffd+dz/4cZ1JYDIK/nv9kZfXps7Xn6y9evnr9prWx+dPq3HAYcC21uYiZBSkUDFCghIvMAEtjCefx5VGln/8GY4VWfZxnMErZVImJ4AwdNW7p/o7apdFn+sXlnis+dKvyvaui/gyQ1fJVzhIaidTdCGxT3g3SB72/ikjq6bhL98p78VvFh5393n65O261g05QB30Mwga0SRNn4w1vM0o0z1NQyCWzdhgGGY4KZlBwCeV6lFvIGL9kUxg6qFgKdlTUzpT0nWMSOtHGpUJasw8nCpZaO09j15kynNllrSL/pw1znPRGhVBZjqD47aJJLilqWtlME2GAo5w7wLgR7q6Uz5hhHN1nLGypz86AL7ykuM6V4DqBJVbiNRpWOhfDZc8eg0G3c9AJv39sH/YaO9fIFtkmOyQkn8ghOSFnZEA4+UP+eSveqr/lf/WP/dPbVt9rZt6ShfB/3ABdlsTC</latexit>

T (n) =

(1) n = 1

3T ( n/2 ) + (n) n > 1

<latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit><latexit sha1_base64="cfD19qNLAoyZEyBBTBBK3PkUcFM=">AAACvnicbVFdb9MwFHUyPkb5WDceeTF0oI5JJRlIG5pAQ7zwOKSWTaqjynFuWmuOHdk3U6uoP4Mfx2/hBacLEm25kq2jc+891/c4LZV0GEW/gnDn3v0HD3cfdR4/efpsr7t/8MOZygoYCaOMvU65AyU1jFCiguvSAi9SBVfpzdcmf3UL1kmjh7goISn4VMtcCo6emnR/Dvv6iLJz+qm5OiyFqdS18Ipu2WHnbDgD5P34iL6hDGGONZU5PdS+PD6kS8rYOBqcQpH42vd02GdKgFRUvzuhzDaw0T72yq2Q3hL6/Feow0Bn7eRJtxcNolXQbRC3oEfauJzsBwcsM6IqQKNQ3LlxHJWY1NyiFAr8KpWDkosbPoWxh5oX4JJ65d+SvvZMRnNj/dFIV+y/HTUvnFsUqa8sOM7cZq4h/5cbV5ifJbXUZYWgxd2gvFIUDW0+g2bSgkC18IALK/1bqZhxywX6L1ubstIuQaxtUs8rLYXJYINVOEfLGxfjTc+2wehk8HEQf//Quzhr7dwlL8gr0icxOSUX5Bu5JCMiyO/gZfA2OA6/hNOwCM1daRi0Pc/JWoTzP3z/0Zs=</latexit>

(24)

Integer arithmetic reductions

Integer multiplication. Given two n-bit integers, compute their product.

arithmetic problem formula bit complexity

integer multiplication a × b M(n)

integer square a ² Θ(M(n))

integer division ⎣a / b⎦, a mod b Θ(M(n))

integer square root ⎣√a ⎦ Θ(M(n))

integer arithmetic problems with the same bit complexity M(n) as integer multiplication

ab = (a + b) ² a ² b ² 2

<latexit sha1_base64="vEhGWAG7lUjl6rsRHboNGxBfdYk=">AAACTXicdVBNa9tAEF05bfPVDyc55rLULaSUCkmxY+cQCPTSYwp1E7AdM1qPkiWrldgdlRjhP5Bfk2vyK3rtH8kplK4UB+rQPtjl8d7Mzs6LcyUtBcEvr7H07PmL5ZXVtfWXr16/aW5sfrdZYQT2RaYycxKDRSU19kmSwpPcIKSxwuP44nPlH/9AY2Wmv9E0x1EKZ1omUgA5adx8BzE/4MPEgCh3gH/k8YfTiH/iUN/xaTQro9m42Qr8zt5ut9fmgR/ud3ajqCLtTq/b5qEf1GixOY7GG97mcJKJIkVNQoG1gzDIaVSCISkUztaGhcUcxAWc4cBRDSnaUVmvM+PvnTLhSWbc0cRr9e+OElJrp2nsKlOgc/vUq8R/eYOCkt6olDovCLV4GJQUilPGq2z4RBoUpKaOgDDS/ZWLc3DRkEtwYUr9do5iYZPystBSZBN8oiq6JANVio9R8f+TfuTv++HXduuwN49zhW2zt2yHhazLDtkXdsT6TLArds1u2K3307vz7r3fD6UNb96zxRbQWP4DfheyBQ==</latexit><latexit sha1_base64="vEhGWAG7lUjl6rsRHboNGxBfdYk=">AAACTXicdVBNa9tAEF05bfPVDyc55rLULaSUCkmxY+cQCPTSYwp1E7AdM1qPkiWrldgdlRjhP5Bfk2vyK3rtH8kplK4UB+rQPtjl8d7Mzs6LcyUtBcEvr7H07PmL5ZXVtfWXr16/aW5sfrdZYQT2RaYycxKDRSU19kmSwpPcIKSxwuP44nPlH/9AY2Wmv9E0x1EKZ1omUgA5adx8BzE/4MPEgCh3gH/k8YfTiH/iUN/xaTQro9m42Qr8zt5ut9fmgR/ud3ajqCLtTq/b5qEf1GixOY7GG97mcJKJIkVNQoG1gzDIaVSCISkUztaGhcUcxAWc4cBRDSnaUVmvM+PvnTLhSWbc0cRr9e+OElJrp2nsKlOgc/vUq8R/eYOCkt6olDovCLV4GJQUilPGq2z4RBoUpKaOgDDS/ZWLc3DRkEtwYUr9do5iYZPystBSZBN8oiq6JANVio9R8f+TfuTv++HXduuwN49zhW2zt2yHhazLDtkXdsT6TLArds1u2K3307vz7r3fD6UNb96zxRbQWP4DfheyBQ==</latexit><latexit sha1_base64="vEhGWAG7lUjl6rsRHboNGxBfdYk=">AAACTXicdVBNa9tAEF05bfPVDyc55rLULaSUCkmxY+cQCPTSYwp1E7AdM1qPkiWrldgdlRjhP5Bfk2vyK3rtH8kplK4UB+rQPtjl8d7Mzs6LcyUtBcEvr7H07PmL5ZXVtfWXr16/aW5sfrdZYQT2RaYycxKDRSU19kmSwpPcIKSxwuP44nPlH/9AY2Wmv9E0x1EKZ1omUgA5adx8BzE/4MPEgCh3gH/k8YfTiH/iUN/xaTQro9m42Qr8zt5ut9fmgR/ud3ajqCLtTq/b5qEf1GixOY7GG97mcJKJIkVNQoG1gzDIaVSCISkUztaGhcUcxAWc4cBRDSnaUVmvM+PvnTLhSWbc0cRr9e+OElJrp2nsKlOgc/vUq8R/eYOCkt6olDovCLV4GJQUilPGq2z4RBoUpKaOgDDS/ZWLc3DRkEtwYUr9do5iYZPystBSZBN8oiq6JANVio9R8f+TfuTv++HXduuwN49zhW2zt2yHhazLDtkXdsT6TLArds1u2K3307vz7r3fD6UNb96zxRbQWP4DfheyBQ==</latexit><latexit sha1_base64="vEhGWAG7lUjl6rsRHboNGxBfdYk=">AAACTXicdVBNa9tAEF05bfPVDyc55rLULaSUCkmxY+cQCPTSYwp1E7AdM1qPkiWrldgdlRjhP5Bfk2vyK3rtH8kplK4UB+rQPtjl8d7Mzs6LcyUtBcEvr7H07PmL5ZXVtfWXr16/aW5sfrdZYQT2RaYycxKDRSU19kmSwpPcIKSxwuP44nPlH/9AY2Wmv9E0x1EKZ1omUgA5adx8BzE/4MPEgCh3gH/k8YfTiH/iUN/xaTQro9m42Qr8zt5ut9fmgR/ud3ajqCLtTq/b5qEf1GixOY7GG97mcJKJIkVNQoG1gzDIaVSCISkUztaGhcUcxAWc4cBRDSnaUVmvM+PvnTLhSWbc0cRr9e+OElJrp2nsKlOgc/vUq8R/eYOCkt6olDovCLV4GJQUilPGq2z4RBoUpKaOgDDS/ZWLc3DRkEtwYUr9do5iYZPystBSZBN8oiq6JANVio9R8f+TfuTv++HXduuwN49zhW2zt2yHhazLDtkXdsT6TLArds1u2K3307vz7r3fD6UNb96zxRbQWP4DfheyBQ==</latexit>

(25)

History of asymptotic complexity of integer multiplication

Remark. GNU Multiple Precision library uses one of first five algorithms depending on n.

year algorithm bit operations

12xx grade school O (n ² )

1962 Karatsuba–Ofman O(n ^1.585 )

1963 Toom-3, Toom-4 O (n ^1.465 ) ^, O (n ^1.404 )

1966 Toom–Cook O (n ^{1 + ε} )

1971 Schönhage–Strassen O (n log n ⋅ log log n)

2007 Fürer n log n 2 *^O(logn)

2019 Harvey–van der Hoeven O (n log n)

O (n)

number of bit operations to multiply two n-bit integers

(26)

D ÎVIDE ÂND C ÔNQUER II

‣ master theorem

‣ integer multiplication

‣ matrix multiplication

‣ convolution and FFT

S ^ECTION 4.2

(27)

Dot product

Dot product. Given two length-n vectors a and b, compute c = a ⋅ b.

Grade-school. Θ(n) arithmetic operations.

Remark. “Grade-school” dot product algorithm is asymptotically optimal.

€

a = .70 .20 .10 [ ]

b = .30 .40 .30 [ ]

a ⋅ b = (.70 × .30) + (.20 × .40) + (.10 × .30) = .32

a · b =

n

i=1

a _i b _i

(28)

Matrix multiplication

Matrix multiplication. Given two n-by-n matrices A and B, compute C = AB.

Grade-school. Θ(n ³ ) arithmetic operations.

Q. Is “grade-school” matrix multiplication algorithm asymptotically optimal?

€

c ₁₁ c ₁₂ ! c _1n c ₂₁ c ₂₂ ! c 2n

" " # "

c _n1 c _n2 ! c nn

"

#

$

%

&

' ' ' '

=

a ₁₁ a ₁₂ ! a _1n a ₂₁ a ₂₂ ! a 2n

" " # "

a _n1 a _n2 ! a nn

"

#

$

%

&

' ' ' '

×

b ₁₁ b ₁₂ ! b _1n b ₂₁ b ₂₂ ! b 2n

" " # "

b _n1 b _n2 ! b nn

"

#

$

%

&

' ' ' '

€

.59 .32 .41 .31 .36 .25 .45 .31 .42

"

#

$

%

&

' ' '

=

.70 .20 .10 .30 .60 .10 .50 .10 .40

"

#

$

%

&

' ' '

×

.80 .30 .50 .10 .40 .10 .10 .30 .40

"

#

$

%

&

' ' '

c _ij =

n

k=1

a _ik b _kj

(29)

Block matrix multiplication

C ₁₁ = A ₁₁ × B ₁₁ + A ₁₂ × B ₂₁ = 0 1 4 5

#

$ %

&

' ( × 16 17 20 21

#

$ %

&

' ( + 2 3 6 7

#

$ %

&

' ( × 24 25 28 29

#

$ %

&

' ( = 152 158 504 526

#

$ %

&

' (

€

152 158 164 170

504 526 548 570

856 894 932 970

1208 1262 1316 1370

"

#

$

%

&

' ' ' '

=

0 1 2 3

4 5 6 7

8 9 10 11

12 13 14 15

"

#

$

%

&

' ' ' '

×

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

"

#

$

%

&

' ' ' '

C ₁₁ A ₁₁ A ₁₂ B ₁₁

B ₂₁

(30)

Block matrix multiplication: warmup

To multiply two n-by-n matrices A and B:

・ Divide: partition A and B into ½n-by-½n blocks.

・ ^Conquer: multiply 8 pairs of ½n-by-½n matrices, recursively.

・ Combine: add appropriate products using 4 matrix additions.

Running time. Apply Case 3 of the master theorem.

30 €

C ₁₁ = ( A ₁₁ × B ₁₁ ) ^{+ A} ( ¹² ^{× B} ²¹ )

C ₁₂ = ( A ₁₁ × B ₁₂ ) ^{+ A} ( ¹² ^{× B} ²² )

C ₂₁ = ( A ₂₁ × B ₁₁ ) ^{+ A} ( ²² ^{× B} ²¹ )

C ₂₂ = ( A ₂₁ × B ₁₂ ) ^{+ A} ( ²² ^{× B} ²² )

T (n) = 8T n /2 ( )

recursive calls

! " # $ # + Θ(n ² )

add, form submatrices

! # # " # # ⇒ $ T (n) = Θ(n ³ )

€

C ₁₁ C ₁₂ C ₂₁ C ₂₂

"

# $

%

&

' =

A ₁₁ A ₁₂ A ₂₁ A ₂₂

"

# $

%

&

' ×

B ₁₁ B ₁₂ B ₂₁ B ₂₂

"

# $

%

&

'

½n-by-½n matrices

C = A B

n-by-n matrices

8 matrix multiplications (of ½n-by-½n matrices)

4 matrix additions

(of ½n-by-½n matrices)

(31)

Strassen’s trick

Key idea. Can multiply two 2-by-2 matrices via 7 scalar multiplications (plus 11 additions and 7 subtractions).

Pf. C 12 = P 1 + P 2

= A 11 𐄂 (B 12 – B 22 ) + (A 11 + A 12 ) 𐄂 B 22

€

C ₁₁ C ₁₂ C ₂₁ C ₂₂

"

# $

%

&

' =

A ₁₁ A ₁₂ A ₂₁ A ₂₂

"

# $

%

&

' ×

B ₁₁ B ₁₂ B ₂₁ B ₂₂

"

# $

%

&

'

P 1 ← A 11 𐄂 (B 12 – B 22 ) P 2 ← (A 11 + A 12 ) 𐄂 B 22

P 3 ← (A 21 + A 22 ) 𐄂 B ¹¹ P 4 ← A 22 𐄂 (B 21 – B 11 )

P 5 ← (A 11 + A 22 ) 𐄂 (B 11 + B 22 ) P 6 ← (A 12 – A 22 ) 𐄂 (B 21 + B 22 ) P 7 ← (A 11 – A 21 ) 𐄂 (B 11 + B 12 ) C 11 = P 5 + P 4 – P 2 + P 6

C 12 = P 1 + P 2

C 21 = P 3 + P 4

C 22 = P 1 + P 5 – P 3 – P 7

7 scalar multiplications

scalars

(32)

Strassen’s trick

Key idea. Can multiply two 2-by-2 matrices via 7 scalar multiplications (plus 11 additions and 7 subtractions).

Pf. C 12 = P 1 + P 2

= A 𐄂 (B – B ) + (A + A ) 𐄂 B

€

C ₁₁ C ₁₂ C ₂₁ C ₂₂

"

# $

%

&

' =

A ₁₁ A ₁₂ A ₂₁ A ₂₂

"

# $

%

&

' ×

B ₁₁ B ₁₂ B ₂₁ B ₂₂

"

# $

%

&

'

P 1 ← A 11 𐄂 (B 12 – B 22 ) P 2 ← (A 11 + A 12 ) 𐄂 B 22

P 3 ← (A 21 + A 22 ) 𐄂 B ¹¹ P 4 ← A 22 𐄂 (B 21 – B 11 )

P 5 ← (A 11 + A 22 ) 𐄂 (B 11 + B 22 ) P 6 ← (A 12 – A 22 ) 𐄂 (B 21 + B 22 ) P 7 ← (A 11 – A 21 ) 𐄂 (B 11 + B 12 ) C 11 = P 5 + P 4 – P 2 + P 6

C 12 = P 1 + P 2

C 21 = P 3 + P 4

C 22 = P 1 + P 5 – P 3 – P 7

7 matrix multiplications (of ½n-by-½n matrices)

½n-by-½n matrices

n ^-by- n ½n ^-by- ½n ^matrix

(33)

Strassen’s algorithm

33 S ^TRASSEN (n, A, B)

______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

I ^F (n = 1) R ^ETURN A 𐄂 B.

Partition A and B into ½n-by-½n blocks.

P 1 ← S ^TRASSEN (n / 2, A 11 , (B 12 – B 22 )).

P 2 ← S ^TRASSEN (n / 2, (A 11 + A 12 ), B 22 ).

P 3 ← S ^TRASSEN (n / 2, (A 21 + A 22 ), B 11 ).

P 4 ← S TRASSEN (n / 2, A 22 , (B 21 – B 11 )).

P 5 ← S ^TRASSEN (n / 2, (A 11 + A 22 ), (B 11 + B 22 )).

P 6 ← S ^TRASSEN (n / 2, (A 12 – A 22 ), (B 21 + B 22 )).

P 7 ← S ^TRASSEN (n / 2, (A 11 – A 21 ), (B 11 + B 12 )).

C 11 = P 5 + P 4 – P 2 + P 6 . C 12 = P 1 + P 2 .

C 21 = P 3 + P 4 .

C 22 = P 1 + P 5 – P 3 – P 7 .

R ETURN C.

C ₁₁ C ₁₂ C ₂₁ C ₂₂

"

# $ %

&

' = A ₁₁ A ₁₂ A ₂₁ A ₂₂

"

# $ %

&

' × B ₁₁ B ₁₂ B ₂₁ B ₂₂

"

# $ %

&

'

7 T(n / 2) + Θ(n ² )

Θ(n ² )

assume n is a power of 2

‣ master theorem ‣ integer multiplication ‣ matrix multiplication ‣ convolution and FFT D C II

Lecture slides by Kevin Wayne

Copyright © 2005 Pearson-Addison Wesley

http://www.cs.princeton.edu/~wayne/kleinberg-tardos

D IVIDE AND C ONQUER II

‣ master theorem

‣ integer multiplication

‣ matrix multiplication

‣ convolution and FFT

D IVIDE AND C ONQUER II

‣ master theorem

‣ integer multiplication

‣ matrix multiplication

‣ convolution and FFT

S ECTIONS 4.4–4.6

Divide-and-conquer recurrences

Goal. Recipe for solving common divide-and-conquer recurrences:

with T(0) = 0 and T(1) = Θ(1).

Terms.

・ a ≥ 1 is the number of subproblems.

・ b ≥ 2 is the factor by which the subproblem size decreases.

・ f (n) ≥ 0 is the work to divide and combine subproblems.

Recursion tree. [ assuming n is a power of b ]

・ a = branching factor.

・ a i = number of subproblems at level i.

・ 1 + log b n levels.

・ n / b i = size of subproblem at level i.

T (n) = a T n

b + f (n)

T (n)

T (n / b) T (n / b) ... T (n / b)

Divide-and-conquer recurrences: recursion tree

Suppose T (n) satisfies T (n) = a T (n / b) + n c with T (1) = 1, for n a power of b.

1 + log b n

n c

a (n / b) c

a i (n / b i ) c

⋮

T (n)

a 2 (n / b 2 ) c

⋮

T (1) T (1)

T (1) T (1) T (1) T (1) T (1) T (1) T (1) T (1) ... T (1) T (1) T (1) T (n / b)

T (n / b 2 ) T (n / b 2 ) T (n / b 2 )

T (n / b)

T (n / b 2 ) T (n / b 2 ) T (n / b 2 )

T (n / b)

T (n / b 2 ) T (n / b 2 ) T (n / b 2 )

...

... ... ...

n log b a

a log b n = n log b a

log n

Divide-and-conquer recurrences: recursion tree analysis

Suppose T (n) satisfies T (n) = a T (n / b) + n c with T (1) = 1, for n a power of b.

Let r = a / b c . Note that r < 1 iff c > log b a.

Geometric series.

・ If 0 < r < 1, then 1 + r + r 2 + r 3 + … + r k ≤ 1 / (1 − r).

・ If r = 1, then 1 + r + r 2 + r 3 + … + r k = k + 1.

・ If r > 1, then 1 + r + r 2 + r 3 + … + r k = (r k+1 − 1) / (r − 1).

c < log b a cost dominated by cost of leaves

c = log b a cost evenly

distributed in tree

c > log b a cost dominated by cost of root

T (n) = n c

log b n

i=0

r i =

(n c ) r < 1 (n c log n) r = 1 (n log b a ) r > 1

Divide-and-conquer recurrences: master theorem

Master theorem. Let a ≥ 1, b ≥ 2, and c ≥ 0 and suppose that T (n) is a function on the non-negative integers that satisfies the recurrence

with T(0) = 0 and T(1) = Θ(1), where n / b means either ⎣n / b⎦ or ⎡n / b⎤. Then,

Case 1. If c > log b a, then T (n) = Θ(n c ).

Case 2. If c = log b a, then T (n) = Θ(n c log n).

Case 3. If c < log b a, then T (n) = Θ(n log b a ).

Pf sketch.

・ Prove when b is an integer and n is an exact power of b.

・ Extend domain of recurrences to reals (or rationals).

・ Deal with floors and ceilings.

T (n) = a T n

Lecture slides by Kevin Wayne 

Copyright © 2005 Pearson-Addison Wesley 

D ÎVIDE ÂND C ÔNQUER II

D ÎVIDE ÂND C ÔNQUER II

S ^ECTIONS 4.4–4.6

・ ^f (n) ≥ 0 is the work to divide and combine subproblems.

・ ^a ⁱ = number of subproblems at level i.

・ ^{1 + log} ^b n levels.

・ ^{n / b} ⁱ = size of subproblem at level i.

Suppose T (n) satisfies T (n) = a T (n / b) + n ^c with T (1) = 1, for n a power of b.

1 + log _b n

n ^c

a (n / b) ^c

a ⁱ (n / b ⁱ ) ^c

a ² (n / b ² ) ^c

T (n / b ² ) T (n / b ² ) T (n / b ² )

T (n / b ² ) T (n / b ² ) T (n / b ² )

T (n / b ² ) T (n / b ² ) T (n / b ² )

n ^log ^b ^a

a ^log ^b ⁿ = n ^log ^b ^a

Suppose T (n) satisfies T (n) = a T (n / b) + n ^c with T (1) = 1, for n a power of b.

Let r = a / b ^c . Note that r < 1 iff c > log b a.

・ If 0 < r < 1, then 1 + r + r ² + r ³ + … + r ^k ≤ 1 / (1 − r).

・ If r = 1, then 1 + r + r ² + r ³ + … + r ^k = k + 1.

・ If r > 1, then 1 + r + r ² + r ³ + … + r ^k = (r ^k+1 − 1) / (r − 1).

T (n) = n ^c

log _b n

r ⁱ =

(n ^c ) r < 1 (n ^c log n) r = 1 (n ^log ^b ^a ) r > 1

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

b + (n ^c )

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

b + (n ^c )

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

・ a = 3, b = 2, c = 1 < log _b a = 1.5849....

・ ^{T(n) = Θ(n} ^log ² ³ ^{) = O(n} ^1.58 ^).

b + (n ^c )

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

・ a = 2, b = 2, c = 1 = log _b a.

・ ^T (n) = Θ(n log n).

b + (n ^c )

Case 1. If c > log b a, then T (n) = Θ(n ^c ).

Case 2. If c = log b a, then T (n) = Θ(n ^c log n).

Case 3. If c < log b a, then T (n) = Θ(n ^log ^b ^a ).

Ex. [Case 3] T (n) = 48 T(⎣n / 4⎦) + n ³ .

・ a = 48, b = 4, c = 3 > log _b a = 2.7924....

・ ^T ^{(n) = Θ(n} ³ ^).

b + (n ^c )

・ Work to divide and combine subproblems is not Θ(n ^c ).

T (n) = n T (n/2) + n ²

2 T (n/2) + n ²

Consider the following recurrence. Which case of the master theorem? 

C. Case 1: T(n) = Θ(n ^log ² ³ ) = O(n ^1.585 ).