IDSSSeminar,27.10.2009 WojciechJa´skowskiKrzysztofKrawiec FormalAnalysisandAlgorithmsforExtractingUnderlyingObjectivesofTest-BasedProblems

(1)

Underlying Objectives

Wojciech Ja´skowski, Krzysztof Krawiec

Problem Background & Motivation Coordinate System Example Theoretical Results Theorems Algorithms Computational Complexity Computational Experiments Random Game Compare-on-one Game Summary

Formal Analysis and Algorithms

for Extracting Underlying Objectives

of Test-Based Problems

Wojciech Ja´skowski Krzysztof Krawiec

Institute of Computing Science, Poznan University of Technology, Poland

(2)

Outline

Problem

Background & Motivation Coordinate System Example Theoretical Results Theorems Algorithms Computational Complexity Computational Experiments Random Game Compare-on-one Game

(3)

Problem

Background & Motivation

Coordinate System Example Theoretical Results Theorems Algorithms Computational Complexity Computational Experiments Random Game Compare-on-one Game Summary

Test-Based Problem

I G = (S,T ,G) that consists of:

I _{set S of}_{solutions (a.k.a. candidate solutions),} I _{set T of}_tests,

I _{interaction function G : S}× T → R.

I The codomain of g is a binary set_{0,1}.

I _{If G(s, t) = 1, we say that solution s solves test t;} I If G(s, t) = 0, we say that s fails test t.

I G is a game; G is a payoff matrix I Goal: find the best solution in S

Chess

I S — the set of all first player strategies I T — the set of all second player strategies I G(s, t) — does s win against t?

(4)

Problem

Test-Based Problem

I G = (S,T ,G) that consists of:

I _{set S of}_{solutions (a.k.a. candidate solutions),} I _{set T of}_tests,

I _{interaction function G : S}× T → R.

I The codomain of g is a binary set_{0,1}.

I _{If G(s, t) = 1, we say that solution s solves test t;} I If G(s, t) = 0, we say that s fails test t.

I G is a game; G is a payoff matrix I Goal: find the best solution in S

Chess

I S — the set of all first player strategies

I T — the set of all second player strategies

(5)

Problem

Example: Density Classification Task

0 1 1 0 1 0 0 1 0

1 0 1 1 0 0 0 0 0

2r + 1 n

Density Classification Task I S — set of all CA rules

I T — set of all initial configurations

I G(s, t) — does s converge to the majority from the initial configuration t?

(6)

Problem

Coevolution

I Idea:

I Evolve thepopulation of solutions and the population of tests in parallel

I Tests are evolvedagainst solutions and vice versa

(evaluation function)

I _{Expects gradual increase of difficulty of tests and}

competence of solutions

I (Sometimes) Successful

I Problem: does not always make progress: I _{Pathologies: cycling, stalling, etc.}

(7)

Problem

Pareto-Coevolution

I [Partial] Answer: Pareto-coevolution

I Treatsevery test as an objective

I Formulates the problem as amultiobjective optimization

I Solution a isbetter than b when a dominates b, I _{i.e., it is better/not worse on all tests from T}

I There exist algorithms that guarantee progress (e.g. IPCA, LAPCA) t1 t2 s1 s2 s3

(8)

Problem

Underlying Objectives

I A huge number of objectives

I We have to search a|T |-dimensional space.

I Q: Is it possible to reduce the number of objectives?

I Some tests examine the same skill/aspect of a solution but with different intensity

I Such tests could be grouped and put on acommon

axis

I _{ordered with increasing difficulty}

I Axis =underlying objective,

teasytmediumthard

Example

Possible underlying objectives in chess

I How well the player (1) controls the center of the board, (2) uses knights, (3) plays endgames, etc.

(9)

Problem

Underlying Objectives

I Such tests could be grouped and put on acommon axis

I Axis =underlying objective, teasytmediumthard

Example

(10)

Problem

Underlying Objectives

I Such tests could be grouped and put on acommon axis

I Axis =underlying objective, teasytmediumthard

Example

(11)

Problem

Underlying Objectives

Motivation

I Motivations forextracting the underlying objectives: I _{Faster (co)evolutionary algorithms (fewer objectives)} I _{Can learn something about the game:}

I What are thetrue, underlying objectives of the

game?

I What is thenumber of underlying objectives of a

game?

I _The_{dimension is an inherent property of a game} Research questions here:

I How to extract the underlying objectives?

(12)

Problem

Underlying Objectives

Motivation

I Motivations forextracting the underlying objectives: I _{Faster (co)evolutionary algorithms (fewer objectives)} I _{Can learn something about the game:}

I What are thetrue, underlying objectives of the

game?

I What is thenumber of underlying objectives of a

game?

I _The_{dimension is an inherent property of a game}

Research questions here:

I How to extract the underlying objectives?

(13)

Problem Background & Motivation

Underlying Structure of a Game

I Set of underlying objectives = the underlying structure of a problem = acoordinate system.

I Two definitions proposed so far: I Bucci et al. (2004)

I de Jong & Bucci (2008)

(14)

Coordinate System

Formal Definition Definition

I s1is (weakly) dominated by s2(s1≤ s2), if for all

tests G(s1, t)≤ G(s2, t)

I t1is (weakly) dominated by t2(t1≤ t2), if for all

solutions G(s, t1)≥ G(s,t2)

I (S,_{≤) is a poset if we assume that}

s1∼ s2 ⇐⇒ s1=s2((T,≤), analogously).

Definition

Thecoordinate system C for a gameG is a set of axes

(Ai), where:

I each axis Ai⊆ T ,

(15)

Coordinate System

Formal Definition Definition

I s1is (weakly) dominated by s2(s1≤ s2), if for all

tests G(s1, t)≤ G(s2, t)

I t1is (weakly) dominated by t2(t1≤ t2), if for all

solutions G(s, t1)≥ G(s,t2)

I (S,_{≤) is a poset if we assume that}

s1∼ s2 ⇐⇒ s1=s2((T,≤), analogously).

Definition

Thecoordinate system C for a gameG is a set of axes (Ai), where:

I each axis Ai⊆ T ,

(16)

Coordinate System

Position Function Definition

Position function p_i:S_{→ A}i assigns a test from Ai to

solution s_{∈ S:}

pi(s) = max{t ∈ Ai|G(s,t) = 1},

where the maximum is taken with respect to relation<. A property of an axis

if Ai={t1< t2<··· < tki} is an axis and pi(s) = tj: G(s, t) 1 . . . 1 0 . . . 0 Ai t1< . . . < tj < tj+1< . . . < tki

(17)

Coordinate System

Position Function Definition

Position function p_i:S_{→ A}i assigns a test from Ai to

solution s_{∈ S:}

pi(s) = max{t ∈ Ai|G(s,t) = 1},

where the maximum is taken with respect to relation<.

A property of an axis

if Ai={t1< t2<··· < tki} is an axis and pi(s) = tj:

G(s, t) 1 . . . 1 0 . . . 0

(18)

Coordinate System

Game Dimension Definition

The coordinate system C iscorrect for a game

G = (S,T ,G) iff the dominance relation is preserved for all s1, s2∈ S, i.e.

s1≤ s2 ⇐⇒ ∀ipi(s1)≤ pi(s2)

Definition

A correct coordinate system C is aminimum coordinate system for G if it has a minimum number of axes.

Definition

Thedimension of a game G, dimB(G) is the size of the

(19)

Coordinate System

s1≤ s2 ⇐⇒ ∀ipi(s1)≤ pi(s2)

Definition

(20)

Coordinate System

s1≤ s2 ⇐⇒ ∀ipi(s1)≤ pi(s2)

Definition

(21)

Coordinate System for Nim [1,3]

Example

I Nim [1,3]: two piles: 1 stone and 3 stones

I Total:

I _{6 unique strategies for the first player (solutions)} I _{9 unique strategies for the second player (tests)}

I The payoff matrix:

t1 t2 t3 t4 t5 t6 t7 t8 t9 s1 1 1 1 1 1 1 s2 s3 1 1 1 1 1 1 1 1 1 s4 1 1 1 1 1 1 s5 1 1 1 s6 1 1 1

(22)

Minimal Coordinate System for Nim [1,3]

t1 t2 t3 t4 t5 t6 t7 t8 t9 s1 1 1 1 1 1 1 s2 s3 1 1 1 1 1 1 1 1 1 s4 1 1 1 1 1 1 s5 1 1 1 s6 1 1 1 I s1< s3

I s1solves t4and t9, but

not t8nor t2 t9 t8 t2 t4 s1 s2 s3 s4 s5 s6 Observation:

I Not all tests have to appear in the minimal coordinate system

(23)

Minimal Coordinate System for Nim [1,3]

t1 t2 t3 t4 t5 t6 t7 t8 t9 s1 1 1 1 1 1 1 s2 s3 1 1 1 1 1 1 1 1 1 s4 1 1 1 1 1 1 s5 1 1 1 s6 1 1 1 I s1< s3

I s1solves t4and t9, but

not t8nor t2 t9 t8 t2 t4 s1 s2 s3 s4 s5 s6 Observation:

I Not all tests have to appear in the minimal coordinate system

(24)

Minimal Coordinate System for Nim [1,3]

Example summary

I Before: 6 solutions and 9 tests/objectives

I After: 2 underlying objectives/axes and only 4 tests

(25)

Outline

Theoretical Results

(26)

Theorem: repeated tests

Theorem

If a test lies on two different axes, it can be removed from one of the axes and the coordinate system will remain correct.

Corollary

To find the dimension of a game, consider only such a coordinate systems C that every test from T appears on atmost one axis in C

t9 t8 t2 t4 s1 s2 s3 s4 s5 s6

(27)

Theorem: redundant tests

Definition

We will say that test t orders solution s1before

solution s₂, written s1<t s2, if:

I G(s1, t) = 0 (s1fails t)

I G(s₂, t) = 1 (s2solves t)

Theorem

Let C be a correct coordinate system forG = (S,T ,G). LetS

D =S

C_{\ {u}. D is a correct coordinate system iff} all ordered pairs ‘covered’ in C are also covered in D, i.e.,

(28)

Theorem: redundant tests

Definition

We will say that test t orders solution s1before

solution s₂, written s1<t s2, if:

I G(s1, t) = 0 (s1fails t)

I G(s₂, t) = 1 (s2solves t)

Theorem

Let C be a correct coordinate system forG = (S,T ,G). LetS

D =S

C_{\ {u}. D is a correct coordinate system iff} all ordered pairs ‘covered’ in C are also covered in D, i.e.,

(29)

Example: redundant tests

t1 t2 t3 t4 t5 t6 t7 t8 t9 s1 1 1 1 1 1 1 s2 s3 1 1 1 1 1 1 1 1 1 s4 1 1 1 1 1 1 s5 1 1 1 s6 1 1 1 I s2<t3s3, but also s2<t4s3 t9 t8 t2 t4 s1 s2 s3 s4 s5 s6

(30)

Theorem: Game Dimension and Poset Width

t0 t1 t2 t4 t5 t6 t3 t7 Theorem

Let C = (Ai)i∈I be a minimal coordinate system for game

G . Then

dimB(G ) = width(

[ C,_≤)

(31)

Theorem: Game Dimension and Poset Width

t0 t1 t2 t4 t5 t6 t3 t7 Theorem

Let C = (Ai)i∈I be a minimal coordinate system for game

G . Then

dimB(G ) = width(

[

(32)

Algorithms

I Heuristic proposed by Bucci et. al (2004)

I _{Does not guarantee finding the minimal coordinate}

system.

I Here, an exact algorithm:

I Exponential, but much faster than a trivial one I _{Founded on theorems proved in our paper}

(33)

Exact Algorithm

procedure EXACT(S, T , g) U ← AllMinimalSubsets(T )

return such U_{∈ U that minimizes ChainPartition(U,≤)} end procedure

I ALLMINIMALSUBSETS:

I Returns all minimal correct subsets of T I _Exponential

I CHAINPARTITION:

I Returns a set of chains C I O(n3)

(34)

Decision Problem

Q: How hard it is to compute dimension for a given game?

Game Dimension Problem

Given a gameG = (S,T ,G), where S and T are finite and a positive integer n, does it exist a correct coordinate systemC for game G of size n or less?

Theorem

(35)

Decision Problem

Q: How hard it is to compute dimension for a given game?

Game Dimension Problem

Given a gameG = (S,T ,G), where S and T are finite and a positive integer n, does it exist a correct coordinate systemC for game G of size n or less?

Theorem

(36)

Proof outline

Set Covering Problem

Given a universeU = (ui), a familyR = (Rj)of subsets

ofU , a cover is a subfamily V ⊆ R of sets whose union isU . Given U and R and an integer m, the question is whether there exists a cover of size m or less.

Example LetU ={1,2,3,4}, u = 4 r = 3, R1={1,2}, R2={1,3}, R3={1,3,4}. Then, e.g.,V = {R1, R3}

(37)

Proof outline

Set Covering Problem

Given a universeU = (ui), a familyR = (Rj)of subsets

ofU , a cover is a subfamily V ⊆ R of sets whose union isU . Given U and R and an integer m, the question is whether there exists a cover of size m or less.

Example LetU ={1,2,3,4}, u = 4 r = 3, R1={1,2}, R2={1,3}, R3={1,3,4}. Then, e.g.,V = {R1, R3}

(38)

Transformation

Proof.

Given an instance I2∈ Dπ2, we construct I1∈ Dπ1 :

1. n = m + u + r

2. S =_{s0} ∪ A ∪ B ∪ C ∪ D, where

A = (ai)i=1...u, B = (bi)i=1...r, C = (ci)i=1...u, and

D = (di)i=1...r are such sets that|S| = 1 + 2r + 2u.

3. T = X_{∪ Y ∪ Z , where}

X = (xi)i=1...r, Y = (yi)i=1...u, and Z = (zi)i=1...r, are

such sets that_{|T | = 2r + u}

4. G is defined as follows:            G(s0, yi) G(ai, xj) ⇐⇒ ui ∈ Rj G(ai, yj) ⇐⇒ i 6= j and G(bi, zj) ⇐⇒ i 6= j G(bi, xj) ⇐⇒ i = j and G(ci, yj) ⇐⇒ i = j and G(di, zj) ⇐⇒ i = j

(39)

Transformation Example

LetU ={1,2,3,4}, R1={1,2}, R2={1,3}, R3={1,3,4} x₁ x₂ x₃ y₁ y₂ y₃ y₄ z₁ z₂ z₃ s₀ 1 1 1 1 a1 1 1 1 1 1 1 a₂ 1 1 1 1 a3 1 1 1 1 1 a4 1 1 1 1 b1 1 1 1 b₂ 1 1 1 b3 1 1 1 c1 1 c2 1 c₃ 1 c4 1 d1 1 d2 1 d3 1

(40)

Corollary

Finding a minimal coordinate system for a game is NP-hard

(41)

“Our” Greedy Heuristics

I Based on the greedy heuristics for the Set Covering Problem.

1. Greedily find a minimal set of tests that “cover” all pairs of solutions.

2. Construct the poset and return its width.

I Approximation ratio of H(s)_{≤ lns + 1, where s is the} size of the largest set inR

I _{Best possible approximation algorithm working in}

polynomial-time (for SCP).

I It should to be also a very good solution for our problem.

(42)

Outline

Computational Experiments

(43)

Heuristics vs. Exact on a Random Payoff

Matrix

I A random game (n_{× n payoff matrix, n = 2...360)}

0 5 10 15 20 25 30 value 0 50 100 150 200 250 300 350 problem size SmartExactWithTimeout1.0 SmartExact GreedyHeuristics BucciHeuristics y(x) = 4.48 * log(x) - 2.43

(44)

Compare-on-one Game

Setup

I Artificial numbers game

I Strategies represented as real-number vectors s[1..d] and t[1..d]:

I d — thea priori dimension of the game.

I Interaction function: g(s, t) =

(

1 if s[m]≥ t[m] 0 otherwise

I where m = arg maxit[i]

(45)

Compare-on-one Game

Results

I Randomly generated n solution and n test strategies;

I Results for d = 2 and d = 3

0 1 2 3 value 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 problem size dimB(G) (exact) width(T, ≤) dim(S, ≤) 0 1 2 3 value 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 problem size dimB(G) (exact) width(T, ≤) dim(S, ≤) Observations:

I Computeddimension converges to the d — a priori dimension of a game

(46)

Compare-on-all

The interaction function:

G(s, t) _{⇐⇒ ∀}is[i]≥ t[i] 0 2 4 6 8 value 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 problem size dimB(G) (exact) width(T, ≤) dim(S, ≤)

(47)

Tic Tac Toe

0 5 10 15 20 25 30 value 0 20 40 60 80 100 problem size TTT win (heuristic) TTT win (exact)

(48)

Summary

I Each test-based-problem has: I _{a dimension, and}

I _{an inherent structure in the form of underlying}

objectives.

I The dimension of a game is (usually) much smaller than the number of strategies.

I The knowledge of the structure could help: I _{investigate essential properties of the game, and} I _{(potentially) design better coevolutionary algorithms.}

I Further work:

(49)

An Infinite Game Can Have Infinite

Dimension

Do there exist highly-dimensional games? Yes.

Example Consider a gameG (n) = (S,T ,G): S = (si)i=1...n T = (t_j)j=1...n G(si, tj) ⇐⇒ i = j I The dimension ofG (n) is n.

I For each n there exists a game with dimension n.

(50)

Infinite Game Can Have Finite Dimension

Do all inifinite games have infinite dimension? No.

Example

Consider aG (n) = (S,T ,G) S = (si)

T = (t_j) G(si, tj) ⇐⇒ i ≥ j

The dimension of this game is 1, because A1= (t1< t2< . . . ) and p1(si) =ti.

Corollary

When a game is finite, it always has a dimension; when a game is infinite, it can either have or not have a

(51)

Infinite Game Can Have Finite Dimension

Do all inifinite games have infinite dimension? No.

Example

Consider aG (n) = (S,T ,G) S = (si)

T = (t_j) G(si, tj) ⇐⇒ i ≥ j

The dimension of this game is 1, because A1= (t1< t2< . . . ) and p1(si) =ti.

Corollary

When a game is finite, it always has a dimension; when a game is infinite, it can either have or not have a

(52)

Lower and Upper Bound

Theorem

For every gameG = (S,T ,G)

(53)

Formally

I The compare-on-one coordinate system for d = 2.

I Tests solved by an exemplary test s1marked in

yellow.

t[1, 0] t[2, 0] t[3, 0] t[0, 1]

t[0, 2]