PARETO SUBOPTIMAL SOLUTIONS TO
LARGE-SCALE MULTIOBJECTIVE
MULTIDIMENSIONAL KNAPSACK
PROBLEMS WITH ASSESSMENTS OF
PARETO OPTIMALITY GAPS
IGNACY KALISZEWSKI
Systems Research Institute, Polish Academy of Sciences; Warsaw School of Information Technology
Presentation outline
This research has been inspired by a bare fact that when solving large scale multiobjective optimization (MO) problems, whether for its own or as the underlying model for Multiple Criteria Decision Making (MCDM), it may happen that Pareto optimal (e cient) solutions cannot be derived.
The reasons for that can be twofold.
When making use of heuristics, such as Evolutionary Multiobjective Optimization (EMO) or other, e ciency of solutions derived is not guaranteed.
When making use of exact methods, such as MIP optimizers, memory limits and, less often, time limits, preclude Pareto optimality derivation.
In both cases, the user is left with a feasible solution which may be believed to be Pareto optimal or close to Pareto optimality, but no information how wide is the Pareto optimality gap is oered.
Presentation outline
This research has been inspired by a bare fact that when solving large scale multiobjective optimization (MO) problems, whether for its own or as the underlying model for Multiple Criteria Decision Making (MCDM), it may happen that Pareto optimal (e cient) solutions cannot be derived.
The reasons for that can be twofold.
When making use of heuristics, such as Evolutionary Multiobjective Optimization (EMO) or other, e ciency of solutions derived is not guaranteed.
When making use of exact methods, such as MIP optimizers, memory limits and, less often, time limits, preclude Pareto optimality derivation.
In both cases, the user is left with a feasible solution which may be believed to be Pareto optimal or close to Pareto optimality, but no information how wide is the Pareto optimality gap is oered.
Presentation outline
This research has been inspired by a bare fact that when solving large scale multiobjective optimization (MO) problems, whether for its own or as the underlying model for Multiple Criteria Decision Making (MCDM), it may happen that Pareto optimal (e cient) solutions cannot be derived.
The reasons for that can be twofold.
When making use of heuristics, such as Evolutionary Multiobjective Optimization (EMO) or other, e ciency of solutions derived is not guaranteed.
When making use of exact methods, such as MIP optimizers, memory limits and, less often, time limits, preclude Pareto optimality derivation.
In both cases, the user is left with a feasible solution which may be believed to be Pareto optimal or close to Pareto optimality, but no information how wide is the Pareto optimality gap is oered.
Presentation outline
This research has been inspired by a bare fact that when solving large scale multiobjective optimization (MO) problems, whether for its own or as the underlying model for Multiple Criteria Decision Making (MCDM), it may happen that Pareto optimal (e cient) solutions cannot be derived.
The reasons for that can be twofold.
When making use of heuristics, such as Evolutionary Multiobjective Optimization (EMO) or other, e ciency of solutions derived is not guaranteed.
When making use of exact methods, such as MIP optimizers, memory limits and, less often, time limits, preclude Pareto optimality derivation.
In both cases, the user is left with a feasible solution which may be believed to be Pareto optimal or close to Pareto optimality, but no information how wide is the Pareto optimality gap is oered.
Presentation outline
This research has been inspired by a bare fact that when solving large scale multiobjective optimization (MO) problems, whether for its own or as the underlying model for Multiple Criteria Decision Making (MCDM), it may happen that Pareto optimal (e cient) solutions cannot be derived.
The reasons for that can be twofold.
When making use of heuristics, such as Evolutionary Multiobjective Optimization (EMO) or other, e ciency of solutions derived is not guaranteed.
When making use of exact methods, such as MIP optimizers, memory limits and, less often, time limits, preclude Pareto optimality derivation.
In both cases, the user is left with a feasible solution which may be believed to be Pareto optimal or close to Pareto
Presentation outline
This research has been inspired by a bare fact that when solving large scale multiobjective optimization (MO) problems, whether for its own or as the underlying model for Multiple Criteria Decision Making (MCDM), it may happen that Pareto optimal (e cient) solutions cannot be derived.
The reasons for that can be twofold.
When making use of heuristics, such as Evolutionary Multiobjective Optimization (EMO) or other, e ciency of solutions derived is not guaranteed.
When making use of exact methods, such as MIP optimizers, memory limits and, less often, time limits, preclude Pareto optimality derivation.
In both cases, the user is left with a feasible solution which may be believed to be Pareto optimal or close to Pareto optimality, but no information how wide is the Pareto optimality gap is oered.
Presentation outline
LARGE-SCALE MULTIOBJECTIVE
MULTIDIMENSIONAL KNAPSACK PROBLEMS
Presentation outline
INTEL, the producer of integrated circuits, uses multiobjective multidimensional knapsack model for R & D investment planning;
(according to Wagner Prize Presentations in San Francisco, INFORMS Conference 2014, Intel: Project Portfolio Planning at Intel Corporation,
https://www. pathlms.com/informs/events/ 219/thumbnail_video_presentations/5939 ).
Derivation of e cient solutions
Let x denote a solution, X a space of solutions, X0 a set of
feasible solutions, X0⊆ X .
Then the multiobjective optimization problem is de ned as: vmax f (x )
(∗) x ∈ X0,
where f : X → Rk, f = (f
1, ..., fk),
fl :→ R, l =1, ..., k, k ≥ 2, are objective functions, and vmax
denotes the operator of deriving all Pareto optimal solutions in X0.
Solution ¯x is Pareto optimal if fl(x ) ≥ fl(¯x ), l =1, ..., k,
Derivation of e cient solutions
Let x denote a solution, X a space of solutions, X0 a set of
feasible solutions, X0⊆ X .
Then the multiobjective optimization problem is de ned as: vmax f (x )
(∗) x ∈ X0,
where f : X → Rk, f = (f
1, ..., fk),
fl :→ R, l =1, ..., k, k ≥ 2, are objective functions, and vmax
denotes the operator of deriving all Pareto optimal solutions in X0.
Solution ¯x is Pareto optimal if fl(x ) ≥ fl(¯x ), l =1, ..., k,
Derivation of e cient solutions
Let x denote a solution, X a space of solutions, X0 a set of
feasible solutions, X0⊆ X .
Then the multiobjective optimization problem is de ned as: vmax f (x )
(∗) x ∈ X0,
where f : X → Rk, f = (f
1, ..., fk),
fl :→ R, l =1, ..., k, k ≥ 2, are objective functions, and vmax
denotes the operator of deriving all Pareto optimal solutions in X0.
Derivation of e cient solutions
It is a well-established result that solution x is Pareto optimal if and only if it solves the Tchebyche weighted optimization problem
minx ∈X0maxl[λl(yl∗− fl(x )) + ρek(y∗− f (x))] (∗∗)
where weights λl >0, l = 1, ..., k, ek = (1, 1, ..., 1),
yl∗ = ˆyl+ ε, ˆyl =maxx ∈X0fl(x ), l =1, ..., k, ε > 0, and ρ is a
positive su ciently small number.
By the only if part of this result, no Pareto optimal solution is a priori excluded from being derived by solving an instance of optimization problem (**).
In contrast to that, maximization of a weighted sum of objective functions over X0 does not possess, in general (and
especially in the case of problems with discrete variables), this property.
Derivation of e cient solutions
It is a well-established result that solution x is Pareto optimal if and only if it solves the Tchebyche weighted optimization problem
minx ∈X0maxl[λl(yl∗− fl(x )) + ρek(y∗− f (x))] (∗∗)
where weights λl >0, l = 1, ..., k, ek = (1, 1, ..., 1),
yl∗ = ˆyl+ ε, ˆyl =maxx ∈X0fl(x ), l =1, ..., k, ε > 0, and ρ is a
positive su ciently small number.
By the only if part of this result, no Pareto optimal solution is a priori excluded from being derived by solving an instance of optimization problem (**).
In contrast to that, maximization of a weighted sum of objective functions over X0 does not possess, in general (and
especially in the case of problems with discrete variables), this property.
Derivation of e cient solutions
It is a well-established result that solution x is Pareto optimal if and only if it solves the Tchebyche weighted optimization problem
minx ∈X0maxl[λl(yl∗− fl(x )) + ρek(y∗− f (x))] (∗∗)
where weights λl >0, l = 1, ..., k, ek = (1, 1, ..., 1),
yl∗ = ˆyl+ ε, ˆyl =maxx ∈X0fl(x ), l =1, ..., k, ε > 0, and ρ is a
positive su ciently small number.
By the only if part of this result, no Pareto optimal solution is a priori excluded from being derived by solving an instance of optimization problem (**).
In contrast to that, maximization of a weighted sum of objective functions over X0 does not possess, in general (and
Derivation of e cient solutions
On the rst glance, the objective function in (*) seems to be di cult to handle.
However, the optimization problem (*) is equivalent to min s ,
s ≥ λl(yl∗− fl(x )) + ρek(y∗− f (x)), l =1, ..., k , (∗ ∗ ∗)
Derivation of e cient solutions
On the rst glance, the objective function in (*) seems to be di cult to handle.
However, the optimization problem (*) is equivalent to min s ,
s ≥ λl(yl∗− fl(x )) + ρek(y∗− f (x)), l =1, ..., k , (∗ ∗ ∗)
Beasley OR Library
Beasley OR Library
Beasley OR Library multidimensional
knapsack problems
Beasley OR library contains a set multidimensional knapsack test problems.
There are 9 groups of problems with numbers of constraints m set to 5, 10, 30, and the number of variables n set to
100, 250, 500 and each group contains 30 problems, giving a total of 270 problems.
Beasley OR Library multidimensional
knapsack problems
Beasley OR library contains a set multidimensional knapsack test problems.
There are 9 groups of problems with numbers of constraints m set to 5, 10, 30, and the number of variables n set to
100, 250, 500 and each group contains 30 problems, giving a total of 270 problems.
Beasley OR Library multidimensional
knapsack problems
Data are as described in Chu and Beasley (1998).
Coe cients ai ,j are integers drawn at random but uniformly
from the interval (0, 1000).
For each m − n combination, the right-hand side coe cients bi
are de ned as follows:
bi = α n
X
j
ai ,j
where α is a tightness ratio where α is 0.25, 0.50, 0.75 for the rst ten, for the next ten, and for the remaining ten problems, respectively.
Beasley OR Library multidimensional
knapsack problems
Data are as described in Chu and Beasley (1998).
Coe cients ai ,j are integers drawn at random but uniformly
from the interval (0, 1000).
For each m − n combination, the right-hand side coe cients bi
are de ned as follows:
bi = α n
X
j
ai ,j
where α is a tightness ratio where α is 0.25, 0.50, 0.75 for the rst ten, for the next ten, and for the remaining ten problems, respectively.
Beasley OR Library multidimensional
knapsack problems
Data are as described in Chu and Beasley (1998).
Coe cients ai ,j are integers drawn at random but uniformly
from the interval (0, 1000).
For each m − n combination, the right-hand side coe cients bi
are de ned as follows:
bi = α n
X
j
ai ,j
where α is a tightness ratio where α is 0.25, 0.50, 0.75 for the rst ten, for the next ten, and for the remaining ten problems, respectively.
Beasley OR Library multidimensional
knapsack problems
The objective function coe cients cj are integers correlated to
ai ,j and generated as follows:
cj = b500qj + 1 m m X i =1 ai ,jc, j =1, ..., n,
where qj is a real number drawn from the continuous uniform
generator U(0, 1) and b·c denotes rounding down to the nearest integer.
Beasley OR Library multidimensional
knapsack problems
To solve the problems, Chu and Beasley used CPLEX version 4.0 on Silicon Graphics Indigo workstation (R4000, 100 MHz, 48 MB memory).
The solving process was stopped whenever tree memory usage exceeded 42 MB or after 1800 CPU seconds.
Chu and Beasley were able to solve all problems from (5 − 100), (5 − 250) and (10 − 100) groups to optimality. For all the remaining problems from (5 − 500), (10 − 250), (10 − 500), (300 − 100), (300 − 250), (300 − 500) groups, no optimal solution was obtained because the solving precesses reached one of the limits mentioned above.
For each problem the optimal solution or the incumbent (i.e. the best feasible solution) and the corresponding objective values were recorded.
Below, we shall use these problems to construct a set of biobjective problems, just by adding an extra objective function.
Beasley OR Library multidimensional
knapsack problems
To solve the problems, Chu and Beasley used CPLEX version 4.0 on Silicon Graphics Indigo workstation (R4000, 100 MHz, 48 MB memory).
The solving process was stopped whenever tree memory usage exceeded 42 MB or after 1800 CPU seconds.
Chu and Beasley were able to solve all problems from (5 − 100), (5 − 250) and (10 − 100) groups to optimality. For all the remaining problems from (5 − 500), (10 − 250), (10 − 500), (300 − 100), (300 − 250), (300 − 500) groups, no optimal solution was obtained because the solving precesses reached one of the limits mentioned above.
For each problem the optimal solution or the incumbent (i.e. the best feasible solution) and the corresponding objective values were recorded.
Below, we shall use these problems to construct a set of biobjective problems, just by adding an extra objective function.
Beasley OR Library multidimensional
knapsack problems
To solve the problems, Chu and Beasley used CPLEX version 4.0 on Silicon Graphics Indigo workstation (R4000, 100 MHz, 48 MB memory).
The solving process was stopped whenever tree memory usage exceeded 42 MB or after 1800 CPU seconds.
Chu and Beasley were able to solve all problems from (5 − 100), (5 − 250) and (10 − 100) groups to optimality.
For all the remaining problems from (5 − 500), (10 − 250), (10 − 500), (300 − 100), (300 − 250), (300 − 500) groups, no optimal solution was obtained because the solving precesses reached one of the limits mentioned above.
For each problem the optimal solution or the incumbent (i.e. the best feasible solution) and the corresponding objective values were recorded.
Below, we shall use these problems to construct a set of biobjective problems, just by adding an extra objective function.
Beasley OR Library multidimensional
knapsack problems
To solve the problems, Chu and Beasley used CPLEX version 4.0 on Silicon Graphics Indigo workstation (R4000, 100 MHz, 48 MB memory).
The solving process was stopped whenever tree memory usage exceeded 42 MB or after 1800 CPU seconds.
Chu and Beasley were able to solve all problems from (5 − 100), (5 − 250) and (10 − 100) groups to optimality. For all the remaining problems from (5 − 500), (10 − 250), (10 − 500), (300 − 100), (300 − 250), (300 − 500) groups, no optimal solution was obtained because the solving precesses reached one of the limits mentioned above.
For each problem the optimal solution or the incumbent (i.e. the best feasible solution) and the corresponding objective values were recorded.
Below, we shall use these problems to construct a set of biobjective problems, just by adding an extra objective function.
Beasley OR Library multidimensional
knapsack problems
To solve the problems, Chu and Beasley used CPLEX version 4.0 on Silicon Graphics Indigo workstation (R4000, 100 MHz, 48 MB memory).
The solving process was stopped whenever tree memory usage exceeded 42 MB or after 1800 CPU seconds.
Chu and Beasley were able to solve all problems from (5 − 100), (5 − 250) and (10 − 100) groups to optimality. For all the remaining problems from (5 − 500), (10 − 250), (10 − 500), (300 − 100), (300 − 250), (300 − 500) groups, no optimal solution was obtained because the solving precesses reached one of the limits mentioned above.
For each problem the optimal solution or the incumbent (i.e. the best feasible solution) and the corresponding objective values were recorded.
Below, we shall use these problems to construct a set of biobjective problems, just by adding an extra objective function.
Beasley OR Library multidimensional
knapsack problems
To solve the problems, Chu and Beasley used CPLEX version 4.0 on Silicon Graphics Indigo workstation (R4000, 100 MHz, 48 MB memory).
The solving process was stopped whenever tree memory usage exceeded 42 MB or after 1800 CPU seconds.
Chu and Beasley were able to solve all problems from (5 − 100), (5 − 250) and (10 − 100) groups to optimality. For all the remaining problems from (5 − 500), (10 − 250), (10 − 500), (300 − 100), (300 − 250), (300 − 500) groups, no optimal solution was obtained because the solving precesses reached one of the limits mentioned above.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
As to solve multiobjective problems (*) we rely on optimization solvers, we decided to investigate at rst the eectiveness of contemporary MILP packages.
Our choice has been CPLEX, because it is renown as an eective and robust optimizer for MILP problems.
Morever, it is accessible free of charge via NEOS platform (https://neos-server.org/neos/solvers/lp:CPLEX/LP.html). Running optimization problems on an open access platform ensures perfect reproducibility of results.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
As to solve multiobjective problems (*) we rely on optimization solvers, we decided to investigate at rst the eectiveness of contemporary MILP packages.
Our choice has been CPLEX, because it is renown as an eective and robust optimizer for MILP problems.
Morever, it is accessible free of charge via NEOS platform (https://neos-server.org/neos/solvers/lp:CPLEX/LP.html).
Running optimization problems on an open access platform ensures perfect reproducibility of results.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
As to solve multiobjective problems (*) we rely on optimization solvers, we decided to investigate at rst the eectiveness of contemporary MILP packages.
Our choice has been CPLEX, because it is renown as an eective and robust optimizer for MILP problems.
Morever, it is accessible free of charge via NEOS platform (https://neos-server.org/neos/solvers/lp:CPLEX/LP.html). Running optimization problems on an open access platform ensures perfect reproducibility of results.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
At the time of numerical experiments, this platform run CPLEX version 12.6.2.0.
With NEOS, jobs are terminated whenever time limit 8 hours of CPU or memory limit 3 Gb is reached.
However, as NEOS reaching one of those limits provides no output log, we put the memory limit to 2.048 Gb.
Adviced by the NEOS support team, in all CPLEX runs we used the following CPLEX parameter settings: set workmem 128, set mip strategy le 2, set mip limits treememory 2048.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
At the time of numerical experiments, this platform run CPLEX version 12.6.2.0.
With NEOS, jobs are terminated whenever time limit 8 hours of CPU or memory limit 3 Gb is reached.
However, as NEOS reaching one of those limits provides no output log, we put the memory limit to 2.048 Gb.
Adviced by the NEOS support team, in all CPLEX runs we used the following CPLEX parameter settings: set workmem 128, set mip strategy le 2, set mip limits treememory 2048.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
At the time of numerical experiments, this platform run CPLEX version 12.6.2.0.
With NEOS, jobs are terminated whenever time limit 8 hours of CPU or memory limit 3 Gb is reached.
However, as NEOS reaching one of those limits provides no output log, we put the memory limit to 2.048 Gb.
Adviced by the NEOS support team, in all CPLEX runs we used the following CPLEX parameter settings: set workmem 128, set mip strategy le 2, set mip limits treememory 2048.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
At the time of numerical experiments, this platform run CPLEX version 12.6.2.0.
With NEOS, jobs are terminated whenever time limit 8 hours of CPU or memory limit 3 Gb is reached.
However, as NEOS reaching one of those limits provides no output log, we put the memory limit to 2.048 Gb.
Adviced by the NEOS support team, in all CPLEX runs we used the following CPLEX parameter settings: set workmem 128, set mip strategy le 2, set mip limits treememory 2048.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
To prepare input data for CPLEX we converted all the 270 knapsack problems to the LP format. We made the converted problems publicly available via the web:
http://www.ibspan.waw.pl/∼kaliszew/main_page.html .
With CPLEX running on NEOS we were able to solve to optimality almost all problems from (5, 500) group, a few problems from (10, 250) group and no problem from (10, 500) group.
No attempt was made to solve problems from (30,100), (30, 250), (30, 500) group.
The gap, as provided by CPLEX, is calculated as
gap = current MIP best bound - objective function value for incumbentobjective function value for incumbent
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
To prepare input data for CPLEX we converted all the 270 knapsack problems to the LP format. We made the converted problems publicly available via the web:
http://www.ibspan.waw.pl/∼kaliszew/main_page.html . With CPLEX running on NEOS we were able to solve to optimality almost all problems from (5, 500) group, a few problems from (10, 250) group and no problem from (10, 500) group.
No attempt was made to solve problems from (30,100), (30, 250), (30, 500) group.
The gap, as provided by CPLEX, is calculated as
gap = current MIP best bound - objective function value for incumbentobjective function value for incumbent
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
To prepare input data for CPLEX we converted all the 270 knapsack problems to the LP format. We made the converted problems publicly available via the web:
http://www.ibspan.waw.pl/∼kaliszew/main_page.html . With CPLEX running on NEOS we were able to solve to optimality almost all problems from (5, 500) group, a few problems from (10, 250) group and no problem from (10, 500) group.
No attempt was made to solve problems from (30,100), (30, 250), (30, 500) group.
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
Problem CPLEX (Chu & Beasley) CPLEX (NEOS)
Av. Av. Av. Av. Av. Av. Av. No. sol. no. gap. sol. sol. no. gap opt. m, n α time nodes time time nodes
(sec.) (%) (sec.) (ticks) (%) 5, 500 0.25 (981.6) 64398 0.99 351.0 53924.7 9368662 0.01 10 0.50 (1048.0) 72744 0.45 181.5 26629.6 4714172 0.01 9 0.75 (1129.8) 80101 0.32 66.5 6889.8 1443435 0.01 10 10, 250 0.25 (1006.2) 69545 4.80 362.3 53153.3 8618316 0.12 3 0.50 (1054.7) 78502 5.41 423.3 678234.5 9532148 0.07 1 0.75 (1195.2) 81475 1.85 331.5 53503.0 7864490 0.02 9 10, 500 0.25 (1738.2) 68723 4.88 260.9 47926.7 4676405 0.15 0 0.50 (1651.2) 68929 5.50 227.9 48963.5 4820390 0.07 0
Solving Beasley OR Library multidimensional
knapsack problems on the NEOS platform
In the case when one is not able to solve a problem to optimality, one is left with an absolute upper bound on the optimal (here: maximal) objective function value, namely
Biobjective multidimensional knapsack
problems
We generated biobjective multidimensional knapsack (BMK) problems from Chu and Beasley problems.
For each original Chu and Beasley problem we added the second objective function, coe cients of which were a random permutation of the coe cients of the rst objective function. To solve BMK problems obtained in that manner, we made use of transformation (***) of optimization problem (**).
In case of knapsack problems, the resulting problems are mixed-integer linear problems, thus they can be handled by CPLEX.
Biobjective multidimensional knapsack
problems
We generated biobjective multidimensional knapsack (BMK) problems from Chu and Beasley problems.
For each original Chu and Beasley problem we added the second objective function, coe cients of which were a random permutation of the coe cients of the rst objective function.
To solve BMK problems obtained in that manner, we made use of transformation (***) of optimization problem (**).
In case of knapsack problems, the resulting problems are mixed-integer linear problems, thus they can be handled by CPLEX.
Biobjective multidimensional knapsack
problems
We generated biobjective multidimensional knapsack (BMK) problems from Chu and Beasley problems.
For each original Chu and Beasley problem we added the second objective function, coe cients of which were a random permutation of the coe cients of the rst objective function. To solve BMK problems obtained in that manner, we made use of transformation (***) of optimization problem (**).
In case of knapsack problems, the resulting problems are mixed-integer linear problems, thus they can be handled by CPLEX.
Biobjective multidimensional knapsack
problems
We generated biobjective multidimensional knapsack (BMK) problems from Chu and Beasley problems.
For each original Chu and Beasley problem we added the second objective function, coe cients of which were a random permutation of the coe cients of the rst objective function. To solve BMK problems obtained in that manner, we made use of transformation (***) of optimization problem (**).
In case of knapsack problems, the resulting problems are mixed-integer linear problems, thus they can be handled by CPLEX.
Biobjective multidimensional knapsack
problems
To simplify modi cations of the input LP les re ecting varying weights λ, we introduced two variables f 1 and f 2, equal to the rst and the second objective function, respectively,
−f1+ N X j =1 c1,jxj =0 , −f2+ N X j =1 c2,jxj =0 .
Biobjective multidimensional knapsack
problems
To simplify modi cations of the input LP les re ecting varying weights λ, we introduced two variables f 1 and f 2, equal to the rst and the second objective function, respectively,
−f1+ N X j =1 c1,jxj =0 , −f2+ N X j =1 c2,jxj =0 .
Biobjective multidimensional knapsack
problems
With that convention, the transformed problems have the following form: min s , s.t. s + (λ1+ ρ)f1+ ρf2≥ λ1y1∗+ ρ(y1∗+ y2∗), s + ρf1+ (λ2+ ρ)f2≥ λ2y2∗+ ρ(y1∗+ y2∗), −f1+PN j =1c1,jxj =0 , −f2+PN j =1c2,jxj =0 , Pn i ,jai ,jxj ≤ bi, i =1, ..., m,
Biobjective multidimensional knapsack
problems
We made the transformed problems publicly available via the web: http://www.ibspan.waw.pl/∼kaliszew/main_page.html .
For each problem λ1 and λ2 were set by default to 0.5 and for
dierent settings of λl the rst two constraints on the previous
Biobjective multidimensional knapsack
problems
We made the transformed problems publicly available via the web: http://www.ibspan.waw.pl/∼kaliszew/main_page.html . For each problem λ1 and λ2 were set by default to 0.5 and for
dierent settings of λl the rst two constraints on the previous
Deriving upper bounds for biobjective multidimensional
knapsack problems
We consider the case when a BMK problem, formulated as problem (***), cannot be solved to optimality because a solver reaches time or memory limit.
For example, this is the case of NEOS CPLEX and biobjective multidimensional knapsack problems steaming from the group (10, 500) of single objective problems.
It is the rst group of BMK problems (starting from low dimensions up) in which no problem (***) (we checked this for λ1= λ2 =0.5) was solved to optimality by NEOS CPLEX, because of memory limit hit.
For such cases, to assess the quality of incumbent, an upper
Deriving upper bounds for biobjective multidimensional
knapsack problems
We consider the case when a BMK problem, formulated as problem (***), cannot be solved to optimality because a solver reaches time or memory limit.
For example, this is the case of NEOS CPLEX and biobjective multidimensional knapsack problems steaming from the group (10, 500) of single objective problems.
It is the rst group of BMK problems (starting from low dimensions up) in which no problem (***) (we checked this for λ1= λ2 =0.5) was solved to optimality by NEOS CPLEX, because of memory limit hit.
For such cases, to assess the quality of incumbent, an upper
Deriving upper bounds for biobjective multidimensional
knapsack problems
We consider the case when a BMK problem, formulated as problem (***), cannot be solved to optimality because a solver reaches time or memory limit.
For example, this is the case of NEOS CPLEX and biobjective multidimensional knapsack problems steaming from the group (10, 500) of single objective problems.
It is the rst group of BMK problems (starting from low dimensions up) in which no problem (***) (we checked this for λ1 = λ2 =0.5) was solved to optimality by NEOS CPLEX, because of memory limit hit.
For such cases, to assess the quality of incumbent, an upper
Deriving upper bounds for biobjective multidimensional
knapsack problems
We consider the case when a BMK problem, formulated as problem (***), cannot be solved to optimality because a solver reaches time or memory limit.
For example, this is the case of NEOS CPLEX and biobjective multidimensional knapsack problems steaming from the group (10, 500) of single objective problems.
It is the rst group of BMK problems (starting from low dimensions up) in which no problem (***) (we checked this for λ1 = λ2 =0.5) was solved to optimality by NEOS CPLEX, because of memory limit hit.
For such cases, to assess the quality of incumbent, an upper
Deriving upper bounds for biobjective multidimensional
knapsack problems
In contrast to the single objective case, the upper bound
optimal objective function value ≤ current MIP best bound.
on the optimal value of objective function s in optimization problem (***) provides no information on upper bounds for f1(x )or f2(x ).
Deriving upper bounds for biobjective multidimensional
knapsack problems
The upper bounds can be established by the general method by Kaliszewski and Miroforidis described in a series of publications.
Here, we adapt this method to the case when a BMK (k = 2) problem is solved by a MIP solver.
Dominance relation ≺ on X is de ned as x0≺ x ⇔ f (x0) f (x ),
where denotes fl(x0) ≤ fl(x ), l =1, 2, and fl(x0) < fl(x )for
Deriving upper bounds for biobjective multidimensional
knapsack problems
The upper bounds can be established by the general method by Kaliszewski and Miroforidis described in a series of publications. Here, we adapt this method to the case when a BMK (k = 2) problem is solved by a MIP solver.
Dominance relation ≺ on X is de ned as x0≺ x ⇔ f (x0) f (x ),
where denotes fl(x0) ≤ fl(x ), l =1, 2, and fl(x0) < fl(x )for
Deriving upper bounds for biobjective multidimensional
knapsack problems
In our method, we make use of a nite nonempty set SU ⊆ X \ X0, elements of which satisfy
∀x ∈SU ¬∃x0∈S U x 0 ≺ x, ∀x ∈SU fl(x inc) ≤ f l(x ), l =1, 2 .
Deriving upper bounds for biobjective multidimensional
knapsack problems
Deriving upper bounds for biobjective multidimensional
knapsack problems
Given λ = (λ1, λ2), let xPopt(λ) denote the Pareto optimal solution to problem (***) which would be derived if problem (***) were solved to optimality.
Then fl(xPopt(λ)) ≤ Ui(SU, λ) = min ¯ x ∈SU min i ∈I (λ,¯x ,l )fi(¯x ), l =1, 2.
where I (λ, ¯x, l) is a subset of indices {1, 2} such that l ∈ I (λ, ¯x , l ) if tl =min{t1, t1}, otherwise I (λ, ¯x) = ∅,
Deriving upper bounds for biobjective multidimensional
knapsack problems
Given λ = (λ1, λ2), let xPopt(λ) denote the Pareto optimal solution to problem (***) which would be derived if problem (***) were solved to optimality.
Then fl(xPopt(λ)) ≤ Ui(SU, λ) = min ¯ x ∈SU min i ∈I (λ,¯x ,l )fi(¯x ), l =1, 2.
where I (λ, ¯x, l) is a subset of indices {1, 2} such that l ∈ I (λ, ¯x , l ) if tl =min{t1, t1}, otherwise I (λ, ¯x) = ∅,
Deriving upper bounds for biobjective multidimensional
knapsack problems
To populate SU for BMK problems, we have proposed a simple
heuristic procedure consisting of pairwise replacements of 0s by 1s in the incumbent.
All the resulting infeasible solutions are stored.
Next, the infeasible solutions thus obtained, which do not satisfy the condition
∀x ∈SU fl(xinc) ≤ fl(x ), l =1, 2 ,
are ltered out.
Also, all dominating infeasible solutions in the remaining set are ltered out.
Deriving upper bounds for biobjective multidimensional
knapsack problems
To populate SU for BMK problems, we have proposed a simple
heuristic procedure consisting of pairwise replacements of 0s by 1s in the incumbent.
All the resulting infeasible solutions are stored.
Next, the infeasible solutions thus obtained, which do not satisfy the condition
∀x ∈SU fl(xinc) ≤ fl(x ), l =1, 2 ,
are ltered out.
Also, all dominating infeasible solutions in the remaining set are ltered out.
Deriving upper bounds for biobjective multidimensional
knapsack problems
To populate SU for BMK problems, we have proposed a simple
heuristic procedure consisting of pairwise replacements of 0s by 1s in the incumbent.
All the resulting infeasible solutions are stored.
Next, the infeasible solutions thus obtained, which do not satisfy the condition
∀x ∈SU fl(xinc) ≤ fl(x ), l =1, 2 ,
are ltered out.
Also, all dominating infeasible solutions in the remaining set are ltered out.
Deriving upper bounds for biobjective multidimensional
knapsack problems
To populate SU for BMK problems, we have proposed a simple
heuristic procedure consisting of pairwise replacements of 0s by 1s in the incumbent.
All the resulting infeasible solutions are stored.
Next, the infeasible solutions thus obtained, which do not satisfy the condition
∀x ∈SU fl(xinc) ≤ fl(x ), l =1, 2 ,
are ltered out.
Also, all dominating infeasible solutions in the remaining set are
Deriving upper bounds for biobjective multidimensional
knapsack problems
To populate SU for BMK problems, we have proposed a simple
heuristic procedure consisting of pairwise replacements of 0s by 1s in the incumbent.
All the resulting infeasible solutions are stored.
Next, the infeasible solutions thus obtained, which do not satisfy the condition
∀x ∈SU fl(xinc) ≤ fl(x ), l =1, 2 ,
are ltered out.
Also, all dominating infeasible solutions in the remaining set are ltered out.
Deriving upper bounds for biobjective multidimensional
knapsack problems
This procedure works if the incumbent is good enough", i.e. in terms of f (·) it is su ciently close to the Pareto optimal solution.
To illustrate the working of the SU populating procedure and
the process of establishing upper bounds, we start with the rst problem from BMK (10, 500) group, which in the set of BMK problems we have made available is denoted as
mknapcb_6_1_bicrit_LP.txt.
This problem has been derived as described above, from the rst problem in the le mknapcb6.txt from the OR Beasley Library.
Deriving upper bounds for biobjective multidimensional
knapsack problems
This procedure works if the incumbent is good enough", i.e. in terms of f (·) it is su ciently close to the Pareto optimal solution.
To illustrate the working of the SU populating procedure and
the process of establishing upper bounds, we start with the rst problem from BMK (10, 500) group, which in the set of BMK problems we have made available is denoted as
mknapcb_6_1_bicrit_LP.txt.
This problem has been derived as described above, from the rst problem in the le mknapcb6.txt from the OR Beasley Library.
Deriving upper bounds for biobjective multidimensional
knapsack problems
We can assume y∗ l = Pn j =1c1,j, l =1, 2, which clearlysatis es the requirement for y∗.
Here we opt to use a rather conservative estimates of ˆyl
because they are easily calculable and allow avoid computing the exact values of ˆyl .
Such conservative estimates can be a problem in decisional contexts, where ˆy is used as a reference ideal point and than y∗ should be its close estimate.
Deriving upper bounds for biobjective multidimensional
knapsack problems
We can assume y∗ l = Pn j =1c1,j, l =1, 2, which clearlysatis es the requirement for y∗.
Here we opt to use a rather conservative estimates of ˆyl
because they are easily calculable and allow avoid computing the exact values of ˆyl .
Such conservative estimates can be a problem in decisional contexts, where ˆy is used as a reference ideal point and than y∗ should be its close estimate.
Deriving upper bounds for biobjective multidimensional
knapsack problems
We can assume y∗ l = Pn j =1c1,j, l =1, 2, which clearlysatis es the requirement for y∗.
Here we opt to use a rather conservative estimates of ˆyl
because they are easily calculable and allow avoid computing the exact values of ˆyl .
Such conservative estimates can be a problem in decisional contexts, where ˆy is used as a reference ideal point and than y∗ should be its close estimate.
Deriving upper bounds for biobjective multidimensional
knapsack problems
We can assume y∗ l = Pn j =1c1,j, l =1, 2, which clearlysatis es the requirement for y∗.
Here we opt to use a rather conservative estimates of ˆyl
because they are easily calculable and allow avoid computing the exact values of ˆyl .
Such conservative estimates can be a problem in decisional contexts, where ˆy is used as a reference ideal point and than y∗ should be its close estimate.
Deriving upper bounds for biobjective multidimensional
knapsack problems
By solving this problem, formulated as single objective
optimization problem (***) with default values λ = (0.5, 0.5), we have hoped to get a Pareto optimal solution.
However, with the memory limit exceeded, we get only the incumbent with f1(xinc) =123599 and f2(xinc) =123637.
To get set SU we applied the procedure proposed above.
In the incumbent there are 148 variables equal to 1, so the procedure has yielded 148 × (500 − 148) = 50926 solutions, all infeasible, among which 1872 satisfy f1(xinc) ≤ f1(x )and
f2(xinc) ≤ f2(x ), f (xinc) 6= f (x ).
Figure on the next slide shows these solutions represented by their corresponding f1(·) and f2(·).
Deriving upper bounds for biobjective multidimensional
knapsack problems
By solving this problem, formulated as single objective
optimization problem (***) with default values λ = (0.5, 0.5), we have hoped to get a Pareto optimal solution.
However, with the memory limit exceeded, we get only the incumbent with f1(xinc) =123599 and f2(xinc) =123637.
To get set SU we applied the procedure proposed above.
In the incumbent there are 148 variables equal to 1, so the procedure has yielded 148 × (500 − 148) = 50926 solutions, all infeasible, among which 1872 satisfy f1(xinc) ≤ f1(x )and
f2(xinc) ≤ f2(x ), f (xinc) 6= f (x ).
Figure on the next slide shows these solutions represented by their corresponding f1(·) and f2(·).
Deriving upper bounds for biobjective multidimensional
knapsack problems
By solving this problem, formulated as single objective
optimization problem (***) with default values λ = (0.5, 0.5), we have hoped to get a Pareto optimal solution.
However, with the memory limit exceeded, we get only the incumbent with f1(xinc) =123599 and f2(xinc) =123637.
To get set SU we applied the procedure proposed above.
In the incumbent there are 148 variables equal to 1, so the procedure has yielded 148 × (500 − 148) = 50926 solutions, all infeasible, among which 1872 satisfy f1(xinc) ≤ f1(x )and
f2(xinc) ≤ f2(x ), f (xinc) 6= f (x ).
Figure on the next slide shows these solutions represented by their corresponding f1(·) and f2(·).
Deriving upper bounds for biobjective multidimensional
knapsack problems
By solving this problem, formulated as single objective
optimization problem (***) with default values λ = (0.5, 0.5), we have hoped to get a Pareto optimal solution.
However, with the memory limit exceeded, we get only the incumbent with f1(xinc) =123599 and f2(xinc) =123637.
To get set SU we applied the procedure proposed above.
In the incumbent there are 148 variables equal to 1, so the procedure has yielded 148 × (500 − 148) = 50926 solutions, all infeasible, among which 1872 satisfy f1(xinc) ≤ f1(x )and
f2(xinc) ≤ f2(x ), f (xinc) 6= f (x ).
Figure on the next slide shows these solutions represented by their corresponding f1(·) and f2(·).
Deriving upper bounds for biobjective multidimensional
knapsack problems
By solving this problem, formulated as single objective
optimization problem (***) with default values λ = (0.5, 0.5), we have hoped to get a Pareto optimal solution.
However, with the memory limit exceeded, we get only the incumbent with f1(xinc) =123599 and f2(xinc) =123637.
To get set SU we applied the procedure proposed above.
In the incumbent there are 148 variables equal to 1, so the procedure has yielded 148 × (500 − 148) = 50926 solutions, all infeasible, among which 1872 satisfy f1(xinc) ≤ f1(x )and
Deriving upper bounds for biobjective multidimensional
knapsack problems
Among all these solutions there are only 5 solutions which are not dominated, so they form set S .
Deriving upper bounds for biobjective multidimensional
knapsack problems
Deriving upper bounds for biobjective multidimensional
knapsack problems
Now, with SU derived, we can calculate upper bounds for
f1(xPopt(0.5, 0.5)), f
2(xPopt(0.5, 0.5)), as presented in the table on the next slide.
For the considered problem y∗
Deriving upper bounds for biobjective multidimensional
knapsack problems
Now, with SU derived, we can calculate upper bounds for
f1(xPopt(0.5, 0.5)), f
2(xPopt(0.5, 0.5)), as presented in the table on the next slide.
For the considered problem y∗
Deriving upper bounds for biobjective multidimensional
knapsack problems
SU f1(¯x ) f2(¯x ) y1∗− f1(¯x ) y2∗− f2(¯x ) t1 t2 I( λ, ¯x ,1 ) I( λ, ¯x ,2 ) min i∈ I( λ, ¯x ,1 ) min i∈ I( λ, ¯x ,2 ) ¯ x1 123599 123653 0.397428 0.397513 125809.00 125782.00 2 ∅ 123653 +∞ ¯ x2 123608 123638 0.397442 0.397489 125804.50 125789.50 2 ∅ 123638 +∞ ¯ x3 123601 123644 0.397431 0.397499 125808.00 125786.50 2 ∅ 123644 +∞ ¯ x4 123600 123648 0.397429 0.397505 125808.50 125784.50 2 ∅ 123648 +∞ ¯ x5 123640 123637 0.397493 0.397488 125788.50 125790.00 ∅ 1 +∞ 123640 min 123638 123640Deriving upper bounds for biobjective multidimensional
knapsack problems
We get f1(xPopt(λ)) ≤ U 1(SU, λ) =123638 , f2(xPopt(λ)) ≤ U 2(SU, λ) =123640 .We have repeated the calculation for the same problem with dierent values of λ, namely λ = (0.3, 0.7) and λ = (0.7, 0.3). The table on the next slide summarizes the results.
Deriving upper bounds for biobjective multidimensional
knapsack problems
We get f1(xPopt(λ)) ≤ U 1(SU, λ) =123638 , f2(xPopt(λ)) ≤ U 2(SU, λ) =123640 .We have repeated the calculation for the same problem with dierent values of λ, namely λ = (0.3, 0.7) and λ = (0.7, 0.3). The table on the next slide summarizes the results.
Deriving upper bounds for biobjective multidimensional
knapsack problems
λ1 λ2 f1(xinc) f 2(xinc) |S U| U1(SU, λ) U2(SU, λ) 0.3 0.7 107766 129837 7 107766 +∞ 0.5 0.5 123599 123637 5 123638 123640 0.7 0.3 128730 108468 3 +∞ 108469Deriving upper bounds for biobjective multidimensional
knapsack problems
In the table on the previous slide we see that with λ = (0.3, 0.7) the upper bound for f2(xP
opt(λ))is tight but for
f2(xoptP (λ))no upper bound is available.
The reason is that with λ = (0.3, 0.7), for no x ∈ SU, f (x) is
above the compromise half line {y | y = y∗− t1
λ, t ≥0}, where 1
Deriving upper bounds for biobjective multidimensional
knapsack problems
In the table on the previous slide we see that with λ = (0.3, 0.7) the upper bound for f2(xP
opt(λ))is tight but for
f2(xoptP (λ))no upper bound is available.
The reason is that with λ = (0.3, 0.7), for no x ∈ SU, f (x) is
above the compromise half line {y | y = y∗− t1
λ, t ≥0}, where 1
Deriving upper bounds for biobjective multidimensional
knapsack problems
Indeed, for λ = (0.3, 0.7) the compromise half line intersects the line f1(x ) = f1(xinc) (=107766) at f2(x ) =260595.1429,
whereas the maximal value for f2(x )over the feasible set is 129837 (we know that from solving problem with
λ1 =0, λ2=1, which amounts to optimization of f2(x )).
In other words, for λ = (0.3, 0.7) the compromise half line misses set f (X0). With y∗ set so "far away"form f (X0)(to get
yi∗ we just summed the coe cient of the corresponding
objective function) the set of λ with which the compromise half line intersects f (X0)is signi cantly reduced.
Deriving upper bounds for biobjective multidimensional
knapsack problems
Indeed, for λ = (0.3, 0.7) the compromise half line intersects the line f1(x ) = f1(xinc) (=107766) at f2(x ) =260595.1429,
whereas the maximal value for f2(x )over the feasible set is 129837 (we know that from solving problem with
λ1 =0, λ2=1, which amounts to optimization of f2(x )). In other words, for λ = (0.3, 0.7) the compromise half line misses set f (X0). With y∗ set so "far away"form f (X0)(to get
yi∗ we just summed the coe cient of the corresponding
objective function) the set of λ with which the compromise half line intersects f (X0)is signi cantly reduced.
Deriving upper bounds for biobjective multidimensional
knapsack problems
With f (xinc)and the corresponding S
U derived, we
immediately have upper bounds on f (xP
opt(˜λ)), where ˜λ is such
that the half line hits f (˜x), namely ˜ λi = 1 yi∗− fi(˜x ). For f (˜x) = (107766, 129837) we have ˜ λ1 = 1000000 375217 − 107766=3.739002658, ˜ λ2 = 1000000 375217 − 129837 =4.075311761 (the scaling factor, here 1000000, can be arbitrary).
We repeated these calculations for f (˜x) = (128730, 108468). The resulting upper bounds are shown on the next slide.
Deriving upper bounds for biobjective multidimensional
knapsack problems
With f (xinc)and the corresponding S
U derived, we
immediately have upper bounds on f (xP
opt(˜λ)), where ˜λ is such
that the half line hits f (˜x), namely ˜ λi = 1 yi∗− fi(˜x ). For f (˜x) = (107766, 129837) we have ˜ λ1= 1000000 375217 − 107766=3.739002658, ˜ λ2 = 1000000 375217 − 129837 =4.075311761
We repeated these calculations for f (˜x) = (128730, 108468). The resulting upper bounds are shown on the next slide.
Deriving upper bounds for biobjective multidimensional
knapsack problems
With f (xinc)and the corresponding S
U derived, we
immediately have upper bounds on f (xP
opt(˜λ)), where ˜λ is such
that the half line hits f (˜x), namely ˜ λi = 1 yi∗− fi(˜x ). For f (˜x) = (107766, 129837) we have ˜ λ1= 1000000 375217 − 107766=3.739002658, ˜ λ2 = 1000000 375217 − 129837 =4.075311761 (the scaling factor, here 1000000, can be arbitrary).
Deriving upper bounds for biobjective multidimensional
knapsack problems
λ1 λ2 f1(xinc) f 2(xinc) |S U| U1(SU, λ) U2(SU, λ) 3.739 4.075 107766 129837 7 107768 129848 4.057 3.749 128730 108468 3 128739 108475Calculations of upper bounds for f (xPopt(λinc))in the problem mknapcb_6_1_bicrit_LP.txt.
Deriving upper bounds for biobjective multidimensional
knapsack problems
Upper bounds on objective functions of Pareto optimal solutions allow to take conscious decisions whether to stop the optimization
process at a satisfactory Pareto gap.
ThePareto gapis de ned as
Ul(SU, λ) − fl(xinc)
fl(xinc)
, l =1, k, (it is a vector measure).
Deriving upper bounds for biobjective multidimensional
knapsack problems
λ1 λ2 time to xinc Pareto gap at xinc solution time
(sec.) (ticks) (%) (%) (sec.) (ticks) l =1 l =2
0.5 0.5 16.65 5663.02 0.032 0.002 299.73 63924.96 Time to the last xinc, Pareto gap at xinc and the solution time. In this case, time spend on improving the last incumbent or proving its optimality, whatever is the case, is one order of magnitude higher than time spend to derive it. On the other
References
Chu PC, Beasley JE. A genetic algorithm for the multidimensional knapsack problem. Journal of Heuristics 1998; 4: 63-86.
Kaliszewski I. Soft computing for complex multiple criteria decision making. Springer; 2006.
Kaliszewski I., Miroforidis J., Podkopaev D. Interactive Multiple Criteria Decision Making based on preference driven Evolutionary Multiobjective Optimization with controllable accuracy - the case of ρe ciency. Systems Research Institute Report, RB/1/2011. Kaliszewski I., Miroforidis J., Podkopaev D. Interactive Multiple Criteria Decision Making based on preference driven Evolutionary Multiobjective Optimization with controllable accuracy. European Journal of Operational Research 2012; 216: 188-199.
Kaliszewski I., Miroforidis J., Podkopaev D. Multiple criteria decision making by multiobjective optimization - a toolbox. Springer; 2016. Miettinen KM, Nonlinear Multiobjective Optimization. Kluwer
