Download Multi-Objective and Constrained Bayesian Optimization: Extending GP-UCB Algorithm and more Summaries Machine Learning in PDF only on Docsity!
Mean-Variance Analysis
in Bayesian Optimization under Uncertainty
Shogo Iwazaki Yu Inatsu Ichiro Takeuchi Nagoya Institute of Technology Nagoya Institute of Technology Nagoya Institute of Technology RIKEN [email protected]
Abstract
We consider active learning (AL) in an uncer- tain environment in which trade-off between multiple risk measures need to be considered. As an AL problem in such an uncertain en- vironment, we study Mean-Variance Analy- sis in Bayesian Optimization (MVA-BO) set- ting. Mean-variance analysis was developed in the field of financial engineering and has been used to make decisions that take into ac- count the trade-off between the average and variance of investment uncertainty. In this paper, we specifically focus on BO setting with an uncertain component and consider multi-task, multi-objective, and constrained optimization scenarios for the mean-variance trade-off of the uncertain component. When the target blackbox function is modeled by Gaussian Process (GP), we derive the bounds of the two risk measures and propose AL al- gorithm for each of the above three scenar- ios based on the risk measure bounds. We show the effectiveness of the proposed AL algorithms through theoretical analysis and numerical experiments.
1 Introduction
Decision making in an uncertain environment has been studied in various domains. For example, in financial engineering, the mean-variance analysis (Markowitz, 1952; Markowitz and Todd, 2000; Keeley and Furlong,
- has been introduced as a framework for making investment decisions, taking into account the trade-off
Proceedings of the 24th^ International Conference on Artifi- cial Intelligence and Statistics (AISTATS) 2021, San Diego, California, USA. PMLR: Volume 130. Copyright 2021 by the author(s).
between the return (mean) and the risk (variance) of the investment. In this paper we study active learning (AL) in an uncertain environment. In many practical AL problems, there are two types of parameters called design parameters and environmental parameters. For example, in a product design, while the design param- eters are fully controllable, the environmental param- eters vary depending on the environment in which the product is used. In this paper, we examine AL prob- lems under such an uncertain environment, where the goal is to efficiently find the optimal design parameters by properly taking into account the uncertainty of the environmental parameters. Concretely, let f (x, w) be a blackbox function indicat- ing the performance of a product, where x ∈ X is the set of controllable design parameters and w ∈ Ω is the set of uncontrollable environmental parameters whose uncertainty is characterized by a probability distribu- tion p(w). We particularly focus on the AL problem where the mean and the variance of the environmental parameters,
Ew[f (x, w)] =
Ω
f (x, w)p(w)dw, (1a)
Vw[f (x, w)] =
Ω
(f (x, w) − Ew[f (x, w)])^2 p(w)dw, (1b)
respectively, are taken into account. Specifically, we work on these two measures in three different scenar- ios: multi-task learning scenario, multi-objective op- timization scenario, and constrained optimization sce- nario. In the first scenario, we study one of multi- task formulations in which a weighted sum of these two measures is maximized 1. In the second scenario, we discuss how to obtain the Pareto frontier of these two measures in an AL setting. In the third scenario, we consider optimizing one of the two measures under some constraint on the other measure. We refer to (^1) This formulation is also referred as a scalarization ob- jective in multi-objective optimization.
Mean-Variance Analysis in Bayesian Optimization under Uncertainty
these problems and the proposed framework for solv- ing them as Mean-Variance Analysis in Bayesian Op- timization (MVA-BO). Figure 1 shows an illustration of a multi-task learning scenario.
In this study, we employ a Gaussian process (GP) to model the uncertainty of the blackbox function f (x, w). In a conventional GP-based AL problem (without uncontrollable environmental parameters w), the acquisition function (AF) is designed based on how the uncertainty of the blackbox function changes when an input point is selected and the blackbox function is evaluated. On the other hand, in MVA-BO, we need to know how the uncertainties of the mean function (1a) and the variance function (1b) change by evalu- ating the blackbox function at the selected input point. Note that we face the difficulty of not being able to di- rectly evaluate the target functions (1a) and (1b). It has been shown in a previous study (O’Hagan, 1991) that, when f (x, w) follows a GP, the mean function (1a) also follows a GP. Unfortunately, however, the variance function (1b) does not follow a GP, indicat- ing that we need to develop a new method to quantify how the uncertainty of the variance function changes by evaluating the blackbox function at the selected input point. In this study, we extend the GP-UCB algorithm (Srinivas et al., 2010) to realize MVA-BO in the above mentioned three scenarios by overcoming these technical difficulties. We demonstrate the effec- tiveness of the proposed MVA-BO framework through theoretical analyses and numerical experiments.
Related Work Various problem setups and meth- ods have been studied for AL and Bayesian optimiza- tion (BO) problems when there are multiple target functions. One of such problem setup is multi-task BO (Swersky et al., 2013). In this problem setup, the AF is designed to select input points that commonly contribute to optimizing multiple target functions. Another popular problem setup is multi-objective BO (Emmerich, 2005; Zuluaga et al., 2016; Suzuki et al., 2020). The goal of a multi-objective optimiza- tion is to obtain so-called Pareto-optimal solutions. The AF in this problem setup is designed to efficiently identify solutions on the Pareto frontier. Another com- mon problem setup is constrained BO (Gardner et al., 2014; Gelbart et al., 2014; Hern´andez-Lobato et al., 2016; Takeno et al., 2021). The goal of this problem setup is to find the optimal solution to a constrained optimization problem in a situation where both the objective function and constraint function are black- box functions that are costly to evaluate. The AF in this problem setup is designed to select input points that are useful not only for maximizing the objective function but also for identifying the feasible region. In this paper, we study these three scenarios as concrete
examples of MVA-BO. Unlike conventional multi-task, multi-objective and constrained BOs, the main tech- nical challenges of MVA-BO are that the two target functions (1a) and (1b) cannot be directly evaluated and that the latter does not follow a GP. Various studies have been published on BO under various types of uncertainty. The most relevant one to our study is on Bayesian quadrature optimization (BQO) (Toscano-Palmerin and Frazier, 2018), the goal of which is to optimize the mean function (1a). When the blackbox function follows a GP, the mean func- tion (1a) also follows a GP, suggesting that one can efficiently solve BQO problems by properly modifying the AFs in conventional BO. By replacing the inte- grand in (1a) with different measures, one can consider various types of AL problems under uncertainty (Be- land and Nair, 2017; Iwazaki et al., 2020a,b; Cakmak et al., 2020). Another line of research dealing with uncontrollable and uncertain factors in BO is known as robust BO (Bogunovic et al., 2018; Nguyen et al., 2020; Kirschner et al., 2020; Bogunovic et al., 2020; Inatsu et al., 2021). The goal of robust BO is to make robust decisions that appropriately take into account various types of uncertainty. For example, input un- certainty in BO has been studied, in which probabilis- tic noise is inevitably added to the input points when evaluating the target blackbox function. Although re- search on BO in an uncertain environment has steadily progressed over the past few years, to our knowledge, there are no AL nor BO studies that take into ac- count the trade-offs between multiple measures such as mean-variance analysis. Decision making under uncertainty is being examined in the field of robust optimization (Ben-Tal et al., 2009; Beyer and Sendhoff, 2007; Ben-Tal and Nemirovski, 2002), with especially applications to financial engi- neering in mind (Schied, 2006; Alexander and Bap- tista, 2002; Fabozzi et al., 2007). It has been pointed out that when making decisions under uncertainty, it is important to balance multiple measures appropri- ately, as represented by the Nobel prize-winning mean- variance analysis in portfolio theory (Markowitz, 1952; Markowitz and Todd, 2000; Keeley and Furlong, 1990). Various risk measures, such as Value at Risk (VaR), have been proposed in financial engineering, and these multiple risk measures are used in combination, de- pending on the purpose of the decision making. How- ever, to our knowledge, there have not been AL or BO studies that have appropriately taken into account multiple measures.
Mean-Variance Analysis in Bayesian Optimization under Uncertainty
Multi-task (MT)-Scenario First, we formulate the problem as a single-objective optimization problem whose objective function is defined as a weighted sum of F 1 and F 2. Given a user-specified weight α ∈ [0, 1], let G be a new objective function defined as
G(x) = αF 1 (x) + (1 − α)F 2 (x).
Here, the goal is to find x∗^ := argmaxx∈X G(x) effi- ciently. For analyzing the theoretical properties, we introduce the notion of an -accurate solution. Let ˆxt be an estimated solution obtained by the algorithm at step t. Given a fixed constant ≥ 0, we say that ˆxt is -accurate if the following inequality holds:
G( ˆxt) ≥ G(x∗) − .
In §4, for an arbitrarily small , we show that our algorithm can find an -accurate solution with high probability after finite step T.
Multi-objective (MO)-Scenario We also con- sider another formulation based on the Pareto op- timality criterion. Hereafter, we use the vector representation of the objective functions: F (x) = (F 1 (x), F 2 (x)). First, let be a relational operator defined over X × X or R^2 × R^2. Given x, x′^ ∈ X , we write x x′^ or F (x) F (x′) provided that F 1 (x) ≤ F 1 (x′) and F 2 (x) ≤ F 2 (x′) hold simultane- ously. An operator ≺ is similarly defined.
The goal of this scenario is to identify the following Pareto set Π efficiently:
Π = {x ∈ X | ∀x′^ ∈ Ex, F (x) F (x′)},
where Ex = {x′^ ∈ X | F (x) 6 = F (x′)}. Moreover, Pareto front Z is defined by
Z = ∂{y ∈ R^2 | ∃x ∈ X , y F (x)},
where ∂A denote the boundary of the set A.
Next, we introduce the notion of an -accurate Pareto set (Zuluaga et al., 2016), which is an idea similar to the -accurate solution in the multi-task scenario. Given a non-negative vector = ( 1 , 2 ), we define the relational operator , which is the relaxed version of . For x, x′^ ∈ X , we write x x′^ or F (x) F (x′) if F 1 (x) ≤ F 1 (x′) + 1 and F 2 (x) ≤ F 2 (x′) + 2 hold simultaneously. Then, the -Pareto front is defined as:
Z = {y ∈ R^2 | ∃y′^ ∈ Z, y y′^ and ∃y′′^ ∈ Z, y′′^ y}.
We say that the estimated Pareto set Πˆt of the algo- rithm is an -accurate Pareto set if the following two conditions are satisfied:
- F ( Πˆt) ⊂ Z, where F ( Πˆt) :=
F (x) | x ∈ Πˆt
- For any x ∈ Π, there is at least one point x′^ ∈ Πˆt such that x x′.
Intuitively, these conditions indicate that the differ- ence of true Pareto front Z and estimated Pareto front, which is constructed by Πˆt, is at most . We refer to Zuluaga et al. (2016) for more detail explanations of -Pareto front.
Constrained (Co)-Scenario As an example of constrained optimization scenario, we consider the fol- lowing problem:
x∗^ = arg max x∈X
F 1 (x) s.t. F 2 (x) ≥ h,
where h > 0 is a user-specified known threshold pa- rameter. Moreover, to provide theoretical guarantees, we define an -accurate solution to be a solution ˆx which satisfies F 1 ( ˆx) ≥ F 1 (x∗)− 1 and F 2 ( ˆx) ≥ h− 2 for a non-negative vector = ( 1 , 2 ). We emphasize that, although there are many existing studies on multi-task, multi-objective, and constrained BO, these existing methods cannot be directly applied to our problem setups because the objective functions F 1 and F 2 are not observed directly.
3 Proposed Method
First, we explain the basic idea of our proposed algo- rithms. To handle F 1 and F 2 efficiently, one simple way is to consider the predicted distributions of F 1 and F 2 , and apply existing methods. However, it is difficult to handle the predicted distribution of F 2 al- though f is modeled by a GP. In this paper, we first derive the intervals in which F 1 and F 2 exist with high probability from the confidence bound of f , and con- struct the algorithm based on these derived intervals. Hereafter, with a slight abuse of notation, we refer to these derived intervals as the confidence bounds of F 1 and F 2.
3.1 Confidence Bounds of Target Functions
The following Lemma 3.1 plays a central role in our proposed methods.
Lemma 3.1. Let βt =
2(γt− 1 + ln 1/δ) + B
and let
˜l( tsq )(x, w)
0 , ˜lt(x, w) ≤ 0 ≤ u˜t(x, w) min
˜l t^2 (x, w), ˜u^2 t (x, w)
otherwise
˜u( t sq)(x, w) = max
˜l^2 t (x,^ w),^ u˜
2 t (x,^ w)
Shogo Iwazaki, Yu Inatsu, Ichiro Takeuchi
where ˜lt(x, w) = lt(x, w) − Ew[ut(x, w)] and u ˜t(x, w) = ut(x, w) − Ew[lt(x, w)]. Then, with prob- ability at least 1 − δ, for any x ∈ X , t ≥ 1 ,
F 1 (x) ∈ Q( tF 1 )(x) := [l( t F^1 )(x), u( tF 1 )(x)], F 2 (x) ∈ Q( tF 2 )(x) := [l( t F^2 )(x), u( tF 2 )(x)]
where
l( tF 1 )(x) =
Ω
lt(x, w)p(w)dw,
u( tF 1 )(x) =
Ω
ut(x, w)p(w)dw,
l( tF 2 )(x) = −
Ω
u ˜( tsq )(x, w)p(w)dw,
u( tF 2 )(x) = −
Ω
˜l t( sq)(x, w)p(w)dw.
Lemma 3.1 is derived by considering the intervals where target functions F 1 and F 2 exist when the state- ment (which occurs with probability 1 − δ) in Lemma 2.1 holds. The details of proofs are in Appendix A.
3.2 Algorithms
Multi-task (MT)-Scenario In the multi-task sce- nario, our algorithm chooses the next input point xt based on the upper confidence bound (UCB) of the function G in which the lower and the upper bounds are given by
l (G) t (x) =^ αl
(F 1 ) t (x) + (1^ −^ α)l
(F 2 ) t (x), u (G) t (x) =^ αu
(F 1 ) t (x) + (1^ −^ α)u
(F 2 ) t (x).
At every step t, the next input point xt of our algo-
rithm is defined by xt = argmaxx∈X u( tG )(x). For the- oretical discussion, we define the estimated solution at step t as ˆxt = xˆt with ˆt = argmaxt′∈{ 1 ,...,t}lt′^ (xt′^ ). This pessimistic estimated solution is often employed in the other GP-based optimization literatures (e.g., Bogunovic et al., 2018; Kirschner et al., 2020). Here- after, we call this strategy Multi-Task (MT)-MVA-BO and the pseudo-code is presented in algorithm 1.
Multi-objective (MO)-Scenario From the confi-
dence bounds of F 1 and F 2 , we define F (^) t(opt) and F (^) t(pes)
by F (^) t(opt) (x) =
u( tF 1 )(x), u( tF 2 )(x)
and F (^) t(pes) (x) = ( l( tF 1 )(x), l t(F 2 )(x)
, which respectively represent the
optimistic and pessimistic predictions of the objective functions at step t. First, based on pessimistic predic- tions, we define the estimated Pareto set Πˆt at step t by
Πˆt =
x ∈ X
∣ ∀x′^ ∈ E (pes) t,x ,^ F^
(pes) t (x)^ ^ F^
(pes) t (x
Algorithm 1 Multi-task MVA-BO (MT-MVA-BO) Input: GP prior GP(0, k), {βt}t≤T , α ∈ (0, 1). for t = 0 to T do Compute u (G) t (x) for any^ x^ ∈ X^. Choose xt = argmaxx∈X u( tG )(x). Sample wt ∼ p(w). Observe yt ← f (xt, wt) + ηt. Update the GP by adding ((xt, wt), yt). end for x ˆT = xˆt where ˆt = argmaxt′∈{ 1 ,...,T }l( tG′ )(xt′ ). Output: xˆT.
where E t,(pes)x =
x′^ ∈ X
∣ F^
(pes) t (x)^6 =^ F^
(pes) t (x
Furthermore, using Πˆt, the potential Pareto set Mt is defined by
Mt =
x ∈ X \ Πˆt
∣ ∀x′^ ∈ Πˆt, F (opt) t (x)^ ^ F^
(pes) t (x
Here, Mt is the set which excludes the points that are -dominated by other points with high probabil- ity. At every step t, our algorithm chooses xt based on the uncertainty defined by the confidence bounds of F 1 and F 2. In this paper, we adopt the diameter λt(x) of rectangle Rectt(x) =
[
l (F 1 ) t (x), u
(F 1 ) t (x)
]
×
[
l( tF 2 )(x), u( tF 2 )(x)
]
as the uncertainty of x:
λt(x) = max y,y′∈Rectt(x)
‖y − y′‖ 2. (4)
Namely, the next input point xt is defined by xt = argmaxx∈Mt∪ Πˆt λt(x) at every step t. Our proposed algorithm terminates when the esti- mated Pareto set Πˆt is guaranteed to be an -Pareto set with high probability. To this end, our algorithm checks the uncertainty set Ut defined by
Ut =
x ∈ Πˆt |∃x′^ ∈ Πˆt \ {x},
F (^) t(pes) (x) + ≺ F (^) t(opt) (x′)
Here, Ut is the set of points where it is not possible to decide whether it is an -Pareto solution based on the current confidence bounds. Our algorithm terminates at step t where both Mt = ∅ and Ut = ∅ hold. Here- after, we call this algorithm Multi-Objective (MO)- MVA-BO.
Constrained (Co)-Scenario Let
M
(cons) t =
x ∈ X | u (F 2 ) t (x)^ ≥^ h^ −^ ^2
M
(obj) t =
x ∈ X | u (F 1 ) t (x)^ ≥^ xmax′∈St^ l
(F 1 ) t (x
1
Shogo Iwazaki, Yu Inatsu, Ichiro Takeuchi
〈f, k((x, w), ·)〉Hk ≤ B for any (x, w) ∈ X × Ω by ap- plying the Schwarz’s inequality. Therefore, we can find that the parameter B˜ is roughly bounded as B˜ ≤ 2 B.
Furthermore, we can obtain more explicit form of The- orem 4.1 by substituting bounds on γT. For example, γT = O((log T )d^1 +d^2 +1) in Gaussian kernel. In this case, RT becomes sub-linear (i.e. limT →∞ RT /T = 0). Hence, for arbitrarily small > 0, Theorem 4.1 guar- antees that MT-MVA-BO returns an -accurate solu- tion within finite steps with high probability.
Finally, we present the similar convergence results for MO-MVA-BO and Co-MVA-BO in Theorem 4.2 and 4.3, respectively.
Theorem 4.2. Fix positive definite kernel k, and assume f ∈ Hk with ‖f ‖Hk ≤ B. Let δ ∈ (0, 1) and > 0 , and set βt according to βt = (√ 2(γt− 1 + ln(3/δ)) + B
at every step t. When ap-
plying MO-MVA-BO under the above conditions, the following 1. and 2. hold with probability at least 1 − δ:
- The algorithm terminates at most step T where T is the smallest positive integer that satisfies the following inequality: √ 4 T Bβ˜ 1 / 2 T
2 T C 1 γT + C 2
2 T C 1 γT + C 2
≤ T min{ 1 , 2 }.
- When the algorithm terminates, estimated Pareto set Πˆt is an -accurate Pareto set.
Theorem 4.3. Let k be a positive-definite kernel, and let f ∈ Hk with ‖f ‖Hk ≤ B. Also let δ ∈ (0, 1) and 1 > 0 , 2 > 0 , and define βt = (√ 2(γt− 1 + ln(3/δ)) + B
. Then, with probability at
least 1 − δ, the following 1. and 2. hold:
- Co-MVA-BO algorithm in §3 terminates after at most T iterations, where T is the smallest positive integer satisfying √ 2 T Bβ˜ 1 / 2 T
8 T C 1 γT + 2C 2
2 T C 1 γT + C 2
≤ T min{ 1 , 2 }.
- If x∗^ exists, then St′^6 = ∅ at the termination step t′^ ≤ T. Moreover, xˆt′ = argmaxx∈St′ l t( F^1 )(x) is an -accurate solution.
5 Numerical Experiments
We conducted intensive numerical experiments on a variety of artificial and real datasets. Due to the space
limitation, we only show a part of the results. See Ap- pendix E for more experimental results and detailed experimental setups. As the common baselines in the three scenarios, we adopted random sampling (RS) and uncertainty sampling (US). RS chooses xt from X uni- formly at random, and US chooses xt such that it achieves the largest average posterior variance. In the multi-task scenario, we computed the regret, G(x∗) − G( ˆxt), at every step t, where xt is the esti- mated solution defined by the algorithms. We defined xˆt as ˆxt = argmaxt′=1,...,tl( t G)(xt′^ ) 5 in RS, US, and proposed method (MT-MVA-BO). To show the effect of difference of objective functions, we also considered two methods BQOUCB and BO-VO. The former is the method designed to maximize F 1 , and this method is also considered for comparison in existing BO stud- ies under uncertainty (Nguyen et al., 2020; Kirschner et al., 2020). The latter is the variant of our method corresponding to the case of α = 0. These methods choose xt as the maximizing point of u( tF 1 )(x) and u( tF 2 )(x) respectively. In addition, estimated solution xˆt is defined by xˆt = argmaxt′=1,...,tl (F 1 ) t (xt′^ ) and xˆt = argmaxt′=1,...,tl( tF 2 )(xt′^ ), respectively. We also compared with the adaptive versions of these meth- ods ADA-BQOUCB and ADA-BO-VO, which choose xt in the same way as BQOUCB and BO-VO, but the estimated solutions are defined as ˆxt = argmaxt′=1,...,tl( tG )(xt′ ). We set α so that maximums of the mean function F 1 , the variance function F 2 and the target function G be- come different. The values of α that we used in each experiments are in appendix. In the multi-objective scenario, we adopted RS and US for comparison. We computed the gap of hyper- volume (Emmerich, 2005), HV − HVˆt to measure the performance, where HV and HVˆt denote the hyper- volumes computed based on the true Pareto set Π and the estimated Pareto set Πˆt, respectively. In the constrained optimization scenario, we adopted RS, US, and BQOUCB for comparison. We defined xˆt = argmaxx∈St l( tF 1 )(x) in RS and US, whereas xˆt = argmaxx∈X l( tF 1 )(x) in BQOUCB. We considered ADA-BQO-UCB, an adaptive version of BQOUCB, which chooses the next point in the same way as what BQOUCB does, but ˆxt is defined as ˆxt = argmaxx∈St l( tF 1 )(x). To measure performances, we used the utility gap measure which is commonly used as a performance measure in constrained BO (Hern´andez-Lobato et al., 2016). As a performance measure, at every step t, we report the
(^5) Note that this definition is slightly different from that of Theorem 4.1. As described by Bogunovic et al. (2018), this definition is more suitable than that of Theorem 4. when the kernel hyperparameters are updated online.
Mean-Variance Analysis in Bayesian Optimization under Uncertainty
utility gap defined as F 1 (x∗) − F 1 ( ˆxt) if F 2 ( ˆxt) ≥ h, whereas F 1 (x∗) − minx∈X F 1 (x) otherwise.
GP Test Functions First, we conducted experi- ments with true oracle functions f generated from 2D GP prior. We divided [− 1 , 1]^2 into 25 uniformly spaced grid points in each dimension and generated the sample path from the GP prior. Then, we cre- ated the GP model with these grid points and set the true oracle function as its GP posterior mean. Here, we created 50 sample paths from different seeds, and conducted 10 runs for each function and report the av- erage performance of a total of 500 experiments. To create a GP sample path, we used the Gaussian ker- nel k((x, w), (x′, w′)) = σ^2 ker exp((‖x − x′‖^22 + ‖w − w′‖^22 )/(2l^2 )) with σker = 1, l = 0.25 and assume that it is known in all the algorithms. The noise variance was set to be σ^2 = 10−^4. The domain spaces X and Ω are set to be 100 grid points evenly allocated in [− 1 , 1]. The density was set to be p(w) =
w∈Ω φ(w)/Z, Z =
w∈Ω φ(w) where^ φ^ is p.d.f.^ of standard nor- mal distribution.
Benchmark Functions We also conducted exper- iments with 6 benchmark functions commonly used in BO study. Due to the space limitation, we only show the results of 3D Rosenbrock function. First, we scaled the input domain to [− 1 , 1]^3 and divided it with 100 grid points in each dimension. Here, we set the first two dimensions as X and the remaining one dimension as Ω. Furthermore, we set p(w) in the same way as the experiments with GP test functions. We used ARD Gaussian kernel k((x, w), (x′, w′)) = σ^2 ker exp(
∑d 1 i=1(xi^ −^ x
′ i) (^2) / 2 l(x) ∑^ i^ + d 2 j=1(wj^ −w ′ j ) (^2) / 2 l(w) j ). Unlike the experiments with GP test functions, we assumed that the hyperparam- eters are unknown, and they are estimated by maxi- mizing the marginal likelihood at every 10 step in the algorithms. We set the noise variance as σ^2 = 10−^4 and report the average performance of 50 simulations with different seeds.
Real-data We applied the proposed methods to two real-world problems: Portfolio optimization problem and Newsvendor problem under dynamic consumer substitution (Mahajan and Van Ryzin, 2001). The goal of the former problem is to optimize hyperparamters of a trading strategy under uncertainty of market con- ditions, while the goal of the latter problem is to op- timize the initial inventory levels under uncertainty of customer behaviors. In the former problem, the con- trol parameter x corresponds to the risk and trade aversion parameters, and the holding cost multiplier, respectively. The environmental parameters w are the bid-ask spread and the borrow cost, which are assumed
to be uniformly distributed over certain ranges. In the latter problem, the control parameter x and w respec- tively correspond to the initial inventory level of prod- ucts and the uncertain purchasing behaviors of cus- tomers, which follow mutually independent Gamma distributions. These problems are also considered in existing BO studies under uncertainty environment (Toscano-Palmerin and Frazier, 2018; Cakmak et al., 2020). As in these previous studies (Toscano-Palmerin and Frazier, 2018; Cakmak et al., 2020), we conducted the experiments in the simulator-based setting de- scribed in section 3.3.2. The average performances of 30 simulations with different seeds are reported.
Results Figure 2 shows the results of a GP test func- tion, a benchmark data (Rosenbrock), and two real data (Portfolio and Newsvendor). In all the datasets and all the three scenarios, the proposed methods MVA-BO (in red color) showed better or at least com- parable performances than other methods. In the ex- periments of the multi-task and constraint optimiza- tion scenario, we also confirmed that the regrets of BQOUCB, BO-VO, ADA-BQOUCB, and ADA-BO-VO stop de- creasing at an early stage. Note that these are reason- able results because target functions of these methods are inconsistent with our settings.
6 Conclusion
We introduced mean-variance analysis within the con- text of Bayesian Optimization under uncertainty. We developed algorithms for multi-task, multi-objective and constrained optimization scenarios, analyzed their convergence properties, and demonstrated their effec- tiveness through numerical experiments.
Acknowledgement
This work was partially supported by MEXT KAK- ENHI (20H00601, 16H06538), JST CREST (JP- MJCR1502), and RIKEN Center for Advanced Intel- ligence Project.
Mean-Variance Analysis in Bayesian Optimization under Uncertainty
References
Yasin Abbasi-Yadkori. Online learning for linearly parametrized control problems. 2013.
Gordon J Alexander and Alexandre M Baptista. Economic implications of using a mean-var model for portfolio selection: A comparison with mean- variance analysis. Journal of Economic Dynamics and Control, 26(7-8):1159–1193, 2002.
Justin J Beland and Prasanth B Nair. Bayesian opti- mization under uncertainty. In NIPS BayesOpt 2017 workshop, 2017.
Aharon Ben-Tal and Arkadi Nemirovski. Robust optimization–methodology and applications. Math- ematical programming, 92(3):453–480, 2002.
Aharon Ben-Tal, Laurent El Ghaoui, and Arkadi Ne- mirovski. Robust optimization, volume 28. Princeton University Press, 2009.
Hans-Georg Beyer and Bernhard Sendhoff. Robust optimization–a comprehensive survey. Computer methods in applied mechanics and engineering, 196 (33-34):3190–3218, 2007.
Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, and Volkan Cevher. Adversarially robust optimiza- tion with Gaussian processes. In Advances in neural information processing systems, pages 5760–5770,
Ilija Bogunovic, Andreas Krause, and Jonathan Scar- lett. Corruption-tolerant Gaussian process bandit optimization. In The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020 , volume 108 of Proceedings of Machine Learn- ing Research, pages 1071–1081, 2020.
Sait Cakmak, Raul Astudillo, Peter Frazier, and Enlu Zhou. Bayesian optimization of risk measures. arXiv preprint arXiv:2007.05554, 2020.
Sayak Ray Chowdhury and Aditya Gopalan. On kernelized multi-armed bandits. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 844–853, International Convention Centre, Sydney, Australia, 06–11 Aug
- URL http://proceedings.mlr.press/v70/ chowdhury17a.html.
Michael Emmerich. Single-and multi-objective evo- lutionary design optimization assisted by Gaussian random field metamodels. Dissertation, LS11, FB Informatik, Universit¨at Dortmund, Germany, 2005.
Frank J Fabozzi, Petter N Kolm, Dessislava A Pachamanova, and Sergio M Focardi. Robust port- folio optimization. The Journal of portfolio manage- ment, 33(3):40–48, 2007.
Lukas Frhlich, Edgar Klenske, Julia Vinogradska, Christian Daniel, and Melanie Zeilinger. Noisy- input entropy search for efficient robust Bayesian optimization. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 2262–2272, Online, 26– Aug 2020. URL http://proceedings.mlr.press/ v108/frohlich20a.html. Jacob R Gardner, Matt J Kusner, Zhixiang Eddie Xu, Kilian Q Weinberger, and John P Cunningham. Bayesian optimization with inequality constraints. In ICML, volume 2014, pages 937–945, 2014. Michael A. Gelbart, Jasper Snoek, and Ryan P. Adams. Bayesian optimization with unknown con- straints. In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence, UAI’14, page 250259, Arlington, Virginia, USA, 2014. AUAI Press. ISBN 9780974903910. Jos´e Miguel Hern´andez-Lobato, Michael A Gelbart, Ryan P Adams, Matthew W Hoffman, and Zoubin Ghahramani. A general framework for con- strained Bayesian optimization using information- based search. The Journal of Machine Learning Re- search, 17(1):5549–5601, 2016. Yu Inatsu, Shogo Iwazaki, and Ichiro Takeuchi. Active learning for distributionally robust level-set estima- tion. arXiv preprint arXiv:2102.04000, 2021. Shogo Iwazaki, Yu Inatsu, and Ichiro Takeuchi. Bayesian experimental design for finding reliable level set under input uncertainty. IEEE Access, 8: 203982–203993, 2020a. doi: 10.1109/ACCESS.2020.
Shogo Iwazaki, Yu Inatsu, and Ichiro Takeuchi. Bayesian quadrature optimization for probability threshold robustness measure. arXiv preprint arXiv:2006.11986, 2020b. Michael C Keeley and Frederick T Furlong. A reex- amination of mean-variance analysis of bank capital regulation. Journal of Banking & Finance, 14(1): 69–84, 1990. Johannes Kirschner, Ilija Bogunovic, Stefanie Jegelka, and Andreas Krause. Distributionally robust Bayesian optimization. In The 23rd International Conference on Artificial Intelligence and Statis- tics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy], volume 108 of Proceed- ings of Machine Learning Research, pages 2174– 2184, 2020. URL http://proceedings.mlr. press/v108/kirschner20a.html. Siddharth Mahajan and Garrett Van Ryzin. Stocking retail assortments under dynamic consumer substi- tution. Operations Research, 49(3):334–351, 2001.
Shogo Iwazaki, Yu Inatsu, Ichiro Takeuchi
Harry Markowitz. Portfolio selection. Journal of Fi- nance, 7(1):77–91, 1952.
Harry M Markowitz and G Peter Todd. Mean-variance analysis in portfolio choice and capital markets, vol- ume 66. John Wiley & Sons, 2000.
Thanh Nguyen, Sunil Gupta, Huong Ha, Santu Rana, and Svetha Venkatesh. Distributionally robust Bayesian quadrature optimization. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 1921–1931, Online, 26–28 Aug 2020. URL http:// proceedings.mlr.press/v108/nguyen20a.html.
Anthony O’Hagan. Bayes–hermite quadrature. Jour- nal of statistical planning and inference, 29(3):245– 260, 1991.
Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learn- ing. MIT Press, 2006.
Alexander Schied. Risk measures and robust optimiza- tion problems. Stochastic Models, 22(4):753–831,
Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias W. Seeger. Gaussian process opti- mization in the bandit setting: No regret and exper- imental design. In Proceedings of the 27th Interna- tional Conference on Machine Learning (ICML-10), June 21-24, 2010, Haifa, Israel, pages 1015–1022,
- URL https://icml.cc/Conferences/2010/ papers/422.pdf.
Yanan Sui, Alkis Gotovos, Joel Burdick, and An- dreas Krause. Safe exploration for optimization with Gaussian processes. In International Conference on Machine Learning, pages 997–1005, 2015.
Shinya Suzuki, Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Multi- objective Bayesian optimization using Pareto- frontier entropy. In Proceedings of Machine Learning and Systems 2020, pages 10841–10850. 2020.
Kevin Swersky, Jasper Snoek, and Ryan P Adams. Multi-task Bayesian optimization. In Advances in neural information processing systems, pages 2004– 2012, 2013.
Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Sequential- and parallel- constrained max-value entropy search via informa- tion lower bound. arXiv preprint arXiv:2102.09788,
Saul Toscano-Palmerin and Peter I. Frazier. Bayesian optimization with expensive integrands. CoRR, abs/1803.08661, 2018. URL http://arxiv.org/ abs/1803.08661.
Marcela Zuluaga, Andreas Krause, and Markus P¨uschel. e-pal: An active learning approach to the multi-objective optimization problem. Jour- nal of Machine Learning Research, 17(104):1– 32, 2016. URL http://jmlr.org/papers/v17/ 15-047.html.
