emukit.bayesian_optimization.acquisitions package

Submodules

class emukit.bayesian_optimization.acquisitions.entropy_search.EntropySearch(model, space, sampler=None, num_samples=100, num_representer_points=50, proposal_function=None, burn_in_steps=50)

Bases: Acquisition

update_parameters()

Updates p_min parameter

Return type:

None

update_pmin()

Return type:

ndarray

Approximates the distribution of the global optimum p(x=x_star|D) by doing the following steps:

• discretizing the input space by representer points sampled from a proposal measure (default EI)

• predicting mean and the covariance matrix of these representer points

• uses EPMGP algorithm to compute the probability of each representer point being the minimum

evaluate(x)

Computes the information gain, i.e the change in entropy of p_min if we would evaluate x.

Parameters:

x (ndarray) – points where the acquisition is evaluated.

Return type:

ndarray

property has_gradients: bool

Returns that this acquisition has gradients

class emukit.bayesian_optimization.acquisitions.entropy_search.MultiInformationSourceEntropySearch(model, space, target_information_source_index=None, num_samples=100, num_representer_points=50, burn_in_steps=50)

Bases: EntropySearch

Entropy search acquisition for multi-information source problems where the objective function is the output of one of the information sources. The other information sources provide auxiliary information about the objective function

class emukit.bayesian_optimization.acquisitions.expected_improvement.ExpectedImprovement(model, jitter=0.0)

Bases: Acquisition

evaluate(x)

Computes the Expected Improvement.

Parameters:

x (ndarray) – points where the acquisition is evaluated.

Return type:

ndarray

evaluate_with_gradients(x)

Computes the Expected Improvement and its derivative.

Parameters:

x (ndarray) – locations where the evaluation with gradients is done.

Return type:

Tuple

property has_gradients: bool

Returns that this acquisition has gradients

class emukit.bayesian_optimization.acquisitions.expected_improvement.MeanPluginExpectedImprovement(model, jitter=0.0)

Bases: ExpectedImprovement

emukit.bayesian_optimization.acquisitions.expected_improvement.get_standard_normal_pdf_cdf(x, mean, standard_deviation)

Returns pdf and cdf of standard normal evaluated at (x - mean)/sigma

Parameters:

• x (array) – Non-standardized input

• mean (array) – Mean to normalize x with

• standard_deviation (array) – Standard deviation to normalize x with

Return type:

Tuple[array, array, array]

Returns:

(normalized version of x, pdf of standard normal, cdf of standard normal)

class emukit.bayesian_optimization.acquisitions.expected_improvement.MultipointExpectedImprovement(model, jitter=0.0, fast_compute=False, eps=0.001)

Bases: ExpectedImprovement

evaluate(x)

Computes the multipoint Expected Improvement.

Parameters:

x (ndarray) – points where the acquisition is evaluated.

Return type:

ndarray

Returns:

multipoint Expected Improvement at the input.

evaluate_with_gradients(x)

Computes the multipoint Expected Improvement and its derivative.

Parameters:

x (ndarray) – locations where the evaluation with gradients is done.

Return type:

Tuple[ndarray, ndarray]

Returns:

multipoint Expected Improvement and its gradient at the input.

emukit.bayesian_optimization.acquisitions.expected_improvement.get_covariance_given_smallest(Sigma, k)

Compute covariance of Y - Y_min given that Y_k is smaller than any other Y_i, i not k

Parameters:

• Sigma (ndarray) – Covariance of the (not conditioned) multivariate gaussian.

• k (int) – Index of random variable.

Return type:

ndarray

Returns:

Covariance of Y - Y_min given that Y_k is smaller than any other Y_i

emukit.bayesian_optimization.acquisitions.expected_improvement.get_covariance_given_value_of_i(Sigma, i)

Covariances of variables k knowing the value of variable i.

Parameters:

• Sigma (ndarray) – Covariance of the multivariate Gaussian.

• i (int) – Known variable.

Return type:

ndarray

Returns:

Covariance of Y given that Y_i is known

emukit.bayesian_optimization.acquisitions.expected_improvement.get_correlations_given_value_of_i(b, mu, Sigma, i)

Partial correlations of variables k knowing the value of variable i.

Parameters:

• b (ndarray) – Known values.

• mu (ndarray) – Mean of the multivariate Gaussian knowing i.

• Sigma (ndarray) – Covariance of the multivariate gaussian knowing i.

• i (int) – known variable.

Return type:

ndarray

Returns:

Partial correlations between all variables knowing i.

emukit.bayesian_optimization.acquisitions.expected_improvement.decompose_mvn(x, mu, Sigma, k)

Decompose Multivariate normal to probability of some random variables being smaller than the rest and multiply this with the marginal probability distribution of those random variables.

Parameters:

• x (ndarray) – Location of decomposition.

• mu (ndarray) – Mean of the multivariate Gaussian.

• Sigma (ndarray) – Covariance of the multivariate gaussian.

• k (list) – Indices of decomposition.

Return type:

ndarray

Returns:

Weighted probabilities of each variable being smaller than the rest.

emukit.bayesian_optimization.acquisitions.expected_improvement.Phi_gradient(x, mu, Sigma)

Compute gradient of CDF of multivariate Gaussian distribution.

Parameters:

• x (ndarray) – Location where the gradient is evaluated.

• mu (ndarray) – Mean of the multivariate Gaussian.

• Sigma (ndarray) – Covariance of the multivariate Gaussian.

Return type:

ndarray

Returns:

Gradient of the CDF of multivariate Gaussian.

emukit.bayesian_optimization.acquisitions.expected_improvement.Phi_hessian(x, mu, Sigma, gradient=None)

Compute hessian matrix of CDF of multivariate Gaussian distribution.

Parameters:

• x (ndarray) – Location where the Hessian is evaluated.

• mu (ndarray) – Mean of the multivariate Gaussian.

• Sigma (ndarray) – Covariance of the multivariate Gaussian.

• gradient (Optional[ndarray]) – Gradient of the multivariate Gaussian at x.

Return type:

ndarray

Returns:

Hessian of the CDF of multivariate Gaussian.

class emukit.bayesian_optimization.acquisitions.local_penalization.LocalPenalization(model)

Bases: Acquisition

Penalization function used in the local penalization method. It is the log of the following function:

\[h(x) = \sum_i{\log\left(\Phi\left(\frac{\|x - x_i\|_{2} - r_i}{s_i}\right)\right)}\]

where \(x_i\) are the points already in the batch

property has_gradients: bool

Abstract property. Whether acquisition value has analytical gradient calculation available.

Returns:

True if gradients are available

update_batches(x_batch, lipschitz_constant, f_min)

Updates the batches internally and pre-computes the parameters of the penalization function

evaluate(x)

Evaluates the penalization function value

Return type:

ndarray

evaluate_with_gradients(x)

Evaluates the penalization function value and gradients with respect to x

Return type:

Tuple[ndarray, ndarray]

class emukit.bayesian_optimization.acquisitions.log_acquisition.LogAcquisition(acquisition)

Bases: Acquisition

Takes the log of an acquisition function.

evaluate(x)

Parameters:

x (ndarray) – Input location

Return type:

ndarray

Returns:

log of original acquisition function at input location(s)

evaluate_with_gradients(x)

Return log of original acquisition with gradient

Parameters:

x (ndarray) – Input location

Return type:

Tuple[ndarray, ndarray]

Returns:

Tuple of (log value, gradient of log value)

property has_gradients: bool

Abstract property. Whether acquisition value has analytical gradient calculation available.

Returns:

True if gradients are available

class emukit.bayesian_optimization.acquisitions.max_value_entropy_search.MaxValueEntropySearch(model, space, num_samples=10, grid_size=5000)

Bases: Acquisition

update_parameters()

MES requires acces to a sample of possible minimum values y* of the objective function. To build this sample we approximate the empirical c.d.f of Pr(y*<y) with a Gumbel(a,b) distribution. This Gumbel distribution can then be easily sampled to yield approximate samples of y*

This needs to be called once at the start of each BO step.

evaluate(x)

Computes the information gain, i.e the predicted change in entropy of p_min (the distribution of the minimal value of the objective function) if we evaluate x. :type x: ndarray :param x: points where the acquisition is evaluated.

Return type:

ndarray

property has_gradients: bool

Returns that this acquisition has gradients

class emukit.bayesian_optimization.acquisitions.max_value_entropy_search.MUMBO(model, space, target_information_source_index=None, num_samples=10, grid_size=5000)

Bases: MaxValueEntropySearch

update_parameters()

MUMBO requires acces to a sample of possible minimum values y* of the objective function. To build this sample we approximate the empirical c.d.f of Pr(y*<y) with a Gumbel(a,b) distribution. This Gumbel distribution can then be easily sampled to yield approximate samples of y*

This needs to be called once at the start of each BO step.

evaluate(x)

Computes the information gain, i.e the change in entropy of p_min (the distribution of the minimal value of the objective function) if we would evaluate x. :type x: ndarray :param x: points where the acquisition is evaluated.

Return type:

ndarray

class emukit.bayesian_optimization.acquisitions.negative_lower_confidence_bound.NegativeLowerConfidenceBound(model, beta=1.0)

Bases: Acquisition

evaluate(x)

Computes the negative lower confidence bound

Parameters:

x (ndarray) – points where the acquisition is evaluated.

Return type:

ndarray

evaluate_with_gradients(x)

Computes the negative lower confidence bound and its derivative

Parameters:

x (ndarray) – points where the acquisition is evaluated.

Return type:

Tuple

property has_gradients

Abstract property. Whether acquisition value has analytical gradient calculation available.

Returns:

True if gradients are available

class emukit.bayesian_optimization.acquisitions.probability_of_feasibility.ProbabilityOfFeasibility(model, jitter=0.0)

Bases: Acquisition

evaluate(x)

Computes the probability of of satisfying the constraint C<0. :type x: ndarray :param x: points where the acquisition is evaluated, shape (number of points, number of dimensions). :rtype: ndarray :return: numpy array with the probability of satisfying the constraint at the points x.

evaluate_with_gradients(x)

Computes the probability of of satisfying the constraint C<0. :type x: ndarray :param x: points where the acquisition is evaluated, shape (number of points, number of dimensions). :rtype: Tuple :return: tuple of numpy arrays with the probability of satisfying the constraint at the points x and its gradient.

property has_gradients

Abstract property. Whether acquisition value has analytical gradient calculation available.

Returns:

True if gradients are available

class emukit.bayesian_optimization.acquisitions.probability_of_improvement.ProbabilityOfImprovement(model, jitter=0.0)

Bases: Acquisition

evaluate(x)

Computes the probability of improving over the current best :type x: ndarray :param x: points where the acquisition is evaluated, shape (number of points, number of dimensions).

Return type:

ndarray

evaluate_with_gradients(x)

Computes the probability of improving over the current best and its derivative :type x: ndarray :param x: points where the acquisition is evaluated, shape (number of points, number of dimensions).

Return type:

Tuple

property has_gradients

Abstract property. Whether acquisition value has analytical gradient calculation available.

Returns:

True if gradients are available

Module contents