emukit.model_wrappers package¶

Submodules¶

class emukit.model_wrappers.gpy_model_wrappers.GPyModelWrapper(gpy_model, n_restarts=1)

This is a thin wrapper around GPy models to allow users to plug GPy models into Emukit

predict(X)
Parameters

X (ndarray) – (n_points x n_dimensions) array containing locations at which to get predictions

Return type

Tuple[ndarray, ndarray]

Returns

(mean, variance) Arrays of size n_points x 1 of the predictive distribution at each input location

predict_noiseless(X)
Parameters

X (ndarray) – (n_points x n_dimensions) array containing locations at which to get predictions

Return type

Tuple[ndarray, ndarray]

Returns

(mean, variance) Arrays of size n_points x 1 of the predictive distribution at each input location

predict_with_full_covariance(X)
Parameters

X (ndarray) – (n_points x n_dimensions) array containing locations at which to get predictions

Return type

Tuple[ndarray, ndarray]

Returns

(mean, variance) Arrays of size n_points x 1 and n_points x n_points of the predictive mean and variance at each input location

Parameters

X (ndarray) – (n_points x n_dimensions) array containing locations at which to get gradient of the predictions

Return type

Tuple[ndarray, ndarray]

Returns

(mean gradient, variance gradient) n_points x n_dimensions arrays of the gradients of the predictive distribution at each input location

Computes and returns model gradients of mean and full covariance matrix at given points

Parameters

X (ndarray) – points to compute gradients at, nd array of shape (q, d)

Return type

Tuple[ndarray, ndarray]

Returns

Tuple with first item being gradient of the mean of shape (q) at X with respect to X (return shape is (q, q, d)). The second item is the gradient of the full covariance matrix of shape (q, q) at X with respect to X (return shape is (q, q, q, d)).

set_data(X, Y)

Sets training data in model

Parameters
• X (ndarray) – New training features

• Y (ndarray) – New training outputs

Return type

None

optimize(verbose=False)

Optimizes model hyper-parameters

calculate_variance_reduction(x_train_new, x_test)

Computes the variance reduction at x_test, if a new point at x_train_new is acquired

Return type

ndarray

predict_covariance(X, with_noise=True)

Calculates posterior covariance between points in X :type X: ndarray :param X: Array of size n_points x n_dimensions containing input locations to compute posterior covariance at :type with_noise: bool :param with_noise: Whether to include likelihood noise in the covariance matrix :rtype: ndarray :return: Posterior covariance matrix of size n_points x n_points

get_covariance_between_points(X1, X2)

Calculate posterior covariance between two sets of points. :type X1: ndarray :param X1: An array of shape n_points1 x n_dimensions. This is the first argument of the

posterior covariance function.

Parameters

X2 (ndarray) – An array of shape n_points2 x n_dimensions. This is the second argument of the posterior covariance function.

Return type

ndarray

Returns

An array of shape n_points1 x n_points2 of posterior covariances between X1 and X2. Namely, [i, j]-th entry of the returned array will represent the posterior covariance between i-th point in X1 and j-th point in X2.

property X: ndarray

An array of shape n_points x n_dimensions containing training inputs

Type

return

Return type

ndarray

property Y: ndarray

An array of shape n_points x 1 containing training outputs

Type

return

Return type

ndarray

generate_hyperparameters_samples(n_samples=20, n_burnin=100, subsample_interval=10, step_size=0.1, leapfrog_steps=20)

Generates the samples from the hyper-parameters and returns them. :param n_samples: Number of generated samples. :param n_burnin: Number of initial samples not used. :param subsample_interval: Interval of subsampling from HMC samples. :param step_size: Size of the gradient steps in the HMC sampler. :param leapfrog_steps: Number of gradient steps before each Metropolis Hasting step. :rtype: ndarray :return: A numpy array whose rows are samples of the hyper-parameters.

fix_model_hyperparameters(sample_hyperparameters)

Fix model hyperparameters

Return type

None

emukit.model_wrappers.gpy_model_wrappers.dSigma(x_predict, x_train, kern, w_inv)

Compute the derivative of the posterior covariance with respect to the prediction input

Parameters
• x_predict (ndarray) – Prediction inputs of shape (q, d)

• x_train (ndarray) – Training inputs of shape (n, d)

• kern (<module 'GPy.kern' from '/home/docs/checkouts/readthedocs.org/user_builds/emukit/envs/latest/lib/python3.6/site-packages/GPy/kern/__init__.py'>) – Covariance of the GP model

• w_inv (ndarray) – Woodbury inverse of the posterior fit of the GP

Return type

ndarray

Returns

Gradient of the posterior covariance of shape (q, q, q, d)

emukit.model_wrappers.gpy_model_wrappers.dmean(x_predict, x_train, kern, w_vec)

Compute the derivative of the posterior mean with respect to prediction input

Parameters
• x_predict (ndarray) – Prediction inputs of shape (q, d)

• x_train (ndarray) – Training inputs of shape (n, d)

• kern (<module 'GPy.kern' from '/home/docs/checkouts/readthedocs.org/user_builds/emukit/envs/latest/lib/python3.6/site-packages/GPy/kern/__init__.py'>) – Covariance of the GP model

• w_vec (ndarray) – Woodbury vector of the posterior fit of the GP

Return type

ndarray

Returns

Gradient of the posterior mean of shape (q, q, d)

class emukit.model_wrappers.gpy_model_wrappers.GPyMultiOutputWrapper(gpy_model, n_outputs, n_optimization_restarts, verbose_optimization=True)

A wrapper around GPy multi-output models. X inputs should have the corresponding output index as the last column in the array

calculate_variance_reduction(x_train_new, x_test)

Calculates reduction in variance at x_test due to observing training point x_train_new

Parameters
• x_train_new (ndarray) – New training point

• x_test (ndarray) – Test points to calculate variance reduction at

Return type

ndarray

Returns

Array of variance reduction at each test point

Calculates gradients of predictions with respect to X, excluding with respect to the output index :type X: ndarray :param X: Point at which to predict gradients :rtype: Tuple[ndarray, ndarray] :return: (mean gradient, variance gradient)

predict(X)

Predicts mean and variance for output specified by last column of X :type X: ndarray :param X: point(s) at which to predict :rtype: ndarray :return: predicted (mean, variance) at X

set_data(X, Y)

Updates model with new training data :type X: ndarray :param X: New training features with output index as last column :type Y: ndarray :param Y: New training targets with output index as last column

Return type

None

optimize()

Optimizes hyper-parameters of model. Starts the optimization at random locations equal to the values of the “n_optimization_restarts” attribute.

Return type

None

property X: ndarray
Return type

ndarray

property Y: ndarray
Return type

ndarray

predict_covariance(X, with_noise=True)

Calculates posterior covariance between points in X

Parameters
• X (ndarray) – Array of size n_points x n_dimensions containing input locations to compute posterior covariance at

• with_noise (bool) – Whether to include likelihood noise in the covariance matrix

Return type

ndarray

Returns

Posterior covariance matrix of size n_points x n_points

get_covariance_between_points(X1, X2)

Calculate posterior covariance between two points :type X1: ndarray :param X1: An array of shape 1 x n_dimensions that contains a data single point. It is the first argument of the

posterior covariance function

Parameters

X2 (ndarray) – An array of shape n_points x n_dimensions that may contain multiple data points. This is the second argument to the posterior covariance function.

Return type

ndarray

Returns

An array of shape n_points x 1 of posterior covariances between X1 and X2

generate_hyperparameters_samples(n_samples=10, n_burnin=5, subsample_interval=1, step_size=0.1, leapfrog_steps=1)

Generates the samples from the hyper-parameters, and returns them (a numpy array whose rows are samples of the hyper-parameters). :param n_samples: Number of generated samples. :param n_burnin: Number of initial samples not used. :param subsample_interval: Interval of subsampling from HMC samples. :param step_size: Size of the gradient steps in the HMC sampler. :param leapfrog_steps: Number of gradient steps before each Metropolis Hasting step.

Return type

ndarray

fix_model_hyperparameters(sample_hyperparameters)

Fix model hyperparameters

Return type

None

GPy wrappers for the quadrature package.

Wrapper for GPy’s GPRegression as required for some EmuKit quadrature methods.

An instance of this class can be passed as base_gp to a WarpedBayesianQuadratureModel object.

Note

GPy’s GPRegression cannot take None as initial values for X and Y. Thus, we initialize them with some arbitrary values. These will be re-set in the WarpedBayesianQuadratureModel.

Parameters
• kern (QuadratureKernel) – An EmuKit quadrature kernel.

• gpy_model (GPRegression) – A GPy GP regression model.

• noise_free (bool) – If False, the observation noise variance will be treated as a model parameter, if True the noise is set to 1e-10, defaults to True.

property X: ndarray

The data nodes.

Return type

ndarray

property Y: ndarray

The data evaluations at the nodes.

Return type

ndarray

property observation_noise_variance: float

The variance of the Gaussian observation noise.

Return type

float

set_data(X, Y)

Sets training data in model.

Parameters
• X (ndarray) – New training features, shape (num_points, input_dim).

• Y (ndarray) – New training outputs, shape (num_points, 1).

Return type

None

predict(X_pred)

Predictive mean and covariance at the locations X_pred.

Parameters

X_pred (ndarray) – Points at which to predict, with shape (number of points, input_dim).

Return type

Tuple[ndarray, ndarray]

Returns

Predictive mean, predictive variances shapes (num_points, 1) and (num_points, 1).

predict_with_full_covariance(X_pred)

Predictive mean and covariance at the locations X_pred.

Parameters

X_pred (ndarray) – Points at which to predict, with shape (num_points, input_dim).

Return type

Tuple[ndarray, ndarray]

Returns

Predictive mean, predictive full covariance shapes (num_points, 1) and (num_points, num_points).

solve_linear(z)

Solve the linear system $$Gx=z$$ for $$x$$.

$$G$$ is the Gram matrix $$G := K(X, X) + \sigma^2 I$$, of shape (num_dat, num_dat) and $$z$$ is a matrix of shape (num_dat, num_obs).

Parameters

z (ndarray) – A matrix of shape (num_dat, num_obs).

Return type

ndarray

Returns

The solution $$x$$ of the linear, shape (num_dat, num_obs).

graminv_residual()

The solution $$z$$ of the linear system

$(K_{XX} + \zeta^2 I) z = (Y - m(X))$

where $$X$$ and $$Y$$ are the available nodes and function evaluation, $$m(X)$$ is the predictive mean at $$X$$, and $$\zeta^2$$ the observation noise variance.

Return type

ndarray

Returns

The solution $$z$$ of the linear system, shape (num_dat, 1).

optimize()

Optimize the hyperparameters of the GP.

Return type

None

Bases: IRBF

Wrapper of the GPy RBF kernel as required for some EmuKit quadrature methods.

$k(x, x') = \sigma^2 e^{-\frac{1}{2}\sum_{i=1}^{d}r_i^2},$

where $$d$$ is the input dimensionality, $$r_i = \frac{x_i-x_i'}{\lambda_i}$$ is the scaled vector difference of dimension $$i$$, $$\lambda_i$$ is the $$i$$ th element of the lengthscales property and $$\sigma^2$$ is the variance property.

Parameters

gpy_rbf (RBF) – An RBF kernel from GPy.

property lengthscales: ndarray

The lengthscales $$\lambda$$ of the kernel.

Return type

ndarray

property variance: float

The scale $$\sigma^2$$ of the kernel.

Return type

float

K(x1, x2)

The kernel k(x1, x2) evaluated at x1 and x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

Kernel evaluated at x1, x2, shape (N, M).

Wrapper of the GPy Exponential (a.k.a Matern12) product kernel as required for some EmuKit quadrature methods.

The product kernel is of the form $$k(x, x') = \sigma^2 \prod_{i=1}^d k_i(x, x')$$ where

$k_i(x, x') = e^{-r_i}.$

$$d$$ is the input dimensionality, $$r_i:=\frac{|x_i - x'_i|}{\lambda_i}$$, $$\sigma^2$$ is the variance property and $$\lambda_i$$ is the $$i$$ th element of the lengthscales property.

Parameters
• gpy_matern (Union[Exponential, Prod, None]) – An Exponential (a.k.a. Matern12) product kernel from GPy. For $$d=1$$ this is equivalent to an Exponential kernel. For $$d>1$$, this is not a $$d$$-dimensional Exponential kernel but a product of $$d$$ 1-dimensional Exponential kernels with differing active dimensions constructed as k1 * k2 * … . Make sure to unlink all variances except the variance of the first kernel k1 in the product as the variance of k1 will be used to represent $$\sigma^2$$. If you are unsure what to do, use the lengthscales and variance parameter instead. If gpy_matern is not given, the lengthscales argument is used.

• lengthscales (Optional[ndarray]) – If gpy_matern is not given, a product Matern12 kernel will be constructed with the given lengthscales. The number of elements need to be equal to the dimensionality $$d$$. If gpy_matern is given, this input is disregarded.

• variance (Optional[float]) – The variance of the product kernel. Only used if gpy_matern is not given. Defaults to 1.

property lengthscales: ndarray

The lengthscales $$\lambda$$ of the kernel.

Return type

ndarray

property variance: float

The scale $$\sigma^2$$ of the kernel.

Return type

float

K(x1, x2)

The kernel k(x1, x2) evaluated at x1 and x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

Kernel evaluated at x1, x2, shape (N, M).

dK_dx1(x1, x2)

Gradient of the kernel wrt x1 evaluated at pair x1, x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

The gradient of the kernel wrt x1 evaluated at (x1, x2), shape (input_dim, N, M)

Wrapper of the GPy Matern32 product kernel as required for some EmuKit quadrature methods.

The product kernel is of the form $$k(x, x') = \sigma^2 \prod_{i=1}^d k_i(x, x')$$ where

$k_i(x, x') = (1 + \sqrt{3}r_i ) e^{-\sqrt{3} r_i}.$

$$d$$ is the input dimensionality, $$r_i:=\frac{|x_i - x'_i|}{\lambda_i}$$, $$\sigma^2$$ is the variance property and $$\lambda_i$$ is the $$i$$ th element of the lengthscales property.

Parameters
• gpy_matern (Union[Matern32, Prod, None]) – A Matern32 product kernel from GPy. For $$d=1$$ this is equivalent to a Matern32 kernel. For $$d>1$$, this is not a $$d$$-dimensional Matern32 kernel but a product of $$d$$ 1-dimensional Matern32 kernels with differing active dimensions constructed as k1 * k2 * … . Make sure to unlink all variances except the variance of the first kernel k1 in the product as the variance of k1 will be used to represent $$\sigma^2$$. If you are unsure what to do, use the lengthscales and variance parameter instead. If gpy_matern is not given, the lengthscales argument is used.

• lengthscales (Optional[ndarray]) – If gpy_matern is not given, a product Matern32 kernel will be constructed with the given lengthscales. The number of elements need to be equal to the dimensionality $$d$$. If gpy_matern is given, this input is disregarded.

• variance (Optional[float]) – The variance of the product kernel. Only used if gpy_matern is not given. Defaults to 1.

property lengthscales: ndarray

The lengthscales $$\lambda$$ of the kernel.

Return type

ndarray

property variance: float

The scale $$\sigma^2$$ of the kernel.

Return type

float

K(x1, x2)

The kernel k(x1, x2) evaluated at x1 and x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

Kernel evaluated at x1, x2, shape (N, M).

dK_dx1(x1, x2)

Gradient of the kernel wrt x1 evaluated at pair x1, x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

The gradient of the kernel wrt x1 evaluated at (x1, x2), shape (input_dim, N, M)

Wrapper of the GPy Matern52 product kernel as required for some EmuKit quadrature methods.

The product kernel is of the form $$k(x, x') = \sigma^2 \prod_{i=1}^d k_i(x, x')$$ where

$k_i(x, x') = (1 + \sqrt{5} r_i + \frac{5}{3} r_i^2) \exp(- \sqrt{5} r_i).$

$$d$$ is the input dimensionality, $$r_i:=\frac{|x_i - x'_i|}{\lambda_i}$$, $$\sigma^2$$ is the variance property and $$\lambda_i$$ is the $$i$$ th element of the lengthscales property.

Parameters
• gpy_matern (Union[Matern52, Prod, None]) – A Matern52 product kernel from GPy. For $$d=1$$ this is equivalent to a Matern52 kernel. For $$d>1$$, this is not a $$d$$-dimensional Matern52 kernel but a product of $$d$$ 1-dimensional Matern52 kernels with differing active dimensions constructed as k1 * k2 * … . Make sure to unlink all variances except the variance of the first kernel k1 in the product as the variance of k1 will be used to represent $$\sigma^2$$. If you are unsure what to do, use the lengthscales and variance parameter instead. If gpy_matern is not given, the lengthscales argument is used.

• lengthscales (Optional[ndarray]) – If gpy_matern is not given, a product Matern52 kernel will be constructed with the given lengthscales. The number of elements need to be equal to the dimensionality $$d$$. If gpy_matern is given, this input is disregarded.

• variance (Optional[float]) – The variance of the product kernel. Only used if gpy_matern is not given. Defaults to 1.

property lengthscales: ndarray

The lengthscales $$\lambda$$ of the kernel.

Return type

ndarray

property variance: float

The scale $$\sigma^2$$ of the kernel.

Return type

float

K(x1, x2)

The kernel k(x1, x2) evaluated at x1 and x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

Kernel evaluated at x1, x2, shape (N, M).

dK_dx1(x1, x2)

Gradient of the kernel wrt x1 evaluated at pair x1, x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

The gradient of the kernel wrt x1 evaluated at (x1, x2), shape (input_dim, N, M)

Bases: IBrownian

Wrapper of the GPy Brownian motion kernel as required for some EmuKit quadrature methods.

$k(x, x') = \sigma^2 \operatorname{min}(x, x')\quad\text{with}\quad x, x' \geq 0,$

where $$\sigma^2$$ is the variance property.

Parameters

gpy_brownian (Brownian) – A Brownian motion kernel from GPy.

property variance: float

The scale $$\sigma^2$$ of the kernel.

Return type

float

K(x1, x2)

The kernel k(x1, x2) evaluated at x1 and x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

Kernel evaluated at x1, x2, shape (N, M).

Wrapper of the GPy Brownian product kernel as required for some EmuKit quadrature methods.

The product kernel is of the form $$k(x, x') = \sigma^2 \prod_{i=1}^d k_i(x, x')$$ where

$k_i(x, x') = \operatorname{min}(x_i-c, x_i'-c)\quad\text{with}\quad x_i, x_i' \geq c,$

$$d$$ is the input dimensionality, $$\sigma^2$$ is the variance property and $$c$$ is the offset property.

Parameters
• gpy_brownian (Union[Brownian, Prod, None]) – A Brownian product kernel from GPy. For $$d=1$$ this is equivalent to a Brownian kernel. For $$d>1$$, this is a product of $$d$$ 1-dimensional Brownian kernels with differing active dimensions constructed as k1 * k2 * … . Make sure to unlink all variances except the variance of the first kernel k1 in the product as the variance of k1 will be used to represent $$\sigma^2$$. If you are unsure what to do, use the input_dim and variance parameter instead. If gpy_brownian is not given, the variance and input_dim argument is used.

• offset (float) – The offset $$c$$ of the kernel. Defaults to 0.

• variance (Optional[float]) – The variance of the product kernel. Only used if gpy_brownian is not given. Defaults to 1.

• input_dim (Optional[int]) – The input dimension. Only used if gpy_brownian is not given.

property variance: float

The scale $$\sigma^2$$ of the kernel.

Return type

float

property offset: float

The offset $$c$$ of the kernel.

Return type

float

K(x1, x2)

The kernel k(x1, x2) evaluated at x1 and x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

Kernel evaluated at x1, x2, shape (N, M).

dK_dx1(x1, x2)

Gradient of the kernel wrt x1 evaluated at pair x1, x2.

Parameters
• x1 (ndarray) – First argument of the kernel, shape (n_points N, input_dim)

• x2 (ndarray) – Second argument of the kernel, shape (n_points M, input_dim)

Return type

ndarray

Returns

The gradient of the kernel wrt x1 evaluated at (x1, x2), shape (input_dim, N, M)

dKdiag_dx(x)

The gradient of the diagonal of the kernel (the variance) v(x):=k(x, x) evaluated at x.

Parameters

x (ndarray) – The locations where the gradient is evaluated, shape (n_points, input_dim).

Return type

ndarray

Returns

The gradient of the diagonal of the kernel evaluated at x, shape (input_dim, n_points).

Wraps a GPy model and returns an EmuKit quadrature model.

Parameters
• gpy_model (GPRegression) – A GPy Gaussian process regression model GPy.models.GPRegression.

• integral_bounds (Optional[List[Tuple[float, float]]]) – List of d tuples, where d is the dimensionality of the integral and the tuples contain the lower and upper bounds of the integral i.e., [(lb_1, ub_1), (lb_2, ub_2), …, (lb_d, ub_d)]. Only used if measure is not given in which case the unnormalized Lebesgue measure is used.

• measure (Optional[IntegrationMeasure]) – An integration measure. Either measure or integral_bounds must be given. If both integral_bounds and measure are given, integral_bounds is disregarded.

• integral_name (str) – The (variable) name(s) of the integral.

Return type

BaseGaussianProcessGPy

Returns

An EmuKit GP model for quadrature with GPy backend.

class emukit.model_wrappers.simple_gp_model.SimpleGaussianProcessModel(x, y)

Bases: IModel

This model is a Gaussian process with an RBF kernel, with no ARD. It is used to demonstrate uses of emukit, it does not aim to be flexible, robust or fast.

optimize()

Optimize the three hyperparameters of the model, namely the kernel variance, kernel lengthscale and likelihood variance

Return type

None

predict(x_new)

Predict from model

Parameters

x_new (ndarray) – (n_points, n_dims) array containing points at which the predictive distributions will be computed

Return type

Tuple[ndarray, ndarray]

Returns

Tuple containing two (n_points, 1) arrays representing the mean and variance of the predictive distribution at the specified input locations

set_data(X, Y)

Set training data to new values

Parameters
• X (ndarray) – (n_points, n_dims) array containing training features

• Y (ndarray) – (n_points, 1) array containing training targets

Return type

None

property X: ndarray
Return type

ndarray

property Y: ndarray
Return type

ndarray