Hypothesis learning

gpax.hypo.step(model, model_prior, X_measured, y_measured, X_unmeasured=None, gp_wrap=False, noise_prior=None, gp_kernel='Matern', gp_kernel_prior=None, gp_input_dim=1, num_warmup=2000, num_samples=2000, num_chains=1, num_restarts=1, print_summary=True)

Compute model posterior and use it to derive predictive uncertainty

Parameters:

• model (Callable[[Array, Dict[str, Array]], Array]) – Parametric model in jax.numpy

• model_prior (Callable[[], Dict[str, Array]]) – Prior over model parameters using numpyro.distributions

• X_measured (Array) – Measured points

• y_measured (Array) – Measured values

• X_unmeasured (Optional[Array]) – Unmeasured points

• gp_wrap (Optional[bool]) – Wrap probabilistic model into a Gaussian process (Default: False)

• noise_prior (Optional[Callable[[], Dict[str, Array]]]) – Custom prior for observation noise. Defaults to LogNormal(0,1)

• gp_kernel (str) – Gaussian process kernel (if gp_wrap is True). Defaults to Matern

• gp_kernel_prior (Optional[Callable[[], Dict[str, Array]]]) – Custom priors over kernel hyperparameters. Defaults to LogNormal(0,1)

• gp_input_dim (Optional[int]) – Number of lenghscale dimensions in GP kernel. Equals to number of input dimensions or 1 (default)

• num_warmup (Optional[int]) – Number of warmup steps for HMC. Defaults to 2000

• num_samples (Optional[int]) – Number of HMC samples. Defaults to 2000

• num_chains (Optional[int]) – Number of HMC chains. Defaults to 2000

• num_restarts (Optional[int]) – Number of restarts if r_hat values are not acceptable (>1.1). Defaults to 1

• print_summary (Optional[bool]) – Verbose parameter

Returns:

Predictive uncertainty and trained model object

gpax.hypo.sample_next(rewards, method='softmax', temperature=1.0, eps=0.4)

Sample model or input channel based on softmax or epsilon-greedy policy

Parameters:

• rewards (Union[array, array]) – Array of shape (N,) with running rewards

• method (Optional[str]) – Selection policy, choose between ‘softmax’ and ‘epsilon-greedy’

• temperature (Optional[float]) – Optional temperature parameter for softmax selection policy

• eps (Optional[float]) – Optional epsilon parameter for epsilon-greedy policy

Return type:

int

Returns:

The index of model or input channel to sample next

gpax.hypo.softmax(logits, temperature=1.0)

Softmax selection policy. Based on Zai, A., Brown, B. (2020). Deep reinforcement learning in action. Manning Publications

Return type:

int

gpax.hypo.eps_greedy(rewards, eps=0.4)

Epsilon-greedy selection policy. Based on Zai, A., Brown, B. (2020). Deep reinforcement learning in action. Manning Publications

Return type:

int

gpax.hypo.update_record(record, action, r)

Update the reward record. Based on Zai, A., Brown, B. (2020). Deep reinforcement learning in action. Manning Publications

Return type:

array