bayesmix/hierarchies/updaters

Updaters¶

An Updater implements the machinery to provide a sampling from the full conditional distribution of a given hierarchy.

The only operation performed is draw that samples from the full conditional, either exactly or via Markov chain Monte Carlo.

class AbstractUpdater¶

Base class for an Updater.

An updater is a class able to sample from the full conditional distribution of an Hierarchy, coming from the product of a Likelihood and a Prior, possibly using a Metropolis-Hastings algorithm.

Subclassed by MetropolisUpdater< RandomWalkUpdater >, MetropolisUpdater< MalaUpdater >, SemiConjugateUpdater< UniNormLikelihood, NxIGPriorModel >, SemiConjugateUpdater< MultiNormLikelihood, NWPriorModel >, SemiConjugateUpdater< UniNormLikelihood, NIGPriorModel >, SemiConjugateUpdater< UniLinRegLikelihood, MNIGPriorModel >, FAUpdater, MetropolisUpdater< DerivedUpdater >, SemiConjugateUpdater< Likelihood, PriorModel >

Public Functions

virtual ~AbstractUpdater() = default¶: Default destructor.

inline virtual bool is_conjugate() const¶: Returns whether the current updater is for a (semi)conjugate model or not.

virtual void draw(AbstractLikelihood &like, AbstractPriorModel &prior, bool update_params) = 0¶

Sampling from the full conditional, given the likelihood and the prior model that constitutes the hierarchy

Parameters:

like – The likelihood of the hierarchy
prior – The prior model of the hierarchy
update_params – Save posterior hyperparameters after draw?

inline virtual ProtoHypersPtr compute_posterior_hypers(AbstractLikelihood &like, AbstractPriorModel &prior)¶: Computes the posterior hyperparameters required for the sampling in case of conjugate hierarchies

inline void save_posterior_hypers(ProtoHypersPtr post_hypers_)¶: Stores the posterior hyperparameters in an appropriate container.

Code Structure¶

We distinguish between semi-conjugate updaters and the metropolis-like updaters.

Semi Conjugate Updaters¶

A semi-conjugate updater can be used when the full conditional distribution has the same form of the prior. Therefore, to sample from the full conditional, it is sufficient to call the draw method of the prior, but with an updated set of hyperparameters.

The class SemiConjugateUpdater defines the API

template<class Likelihood, class PriorModel> class SemiConjugateUpdater : public AbstractUpdater ¶

Updater for semi-conjugate hierarchies.

We say that a hierarchy is semi-conjugate if the full conditionals of each parameter is in the same parametric family of the prior distribution of that parameter.

As a consequence, sampling from the full conditional can be done by calling the sample method from the PriorModel class, with updater hyperparameters

Classes inheriting from this one should only implement the compute_posterior_hypers(...) member function

This class is templated with respect to

Template Parameters:

Likelihood – the likelihood of the hierarchy, instance of AbstractLikelihood
PriorModel – the prior of the hierarchy, instance of AbstractPriorModel

Public Functions

virtual void draw(AbstractLikelihood &like, AbstractPriorModel &prior, bool update_params) override¶

Sampling from the full conditional, given the likelihood and the prior model that constitutes the hierarchy

Parameters:

like – The likelihood of the hierarchy
prior – The prior model of the hierarchy
update_params – Save posterior hyperparameters after draw?

Classes inheriting from this one should only implement the compute_posterior_hypers(...) member function.

Metropolis-like Updaters¶

A Metropolis updater uses the Metropolis-Hastings algorithm (or its variations) to sample from the full conditional density.

template<class DerivedUpdater> class MetropolisUpdater : public AbstractUpdater ¶

Base class for updaters using a Metropolis-Hastings algorithm

This class serves as the base for a CRTP. Children of this class should implement the methods template <typename F> Eigen::VectorXd sample_proposal(Eigen::VectorXd curr_state, AbstractLikelihood &like, AbstractPriorModel &prior, F &target_lpdf) and template <typename F> double proposal_lpdf(Eigen::VectorXd prop_state, Eigen::VectorXd curr_state, AbstractLikelihood &like, AbstractPriorModel &prior, F &target_lpdf) where the template parameter is needed to allow the use of stan’s automatic differentiation if the gradient of the full conditional is required.

Public Functions

inline virtual void draw(AbstractLikelihood &like, AbstractPriorModel &prior, bool update_params) override¶: Samples from the full conditional distribution using a Metropolis-Hastings step

Classes inheriting from this one should only implement the sample_proposal(...) method, which samples from the porposal distribution, and the proposal_lpdf one, which evaluates the proposal density log-probability density function.

Updater Classes¶

class RandomWalkUpdater : public MetropolisUpdater<RandomWalkUpdater>¶

Metropolis-Hastings updater using an isotropic proposal function centered in the current value of the parameters (unconstrained). This class requires that the Hierarchy’s state implements the get_unconstrained(), set_from_unconstrained() and log_det_jac() functions.

Given the current value of the unconstrained parameters $x$ , a new value is proposed from

$x_{new} \sim N(x, step\_size \cdot I)$

and then either accepted (in which case the hierarchy’s state is set to $x_{new}$ ) or rejected.

Public Functions

template<typename F> inline Eigen::VectorXd sample_proposal(Eigen::VectorXd curr_state, AbstractLikelihood &like, AbstractPriorModel &prior, F &target_lpdf)¶

Samples from the proposal distribution

Parameters:

curr_state – the current state (unconstrained parametrization)
like – instance of likelihood
prior – instance of prior
target_lpdf – either double or stan::math::var. Needed for stan’s automatic differentiation. It is not used here.

template<typename F> inline double proposal_lpdf(Eigen::VectorXd prop_state, Eigen::VectorXd curr_state, AbstractLikelihood &like, AbstractPriorModel &prior, F &target_lpdf)¶

Evaluates the log probability density function of the proposal

Parameters:

prop_state – the proposed state (at which to evaluate the lpdf)
curr_state – the current state (unconstrained parametrization)
like – instance of likelihood
prior – instance of prior
target_lpdf – either double or stan::math::var. Needed for stan’s automatic differentiation. It is not used here.

inline virtual std::shared_ptr<AbstractUpdater> clone() const override¶: Returns a shared_ptr to a new instance of this

class MalaUpdater : public MetropolisUpdater<MalaUpdater>¶

Metropolis Adjusted Langevin Algorithm.

This class requires that the Hierarchy’s state implements the get_unconstrained(), set_from_unconstrained() and log_det_jac() functions.

Given the current value of the unconstrained parameters $x$ , a new value is proposed from

$x_{new} \sim N(x + step\_size \cdot \text{grad}(full\_cond)(x), \sqrt{2 step\_size} \cdot I)$

and then either accepted (in which case the hierarchy’s state is set to $x_{new}$ ) or rejected.

Public Functions

inline Eigen::VectorXd sample_proposal(Eigen::VectorXd curr_state, AbstractLikelihood &like, AbstractPriorModel &prior, target_lpdf_unconstrained &target_lpdf)¶

Samples from the proposal distribution

Parameters:

curr_state – the current state (unconstrained parametrization)
like – instance of likelihood
prior – instance of prior
target_lpdf – either double or stan::math::var. Needed for stan’s automatic differentiation. It will be filled with the lpdf at the ‘curr_state’

inline double proposal_lpdf(Eigen::VectorXd prop_state, Eigen::VectorXd curr_state, AbstractLikelihood &like, AbstractPriorModel &prior, target_lpdf_unconstrained &target_lpdf)¶

Evaluates the log probability density function of the proposal

Parameters:

prop_state – the proposed state (at which to evaluate the lpdf)
curr_state – the current state (unconstrained parametrization)
like – instance of likelihood
prior – instance of prior
target_lpdf – either double or stan::math::var. Needed for stan’s automatic differentiation. It will be filled with the lpdf at ‘curr_state’

class NNIGUpdater : public SemiConjugateUpdater<UniNormLikelihood, NIGPriorModel>¶

Updater specific for the UniNormLikelihood used in combination with NIGPriorModel, that is the model

$y_i \mid \mu, \sigma^2 &\stackrel{\small\mathrm{iid}}{\sim} N(\mu, \sigma^2) \\ \mu \mid \sigma^2 &\sim N(\mu_0, \sigma^2 / \lambda) \\ \sigma^2 &\sim InvGamma(a, b)$

It exploits the conjugacy of the model to sample the full conditional of $(\mu, \sigma^2)$ by calling NIGPriorModel::sample with updated parameters

Public Functions

inline virtual bool is_conjugate() const override¶: Returns whether the current updater is for a (semi)conjugate model or not.

virtual ProtoHypersPtr compute_posterior_hypers(AbstractLikelihood &like, AbstractPriorModel &prior) override¶: Computes the posterior hyperparameters required for the sampling in case of conjugate hierarchies

class NNxIGUpdater : public SemiConjugateUpdater<UniNormLikelihood, NxIGPriorModel>¶

Updater specific for the UniNormLikelihood used in combination with NxIGPriorModel, that is the model

$y_i \mid \mu, \sigma^2 &\stackrel{\small\mathrm{iid}}{\sim} N(\mu, \sigma^2) \\ \mu &\sim N(\mu_0, \eta^2) \\ \sigma^2 & \sim InvGamma(a,b)$

It exploits the semi-conjugacy of the model to sample the full conditional of $(\mu, \sigma^2)$ by calling NxIGPriorModel::sample with updated parameters

Public Functions

virtual ProtoHypersPtr compute_posterior_hypers(AbstractLikelihood &like, AbstractPriorModel &prior) override¶: Computes the posterior hyperparameters required for the sampling in case of conjugate hierarchies

class NNWUpdater : public SemiConjugateUpdater<MultiNormLikelihood, NWPriorModel>¶

Updater specific for the MultiNormLikelihood used in combination with NWPriorModel, that is the model

$y_i \mid \bm{\mu}, \Sigma &\stackrel{\small\mathrm{iid}}{\sim} N_d(\bm{mu}, \Sigma) \\ \bm{\mu} \mid \Sigma &\sim N_d(\bm{\mu}_0, \Sigma / \lambda) \\ \Sigma^{-1} &\sim Wishart(\nu, \Psi)$

It exploits the conjugacy of the model to sample the full conditional of $(\bm{\mu}, \Sigma)$ by calling NWPriorModel::sample with updated parameters.

Public Functions

inline virtual bool is_conjugate() const override¶: Returns whether the current updater is for a (semi)conjugate model or not.

virtual ProtoHypersPtr compute_posterior_hypers(AbstractLikelihood &like, AbstractPriorModel &prior) override¶: Computes the posterior hyperparameters required for the sampling in case of conjugate hierarchies

class MNIGUpdater : public SemiConjugateUpdater<UniLinRegLikelihood, MNIGPriorModel>¶

Updater specific for the UniLinRegLikelihood used in combination with MNIGPriorModel, that is the model

$y_i \mid \bm{\beta}, \sigma^2 &\stackrel{\small\mathrm{iid}}{\sim} N(\bm{\beta}^T\bm{x}_i, \sigma^2) \\ \bm{\beta} \mid \sigma^2 &\sim N_p(\mu_{0}, \sigma^2 \mathbf{V}^{-1}) \\ \sigma^2 &\sim InvGamma(a, b)$

It exploits the conjugacy of the model to sample the full conditional of $(\bm{\beta}, \sigma^2)$ by calling MNIGPriorModel::sample with updated parameters

Public Functions

inline virtual bool is_conjugate() const override¶: Returns whether the current updater is for a (semi)conjugate model or not.

virtual ProtoHypersPtr compute_posterior_hypers(AbstractLikelihood &like, AbstractPriorModel &prior) override¶: Computes the posterior hyperparameters required for the sampling in case of conjugate hierarchies

class FAUpdater : public AbstractUpdater ¶

Updater specific for the FAHierachy. See Bhattacharya, Anirban, and David B. Dunson. “Sparse Bayesian infinite factor models.” Biometrika (2011): 291-306. for further details

Public Functions

virtual void draw(AbstractLikelihood &like, AbstractPriorModel &prior, bool update_params) override¶

Sampling from the full conditional, given the likelihood and the prior model that constitutes the hierarchy

Parameters:

like – The likelihood of the hierarchy
prior – The prior model of the hierarchy
update_params – Save posterior hyperparameters after draw?