bayesmix/hierarchies/prior_models

Prior Models¶

A PriorModel represents the prior for the parameters in the likelihood, i.e.

$\bm{\tau} \sim G_{0}$

with $G_{0}$ being a suitable prior on the parameters space. We also allow for more flexible priors adding further level of randomness (i.e. the hyperprior) on the parameter characterizing $G_{0}$

Main operations performed¶

A likelihood object must be able to perform the following operations:

a. First of all, lpdf() and lpdf_from_unconstrained() methods evaluate the log-prior density function at the current state $\bm \tau$ or its unconstrained representation. In particular, lpdf_from_unconstrained() is needed by Metropolis-like updaters.

b. The sample() method generates a draw from the prior distribution. If hier_hypers is nullptr, the prior hyperparameter values are used. To allow sampling from the full conditional distribution in case of semi-congugate hierarchies, we introduce the ``hier_hypers parameter, which is a pointer to a Protobuf message storing the hierarchy hyperaprameters to use for the sampling.

The update_hypers() method updates the prior hyperparameters, given the vector of all cluster states.

Code structure¶

As for the Likelihood classes we employ the Curiously Recurring Template Pattern to manage the polymorphic nature of PriorModel classes.

The class AbstractPriorModel defines the API, i.e. all the methods that need to be called from outside of a PrioModel class. A template class BasePriorModel inherits from AbstractPriorModel and implements some of the necessary virtual methods, which need not be implemented by the child classes.

Instead, child classes must implement:

lpdf: evaluates $G_0(\theta_h)$
sample: samples from $G_0$ given a hyperparameters (passed as a pointer). If hier_hypers is nullptr, the prior hyperparameter values are used.
set_hypers_from_proto: sets the hyperparameters from a Probuf message
get_hypers_proto: returns the hyperparameters as a Probuf message
initialize_hypers: provides a default initialization of hyperparameters

In case you want to use a Metropolis-like updater, child classes should also implement:

lpdf_from_unconstrained: evaluates $G_0(\tilde{\theta}_h)$ , where $\tilde{\theta}_h$ is the vector of unconstrained parameters.

Abstract Classes¶

class AbstractPriorModel¶

Abstract class for a generic prior model

This class is the basis for a curiously recurring template pattern (CRTP) for PriorModel objects, ad it is solely composed of interface functions for derived classes to use.

A prior model represents the prior for the parameters in the likelihood. Hence, it can evaluate the log probability density function (lpdf) for a given parameter state.

We also store a pointer to the protobuf object that represents the type of prior used for the parameters in the likelihood.

Subclassed by BasePriorModel< NxIGPriorModel, State::UniLS, Hyperparams::NxIG, bayesmix::NNxIGPrior >, BasePriorModel< NIGPriorModel, State::UniLS, Hyperparams::NIG, bayesmix::NNIGPrior >, BasePriorModel< FAPriorModel, State::FA, Hyperparams::FA, bayesmix::FAPrior >, BasePriorModel< MNIGPriorModel, State::UniLinRegLS, Hyperparams::MNIG, bayesmix::LinRegUniPrior >, BasePriorModel< NWPriorModel, State::MultiLS, Hyperparams::NW, bayesmix::NNWPrior >, BasePriorModel< Derived, State, HyperParams, Prior >

Public Functions

virtual ~AbstractPriorModel() = default¶: Default destructor.

virtual std::shared_ptr<AbstractPriorModel> clone() const = 0¶: Returns an independent, data-less copy of this object.

virtual std::shared_ptr<AbstractPriorModel> deep_clone() const = 0¶: Returns an independent, data-less deep copy of this object.

virtual double lpdf(const google::protobuf::Message &state_) = 0¶

Evaluates the log likelihood for the prior model, given the state of the cluster

Parameters:: state_ – A Protobuf message storing the current state of the cluster
Returns:: The evaluation of the log likelihood

inline virtual double lpdf_from_unconstrained(Eigen::VectorXd unconstrained_params) const¶

Evaluates the log likelihood for unconstrained parameter values. By unconstrained parameters we mean that each entry of the parameter vector can range over (-inf, inf). Usually, some kind of transformation is required from the unconstrained parameterization to the actual parameterization.

Parameters:: unconstrained_params – vector collecting the unconstrained parameters
Returns:: The evaluation of the log likelihood of the prior model

inline virtual stan::math::var lpdf_from_unconstrained(Eigen::Matrix<stan::math::var, Eigen::Dynamic, 1> unconstrained_params) const¶

Parameters:: unconstrained_params – vector collecting the unconstrained parameters
Returns:: The evaluation of the log likelihood of the prior model

virtual std::shared_ptr<google::protobuf::Message> sample_proto(ProtoHypersPtr hier_hypers = nullptr) = 0¶

Sampling from the prior model

Parameters:: hier_hypers – A pointer to a bayesmix::AlgorithmState::hierarchyHypers object, which defines the parameters to use for the sampling. The default behaviour (i.e. hier_hypers = nullptr) uses prior hyperparameters
Returns:: A Protobuf message storing the state sampled from the prior model

virtual void update_hypers(const std::vector<bayesmix::AlgorithmState::ClusterState> &states) = 0¶: Updates hyperparameter values given a vector of cluster states.

virtual google::protobuf::Message *get_mutable_prior() = 0¶: Returns a pointer to the Protobuf message of the prior of this cluster.

virtual void set_hypers_from_proto(const google::protobuf::Message &state_) = 0¶: Read and set hyperparameter values from a given Protobuf message.

virtual void write_hypers_to_proto(google::protobuf::Message *out) const = 0¶: Writes current values of the hyperparameters to a Protobuf message by pointer

virtual std::shared_ptr<bayesmix::AlgorithmState::HierarchyHypers> get_hypers_proto() const = 0¶: Writes current value of hyperparameters to a Protobuf message and return a shared_ptr. New hierarchies have to first modify the field ‘oneof val’ in the AlgoritmState::HierarchyHypers message by adding the appropriate type

template<class Derived, class State, typename HyperParams, typename Prior> class BasePriorModel : public AbstractPriorModel ¶

Base template class of a PriorModel object

This class derives from AbstractPriorModel and is templated over Derived (needed for the curiously recurring template pattern), State (an instance of Basestate), HyperParams (a struct representing the hyperparameters, see hyperparams.h) and Prior: a protobuf message representing the type of prior imposed on the hyperparameters.

Template Parameters:

Derived – Name of the implemented derived class
State – Class name of the container for state values
HyperParams – Class name of the container for hyperparameters
Prior – Class name of the protobuf message for the prior on the hyperparameters.

Public Functions

BasePriorModel() = default¶: Default constructor.

~BasePriorModel() = default¶: Default destructor.

inline virtual double lpdf_from_unconstrained(Eigen::VectorXd unconstrained_params) const override¶

Parameters:: unconstrained_params – vector collecting the unconstrained parameters
Returns:: The evaluation of the log likelihood of the prior model

inline virtual stan::math::var lpdf_from_unconstrained(Eigen::Matrix<stan::math::var, Eigen::Dynamic, 1> unconstrained_params) const override¶

This version using stan::math::var type is required for Stan automatic aifferentiation. Evaluates the log likelihood for unconstrained parameter values. By unconstrained parameters we mean that each entry of the parameter vector can range over (-inf, inf). Usually, some kind of transformation is required from the unconstrained parameterization to the actual parameterization.

Parameters:: unconstrained_params – vector collecting the unconstrained parameters
Returns:: The evaluation of the log likelihood of the prior model

inline virtual std::shared_ptr<google::protobuf::Message> sample_proto(ProtoHypersPtr hier_hypers = nullptr) override¶

Sampling from the prior model

Parameters:: hier_hypers – A pointer to a bayesmix::AlgorithmState::hierarchyHypers object, which defines the parameters to use for the sampling. The default behaviour (i.e. hier_hypers = nullptr) uses prior hyperparameters
Returns:: A Protobuf message storing the state sampled from the prior model

virtual std::shared_ptr<AbstractPriorModel> clone() const override¶: Returns an independent, data-less copy of this object.

virtual std::shared_ptr<AbstractPriorModel> deep_clone() const override¶: Returns an independent, data-less deep copy of this object.

virtual google::protobuf::Message *get_mutable_prior() override¶: Returns a pointer to the Protobuf message of the prior of this cluster.

inline HyperParams get_hypers() const¶: Returns the struct of the current prior hyperparameters.

virtual void write_hypers_to_proto(google::protobuf::Message *out) const override¶: Writes current values of the hyperparameters to a Protobuf message by pointer

void initialize()¶: Initializes the prior model (both prior and hyperparameters)

Non-abstract Classes¶

class NIGPriorModel : public BasePriorModel<NIGPriorModel, State::UniLS, Hyperparams::NIG, bayesmix::NNIGPrior>¶

A conjugate prior model for the univariate normal likelihood, that is

$\mu \mid \sigma^2 &\sim N(\mu_0, \sigma^2 / \lambda) \\ \sigma^2 &\sim InvGamma(a,b)$

With several possibilies for hyper-priors on $\mu$ and $\sigma^2$ . We have considered a normal prior for $mu0$ and a Normal-Gamma-Gamma for $(mu0, a, b)$ in addition to fixing prior hyperparameters

Public Functions

virtual double lpdf(const google::protobuf::Message &state_) override¶

Evaluates the log likelihood for the prior model, given the state of the cluster

Parameters:: state_ – A Protobuf message storing the current state of the cluster
Returns:: The evaluation of the log likelihood

virtual void update_hypers(const std::vector<bayesmix::AlgorithmState::ClusterState> &states) override¶: Updates hyperparameter values given a vector of cluster states.

virtual void set_hypers_from_proto(const google::protobuf::Message &hypers_) override¶: Read and set hyperparameter values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::AlgorithmState::HierarchyHypers> get_hypers_proto() const override¶: Writes current value of hyperparameters to a Protobuf message and return a shared_ptr. New hierarchies have to first modify the field ‘oneof val’ in the AlgoritmState::HierarchyHypers message by adding the appropriate type

Protected Functions

virtual void initialize_hypers() override¶: Initializes hierarchy hyperparameters to appropriate values.

class NxIGPriorModel : public BasePriorModel<NxIGPriorModel, State::UniLS, Hyperparams::NxIG, bayesmix::NNxIGPrior>¶

A semi-conjugate prior model for the univariate normal likelihood, that is

$\mu & \sim N(\mu_0, \eta^2) \\ \sigma^2 & \sim InvGamma(a,b)$

Public Functions

virtual double lpdf(const google::protobuf::Message &state_) override¶

Evaluates the log likelihood for the prior model, given the state of the cluster

Parameters:: state_ – A Protobuf message storing the current state of the cluster
Returns:: The evaluation of the log likelihood

virtual void update_hypers(const std::vector<bayesmix::AlgorithmState::ClusterState> &states) override¶: Updates hyperparameter values given a vector of cluster states.

virtual void set_hypers_from_proto(const google::protobuf::Message &hypers_) override¶: Read and set hyperparameter values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::AlgorithmState::HierarchyHypers> get_hypers_proto() const override¶: Writes current value of hyperparameters to a Protobuf message and return a shared_ptr. New hierarchies have to first modify the field ‘oneof val’ in the AlgoritmState::HierarchyHypers message by adding the appropriate type

Protected Functions

virtual void initialize_hypers() override¶: Initializes hierarchy hyperparameters to appropriate values.

class NWPriorModel : public BasePriorModel<NWPriorModel, State::MultiLS, Hyperparams::NW, bayesmix::NNWPrior>¶

A conjugate prior model for the multivariate normal likelihood, that is

$\bm{\mu} \mid \Sigma &\sim N_p(\bm{\mu}_0, (\Sigma \lambda)^{-1}) \\ \Sigma & \sim Wishart(\nu_0, \Psi_0)$

With some options for hyper-priors on $\bm{\mu}$ and $\Sigma$ . We have considered a normal prior for $\bm{\mu}_0$ in addition to fixing prior hyperparameters

Public Functions

virtual double lpdf(const google::protobuf::Message &state_) override¶

Evaluates the log likelihood for the prior model, given the state of the cluster

Parameters:: state_ – A Protobuf message storing the current state of the cluster
Returns:: The evaluation of the log likelihood

virtual void update_hypers(const std::vector<bayesmix::AlgorithmState::ClusterState> &states) override¶: Updates hyperparameter values given a vector of cluster states.

virtual void set_hypers_from_proto(const google::protobuf::Message &hypers_) override¶: Read and set hyperparameter values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::AlgorithmState::HierarchyHypers> get_hypers_proto() const override¶: Writes current value of hyperparameters to a Protobuf message and return a shared_ptr. New hierarchies have to first modify the field ‘oneof val’ in the AlgoritmState::HierarchyHypers message by adding the appropriate type

Protected Functions

virtual void initialize_hypers() override¶: Initializes hierarchy hyperparameters to appropriate values.

class MNIGPriorModel : public BasePriorModel<MNIGPriorModel, State::UniLinRegLS, Hyperparams::MNIG, bayesmix::LinRegUniPrior>¶

A conjugate prior model for the scalar linear regression likelihood, i.e.

$\bm{\beta} \mid \sigma^2 & \sim N_p(\bm{\mu}, \sigma^2 \Lambda^{-1}) \\ \sigma^2 & \sim InvGamma(a,b)$

Public Functions

virtual double lpdf(const google::protobuf::Message &state_) override¶

Evaluates the log likelihood for the prior model, given the state of the cluster

Parameters:: state_ – A Protobuf message storing the current state of the cluster
Returns:: The evaluation of the log likelihood

virtual void update_hypers(const std::vector<bayesmix::AlgorithmState::ClusterState> &states) override¶: Updates hyperparameter values given a vector of cluster states.

virtual void set_hypers_from_proto(const google::protobuf::Message &hypers_) override¶: Read and set hyperparameter values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::AlgorithmState::HierarchyHypers> get_hypers_proto() const override¶: Writes current value of hyperparameters to a Protobuf message and return a shared_ptr. New hierarchies have to first modify the field ‘oneof val’ in the AlgoritmState::HierarchyHypers message by adding the appropriate type

Protected Functions

virtual void initialize_hypers() override¶: Initializes hierarchy hyperparameters to appropriate values.

class FAPriorModel : public BasePriorModel<FAPriorModel, State::FA, Hyperparams::FA, bayesmix::FAPrior>¶

A priormodel for the factor analyzers likelihood, that is

$\bm{\mu} &\sim N_p(\tilde{\bm{\mu}}, \psi I) \\ \Lambda &\sim DL(\alpha) \\ \Sigma &= \mathrm{diag}(\sigma^2_1, \ldots, \sigma^2_p) \\ \sigma^2_j &\stackrel{\small\mathrm{iid}}{\sim} InvGamma(a,b) \quad j=1,...,p$

Where $DL$ is the Dirichlet-Laplace distribution. See Bhattacharya A., Pati D., Pillai N.S., Dunson D.B. (2015). JASA 110(512), 1479–1490 for details.

Public Functions

virtual double lpdf(const google::protobuf::Message &state_) override¶

Evaluates the log likelihood for the prior model, given the state of the cluster

Parameters:: state_ – A Protobuf message storing the current state of the cluster
Returns:: The evaluation of the log likelihood

virtual void update_hypers(const std::vector<bayesmix::AlgorithmState::ClusterState> &states) override¶: Updates hyperparameter values given a vector of cluster states.

virtual void set_hypers_from_proto(const google::protobuf::Message &hypers_) override¶: Read and set hyperparameter values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::AlgorithmState::HierarchyHypers> get_hypers_proto() const override¶: Writes current value of hyperparameters to a Protobuf message and return a shared_ptr. New hierarchies have to first modify the field ‘oneof val’ in the AlgoritmState::HierarchyHypers message by adding the appropriate type

Protected Functions

virtual void initialize_hypers() override¶: Initializes hierarchy hyperparameters to appropriate values.