bayesmix/hierarchies

Hierarchies¶

In our algorithms, we store a vector of hierarchies, each of which represent a parameter $\theta_h$ . The hierarchy implements all the methods needed to update $\theta_h$ : sampling from the prior distribution $P_0$ , the full-conditional distribution (given the data { $y_i$ such that $c_i = h$ } ) and so on.

In BayesMix, each choice of $G_0$ is implemented in a different PriorModel object and each choice of $k(\cdot \mid \cdot)$ in a Likelihood object, so that it is straightforward to create a new Hierarchy using one of the already implemented priors or likelihoods. The sampling from the full conditional of $\theta_h$ is performed in an Updater class. State classes are used to store parameters $\theta_h$ s of every mixture component. Their main purpose is to handle serialization and de-serialization of the state

API: hierarchies submodules

Main operations performed¶

A hierarchy must be able to perform the following operations:

Sample from the prior distribution: generate $\theta_h \sim P_0$ [sample_prior]
Sample from the ‘full conditional’ distribution: generate theta_h from the distribution $p(\theta_h \mid \cdots ) \propto P_0(\theta_h) \prod_{i: c_i = h} k(y_i | \theta_h)$ [sample_full_conditional]
Update the hyperparameters involved in $P_0$ [update_hypers]
Evaluate the likelihood in one point, i.e. $k(x | \theta_h)$ for theta_h the current value of the parameters [like_lpdf]
When $k$ and $P_0$ are conjugate, we must also be able to compute the marginal/prior predictive distribution in one point, i.e. $m(x) = \int k(x | \theta) P_0(d\theta)$ , and the conditional predictive distribution $m(x | \textbf{y} ) = \int k(x | \theta) P_0(d\theta | \{y_i: c_i = h\})$ [prior_pred_lpdf, conditional_pred_lpdf]

Moreover, the following utilities are needed:

write the current state $\theta_h$ into a appropriately defined Protobuf message [write_state_to_proto]
restore theta_h from a given Protobuf message [set_state_from_proto]
write the values of the hyperparameters in $P_0$ to a Protobuf message [write_hypers_to_proto]

In each hierarchy, we also keep track of which data points are allocated to the hierarchy. For this purpose add_datum and remove_datum are employed. Finally, the update involeved in the full-conditional, especially if $P_0$ and $k$ are conjugate an semi-conjugate can be performed efficiently from a set of sufficient statistics, hence when add_datum or remove_datum are invoked, the method update_summary_statistics is called.

Code structure¶

We employ a Curiously Recurring Template Pattern (CRTP) coupled with an abstract interface, similarly to the Mixing class. The code thus composes of: a virtual class defining the API, a template base class that is the base for the CRTP and derived child classes that fully specialize the template arguments. The class AbstractHierarchy defines the API, i.e. all the methods that need to be called from outside of a Hierarchy class. A template class BaseHierarchy inherits from AbstractHierarchy and implements some of the necessary virtual methods, which need not be implemented by the child classes.

Instead, child classes must implement:

like_lpdf: evaluates $k(x | \theta_h)$
marg_lpdf: evaluates m(x) given some parameters $\theta_h$ (could be both the hyperparameters in $P_0$ or the paramters given by the full conditionals)
draw: samples from $P_0$ given the parameters
clear_summary_statistics: clears all the summary statistics
update_hypers: performs the update of parameters in $P_0$ given all the $\theta_h$ (passed as a vector of protobuf Messages)
initialize_state: initializes the current $\theta_h$ given the hyperparameters in $P_0$
initialize_hypers: initializes the hyperparameters in $P_0$ given their hyperprior
update_summary_statistics: updates the summary statistics when an observation is allocated or de-allocated from the hierarchy
get_posterior_parameters: returns the paramters of the full conditional distribution possible only when $P_0$ and $k$ are conjugate
set_state_from_proto
write_state_to_proto
write_hypers_to_proto

Note that not all of these members are declared virtual in AbstractHierarchy or BaseHierarchy: this is because virtual members are only the ones that must be called from outside the Hierarchy, the other ones are handled via CRTP. Not having them virtual saves a lot of lookups in the vtables. The BaseHierarchy class takes 4 template parameters:

Derived must be the type of the child class (needed for the CRTP)
State is usually a struct representing $\theta_h$
Hyperparams is usually a struct representing the parameters in $P_0$
Prior must be a protobuf object encoding the prior parameters.

Abstract Classes¶

class AbstractHierarchy¶

Abstract base class for a hierarchy object. This class is the basis for a curiously recurring template pattern (CRTP) for Hierarchy objects, and is solely composed of interface functions for derived classes to use. For more information about this pattern, as well the list of methods required for classes in this inheritance tree, please refer to the README.md file included in this folder.

This abstract class represents a Bayesian hierarchical model:

$x_1,\dots,x_n &\sim f(x \mid \theta) \\ \theta &\sim G$

A Hierarchy object can compute the following quantities:

the likelihood log-probability density function
the prior predictive probability: $\int_\Theta f(x \mid \theta) G(d\theta)$ (for conjugate models only)
the posterior predictive probability $\int_\Theta f(x \mid \theta) G(d\theta \mid x_1, ..., x_n)$ (for conjugate models only)

Moreover, the Hierarchy knows how to sample from the full conditional of $\theta$ , possibly in an approximate way.

In the context of our Gibbs samplers, an hierarchy represents the parameter value associated to a certain cluster, and also knows which observations are allocated to that cluster.

Moreover, hyperparameters and (possibly) hyperpriors associated to them can be shared across multiple Hierarchies objects via a shared pointer. In conjunction with a single Mixing object, a collection of Hierarchy objects completely defines a mixture model, and these two parts can be chosen independently of each other.

Communication with other classes, as well as storage of some relevant values, is performed via appropriately defined Protobuf messages (see for instance the proto/ls_state.proto and proto/hierarchy_prior.proto files) and their relative class methods.

Subclassed by BaseHierarchy< LapNIGHierarchy, LaplaceLikelihood, NxIGPriorModel >, BaseHierarchy< FAHierarchy, FALikelihood, FAPriorModel >, BaseHierarchy< NNWHierarchy, MultiNormLikelihood, NWPriorModel >, BaseHierarchy< NNxIGHierarchy, UniNormLikelihood, NxIGPriorModel >, BaseHierarchy< LinRegUniHierarchy, UniLinRegLikelihood, MNIGPriorModel >, BaseHierarchy< NNIGHierarchy, UniNormLikelihood, NIGPriorModel >, BaseHierarchy< Derived, Likelihood, PriorModel >

Public Functions

virtual void set_updater(std::shared_ptr<AbstractUpdater> updater_) = 0¶: Set the update algorithm for the current hierarchy.

virtual std::shared_ptr<AbstractLikelihood> get_likelihood() = 0¶: Returns (a pointer to) the likelihood for the current hierarchy.

virtual std::shared_ptr<AbstractPriorModel> get_prior() = 0¶: Returns (a pointer to) the prior model for the current hierarchy.

virtual ~AbstractHierarchy() = default¶: Default destructor.

virtual std::shared_ptr<AbstractHierarchy> clone() const = 0¶: Returns an independent, data-less copy of this object.

virtual std::shared_ptr<AbstractHierarchy> deep_clone() const = 0¶: Returns an independent, data-less copy of this object.

inline virtual double get_like_lpdf(const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const¶: Public wrapper for like_lpdf() methods.

inline virtual double prior_pred_lpdf(const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const¶

Evaluates the log-prior predictive distribution of data in a single point

Parameters:

datum – Point which is to be evaluated
covariate – (Optional) covariate vector associated to datum

Returns:

The evaluation of the lpdf

inline virtual double conditional_pred_lpdf(const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const¶

Evaluates the log-conditional predictive distr. of data in a single point

Parameters:

datum – Point which is to be evaluated
covariate – (Optional) covariate vector associated to datum

Returns:

The evaluation of the lpdf

virtual Eigen::VectorXd like_lpdf_grid(const Eigen::MatrixXd &data, const Eigen::MatrixXd &covariates = Eigen::MatrixXd(0, 0)) const = 0¶

Evaluates the log-likelihood of data in a grid of points

Parameters:

data – Grid of points (by row) which are to be evaluated
covariates – (Optional) covariate vectors associated to data

Returns:

The evaluation of the lpdf

inline virtual Eigen::VectorXd prior_pred_lpdf_grid(const Eigen::MatrixXd &data, const Eigen::MatrixXd &covariates = Eigen::MatrixXd(0, 0)) const¶

Evaluates the log-prior predictive distr. of data in a grid of points

Parameters:

data – Grid of points (by row) which are to be evaluated
covariates – (Optional) covariate vectors associated to data

Returns:

The evaluation of the lpdf

inline virtual Eigen::VectorXd conditional_pred_lpdf_grid(const Eigen::MatrixXd &data, const Eigen::MatrixXd &covariates = Eigen::MatrixXd(0, 0)) const¶

Evaluates the log-prior predictive distr. of data in a grid of points

Parameters:

data – Grid of points (by row) which are to be evaluated
covariates – (Optional) covariate vectors associated to data

Returns:

The evaluation of the lpdf

virtual void sample_prior() = 0¶: Generates new state values from the centering prior distribution.

virtual void sample_full_cond(const bool update_params = false) = 0¶

Generates new state values from the centering posterior distribution

Parameters:: update_params – Save posterior hypers after the computation?

virtual void sample_full_cond(const Eigen::MatrixXd &data, const Eigen::MatrixXd &covariates = Eigen::MatrixXd(0, 0)) = 0¶: Overloaded version of sample_full_cond(bool), mainly used for debugging.

virtual void update_hypers(const std::vector<bayesmix::AlgorithmState::ClusterState> &states) = 0¶: Updates hyperparameter values given a vector of cluster states.

virtual bayesmix::HierarchyId get_id() const = 0¶: Returns the Protobuf ID associated to this class.

virtual int get_card() const = 0¶: Returns the current cardinality of the cluster.

virtual double get_log_card() const = 0¶: Returns the logarithm of the current cardinality of the cluster.

virtual std::set<int> get_data_idx() const = 0¶: Returns the indexes of data points belonging to this cluster.

virtual google::protobuf::Message *get_mutable_prior() = 0¶: Returns a pointer to the Protobuf message of the prior of this cluster.

virtual std::shared_ptr<bayesmix::AlgorithmState::ClusterState> get_state_proto() const = 0¶: Writes current state to a Protobuf message and return a shared_ptr New hierarchies have to first modify the field ‘oneof val’ in the AlgoritmState::ClusterState message by adding the appropriate type

virtual void write_state_to_proto(google::protobuf::Message *const out) const = 0¶: Writes current state to a Protobuf message by pointer.

virtual void write_hypers_to_proto(google::protobuf::Message *const out) const = 0¶: Writes current hyperparameters to a Protobuf message by pointer.

virtual void set_state_from_proto(const google::protobuf::Message &state_) = 0¶: Read and set state values from a given Protobuf message.

virtual void set_hypers_from_proto(const google::protobuf::Message &state_) = 0¶: Read and set hyperparameter values from a given Protobuf message.

virtual void add_datum(const int id, const Eigen::RowVectorXd &datum, const bool update_params = false, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) = 0¶: Adds a datum and its index to the hierarchy.

virtual void remove_datum(const int id, const Eigen::RowVectorXd &datum, const bool update_params = false, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) = 0¶: Removes a datum and its index from the hierarchy.

inline void update_ss(const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate, const bool add)¶: Public wrapper for update_summary_statistics() methods.

virtual void initialize() = 0¶: Main function that initializes members to appropriate values.

virtual bool is_multivariate() const = 0¶: Returns whether the hierarchy models multivariate data or not.

virtual bool is_dependent() const = 0¶: Returns whether the hierarchy depends on covariate values or not.

virtual bool is_conjugate() const = 0¶: Returns whether the hierarchy represents a conjugate model or not.

virtual void set_dataset(const Eigen::MatrixXd *const dataset) = 0¶: Sets the (pointer to) the dataset in the cluster.

template<class Derived, class Likelihood, class PriorModel> class BaseHierarchy : public AbstractHierarchy ¶

Base template class for a hierarchy object.

This class is a templatized version of, and derived from, the AbstractHierarchy class, and the second stage of the curiously recurring template pattern for Hierarchy objects (please see the docs of the parent class for further information). It includes class members and some more functions which could not be implemented in the non-templatized abstract class. See, for instance, NNIGHierarchy to better understand the CRTP patterns.

Template Parameters:

Derived – Name of the implemented derived class
Likelihood – Class name of the likelihood model for the hierarchy
PriorModel – Class name of the prior model for the hierarchy

Public Functions

inline BaseHierarchy(std::shared_ptr<AbstractLikelihood> like_ = nullptr, std::shared_ptr<AbstractPriorModel> prior_ = nullptr, std::shared_ptr<AbstractUpdater> updater_ = nullptr)¶: Constructor that allows the specification of Likelihood, PriorModel and Updater for a given Hierarchy

~BaseHierarchy() = default¶: Default destructor.

inline void set_likelihood(std::shared_ptr<AbstractLikelihood> like_)¶: Sets the likelihood for the current hierarchy.

inline void set_prior(std::shared_ptr<AbstractPriorModel> prior_)¶: Sets the prior model for the current hierarchy.

inline virtual void set_updater(std::shared_ptr<AbstractUpdater> updater_) override¶: Sets the update algorithm for the current hierarchy.

inline virtual std::shared_ptr<AbstractLikelihood> get_likelihood() override¶: Returns (a pointer to) the likelihood for the current hierarchy.

inline virtual std::shared_ptr<AbstractPriorModel> get_prior() override¶: Returns (a pointer to) the prior model for the current hierarchy.

inline virtual std::shared_ptr<AbstractHierarchy> clone() const override¶: Returns an independent, data-less copy of this object.

inline virtual std::shared_ptr<AbstractHierarchy> deep_clone() const override¶: Returns an independent, data-less deep copy of this object.

inline virtual double get_like_lpdf(const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const override¶: Public wrapper for like_lpdf() methods.

inline virtual Eigen::VectorXd like_lpdf_grid(const Eigen::MatrixXd &data, const Eigen::MatrixXd &covariates = Eigen::MatrixXd(0, 0)) const override¶

Evaluates the log-likelihood of data in a grid of points

Parameters:

data – Grid of points (by row) which are to be evaluated
covariates – (Optional) covariate vectors associated to data

Returns:

The evaluation of the lpdf

inline double get_marg_lpdf(ProtoHypersPtr hier_params, const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate) const¶: Public wrapper for marg_lpdf() methods.

inline virtual double prior_pred_lpdf(const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const override¶

Evaluates the log-prior predictive distribution of data in a single point

Parameters:

datum – Point which is to be evaluated
covariate – (Optional) covariate vector associated to datum

Returns:

The evaluation of the lpdf

inline virtual Eigen::VectorXd prior_pred_lpdf_grid(const Eigen::MatrixXd &data, const Eigen::MatrixXd &covariates) const override¶

Evaluates the log-prior predictive distr. of data in a grid of points

Parameters:

data – Grid of points (by row) which are to be evaluated
covariates – (Optional) covariate vectors associated to data

Returns:

The evaluation of the lpdf

inline virtual double conditional_pred_lpdf(const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const override¶

Evaluates the log-conditional predictive distr. of data in a single point

Parameters:

datum – Point which is to be evaluated
covariate – (Optional) covariate vector associated to datum

Returns:

The evaluation of the lpdf

inline virtual Eigen::VectorXd conditional_pred_lpdf_grid(const Eigen::MatrixXd &data, const Eigen::MatrixXd &covariates) const override¶

Evaluates the log-prior predictive distr. of data in a grid of points

Parameters:

data – Grid of points (by row) which are to be evaluated
covariates – (Optional) covariate vectors associated to data

Returns:

The evaluation of the lpdf

inline virtual void sample_prior() override¶: Generates new state values from the centering prior distribution.

inline virtual void sample_full_cond(bool update_params = false) override¶

Generates new state values from the centering posterior distribution

Parameters:: update_params – Save posterior hypers after the computation?

inline virtual void sample_full_cond(const Eigen::MatrixXd &data, const Eigen::MatrixXd &covariates = Eigen::MatrixXd(0, 0)) override¶: Overloaded version of sample_full_cond(bool), mainly used for debugging.

inline virtual void update_hypers(const std::vector<bayesmix::AlgorithmState::ClusterState> &states) override¶: Updates hyperparameter values given a vector of cluster states.

inline auto get_state() const -> decltype(like->get_state())¶: Returns the class of the current state.

inline virtual int get_card() const override¶: Returns the current cardinality of the cluster.

inline virtual double get_log_card() const override¶: Returns the logarithm of the current cardinality of the cluster.

inline virtual std::set<int> get_data_idx() const override¶: Returns the indexes of data points belonging to this cluster.

inline virtual std::shared_ptr<bayesmix::AlgorithmState::ClusterState> get_state_proto() const override¶: Writes current state to a Protobuf message and return a shared_ptr New hierarchies have to first modify the field ‘oneof val’ in the AlgoritmState::ClusterState message by adding the appropriate type

inline virtual google::protobuf::Message *get_mutable_prior() override¶: Returns a pointer to the Protobuf message of the prior of this cluster.

inline virtual void write_state_to_proto(google::protobuf::Message *out) const override¶: Writes current state to a Protobuf message by pointer.

inline virtual void write_hypers_to_proto(google::protobuf::Message *out) const override¶: Writes current values of the hyperparameters to a Protobuf message by pointer

inline virtual void set_state_from_proto(const google::protobuf::Message &state_) override¶: Read and set state values from a given Protobuf message.

inline virtual void set_hypers_from_proto(const google::protobuf::Message &state_) override¶: Read and set hyperparameter values from a given Protobuf message.

inline virtual void add_datum(const int id, const Eigen::RowVectorXd &datum, const bool update_params = false, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) override¶: Adds a datum and its index to the hierarchy.

inline virtual void remove_datum(const int id, const Eigen::RowVectorXd &datum, const bool update_params = false, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) override¶: Removes a datum and its index from the hierarchy.

inline virtual void initialize() override¶: Main function that initializes members to appropriate values.

inline virtual bool is_multivariate() const override¶: Returns whether the hierarchy models multivariate data or not.

inline virtual bool is_dependent() const override¶: Returns whether the hierarchy depends on covariate values or not.

inline virtual bool is_conjugate() const override¶: Returns whether the hierarchy represents a conjugate model or not.

inline virtual void set_dataset(const Eigen::MatrixXd *const dataset) override¶: Sets the (pointer to the) dataset matrix.

Classes for Conjugate Hierarchies¶

class NNIGHierarchy : public BaseHierarchy<NNIGHierarchy, UniNormLikelihood, NIGPriorModel>¶

Conjugate Normal Normal-InverseGamma hierarchy for univariate data.

This class represents a hierarchical model where data are distributed according to a Normal likelihood (see the UniNormLikelihood class for details). The likelihood parameters have a Normal-InverseGamma centering distribution (see the NIGPriorModel class for details). That is:

$f(x_i \mid \mu, \sigma^2) &= N(\mu,\sigma^2) \\ (\mu,\sigma^2) & \sim NIG(\mu_0, \lambda_0, \alpha_0, \beta_0)$

The state is composed of mean and variance. The state hyperparameters are $(\mu_0, \lambda_0, \alpha_0, \beta_0)$ , all scalar values. Note that this hierarchy is conjugate, thus the marginal distribution is available in closed form

Public Functions

inline virtual bayesmix::HierarchyId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline void set_default_updater()¶: Sets the default updater algorithm for this hierarchy.

inline virtual void initialize_state() override¶: Initializes state parameters to appropriate values.

inline virtual double marg_lpdf(ProtoHypersPtr hier_params, const Eigen::RowVectorXd &datum) const override¶

Evaluates the log-marginal distribution of data in a single point

Parameters:

hier_params – Pointer to the container of (prior or posterior) hyperparameter values
datum – Point which is to be evaluated

Returns:

The evaluation of the lpdf

class NNWHierarchy : public BaseHierarchy<NNWHierarchy, MultiNormLikelihood, NWPriorModel>¶

Normal Normal-Wishart hierarchy for multivariate data.

This class represents a hierarchy whose multivariate data are distributed according to a multivariate normal likelihood (see the MultiNormLikelihood for details). The likelihood parameters have a Normal-Wishart centering distribution (see the NWPriorModel class for details). That is:

$f(\bm{x}_i \mid \bm{\mu},\Sigma) &= N_d(\bm{\mu},\Sigma^{-1}) \\ (\bm{\mu},\Sigma) &\sim NW(\mu_0, \lambda, \Psi_0, \nu_0)$

The state is composed of mean and precision matrix. The Cholesky factor and log-determinant of the latter are also included in the container for efficiency reasons. The state’s hyperparameters are $(\mu_0, \lambda, \Psi_0, \nu_0)$ , which are respectively vector, scalar, matrix, and scalar. Note that this hierarchy is conjugate, thus the marginal distribution is available in closed form

Public Functions

inline virtual bayesmix::HierarchyId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline void set_default_updater()¶: Sets the default updater algorithm for this hierarchy.

inline virtual void initialize_state() override¶: Initializes state parameters to appropriate values.

inline virtual double marg_lpdf(ProtoHypersPtr hier_params, const Eigen::RowVectorXd &datum) const override¶

Evaluates the log-marginal distribution of data in a single point

Parameters:

hier_params – Pointer to the container of (prior or posterior) hyperparameter values
datum – Point which is to be evaluated

Returns:

The evaluation of the lpdf

inline HyperParams get_predictive_t_parameters(ProtoHypersPtr hier_params) const¶

Helper function that computes the predictive parameters for the multivariate t distribution from the current hyperparameter values. It is used to efficiently compute the log-marginal distribution of data.

Parameters:: hier_params – Pointer to the container of (prior or posterior) hyperparameter values
Returns:: A HyperParam object with the predictive parameters

class LinRegUniHierarchy : public BaseHierarchy<LinRegUniHierarchy, UniLinRegLikelihood, MNIGPriorModel>¶

Linear regression hierarchy for univariate data.

This class implements a dependent hierarchy which represents the classical univariate Bayesian linear regression model, i.e.:

$f(y_i \mid \bm{x}_i,\mu,\sigma^2) &= N(\bm{\beta}^T \bm{x}_i, \sigma^2) \\ \bm{\beta} \mid \sigma^2 &\sim N_p(\bm{\mu}, \sigma^2 \Lambda^{-1}) \\ \sigma^2 &\sim InvGamma(a, b)$

The state consists of the regression_coeffs $\bm{\beta}$ , and the var $\sigma^2$ . $\Lambda$ is called the variance-scaling factor. Note that this hierarchy is conjugate, thus the marginal distribution is available in closed form. For more information, please refer to the parent class BaseHierarchy, to the class UniLinRegLikelihood for details on the likelihood model and to MNIGPriorModel for details on the prior model.

Public Functions

inline virtual bayesmix::HierarchyId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline void set_default_updater()¶: Sets the default updater algorithm for this hierarchy.

inline virtual void initialize_state() override¶: Initializes state parameters to appropriate values.

inline virtual double marg_lpdf(ProtoHypersPtr hier_params, const Eigen::RowVectorXd &datum, const Eigen::RowVectorXd &covariate) const override¶

Evaluates the log-marginal distribution of data in a single point

Parameters:

hier_params – Pointer to the container of (prior or posterior) hyperparameter values
datum – Point which is to be evaluated
covariate – Covariate vectors associated to data

Returns:

The evaluation of the lpdf

Classes for Non-Conjugate Hierarchies¶

class NNxIGHierarchy : public BaseHierarchy<NNxIGHierarchy, UniNormLikelihood, NxIGPriorModel>¶

Semi-conjugate Normal Normal x InverseGamma hierarchy for univariate data.

This class represents a hierarchical model where data are distributed according to a Normal likelihood (see the UniNormLikelihood class for details). The likelihood parameters have a Normal x InverseGamma centering distribution (see the NxIGPriorModel class for details). That is:

$f(x_i \mid \mu,\sigma^2) &= N(\mu,\sigma^2) \\ \mu &\sim N(\mu_0, \eta^2) \\ \sigma^2 &\sim InvGamma(a, b)$

The state is composed of mean and variance. The state hyperparameters are $(\mu_0, \eta^2, a, b)$ , all scalar values. Note that this hierarchy is NOT conjugate, meaning that the marginal distribution is not available in closed form

Public Functions

inline virtual bayesmix::HierarchyId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline void set_default_updater()¶: Sets the default updater algorithm for this hierarchy.

inline virtual void initialize_state() override¶: Initializes state parameters to appropriate values.

class LapNIGHierarchy : public BaseHierarchy<LapNIGHierarchy, LaplaceLikelihood, NxIGPriorModel>¶

Laplace Normal-InverseGamma hierarchy for univariate data.

This class represents a hierarchical model where data are distributed according to a Laplace likelihood (see the LaplaceLikelihood class for deatils). The likelihood parameters have a Normal x InverseGamma centering distribution (see the NxIGPriorModel class for details). That is:

$f(x_i \mid \mu,\sigma^2) &= Laplace(\mu,\sqrt{\sigma^2/2})\\ \mu &\sim N(\mu_0,\eta^2) \\ \sigma^2 &\sim InvGamma(a, b)$

The state is composed of mean and variance (thus the scale for the Laplace distribution is $\sqrt{\sigma^2/2})$ . The state hyperparameters are $(mu_0, \sigma^2, a, b)$ , all scalar values. Note that this hierarchy is NOT conjugate, thus the marginal distribution is not available in closed form.

Public Functions

inline virtual bayesmix::HierarchyId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline void set_default_updater()¶: Sets the default updater algorithm for this hierarchy.

inline virtual void initialize_state() override¶: Initializes state parameters to appropriate values.

class FAHierarchy : public BaseHierarchy<FAHierarchy, FALikelihood, FAPriorModel>¶

Mixture of Factor Analysers hierarchy for multivariate data.

This class represents a hierarchical model where data are distributed according to a multivariate Normal likelihood with a specific factorization of the covariance matrix (see the FAHierarchy class for details). The likelihood parameters have a Dirichlet-Laplace distribution x InverseGamma centering distribution (see the FAPriorModel class for details). That is:

$f(x_i \mid \mu, \Sigma, \Lambda) &= N(\mu, \Sigma + \Lambda \Lambda^T) \\ \mu &\sim N_p(\tilde \mu, \psi I) \\ \Lambda &\sim DL(\alpha) \\ \Sigma &= diag(\sigma^2_1, \ldots, \sigma^2_p) \\ \sigma^2_j &\sim IG(a,b) \quad j=1,...,p$

where Lambda is the latent score matrix (size $p \times d$ with $d << p$ ) and $DL(\alpha)$ is the Laplace-Dirichlet distribution. See Bhattacharya et al. (2015) for further details

Public Functions

inline virtual bayesmix::HierarchyId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline void set_default_updater()¶: Sets the default updater algorithm for this hierarchy.

inline virtual void initialize_state() override¶: Initializes state parameters to appropriate values.