bayesmix/mixings

Mixings¶

In the algorithms of the library, we store a single Mixing object that represents a prior for the mixing weights for the mixture models and the induced exchangeable partition probability function (EPPF). There are two types of Mixing: marginal and conditional, each of which can only be used with the matching type of Algorithm. For both of these types, certain API functions are required.

Code structure¶

We employ a Curiously Recurring Template Pattern coupled with an abstract interface, similarly to the Hierarchy class (see here). The code thus composes of: a virtual class defining the API, a template base class that is the base for the CRTP and derived child classes that fully specialize the template arguments. The class AbstractMixing defines the API, i.e. all the methods that need to be called from outside of a Mixing class. A template class BaseMixing inherits from AbstractMixing and implements some of the necessary virtual methods, which need not be implemented by the child classes.

Classes¶

class AbstractMixing¶

Abstract base class for a generic mixture model

This class is the basis for a curiously recurring template pattern (CRTP) for Mixing objects, and is solely composed of interface functions for derived classes to use, similarly to how the Hierarchy objects were implemented. For more information about this pattern, please refer to the README.md file included in the hierarchies subfolder. This class represents a prior for the mixture weights and the induced exchangeable partition probability function (EPPF). See ConditionalAlgorithm and MarginalAlgorithm for further details.

There are two kinds of Mixing objects: marginal and conditional mixings. Any class inheriting from this one must implement the is_conditional() flag accordingly, and can only be used with the same type of Algorithm object. In a conditional mixing, mixing weights for the clusters are part of the state of the algorithm. Their values are stored in some form in this class, and they can be obtained by calling the get_mixing_weights() method. In a marginal mixing, the actual mixing weights have been marginalized out of the model, and information related to them translates to probability masses to assign a data point to an existing cluster, or to a new one. According to the type of mixing which is being implemented, classes inheriting from this one must either implement get_weights(), or both mass_existing_cluster() and mass_new_cluster() methods. Each of these methods has a version with covariates for dependent mixings and one without covariates; please implement the ones that reflect your mixing type. Other required methods are update_state() for a conditional update of the mixing state (if any) given allocations and unique values coming from the library Algorithm classes, and read-write methods involving Protobuf objects.

Subclassed by BaseMixing< TruncatedSBMixing, TruncSB::State, bayesmix::TruncSBPrior >, BaseMixing< PitYorMixing, PitYor::State, bayesmix::PYPrior >, BaseMixing< MixtureFiniteMixing, Mixture_Finite::State, bayesmix::MFMPrior >, BaseMixing< DirichletMixing, Dirichlet::State, bayesmix::DPPrior >, BaseMixing< LogitSBMixing, LogitSB::State, bayesmix::LogSBPrior >, BaseMixing< Derived, State, Prior >

Public Functions

virtual void update_state(const std::vector<std::shared_ptr<AbstractHierarchy>> &unique_values, const std::vector<unsigned int> &allocations) = 0¶

Performs conditional update of state, given allocations and unique values

Parameters:

unique_values – A vector of (pointers to) Hierarchy objects
allocations – A vector of allocations label

inline Eigen::VectorXd get_mixing_weights(const bool log, const bool propto, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const¶: Public wrapper for mixing_weights() methods.

inline double get_mass_existing_cluster(const unsigned int n, const unsigned int n_clust, const bool log, const bool propto, const std::shared_ptr<AbstractHierarchy> hier, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const¶: Public wrapper for mass_existing_cluster() methods.

inline double get_mass_new_cluster(const unsigned int n, const unsigned int n_clust, const bool log, const bool propto, const Eigen::RowVectorXd &covariate = Eigen::RowVectorXd(0)) const¶: Public wrapper for mass_new_cluster() methods.

virtual unsigned int get_num_components() const = 0¶: Returns current number of clusters of the mixture model.

virtual google::protobuf::Message *get_mutable_prior() = 0¶: Returns a pointer to the Protobuf message of the prior of this cluster.

virtual void set_num_components(const unsigned int num_) = 0¶: Sets current number of clusters of the mixture model.

virtual void set_covariates(Eigen::MatrixXd *const covar) = 0¶: Sets pointer to the covariate matrix for the mixture model.

virtual void set_state_from_proto(const google::protobuf::Message &state_) = 0¶: Read and set state values from a given Protobuf message.

virtual void write_state_to_proto(google::protobuf::Message *const out) const = 0¶: Writes current state to a Protobuf message by pointer.

virtual bayesmix::MixingId get_id() const = 0¶: Returns the Protobuf ID associated to this class.

virtual void initialize() = 0¶: Main function that initializes members to appropriate values.

virtual bool is_conditional() const = 0¶: Returns whether the mixing is conditional or marginal.

inline virtual bool is_dependent() const¶: Returns whether the mixing depends on covariate values or not.

template<class Derived, typename State, typename Prior> class BaseMixing : public AbstractMixing ¶

Base template class for a mixing object.

This class is a templatized version of, and derived from, the AbstractMixing class, and the second stage of the curiously recurring template pattern for Mixing objects (please see the file of the parent class for further information). It includes class members and some more functions which could not be implemented in the non-templatized abstract class. When deriving a class from BaseMixing, its own name must be passed to the first template argument, and custom containers for state and prior values must be provided in the remaining ones.

Template Parameters:

Derived – Name of the implemented derived class
State – Class name of the container for state values
Prior – Class name of the container for prior parameters

Public Functions

inline State get_state() const¶: Returns the struct of the current state.

inline virtual unsigned int get_num_components() const override¶: Returns the current number of clusters of the mixture model.

virtual google::protobuf::Message *get_mutable_prior() override¶: Returns a pointer to the Protobuf message of the prior of this cluster.

inline virtual void set_num_components(const unsigned int num_) override¶: Sets the current number of clusters of the mixture model.

inline virtual void set_covariates(Eigen::MatrixXd *const covar) override¶: Sets the (pointer to the) covariate matrix for the mixture model.

virtual void write_state_to_proto(google::protobuf::Message *const out) const override¶: Writes current state to a Protobuf message by pointer.

virtual std::shared_ptr<bayesmix::MixingState> get_state_proto() const = 0¶: Writes current state to a Protobuf message and return a shared_ptr New hierarchies have to first modify the field ‘oneof val’ in the MixingState message by adding the appropriate type

virtual void initialize() override¶: Main function that initializes members to appropriate values.

class DirichletMixing : public BaseMixing<DirichletMixing, Dirichlet::State, bayesmix::DPPrior>¶

Class that represents the EPPF induced by the Dirithclet process (DP) introduced in Ferguson (1973), see also Sethuraman (1994). The EPPF induced by the DP depends on a totalmass parameter M. Given a clustering of n elements into k clusters, each with cardinality $n_j, j=1, ..., k$ the EPPF of the DP gives the following probabilities for the cluster membership of the (n+1)-th observation:

$p(\text{j-th cluster} | ...) &= n_j / (n + M) \\ p(\text{new cluster} | ...) &= M / (n + M)$

The state is solely composed of M, but we also store log(M) for efficiency reasons. For more information about the class, please refer instead to base classes, AbstractMixing and BaseMixing.

Public Functions

virtual void update_state(const std::vector<std::shared_ptr<AbstractHierarchy>> &unique_values, const std::vector<unsigned int> &allocations) override¶

Performs conditional update of state, given allocations and unique values

Parameters:

unique_values – A vector of (pointers to) Hierarchy objects
allocations – A vector of allocations label

virtual void set_state_from_proto(const google::protobuf::Message &state_) override¶: Read and set state values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::MixingState> get_state_proto() const override¶: Writes current state to a Protobuf message and return a shared_ptr New hierarchies have to first modify the field ‘oneof val’ in the MixingState message by adding the appropriate type

inline virtual bayesmix::MixingId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline virtual bool is_conditional() const override¶: Returns whether the mixing is conditional or marginal.

class PitYorMixing : public BaseMixing<PitYorMixing, PitYor::State, bayesmix::PYPrior>¶

Class that represents the Pitman-Yor process (PY) in Pitman and Yor (1997). The EPPF induced by the PY depends on a strength parameter M and a discount paramter d. Given a clustering of n elements into k clusters, each with cardinality $n_j, j=1, ..., k$ , the EPPF of the PY gives the following probabilities for the cluster membership of the (n+1)-th observation:

$p(\text{j-th cluster} | ...) \propto (n_j - d) \\ p(\text{new cluster} | ...) \propto M + k d$

When discount=0, the EPPF of the PY process coincides with the one of the DP with totalmass = strength. For more information about the class, please refer instead to base classes, AbstractMixing and BaseMixing.

Public Functions

virtual void update_state(const std::vector<std::shared_ptr<AbstractHierarchy>> &unique_values, const std::vector<unsigned int> &allocations) override¶

Performs conditional update of state, given allocations and unique values

Parameters:

unique_values – A vector of (pointers to) Hierarchy objects
allocations – A vector of allocations label

virtual void set_state_from_proto(const google::protobuf::Message &state_) override¶: Read and set state values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::MixingState> get_state_proto() const override¶: Writes current state to a Protobuf message and return a shared_ptr New hierarchies have to first modify the field ‘oneof val’ in the MixingState message by adding the appropriate type

inline virtual bayesmix::MixingId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline virtual bool is_conditional() const override¶: Returns whether the mixing is conditional or marginal.

class MixtureFiniteMixing : public BaseMixing<MixtureFiniteMixing, Mixture_Finite::State, bayesmix::MFMPrior>¶

Class that represents the Mixture of Finite Mixtures (MFM) [1] The basic idea is to take usual finite mixture model with Dirichlet weights and put a prior (Poisson) on the number of components. The EPPF induced by MFM depends on a Dirichlet parameter ‘gamma’ and number $V_n(t)$ , where $V_n(t)$ depends on the Poisson rate parameter ‘lambda’.

$V_n(t) = \sum_{k=1}^{\infty} ( k_(t)p_K(k) / (\gamma*k)^(n) )$

Given a clustering of n elements into k clusters, each with cardinality $n_j, j=1, ..., k$ , the EPPF of the MFM gives the following probabilities for the cluster membership of the (n+1)-th observation:

$p(\text{j-th cluster} | ...) &= (n_j + \gamma) / D \\ p(\text{k+1-th cluster} | ...) &= V[k+1]/V[k] \gamma / D \\ D &= n_j + \gamma / (n + \gamma * (k + V[k+1]/V[k]))$

For numerical reasons each value of V is multiplied with a constant C computed as the first term of the series of V_n[0]. For more information about the class, please refer instead to base classes, AbstractMixing and BaseMixing.

[1] “Mixture Models with a Prior on the Number of Components”, J.W.Miller and M.T.Harrison, 2015, arXiv:1502.06241v1

Public Functions

virtual void update_state(const std::vector<std::shared_ptr<AbstractHierarchy>> &unique_values, const std::vector<unsigned int> &allocations) override¶

Performs conditional update of state, given allocations and unique values

Parameters:

unique_values – A vector of (pointers to) Hierarchy objects
allocations – A vector of allocations label

virtual void set_state_from_proto(const google::protobuf::Message &state_) override¶: Read and set state values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::MixingState> get_state_proto() const override¶: Writes current state to a Protobuf message and return a shared_ptr New hierarchies have to first modify the field ‘oneof val’ in the MixingState message by adding the appropriate type

inline virtual bayesmix::MixingId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline virtual bool is_conditional() const override¶: Returns whether the mixing is conditional or marginal.

class TruncatedSBMixing : public BaseMixing<TruncatedSBMixing, TruncSB::State, bayesmix::TruncSBPrior>¶

Class that represents a truncated stick-breaking process, as shown in Ishwaran and James (2001).

A truncated stick-breaking process is a prior for weights $(w_1,...,w_H)$ in the H-1 dimensional unit simplex, and is defined as follows:

$w_1 &= v_1 \\ w_j &= v_j (1 - v_1) ... (1 - v_{j-1}), \quad \text{for } j=1, ... H-1 \\ w_H &= 1 - (w_1 + w_2 + ... + w_{H-1})$

The $v_j$ ‘s are called sticks and we assume them to be independently distributed as $v_j \sim \text{Beta}(a_j, b_j)$ .

When $a_j = 1$ and $b_j = M$ , the stick-breaking process is a truncation of the stick-breaking representation of the DP. When $a_j = 1-d$ and $b_j = M+id$ , it is the trunctation of a PY process. Its state is composed of the weights $w_j$ in log-scale and the sticks $v_j$ . For more information about the class, please refer instead to base classes, AbstractMixing and BaseMixing.

Public Functions

virtual void update_state(const std::vector<std::shared_ptr<AbstractHierarchy>> &unique_values, const std::vector<unsigned int> &allocations) override¶

Performs conditional update of state, given allocations and unique values

Parameters:

unique_values – A vector of (pointers to) Hierarchy objects
allocations – A vector of allocations label

virtual void set_state_from_proto(const google::protobuf::Message &state_) override¶: Read and set state values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::MixingState> get_state_proto() const override¶: Writes current state to a Protobuf message and return a shared_ptr New hierarchies have to first modify the field ‘oneof val’ in the MixingState message by adding the appropriate type

inline virtual bayesmix::MixingId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline virtual bool is_conditional() const override¶: Returns whether the mixing is conditional or marginal.

Eigen::MatrixXd get_prior_shape_parameters() const¶: Returns the prior shape parameters of the Beta-distributed sticks.

double keep_breaking(int num_sticks)¶: Adds num_sticks sticks to the state by keep breaking returns the sum of the new weights

class LogitSBMixing : public BaseMixing<LogitSBMixing, LogitSB::State, bayesmix::LogSBPrior>¶

Class that represents the logit stick-breaking process indroduced in Rigon and Durante (2020). That is, a prior for weights $(w_1,\dots,w_H)$ , depending on covariates $x$ in $\mathbb{R}^p$ , in the H-1 dimensional unit simplex, defined as follows:

$w_1 &= v_1 \\ w_j &= v_j (1 - v_1) ... (1 - v_{j-1}), \quad \text{for } j=1, ... H-1 \\ w_H &= 1 - (w_1 + w_2 + ... + w_{H-1}) \\ v_j(x) &= 1 / exp(- <\alpha_j, x> ), for j = 1, ..., H-1$

The main difference with the mentioned paper is that the authors propose a Gibbs sampler in which the full conditionals are available in close form thanks to a Polya-Gamma augmentation. Here instead, a Metropolis-adjusted Langevin algorithm (MALA) step is used. The step-size of the MALA step must be passed in the LogSBPrior Protobuf message. For more information about the class, please refer instead to base classes, AbstractMixing and BaseMixing.

Public Functions

virtual void update_state(const std::vector<std::shared_ptr<AbstractHierarchy>> &unique_values, const std::vector<unsigned int> &allocations) override¶

Performs conditional update of state, given allocations and unique values

Parameters:

unique_values – A vector of (pointers to) Hierarchy objects
allocations – A vector of allocations label

virtual void set_state_from_proto(const google::protobuf::Message &state_) override¶: Read and set state values from a given Protobuf message.

virtual std::shared_ptr<bayesmix::MixingState> get_state_proto() const override¶: Writes current state to a Protobuf message and return a shared_ptr New hierarchies have to first modify the field ‘oneof val’ in the MixingState message by adding the appropriate type

inline virtual bayesmix::MixingId get_id() const override¶: Returns the Protobuf ID associated to this class.

inline Eigen::VectorXd get_acceptance_rates() const¶: Returns the normalized acceptance rates of the Metropolis steps.

inline virtual bool is_conditional() const override¶: Returns whether the mixing is conditional or marginal.

inline virtual bool is_dependent() const override¶: Returns whether the mixing depends on covariate values or not.