Let us introduce a mixture of two gaussian distributions with density \[f(x) = \omega_1 \varphi(x; \mu_1, \sigma_1^2) + \omega_2 \varphi(x; \mu_2, \sigma_2^2)\] where \(\omega_1 = 1 - \omega_2 \in [0,1]\), \(\mu_1,\mu_2 \in \mathbb{R}\), \(\sigma_1^2, \sigma_2^2 \in \mathbb{R}^+\) and \(\varphi\) is the Gaussian density \[\varphi(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left\{-\frac{1}{2\sigma^2}(x-\mu)^2\right\}\]
How to draw a sample from the mixture distribution?
sample_mixture <- function(size, omega, mean, sd) {
# First sample indices which determine the Gaussian distribution used for
# each sample.
allocations <- sample(x = c(1, 2), size = size, replace = TRUE, prob = omega)
# Sample normal variates using the indices into the mean and variance
# vectors. Here we are giving rnorm vectors of length n for the mean and
# standard deviation. In this case it generates each sample wit the mean and
# variance at the corresponding index.
rnorm(n = size, mean = mu[allocations], sd = sigma[allocations])
}
Let us set \(\omega_1 = 0.3 = 1 - \omega_2\), \(\mu_1 = -2\), \(\mu_2 = 2\), \(\sigma_1^2 = \sigma_2^2 = 1\) and run our sampler.
num_samples <- 10000
omega <- c(0.3, 0.7)
mu <- c(-2, 2)
sigma <- c(1, 1)
samples <- sample_mixture(num_samples, omega, mu, sigma)
# Plot a histogram of the samples
library(ggplot2)
theme_set(theme_bw())
qplot(x = samples, y = ..density.., geom = "histogram", binwidth = 0.15)