```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
knitr::opts_chunk$set(tidy = TRUE)
```
### Mixture models
Let us introduce a mixture of two gaussian distributions with density
\[f(x) = \omega_1 \varphi(x; \mu_1, \sigma_1^2) + \omega_2 \varphi(x; \mu_2, \sigma_2^2)\]
where $\omega_1 = 1 - \omega_2 \in [0,1]$, $\mu_1,\mu_2 \in \mathbb{R}$,
$\sigma_1^2, \sigma_2^2 \in \mathbb{R}^+$ and $\varphi$ is the Gaussian density
\[\varphi(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left\{-\frac{1}{2\sigma^2}(x-\mu)^2\right\}\]
How to draw a sample from the mixture distribution?
```{r function}
sample_mixture <- function(size, omega, mean, sd) {
# First sample indices which determine the Gaussian distribution used for each sample.
allocations <- sample(x=c(1,2), size=size, replace=TRUE, prob=omega)
# Sample normal variates using the indices into the mean and variance vectors.
# Here we are giving rnorm vectors of length n for the mean and standard deviation. In this case
# it generates each sample wit the mean and variance at the corresponding index.
rnorm(n=size, mean=mu[allocations], sd=sigma[allocations])
}
```
Let us set $\omega_1 = 0.3 = 1 - \omega_2$, $\mu_1 = -2$, $\mu_2 = 2$, $\sigma_1^2 = \sigma_2^2 = 1$ and run our sampler.
```{r run}
num_samples <- 1e4
omega <- c(0.3, 0.7)
mu <- c(-2, 2)
sigma <- c(1, 1)
samples <- sample_mixture(num_samples, omega, mu, sigma)
# Plot a histogram of the samples
library(ggplot2)
theme_set(theme_bw())
qplot(x = samples, y = ..density.., geom = "histogram", binwidth = 0.15)
```