```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
knitr::opts_chunk$set(tidy = TRUE)
```

### Mixture models

Let us introduce a mixture of two gaussian distributions with density
\[f(x) =  \omega_1 \varphi(x; \mu_1, \sigma_1^2) +  \omega_2 \varphi(x; \mu_2, \sigma_2^2)\]
where $\omega_1 = 1 - \omega_2 \in [0,1]$, $\mu_1,\mu_2 \in \mathbb{R}$, 
$\sigma_1^2, \sigma_2^2 \in \mathbb{R}^+$ and $\varphi$ is the Gaussian density
\[\varphi(x; \mu, \sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left\{-\frac{1}{2\sigma^2}(x-\mu)^2\right\}\]

How to draw a sample from the mixture distribution?
```{r function}
sample_mixture <- function(size, omega, mean, sd) {
  # First sample indices which determine the Gaussian distribution used for each sample.
  allocations <- sample(x=c(1,2), size=size, replace=TRUE, prob=omega)

  # Sample normal variates using the indices into the mean and variance vectors.
  # Here we are giving rnorm vectors of length n for the mean and standard deviation. In this case
  # it generates each sample wit the mean and variance at the corresponding index.
  rnorm(n=size, mean=mu[allocations], sd=sigma[allocations])
}
```

Let us set $\omega_1 = 0.3 = 1 - \omega_2$, $\mu_1 = -2$, $\mu_2 = 2$, $\sigma_1^2 = \sigma_2^2 = 1$ and run our sampler.
```{r run}
num_samples <- 1e4
omega <- c(0.3, 0.7)
mu <- c(-2, 2)
sigma <- c(1, 1)

samples <- sample_mixture(num_samples, omega, mu, sigma)

# Plot a histogram of the samples
library(ggplot2)
theme_set(theme_bw())
qplot(x = samples, y = ..density.., geom = "histogram", binwidth = 0.15)
```
