Homework 8.1: Hierarchical models are hiding in plain sight (20 pts)

---

Say we have a set of measurements, \(\mathbf{x} = \{x_1, x_2,\ldots\}\). Each measurement has associated with is some measurement error such that \(x_i \sim \text{Norm}(\mu_i, s)\). Furthermore, there is a natural variability from measurement to measurement such that \(\mu_i \sim \text{Norm}(\mu, \sigma)\).

a) Write a mathematical expression for the joint generative probability density, \(\pi(\mathbf{x}, \boldsymbol{\mu}, \mu, s, \sigma)\), where \(\mathbf{x} = \{x_1, x_2,\ldots\}\) and \(\boldsymbol{\mu} = \{\mu_1, \mu_2, \ldots\}\). You may assume that the prior may be separated such that \(g(\mu, \sigma, s) = g(\mu) g(\sigma) g(s)\).

b) Calculate \(\pi(\mathbf{x}, \mu, s, \sigma)\). Then write this expression in the limit where the natural variability is much greater than the measurement error (\(s \ll \sigma\)). Hint: There is a bit of algebraic grunge to this part of the problem, and you may find it useful to complete the square.

c) Finally, in this limit, write the expression for \(\pi(\mathbf{x}, \mu, \sigma)\) and show that for this hierarchical model, the limit of \(s\ll \sigma\) is equivalent to having a likelihood of \(x_i \sim \text{Norm}(\mu, \sigma)\;\forall i\).

This problem shows how a hierarchical model can reduce to a non-hierarchical one. In this case, it helps you see what is being neglected when you choose not to use a hierarchical model (since so many models actually are hierarchical if you don’t make approximations like the one derived here).