(c) 2017 Justin Bois. This work is licensed under a Creative Commons Attribution License CC-BY 4.0. All code contained therein is licensed under an MIT license.
This homework was generated from an Jupyter notebook. You can download the notebook here.
Consider the following hierarchical model.
\begin{align} \phi &\sim \text{To be specified} \\[1em] \tau &\sim \text{To be specified} \\[1em] \theta_i &\sim \text{Norm}(\phi, \tau) \;\;\forall i \in \{1, 2, 3, \ldots k\} \\[1em] x_{ij} &\sim \text{Norm}(\theta_i, \sigma) \;\;\forall i, j. \end{align}
This is a typical two-level hierarchical model for repeated experiments. We may wish to choose uninformative priors the hyperparameters for $\phi$ and $\tau$. We will assume $\phi$ has a Uniform prior, which we will take to be improper, meaning that we just specify $g(\phi) = \text{constant}$.
a) Show that the posterior is improper, that is, cannot be normalized, if we choose an unbounded Jeffreys prior for $\tau$, $g(\tau) \propto 1/\tau$. Note that we have not specifies a prior for $\sigma$. For simplicity, you may assume it is known, or you may assume a Jeffreys prior.
b) Show that the posterior is proper if we have a Uniform prior for $\tau$ ($g(\tau) = \text{constant}$) for some values of $k$, the number of repeats of the experiment. Be sure to indicate for which values of $k$ the posterior is proper.
c) What would be a reasonable modeling strategy in the case where $k$ does not meet the requirements you laid out in part (b)?
Do part (c) of Problem 9.2 using a variational Bayes method. Comment on any difficulties you encounter and the differences between what you get with the variational method and full MCMC and why they might arise. You should do this for a hierarchical model. It might be useful to refer to auxiliary lesson 9.