{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# Homework 8.1: Hierarchical models are hiding in plain sight (20 pts)\n", "\n", "<hr>"]}, {"cell_type": "markdown", "metadata": {}, "source": ["Say we have a set of measurements, $\\mathbf{x} = \\{x_1, x_2,\\ldots\\}$. Each measurement has associated with is some measurement error such that $x_i \\sim \\text{Norm}(\\mu_i, s)$. Furthermore, there is a natural variability from measurement to measurement such that $\\mu_i \\sim \\text{Norm}(\\mu, \\sigma)$.\n", "\n", "**a)** Write a mathematical expression for the joint generative probability density, $\\pi(\\mathbf{x}, \\boldsymbol{\\mu}, \\mu, s, \\sigma)$, where $\\mathbf{x} = \\{x_1, x_2,\\ldots\\}$ and $\\boldsymbol{\\mu} = \\{\\mu_1, \\mu_2, \\ldots\\}$. You may assume that the prior may be separated such that $g(\\mu, \\sigma, s) = g(\\mu) g(\\sigma) g(s)$.\n", "\n", "**b)** Calculate $\\pi(\\mathbf{x}, \\mu, s, \\sigma)$. Then write this expression in the limit where the natural variability is much greater than the measurement error ($s \\ll \\sigma$). *Hint*: There is a bit of algebraic grunge to this part of the problem, and you may find it useful to [complete the square](https://en.wikipedia.org/wiki/Completing_the_square).\n", "\n", "**c)** Finally, in this limit, write the expression for $\\pi(\\mathbf{x}, \\mu, \\sigma)$ and show that for this hierarchical model, the limit of $s\\ll \\sigma$ is equivalent to having a likelihood of $x_i \\sim \\text{Norm}(\\mu, \\sigma)\\;\\forall i$.\n", "\n", "This problem shows how a hierarchical model can reduce to a non-hierarchical one. In this case, it helps you see what is being neglected when you choose not to use a hierarchical model (since so many models actually are hierarchical if you don't make approximations like the one derived here)."]}, {"cell_type": "markdown", "metadata": {}, "source": ["<br />"]}], "metadata": {"anaconda-cloud": {}, "kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"codemirror_mode": {"name": "ipython", "version": 3}, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6"}}, "nbformat": 4, "nbformat_minor": 4}