added proof to delta method in stats/delta-method-demo.ipynb

zyxue · Jul 8, 2024 · f99973a · f99973a
1 parent e676365
commit f99973a
Showing 1 changed file with 27 additions and 1 deletion.
diff --git a/stats/delta-method-demo.ipynb b/stats/delta-method-demo.ipynb
@@ -41,7 +41,33 @@
    "id": "641cc613",
    "metadata": {},
    "source": [
-    "The Delta method: if $Y_n \\sim \\mathcal{N}\\left(\\mu, \\frac{\\sigma^2}{n} \\right)$, then $g(Y_n) \\sim \\mathcal{N}\\left(g(\\mu), \\left( g(\\mu)' \\right)^2\\frac{\\sigma^2}{n} \\right)$, given $g$ is differentiable and $g'(\\mu) \\neq 0$."
+    "The Delta method: if $Y_n \\sim \\mathcal{N}\\left(\\mu, \\frac{\\sigma^2}{n} \\right)$, then $g(Y_n) \\sim \\mathcal{N}\\left(g(\\mu), \\left( g(\\mu)' \\right)^2\\frac{\\sigma^2}{n} \\right)$, given $g$ is differentiable and $g'(\\mu) \\neq 0$.\n",
+    "\n",
+    "ref: Section 5.5 The Delta Method of [All of Statistics](https://link.springer.com/book/10.1007/978-0-387-21736-9).\n",
+    "\n",
+    "Proof:\n",
+    "\n",
+    "Expand $g(Y_n)$ around $\\mu$ with first-order Taylor series,\n",
+    "\n",
+    "\\begin{align*}\n",
+    "g(Y_n) \n",
+    "&\\approx g(\\mu) + g'(\\mu)(Y_n - \\mu) \\\\\n",
+    "\\end{align*}\n",
+    "\n",
+    "Note,\n",
+    "\n",
+    "\\begin{align*}\n",
+    "Y_n - \\mu\n",
+    "&\\sim \\mathcal{N}\\left(0, \\frac{\\sigma^2}{n} \\right) \\\\\n",
+    "g'(\\mu)(Y_n - \\mu)\n",
+    "&\\sim \\mathcal{N}\\left(0, g'(\\mu)^2 \\frac{\\sigma^2}{n} \\right) \\\\\n",
+    "g(\\mu) + g'(\\mu)(Y_n - \\mu)\n",
+    "&\\sim \\mathcal{N}\\left(g(\\mu), g'(\\mu)^2 \\frac{\\sigma^2}{n} \\right)\n",
+    "\\end{align*}\n",
+    "\n",
+    "Q.E.D.\n",
+    "\n",
+    "The name delta could refer to the changes in $Y_n$ when calculating $g'$."
    ]
   },
   {