Maximum_Likelihood_Estimate_answer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Maximum Likelihood Estimate(MLE) 最大似然估计"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Name: [Your name]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<script type=\"text/javascript\" src=\"http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=default\"></script><font size=4>\n",
    "\n",
    "1.设总体概率函数如下, $x_1, \\dots ,x_n$是样本, 试求未知参数的最大似然估计.\n",
    "\n",
    "(1) $p(x; \\theta)=\\sqrt{\\theta}x^{\\sqrt{\\theta}-1}, 0<x<1, \\theta>0$;\n",
    "\n",
    "(2) $p(x; \\theta)=\\theta c^{\\theta}x^{-(\\theta+1)}, x>c, c>0, \\theta>1$;\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "解:  (1)似数函数为L($\\theta$)=$(\\sqrt{\\theta})^n(x_1\\dots x_n)^{\\sqrt{\\theta}-1}$, 其对数似然函数为\n",
    "$$\\ln L(\\theta)=\\frac{n}{2}\\ln\\theta+(\\sqrt{\\theta}-1)(\\ln x_1+\\dots+\\ln x_n)$$.\n",
    "将$\\ln L(\\theta)$关于$\\theta$求导并令其为0即得到似然方程\n",
    "$$\\frac{\\partial\\ln L(\\theta)}{\\partial\\theta}=\\frac{n}{2\\theta}+(\\ln x_1 + \\dots + \\ln x_n)\\frac{1}{2\\sqrt{\\theta}}=0$$\n",
    "解之得\n",
    "$$\\widehat{\\theta}=(\\frac{1}{n}\\sum_{i=1}^{n}\\ln x_i)^{-2}$$\n",
    "由于\n",
    "$$\\frac{\\partial^2\\ln L(\\theta)}{\\partial\\theta^2}\\big|_\\widehat{\\theta}=(-\\frac{n}{2\\theta^2}-\\frac{\\sum\\ln x_i}{4\\theta^{3/2}})\\big|_\\widehat{\\theta}=-\\frac{3(\\sum \\ln x_i)^4}{4n^3}<0$$\n",
    "所以$\\widehat{\\theta}$是$\\theta$的最大似然估计.\n",
    "\n",
    "-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:\n",
    "\n",
    "(2)似然函数为$L(\\theta)=\\theta^nc^{n\\theta}(x_1\\dots x_n)^{-(\\theta+1)}$,其对数似然函数为\n",
    "$$\\ln L(\\theta)=n\\ln\\theta+n\\theta\\ln c-(\\theta+1)(\\ln x_1 + \\dots + \\ln x_n)$$\n",
    "将$\\ln L(\\theta)$关于$\\theta$求导并令其为0得到似然函数方程\n",
    "$$\\frac{\\partial\\ln L(\\theta)}{\\partial\\theta}=\\frac{n}{\\theta}+n\\ln c -(\\ln x_1+\\dots+\\ln x_n)=0$$\n",
    "解之可得\n",
    "$$\\widehat{\\theta}=(\\frac{1}{n}\\sum_{i=1}^{n}\\ln x_i - \\ln c)^{-1}$$\n",
    "由于$\\frac{\\partial^2\\ln L(\\theta)}{\\partial\\theta^2}=\\frac{-n}{\\theta^2}<0$, 这说明$\\widehat{\\theta}$是$\\theta$的最大似然估计."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<script type=\"text/javascript\" src=\"http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=default\"></script><font size=4>\n",
    "\n",
    "2.假如有一个罐子, 里面有黑白两种颜色的球, 数目多少不知, 两种颜色的比例也不知. 我们想知道罐中白球和黑球的比例, 但我们不能把罐中的球全部拿出来数(球太多了, 耽误我上王者). 现在我们可以进行100次实验, 每次实验都拿出10个球, 记录下白球的数量. 假设这100次实验都相与独立的, 请问罐中白球所占的比例最有可能是多少? 实验的数据如下:\n",
    "\n",
    "\n",
    "| 拿出白球的数量 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |\n",
    "|:----------------:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:--:|\n",
    "| 实验次数      | 0 | 1 | 6 | 7 | 23 | 26 | 21 | 12 | 3 | 1 | 0 |\n",
    "\n",
    "tips: 变量X为每次实验中拿出白球的个数, 且X服从二项分布Bin(x|10, p).\n",
    "\n",
    "\n",
    "</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "解: 本题中, 总体X为拿出了的球中白球个数, 且X服从参数为(10,p)的二项分布, 即\n",
    "$$p(X=x)=C_{10}^{x}p^x(1-p)^{10-x}$$.\n",
    "$x_1,x_2,\\dots,x_100$为样本, 则其似然函数为(忽略常数)\n",
    "$$L(p)=p^{\\sum\\limits _{i=1}^{100}x_i}(1-p)^{10\\times100-\\sum\\limits _{i=1}^{100}x_i}$$,\n",
    "对数似然函数为\n",
    "$$\\ln L(p)=\\sum_{i=1}^{100}x_i\\ln p+(10\\times100-\\sum_{i=1}^{100}x_i)\\ln(1-p)$$.\n",
    "将对数似然函数关于p求导并令其为0得到似然方程\n",
    "$$\\frac{\\partial \\ln(p)}{\\partial p}=\\frac{\\sum\\limits_{i=1}^{100}x_i}{p}-\\frac{10\\times100-\\sum\\limits_{i=1}^{100}x_i}{1-p}=0$$,\n",
    "解之得\n",
    "$$\\widehat{p}=\\frac{\\sum\\limits_{i=1}^{100}x_i}{1000}$$.\n",
    "由于\n",
    "$$\\frac{\\partial\\ln^2L(p)}{\\partial p^2}=-\\frac{\\sum\\limits_{i=1}^{100}x_i}{p^2}-\\frac{1000-\\sum\\limits_{i=1}^{100}x_i}{(1-p)^2}<0$$\n",
    "由二阶导数的性质知, p的最大似然估计为\n",
    "$$\\widehat{p}=\\frac{\\sum\\limits_{i=1}^{100}x_i}{1000}=\\frac{499}{1000}=0.499$$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "遇到不会输入的数学符号, 可以参考《LATEX数学符号表.pdf》文献中的说明, Markdown的公式输入与Latex相类似, 现在多掌握一下Markdown的公式输入, 对你以后的Latex写作是很有帮助."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "参考文献:\n",
    "\n",
    "[1]茆诗松, 程依明, 濮晓龙, 等. 概率论与数理统计教程 (第二版). 北京: 高等教育出版社, 2011."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果你有什么疑问或者收获, 你可以在这里记录下来.\n",
    "###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}