dlnd_tv_script_generation-zh .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 生成电视剧剧本\n",
    "\n",
    "在这个项目中，你将使用 RNN 创作你自己的[《辛普森一家》](https://zh.wikipedia.org/wiki/%E8%BE%9B%E6%99%AE%E6%A3%AE%E4%B8%80%E5%AE%B6)电视剧剧本。你将会用到《辛普森一家》第 27 季中部分剧本的[数据集](https://www.kaggle.com/wcukierski/the-simpsons-by-the-data)。你创建的神经网络将为一个在 [Moe 酒馆](https://simpsonswiki.com/wiki/Moe's_Tavern)中的场景生成一集新的剧本。\n",
    "## 获取数据\n",
    "我们早已为你提供了数据。你将使用原始数据集的子集，它只包括 Moe 酒馆中的场景。数据中并不包括酒馆的其他版本，比如 “Moe 的山洞”、“燃烧的 Moe 酒馆”、“Moe 叔叔的家庭大餐”等等。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import helper\n",
    "\n",
    "data_dir = './data/simpsons/moes_tavern_lines.txt'\n",
    "text = helper.load_data(data_dir)\n",
    "# Ignore notice, since we don't use it for analysing the data\n",
    "text = text[81:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 探索数据\n",
    "使用 `view_sentence_range` 来查看数据的不同部分。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset Stats\n",
      "Roughly the number of unique words: 11492\n",
      "Number of scenes: 262\n",
      "Average number of sentences in each scene: 15.248091603053435\n",
      "Number of lines: 4257\n",
      "Average number of words in each line: 11.50434578341555\n",
      "\n",
      "The sentences 0 to 10:\n",
      "Moe_Szyslak: (INTO PHONE) Moe's Tavern. Where the elite meet to drink.\n",
      "Bart_Simpson: Eh, yeah, hello, is Mike there? Last name, Rotch.\n",
      "Moe_Szyslak: (INTO PHONE) Hold on, I'll check. (TO BARFLIES) Mike Rotch. Mike Rotch. Hey, has anybody seen Mike Rotch, lately?\n",
      "Moe_Szyslak: (INTO PHONE) Listen you little puke. One of these days I'm gonna catch you, and I'm gonna carve my name on your back with an ice pick.\n",
      "Moe_Szyslak: What's the matter Homer? You're not your normal effervescent self.\n",
      "Homer_Simpson: I got my problems, Moe. Give me another one.\n",
      "Moe_Szyslak: Homer, hey, you should not drink to forget your problems.\n",
      "Barney_Gumble: Yeah, you should only drink to enhance your social skills.\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "view_sentence_range = (0, 10)\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import numpy as np\n",
    "\n",
    "print('Dataset Stats')\n",
    "print('Roughly the number of unique words: {}'.format(len({word: None for word in text.split()})))\n",
    "scenes = text.split('\\n\\n')\n",
    "print('Number of scenes: {}'.format(len(scenes)))\n",
    "sentence_count_scene = [scene.count('\\n') for scene in scenes]\n",
    "print('Average number of sentences in each scene: {}'.format(np.average(sentence_count_scene)))\n",
    "\n",
    "sentences = [sentence for scene in scenes for sentence in scene.split('\\n')]\n",
    "print('Number of lines: {}'.format(len(sentences)))\n",
    "word_count_sentence = [len(sentence.split()) for sentence in sentences]\n",
    "print('Average number of words in each line: {}'.format(np.average(word_count_sentence)))\n",
    "\n",
    "print()\n",
    "print('The sentences {} to {}:'.format(*view_sentence_range))\n",
    "print('\\n'.join(text.split('\\n')[view_sentence_range[0]:view_sentence_range[1]]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实现预处理函数\n",
    "对数据集进行的第一个操作是预处理。请实现下面两个预处理函数：\n",
    "\n",
    "- 查询表\n",
    "- 标记符号的字符串\n",
    "\n",
    "### 查询表\n",
    "要创建词嵌入，你首先要将词语转换为 id。请在这个函数中创建两个字典：\n",
    "\n",
    "- 将词语转换为 id 的字典，我们称它为 `vocab_to_int`\n",
    "- 将 id 转换为词语的字典，我们称它为 `int_to_vocab`\n",
    "\n",
    "请在下面的元组中返回这些字典\n",
    " `(vocab_to_int, int_to_vocab)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import problem_unittests as tests\n",
    "\n",
    "def create_lookup_tables(text):\n",
    "    \"\"\"\n",
    "    Create lookup tables for vocabulary\n",
    "    :param text: The text of tv scripts split into words\n",
    "    :return: A tuple of dicts (vocab_to_int, int_to_vocab)\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    \n",
    "    vocab = sorted(set(text))\n",
    "    vocab_to_int = {word: index for index, word in enumerate(vocab)}\n",
    "    int_to_vocab = dict(enumerate(vocab))\n",
    "\n",
    "    return vocab_to_int, int_to_vocab\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_create_lookup_tables(create_lookup_tables)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 标记符号的字符串\n",
    "我们会使用空格当作分隔符，来将剧本分割为词语数组。然而，句号和感叹号等符号使得神经网络难以分辨“再见”和“再见！”之间的区别。\n",
    "\n",
    "实现函数 `token_lookup` 来返回一个字典，这个字典用于将 “!” 等符号标记为 “||Exclamation_Mark||” 形式。为下列符号创建一个字典，其中符号为标志，值为标记。\n",
    "\n",
    "- period ( . )\n",
    "- comma ( , )\n",
    "- quotation mark ( \" )\n",
    "- semicolon ( ; )\n",
    "- exclamation mark ( ! )\n",
    "- question mark ( ? )\n",
    "- left parenthesis ( ( )\n",
    "- right parenthesis ( ) )\n",
    "- dash ( -- )\n",
    "- return ( \\n )\n",
    "\n",
    "这个字典将用于标记符号并在其周围添加分隔符（空格）。这能将符号视作单独词汇分割开来，并使神经网络更轻松地预测下一个词汇。请确保你并没有使用容易与词汇混淆的标记。与其使用 “dash” 这样的标记，试试使用“||dash||”。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def token_lookup():\n",
    "    \"\"\"\n",
    "    Generate a dict to turn punctuation into a token.\n",
    "    :return: Tokenize dictionary where the key is the punctuation and the value is the token\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    token_dict = {'.':'||Period||',\n",
    "             ',':'||Comma||', \n",
    "             '\"':'||Quotation_Mark||', \n",
    "             ';':'||Semicolon||',\n",
    "             '!':'||Exclamation_Mark||', \n",
    "             '?':'||Question_Mark||', \n",
    "             '(':'||Left_Parenthesis||',\n",
    "             ')':'||Right_Parenthesis||', \n",
    "             '--':'||Dash||', \n",
    "             '\\n':'||Return||'}\n",
    "    return token_dict\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_tokenize(token_lookup)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预处理并保存所有数据\n",
    "运行以下代码将预处理所有数据，并将它们保存至文件。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "# Preprocess Training, Validation, and Testing Data\n",
    "helper.preprocess_and_save_data(data_dir, token_lookup, create_lookup_tables)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 检查点\n",
    "这是你遇到的第一个检点。如果你想要回到这个 notebook，或需要重新打开 notebook，你都可以从这里开始。预处理的数据都已经保存完毕。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import helper\n",
    "import numpy as np\n",
    "import problem_unittests as tests\n",
    "\n",
    "int_text, vocab_to_int, int_to_vocab, token_dict = helper.load_preprocess()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 创建神经网络\n",
    "你将通过实现下面的函数，来创造用于构建 RNN 的必要元素：\n",
    "\n",
    "- get_inputs\n",
    "- get_init\\_cell\n",
    "- get_embed\n",
    "- build_rnn\n",
    "- build_nn\n",
    "- get_batches\n",
    "\n",
    "### 检查 TensorFlow 版本并访问 GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TensorFlow Version: 1.0.0\n",
      "Default GPU Device: /gpu:0\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "from distutils.version import LooseVersion\n",
    "import warnings\n",
    "import tensorflow as tf\n",
    "\n",
    "# Check TensorFlow Version\n",
    "assert LooseVersion(tf.__version__) >= LooseVersion('1.0'), 'Please use TensorFlow version 1.0 or newer'\n",
    "print('TensorFlow Version: {}'.format(tf.__version__))\n",
    "\n",
    "# Check for a GPU\n",
    "if not tf.test.gpu_device_name():\n",
    "    warnings.warn('No GPU found. Please use a GPU to train your neural network.')\n",
    "else:\n",
    "    print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 输入\n",
    "\n",
    "实现函数 `get_inputs()` 来为神经网络创建 TF 占位符。它将创建下列占位符：\n",
    "\n",
    "- 使用 [TF 占位符](https://www.tensorflow.org/api_docs/python/tf/placeholder) `name` 参量输入 \"input\" 文本占位符。\n",
    "- Targets 占位符\n",
    "- Learning Rate 占位符\n",
    "\n",
    "返回下列元组中的占位符 `(Input, Targets, LearningRate)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def get_inputs():\n",
    "    \"\"\"\n",
    "    Create TF Placeholders for input, targets, and learning rate.\n",
    "    :return: Tuple (input, targets, learning rate)\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    text_input = tf.placeholder(tf.int32, shape=(None,None), name=\"input\")\n",
    "    targets = tf.placeholder(tf.int32, shape=(None,None))\n",
    "    learning_rate = tf.placeholder(tf.float32, shape=None)\n",
    "    return (text_input, targets, learning_rate)\n",
    "\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_get_inputs(get_inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建 RNN Cell 并初始化\n",
    "\n",
    "在 [`MultiRNNCell`](https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/MultiRNNCell) 中堆叠一个或多个 [`BasicLSTMCells`](https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/BasicLSTMCell)\n",
    "\n",
    "- 使用 `rnn_size` 设定 RNN 大小。\n",
    "- 使用 MultiRNNCell 的 [`zero_state()`](https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/MultiRNNCell#zero_state) 函数初始化 Cell 状态\n",
    "- 使用 [`tf.identity()`](https://www.tensorflow.org/api_docs/python/tf/identity) 为初始状态应用名称 \"initial_state\"\n",
    " \n",
    "\n",
    "返回 cell 和下列元组中的初始状态 `(Cell, InitialState)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def get_init_cell(batch_size, rnn_size):\n",
    "    \"\"\"\n",
    "    Create an RNN Cell and initialize it.\n",
    "    :param batch_size: Size of batches\n",
    "    :param rnn_size: Size of RNNs\n",
    "    :return: Tuple (cell, initialize state)\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)\n",
    "    \n",
    "    cell = tf.contrib.rnn.MultiRNNCell([lstm])\n",
    "    \n",
    "    initial_state = cell.zero_state(batch_size,tf.float32)\n",
    "    \n",
    "    initial_state = tf.identity(initial_state,name='initial_state')\n",
    "    return (cell, initial_state)\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_get_init_cell(get_init_cell)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 词嵌入\n",
    "使用 TensorFlow 将嵌入运用到 `input_data` 中。\n",
    "返回嵌入序列。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def get_embed(input_data, vocab_size, embed_dim):\n",
    "    \"\"\"\n",
    "    Create embedding for <input_data>.\n",
    "    :param input_data: TF placeholder for text input.\n",
    "    :param vocab_size: Number of words in vocabulary.\n",
    "    :param embed_dim: Number of embedding dimensions\n",
    "    :return: Embedded input.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    embedding = tf.Variable(tf.random_uniform((vocab_size, embed_dim), -1, 1))\n",
    "    embed = tf.nn.embedding_lookup(embedding, input_data)\n",
    "    return embed\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_get_embed(get_embed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建 RNN\n",
    "你已经在 `get_init_cell()` 函数中创建了 RNN Cell。是时候使用这个 Cell 来创建 RNN了。\n",
    "\n",
    "- 使用 [`tf.nn.dynamic_rnn()`](https://www.tensorflow.org/api_docs/python/tf/nn/dynamic_rnn) 创建 RNN\n",
    "- 使用 [`tf.identity()`](https://www.tensorflow.org/api_docs/python/tf/identity) 将名称 \"final_state\" 应用到最终状态中\n",
    "\n",
    "\n",
    "返回下列元组中的输出和最终状态`(Outputs, FinalState)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def build_rnn(cell, inputs):\n",
    "    \"\"\"\n",
    "    Create a RNN using a RNN Cell\n",
    "    :param cell: RNN Cell\n",
    "    :param inputs: Input text data\n",
    "    :return: Tuple (Outputs, Final State)\n",
    "    \"\"\"\n",
    "    outputs, state = tf.nn.dynamic_rnn(cell, inputs, dtype=tf.float32)        \n",
    "    # Setup initial state\n",
    "    state = tf.identity(state, name='final_state')\n",
    "        \n",
    "    return outputs, state\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_build_rnn(build_rnn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 构建神经网络\n",
    "应用你在上面实现的函数，来：\n",
    "\n",
    "- 使用你的 `get_embed(input_data, vocab_size, embed_dim)` 函数将嵌入应用到 `input_data` 中\n",
    "- 使用 `cell` 和你的 `build_rnn(cell, inputs)` 函数来创建 RNN\n",
    "- 应用一个完全联通线性激活和 `vocab_size` 的分层作为输出数量。\n",
    "\n",
    "返回下列元组中的 logit 和最终状态 `Logits, FinalState`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def build_nn(cell, rnn_size, input_data, vocab_size, embed_dim):\n",
    "    \"\"\"\n",
    "    Build part of the neural network\n",
    "    :param cell: RNN cell\n",
    "    :param rnn_size: Size of rnns\n",
    "    :param input_data: Input data\n",
    "    :param vocab_size: Vocabulary size\n",
    "    :param embed_dim: Number of embedding dimensions\n",
    "    :return: Tuple (Logits, FinalState)\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    inputs = get_embed(input_data, vocab_size, embed_dim)\n",
    "    outputs, final_state = build_rnn(cell, inputs)\n",
    "    logits = tf.contrib.layers.fully_connected(outputs, vocab_size, activation_fn=None)\n",
    "    return (logits, final_state)\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_build_nn(build_nn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 批次\n",
    "\n",
    "实现 `get_batches` 来使用 `int_text` 创建输入与目标批次。这些批次应为 Numpy 数组，并具有形状 `(number of batches, 2, batch size, sequence length)`。每个批次包含两个元素：\n",
    "\n",
    "- 第一个元素为**输入**的单独批次，并具有形状 `[batch size, sequence length]`\n",
    "- 第二个元素为**目标**的单独批次，并具有形状 `[batch size, sequence length]`\n",
    "\n",
    "如果你无法在最后一个批次中填入足够数据，请放弃这个批次。\n",
    "\n",
    "例如 `get_batches([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15], 2, 3)` 将返回下面这个 Numpy 数组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (<ipython-input-13-585e60cf595e>, line 5)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  File \u001b[0;32m\"<ipython-input-13-585e60cf595e>\"\u001b[0;36m, line \u001b[0;32m5\u001b[0m\n\u001b[0;31m    [[ 1  2  3], [ 7  8  9]],\u001b[0m\n\u001b[0m          ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "[\n",
    "  # First Batch\n",
    "  [\n",
    "    # Batch of Input\n",
    "    [[ 1  2  3], [ 7  8  9]],\n",
    "    # Batch of targets\n",
    "    [[ 2  3  4], [ 8  9 10]]\n",
    "  ],\n",
    " \n",
    "  # Second Batch\n",
    "  [\n",
    "    # Batch of Input\n",
    "    [[ 4  5  6], [10 11 12]],\n",
    "    # Batch of targets\n",
    "    [[ 5  6  7], [11 12 13]]\n",
    "  ]\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def get_batches(int_text, batch_size, seq_length):\n",
    "    \"\"\"\n",
    "    Return batches of input and target\n",
    "    :param int_text: Text with the words replaced by their ids\n",
    "    :param batch_size: The size of batch\n",
    "    :param seq_length: The length of sequence\n",
    "    :return: Batches as a Numpy array\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    n_batches = len(int_text)//(batch_size*seq_length)\n",
    "    batches = []\n",
    "    \n",
    "    for b_index in range(n_batches):  \n",
    "        one_seq = []\n",
    "        one_tar =[]\n",
    "        for n in range(batch_size):\n",
    "            index = b_index * seq_length + n * n_batches * seq_length            \n",
    "            seq = int_text[index:index+seq_length]\n",
    "            tar = int_text[index+1:index+seq_length+1]\n",
    "            one_seq.append(seq)\n",
    "            one_tar.append(tar)\n",
    "            \n",
    "        one_batch = [one_seq, one_tar]\n",
    "        batches.append(one_batch)\n",
    "    batches = np.array(batches)\n",
    "    return batches\n",
    "\n",
    "    return None\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_get_batches(get_batches)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 神经网络训练\n",
    "### 超参数\n",
    "调整下列参数:\n",
    "\n",
    "- 将 `num_epochs` 设置为训练次数。\n",
    "- 将 `batch_size` 设置为程序组大小。\n",
    "- 将 `rnn_size` 设置为 RNN 大小。\n",
    "- 将 `embed_dim` 设置为嵌入大小。\n",
    "- 将 `seq_length` 设置为序列长度。\n",
    "- 将 `learning_rate` 设置为学习率。\n",
    "- 将 `show_every_n_batches` 设置为神经网络应输出的程序组数量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Number of Epochs\n",
    "num_epochs = 300\n",
    "# Batch Size\n",
    "batch_size = 128\n",
    "# RNN Size\n",
    "rnn_size = 256\n",
    "# Embedding Dimension Size\n",
    "embed_dim = 1000\n",
    "# Sequence Length\n",
    "seq_length = 20\n",
    "# Learning Rate\n",
    "learning_rate = 0.0025\n",
    "# Show stats for every n number of batches\n",
    "show_every_n_batches = 10\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "save_dir = './save'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 创建图表\n",
    "使用你实现的神经网络创建图表。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "from tensorflow.contrib import seq2seq\n",
    "\n",
    "train_graph = tf.Graph()\n",
    "with train_graph.as_default():\n",
    "    vocab_size = len(int_to_vocab)\n",
    "    input_text, targets, lr = get_inputs()\n",
    "    input_data_shape = tf.shape(input_text)\n",
    "    cell, initial_state = get_init_cell(input_data_shape[0], rnn_size)\n",
    "    logits, final_state = build_nn(cell, rnn_size, input_text, vocab_size, embed_dim)\n",
    "\n",
    "    # Probabilities for generating words\n",
    "    probs = tf.nn.softmax(logits, name='probs')\n",
    "\n",
    "    # Loss function\n",
    "    cost = seq2seq.sequence_loss(\n",
    "        logits,\n",
    "        targets,\n",
    "        tf.ones([input_data_shape[0], input_data_shape[1]]))\n",
    "\n",
    "    # Optimizer\n",
    "    optimizer = tf.train.AdamOptimizer(lr)\n",
    "\n",
    "    # Gradient Clipping\n",
    "    gradients = optimizer.compute_gradients(cost)\n",
    "    capped_gradients = [(tf.clip_by_value(grad, -1., 1.), var) for grad, var in gradients if grad is not None]\n",
    "    train_op = optimizer.apply_gradients(capped_gradients)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练\n",
    "在预处理数据中训练神经网络。如果你遇到困难，请查看这个[表格](https://discussions.udacity.com/)，看看是否有人遇到了和你一样的问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch   0 Batch    0/26   train_loss = 8.821\n",
      "Epoch   0 Batch   10/26   train_loss = 6.148\n",
      "Epoch   0 Batch   20/26   train_loss = 6.073\n",
      "Epoch   1 Batch    4/26   train_loss = 5.518\n",
      "Epoch   1 Batch   14/26   train_loss = 5.257\n",
      "Epoch   1 Batch   24/26   train_loss = 5.263\n",
      "Epoch   2 Batch    8/26   train_loss = 5.008\n",
      "Epoch   2 Batch   18/26   train_loss = 4.938\n",
      "Epoch   3 Batch    2/26   train_loss = 4.750\n",
      "Epoch   3 Batch   12/26   train_loss = 4.895\n",
      "Epoch   3 Batch   22/26   train_loss = 4.540\n",
      "Epoch   4 Batch    6/26   train_loss = 4.587\n",
      "Epoch   4 Batch   16/26   train_loss = 4.471\n",
      "Epoch   5 Batch    0/26   train_loss = 4.300\n",
      "Epoch   5 Batch   10/26   train_loss = 4.276\n",
      "Epoch   5 Batch   20/26   train_loss = 4.204\n",
      "Epoch   6 Batch    4/26   train_loss = 4.121\n",
      "Epoch   6 Batch   14/26   train_loss = 4.045\n",
      "Epoch   6 Batch   24/26   train_loss = 3.989\n",
      "Epoch   7 Batch    8/26   train_loss = 3.948\n",
      "Epoch   7 Batch   18/26   train_loss = 3.933\n",
      "Epoch   8 Batch    2/26   train_loss = 3.817\n",
      "Epoch   8 Batch   12/26   train_loss = 3.923\n",
      "Epoch   8 Batch   22/26   train_loss = 3.669\n",
      "Epoch   9 Batch    6/26   train_loss = 3.695\n",
      "Epoch   9 Batch   16/26   train_loss = 3.661\n",
      "Epoch  10 Batch    0/26   train_loss = 3.509\n",
      "Epoch  10 Batch   10/26   train_loss = 3.530\n",
      "Epoch  10 Batch   20/26   train_loss = 3.424\n",
      "Epoch  11 Batch    4/26   train_loss = 3.394\n",
      "Epoch  11 Batch   14/26   train_loss = 3.365\n",
      "Epoch  11 Batch   24/26   train_loss = 3.255\n",
      "Epoch  12 Batch    8/26   train_loss = 3.268\n",
      "Epoch  12 Batch   18/26   train_loss = 3.256\n",
      "Epoch  13 Batch    2/26   train_loss = 3.178\n",
      "Epoch  13 Batch   12/26   train_loss = 3.213\n",
      "Epoch  13 Batch   22/26   train_loss = 3.077\n",
      "Epoch  14 Batch    6/26   train_loss = 3.057\n",
      "Epoch  14 Batch   16/26   train_loss = 3.063\n",
      "Epoch  15 Batch    0/26   train_loss = 2.956\n",
      "Epoch  15 Batch   10/26   train_loss = 2.972\n",
      "Epoch  15 Batch   20/26   train_loss = 2.842\n",
      "Epoch  16 Batch    4/26   train_loss = 2.875\n",
      "Epoch  16 Batch   14/26   train_loss = 2.848\n",
      "Epoch  16 Batch   24/26   train_loss = 2.746\n",
      "Epoch  17 Batch    8/26   train_loss = 2.759\n",
      "Epoch  17 Batch   18/26   train_loss = 2.776\n",
      "Epoch  18 Batch    2/26   train_loss = 2.731\n",
      "Epoch  18 Batch   12/26   train_loss = 2.701\n",
      "Epoch  18 Batch   22/26   train_loss = 2.655\n",
      "Epoch  19 Batch    6/26   train_loss = 2.613\n",
      "Epoch  19 Batch   16/26   train_loss = 2.645\n",
      "Epoch  20 Batch    0/26   train_loss = 2.568\n",
      "Epoch  20 Batch   10/26   train_loss = 2.575\n",
      "Epoch  20 Batch   20/26   train_loss = 2.444\n",
      "Epoch  21 Batch    4/26   train_loss = 2.485\n",
      "Epoch  21 Batch   14/26   train_loss = 2.466\n",
      "Epoch  21 Batch   24/26   train_loss = 2.377\n",
      "Epoch  22 Batch    8/26   train_loss = 2.390\n",
      "Epoch  22 Batch   18/26   train_loss = 2.393\n",
      "Epoch  23 Batch    2/26   train_loss = 2.383\n",
      "Epoch  23 Batch   12/26   train_loss = 2.324\n",
      "Epoch  23 Batch   22/26   train_loss = 2.318\n",
      "Epoch  24 Batch    6/26   train_loss = 2.254\n",
      "Epoch  24 Batch   16/26   train_loss = 2.317\n",
      "Epoch  25 Batch    0/26   train_loss = 2.237\n",
      "Epoch  25 Batch   10/26   train_loss = 2.244\n",
      "Epoch  25 Batch   20/26   train_loss = 2.134\n",
      "Epoch  26 Batch    4/26   train_loss = 2.182\n",
      "Epoch  26 Batch   14/26   train_loss = 2.153\n",
      "Epoch  26 Batch   24/26   train_loss = 2.053\n",
      "Epoch  27 Batch    8/26   train_loss = 2.093\n",
      "Epoch  27 Batch   18/26   train_loss = 2.075\n",
      "Epoch  28 Batch    2/26   train_loss = 2.091\n",
      "Epoch  28 Batch   12/26   train_loss = 2.025\n",
      "Epoch  28 Batch   22/26   train_loss = 2.026\n",
      "Epoch  29 Batch    6/26   train_loss = 1.952\n",
      "Epoch  29 Batch   16/26   train_loss = 2.032\n",
      "Epoch  30 Batch    0/26   train_loss = 1.959\n",
      "Epoch  30 Batch   10/26   train_loss = 1.958\n",
      "Epoch  30 Batch   20/26   train_loss = 1.863\n",
      "Epoch  31 Batch    4/26   train_loss = 1.909\n",
      "Epoch  31 Batch   14/26   train_loss = 1.866\n",
      "Epoch  31 Batch   24/26   train_loss = 1.772\n",
      "Epoch  32 Batch    8/26   train_loss = 1.828\n",
      "Epoch  32 Batch   18/26   train_loss = 1.797\n",
      "Epoch  33 Batch    2/26   train_loss = 1.810\n",
      "Epoch  33 Batch   12/26   train_loss = 1.733\n",
      "Epoch  33 Batch   22/26   train_loss = 1.762\n",
      "Epoch  34 Batch    6/26   train_loss = 1.672\n",
      "Epoch  34 Batch   16/26   train_loss = 1.763\n",
      "Epoch  35 Batch    0/26   train_loss = 1.704\n",
      "Epoch  35 Batch   10/26   train_loss = 1.701\n",
      "Epoch  35 Batch   20/26   train_loss = 1.615\n",
      "Epoch  36 Batch    4/26   train_loss = 1.654\n",
      "Epoch  36 Batch   14/26   train_loss = 1.612\n",
      "Epoch  36 Batch   24/26   train_loss = 1.526\n",
      "Epoch  37 Batch    8/26   train_loss = 1.590\n",
      "Epoch  37 Batch   18/26   train_loss = 1.550\n",
      "Epoch  38 Batch    2/26   train_loss = 1.578\n",
      "Epoch  38 Batch   12/26   train_loss = 1.497\n",
      "Epoch  38 Batch   22/26   train_loss = 1.518\n",
      "Epoch  39 Batch    6/26   train_loss = 1.436\n",
      "Epoch  39 Batch   16/26   train_loss = 1.537\n",
      "Epoch  40 Batch    0/26   train_loss = 1.476\n",
      "Epoch  40 Batch   10/26   train_loss = 1.475\n",
      "Epoch  40 Batch   20/26   train_loss = 1.406\n",
      "Epoch  41 Batch    4/26   train_loss = 1.434\n",
      "Epoch  41 Batch   14/26   train_loss = 1.398\n",
      "Epoch  41 Batch   24/26   train_loss = 1.303\n",
      "Epoch  42 Batch    8/26   train_loss = 1.378\n",
      "Epoch  42 Batch   18/26   train_loss = 1.332\n",
      "Epoch  43 Batch    2/26   train_loss = 1.373\n",
      "Epoch  43 Batch   12/26   train_loss = 1.280\n",
      "Epoch  43 Batch   22/26   train_loss = 1.309\n",
      "Epoch  44 Batch    6/26   train_loss = 1.238\n",
      "Epoch  44 Batch   16/26   train_loss = 1.340\n",
      "Epoch  45 Batch    0/26   train_loss = 1.294\n",
      "Epoch  45 Batch   10/26   train_loss = 1.279\n",
      "Epoch  45 Batch   20/26   train_loss = 1.220\n",
      "Epoch  46 Batch    4/26   train_loss = 1.235\n",
      "Epoch  46 Batch   14/26   train_loss = 1.216\n",
      "Epoch  46 Batch   24/26   train_loss = 1.126\n",
      "Epoch  47 Batch    8/26   train_loss = 1.208\n",
      "Epoch  47 Batch   18/26   train_loss = 1.167\n",
      "Epoch  48 Batch    2/26   train_loss = 1.187\n",
      "Epoch  48 Batch   12/26   train_loss = 1.102\n",
      "Epoch  48 Batch   22/26   train_loss = 1.142\n",
      "Epoch  49 Batch    6/26   train_loss = 1.058\n",
      "Epoch  49 Batch   16/26   train_loss = 1.151\n",
      "Epoch  50 Batch    0/26   train_loss = 1.128\n",
      "Epoch  50 Batch   10/26   train_loss = 1.098\n",
      "Epoch  50 Batch   20/26   train_loss = 1.050\n",
      "Epoch  51 Batch    4/26   train_loss = 1.069\n",
      "Epoch  51 Batch   14/26   train_loss = 1.040\n",
      "Epoch  51 Batch   24/26   train_loss = 0.961\n",
      "Epoch  52 Batch    8/26   train_loss = 1.032\n",
      "Epoch  52 Batch   18/26   train_loss = 1.006\n",
      "Epoch  53 Batch    2/26   train_loss = 1.019\n",
      "Epoch  53 Batch   12/26   train_loss = 0.947\n",
      "Epoch  53 Batch   22/26   train_loss = 0.984\n",
      "Epoch  54 Batch    6/26   train_loss = 0.904\n",
      "Epoch  54 Batch   16/26   train_loss = 0.999\n",
      "Epoch  55 Batch    0/26   train_loss = 0.978\n",
      "Epoch  55 Batch   10/26   train_loss = 0.955\n",
      "Epoch  55 Batch   20/26   train_loss = 0.900\n",
      "Epoch  56 Batch    4/26   train_loss = 0.929\n",
      "Epoch  56 Batch   14/26   train_loss = 0.898\n",
      "Epoch  56 Batch   24/26   train_loss = 0.819\n",
      "Epoch  57 Batch    8/26   train_loss = 0.888\n",
      "Epoch  57 Batch   18/26   train_loss = 0.868\n",
      "Epoch  58 Batch    2/26   train_loss = 0.868\n",
      "Epoch  58 Batch   12/26   train_loss = 0.803\n",
      "Epoch  58 Batch   22/26   train_loss = 0.862\n",
      "Epoch  59 Batch    6/26   train_loss = 0.778\n",
      "Epoch  59 Batch   16/26   train_loss = 0.852\n",
      "Epoch  60 Batch    0/26   train_loss = 0.848\n",
      "Epoch  60 Batch   10/26   train_loss = 0.838\n",
      "Epoch  60 Batch   20/26   train_loss = 0.784\n",
      "Epoch  61 Batch    4/26   train_loss = 0.788\n",
      "Epoch  61 Batch   14/26   train_loss = 0.774\n",
      "Epoch  61 Batch   24/26   train_loss = 0.708\n",
      "Epoch  62 Batch    8/26   train_loss = 0.767\n",
      "Epoch  62 Batch   18/26   train_loss = 0.731\n",
      "Epoch  63 Batch    2/26   train_loss = 0.739\n",
      "Epoch  63 Batch   12/26   train_loss = 0.686\n",
      "Epoch  63 Batch   22/26   train_loss = 0.725\n",
      "Epoch  64 Batch    6/26   train_loss = 0.660\n",
      "Epoch  64 Batch   16/26   train_loss = 0.743\n",
      "Epoch  65 Batch    0/26   train_loss = 0.716\n",
      "Epoch  65 Batch   10/26   train_loss = 0.695\n",
      "Epoch  65 Batch   20/26   train_loss = 0.648\n",
      "Epoch  66 Batch    4/26   train_loss = 0.672\n",
      "Epoch  66 Batch   14/26   train_loss = 0.660\n",
      "Epoch  66 Batch   24/26   train_loss = 0.589\n",
      "Epoch  67 Batch    8/26   train_loss = 0.642\n",
      "Epoch  67 Batch   18/26   train_loss = 0.619\n",
      "Epoch  68 Batch    2/26   train_loss = 0.632\n",
      "Epoch  68 Batch   12/26   train_loss = 0.562\n",
      "Epoch  68 Batch   22/26   train_loss = 0.611\n",
      "Epoch  69 Batch    6/26   train_loss = 0.564\n",
      "Epoch  69 Batch   16/26   train_loss = 0.633\n",
      "Epoch  70 Batch    0/26   train_loss = 0.597\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch  70 Batch   10/26   train_loss = 0.592\n",
      "Epoch  70 Batch   20/26   train_loss = 0.556\n",
      "Epoch  71 Batch    4/26   train_loss = 0.566\n",
      "Epoch  71 Batch   14/26   train_loss = 0.559\n",
      "Epoch  71 Batch   24/26   train_loss = 0.504\n",
      "Epoch  72 Batch    8/26   train_loss = 0.552\n",
      "Epoch  72 Batch   18/26   train_loss = 0.521\n",
      "Epoch  73 Batch    2/26   train_loss = 0.544\n",
      "Epoch  73 Batch   12/26   train_loss = 0.488\n",
      "Epoch  73 Batch   22/26   train_loss = 0.522\n",
      "Epoch  74 Batch    6/26   train_loss = 0.481\n",
      "Epoch  74 Batch   16/26   train_loss = 0.541\n",
      "Epoch  75 Batch    0/26   train_loss = 0.513\n",
      "Epoch  75 Batch   10/26   train_loss = 0.509\n",
      "Epoch  75 Batch   20/26   train_loss = 0.477\n",
      "Epoch  76 Batch    4/26   train_loss = 0.484\n",
      "Epoch  76 Batch   14/26   train_loss = 0.473\n",
      "Epoch  76 Batch   24/26   train_loss = 0.418\n",
      "Epoch  77 Batch    8/26   train_loss = 0.472\n",
      "Epoch  77 Batch   18/26   train_loss = 0.448\n",
      "Epoch  78 Batch    2/26   train_loss = 0.462\n",
      "Epoch  78 Batch   12/26   train_loss = 0.416\n",
      "Epoch  78 Batch   22/26   train_loss = 0.450\n",
      "Epoch  79 Batch    6/26   train_loss = 0.416\n",
      "Epoch  79 Batch   16/26   train_loss = 0.465\n",
      "Epoch  80 Batch    0/26   train_loss = 0.437\n",
      "Epoch  80 Batch   10/26   train_loss = 0.439\n",
      "Epoch  80 Batch   20/26   train_loss = 0.409\n",
      "Epoch  81 Batch    4/26   train_loss = 0.412\n",
      "Epoch  81 Batch   14/26   train_loss = 0.410\n",
      "Epoch  81 Batch   24/26   train_loss = 0.366\n",
      "Epoch  82 Batch    8/26   train_loss = 0.403\n",
      "Epoch  82 Batch   18/26   train_loss = 0.390\n",
      "Epoch  83 Batch    2/26   train_loss = 0.395\n",
      "Epoch  83 Batch   12/26   train_loss = 0.360\n",
      "Epoch  83 Batch   22/26   train_loss = 0.389\n",
      "Epoch  84 Batch    6/26   train_loss = 0.357\n",
      "Epoch  84 Batch   16/26   train_loss = 0.404\n",
      "Epoch  85 Batch    0/26   train_loss = 0.377\n",
      "Epoch  85 Batch   10/26   train_loss = 0.386\n",
      "Epoch  85 Batch   20/26   train_loss = 0.363\n",
      "Epoch  86 Batch    4/26   train_loss = 0.359\n",
      "Epoch  86 Batch   14/26   train_loss = 0.362\n",
      "Epoch  86 Batch   24/26   train_loss = 0.327\n",
      "Epoch  87 Batch    8/26   train_loss = 0.362\n",
      "Epoch  87 Batch   18/26   train_loss = 0.343\n",
      "Epoch  88 Batch    2/26   train_loss = 0.350\n",
      "Epoch  88 Batch   12/26   train_loss = 0.320\n",
      "Epoch  88 Batch   22/26   train_loss = 0.349\n",
      "Epoch  89 Batch    6/26   train_loss = 0.323\n",
      "Epoch  89 Batch   16/26   train_loss = 0.359\n",
      "Epoch  90 Batch    0/26   train_loss = 0.345\n",
      "Epoch  90 Batch   10/26   train_loss = 0.350\n",
      "Epoch  90 Batch   20/26   train_loss = 0.328\n",
      "Epoch  91 Batch    4/26   train_loss = 0.322\n",
      "Epoch  91 Batch   14/26   train_loss = 0.328\n",
      "Epoch  91 Batch   24/26   train_loss = 0.301\n",
      "Epoch  92 Batch    8/26   train_loss = 0.321\n",
      "Epoch  92 Batch   18/26   train_loss = 0.311\n",
      "Epoch  93 Batch    2/26   train_loss = 0.328\n",
      "Epoch  93 Batch   12/26   train_loss = 0.291\n",
      "Epoch  93 Batch   22/26   train_loss = 0.322\n",
      "Epoch  94 Batch    6/26   train_loss = 0.297\n",
      "Epoch  94 Batch   16/26   train_loss = 0.340\n",
      "Epoch  95 Batch    0/26   train_loss = 0.320\n",
      "Epoch  95 Batch   10/26   train_loss = 0.321\n",
      "Epoch  95 Batch   20/26   train_loss = 0.299\n",
      "Epoch  96 Batch    4/26   train_loss = 0.304\n",
      "Epoch  96 Batch   14/26   train_loss = 0.307\n",
      "Epoch  96 Batch   24/26   train_loss = 0.270\n",
      "Epoch  97 Batch    8/26   train_loss = 0.290\n",
      "Epoch  97 Batch   18/26   train_loss = 0.296\n",
      "Epoch  98 Batch    2/26   train_loss = 0.289\n",
      "Epoch  98 Batch   12/26   train_loss = 0.260\n",
      "Epoch  98 Batch   22/26   train_loss = 0.293\n",
      "Epoch  99 Batch    6/26   train_loss = 0.278\n",
      "Epoch  99 Batch   16/26   train_loss = 0.299\n",
      "Epoch 100 Batch    0/26   train_loss = 0.291\n",
      "Epoch 100 Batch   10/26   train_loss = 0.289\n",
      "Epoch 100 Batch   20/26   train_loss = 0.287\n",
      "Epoch 101 Batch    4/26   train_loss = 0.270\n",
      "Epoch 101 Batch   14/26   train_loss = 0.267\n",
      "Epoch 101 Batch   24/26   train_loss = 0.246\n",
      "Epoch 102 Batch    8/26   train_loss = 0.265\n",
      "Epoch 102 Batch   18/26   train_loss = 0.262\n",
      "Epoch 103 Batch    2/26   train_loss = 0.254\n",
      "Epoch 103 Batch   12/26   train_loss = 0.243\n",
      "Epoch 103 Batch   22/26   train_loss = 0.268\n",
      "Epoch 104 Batch    6/26   train_loss = 0.243\n",
      "Epoch 104 Batch   16/26   train_loss = 0.272\n",
      "Epoch 105 Batch    0/26   train_loss = 0.262\n",
      "Epoch 105 Batch   10/26   train_loss = 0.268\n",
      "Epoch 105 Batch   20/26   train_loss = 0.249\n",
      "Epoch 106 Batch    4/26   train_loss = 0.244\n",
      "Epoch 106 Batch   14/26   train_loss = 0.245\n",
      "Epoch 106 Batch   24/26   train_loss = 0.222\n",
      "Epoch 107 Batch    8/26   train_loss = 0.237\n",
      "Epoch 107 Batch   18/26   train_loss = 0.239\n",
      "Epoch 108 Batch    2/26   train_loss = 0.232\n",
      "Epoch 108 Batch   12/26   train_loss = 0.219\n",
      "Epoch 108 Batch   22/26   train_loss = 0.234\n",
      "Epoch 109 Batch    6/26   train_loss = 0.225\n",
      "Epoch 109 Batch   16/26   train_loss = 0.240\n",
      "Epoch 110 Batch    0/26   train_loss = 0.222\n",
      "Epoch 110 Batch   10/26   train_loss = 0.239\n",
      "Epoch 110 Batch   20/26   train_loss = 0.220\n",
      "Epoch 111 Batch    4/26   train_loss = 0.210\n",
      "Epoch 111 Batch   14/26   train_loss = 0.219\n",
      "Epoch 111 Batch   24/26   train_loss = 0.196\n",
      "Epoch 112 Batch    8/26   train_loss = 0.209\n",
      "Epoch 112 Batch   18/26   train_loss = 0.206\n",
      "Epoch 113 Batch    2/26   train_loss = 0.207\n",
      "Epoch 113 Batch   12/26   train_loss = 0.195\n",
      "Epoch 113 Batch   22/26   train_loss = 0.211\n",
      "Epoch 114 Batch    6/26   train_loss = 0.198\n",
      "Epoch 114 Batch   16/26   train_loss = 0.220\n",
      "Epoch 115 Batch    0/26   train_loss = 0.203\n",
      "Epoch 115 Batch   10/26   train_loss = 0.217\n",
      "Epoch 115 Batch   20/26   train_loss = 0.201\n",
      "Epoch 116 Batch    4/26   train_loss = 0.192\n",
      "Epoch 116 Batch   14/26   train_loss = 0.206\n",
      "Epoch 116 Batch   24/26   train_loss = 0.178\n",
      "Epoch 117 Batch    8/26   train_loss = 0.192\n",
      "Epoch 117 Batch   18/26   train_loss = 0.192\n",
      "Epoch 118 Batch    2/26   train_loss = 0.197\n",
      "Epoch 118 Batch   12/26   train_loss = 0.182\n",
      "Epoch 118 Batch   22/26   train_loss = 0.199\n",
      "Epoch 119 Batch    6/26   train_loss = 0.187\n",
      "Epoch 119 Batch   16/26   train_loss = 0.208\n",
      "Epoch 120 Batch    0/26   train_loss = 0.194\n",
      "Epoch 120 Batch   10/26   train_loss = 0.204\n",
      "Epoch 120 Batch   20/26   train_loss = 0.191\n",
      "Epoch 121 Batch    4/26   train_loss = 0.184\n",
      "Epoch 121 Batch   14/26   train_loss = 0.194\n",
      "Epoch 121 Batch   24/26   train_loss = 0.167\n",
      "Epoch 122 Batch    8/26   train_loss = 0.185\n",
      "Epoch 122 Batch   18/26   train_loss = 0.182\n",
      "Epoch 123 Batch    2/26   train_loss = 0.186\n",
      "Epoch 123 Batch   12/26   train_loss = 0.172\n",
      "Epoch 123 Batch   22/26   train_loss = 0.189\n",
      "Epoch 124 Batch    6/26   train_loss = 0.181\n",
      "Epoch 124 Batch   16/26   train_loss = 0.197\n",
      "Epoch 125 Batch    0/26   train_loss = 0.179\n",
      "Epoch 125 Batch   10/26   train_loss = 0.194\n",
      "Epoch 125 Batch   20/26   train_loss = 0.182\n",
      "Epoch 126 Batch    4/26   train_loss = 0.172\n",
      "Epoch 126 Batch   14/26   train_loss = 0.181\n",
      "Epoch 126 Batch   24/26   train_loss = 0.158\n",
      "Epoch 127 Batch    8/26   train_loss = 0.175\n",
      "Epoch 127 Batch   18/26   train_loss = 0.173\n",
      "Epoch 128 Batch    2/26   train_loss = 0.175\n",
      "Epoch 128 Batch   12/26   train_loss = 0.163\n",
      "Epoch 128 Batch   22/26   train_loss = 0.180\n",
      "Epoch 129 Batch    6/26   train_loss = 0.170\n",
      "Epoch 129 Batch   16/26   train_loss = 0.187\n",
      "Epoch 130 Batch    0/26   train_loss = 0.170\n",
      "Epoch 130 Batch   10/26   train_loss = 0.185\n",
      "Epoch 130 Batch   20/26   train_loss = 0.173\n",
      "Epoch 131 Batch    4/26   train_loss = 0.164\n",
      "Epoch 131 Batch   14/26   train_loss = 0.173\n",
      "Epoch 131 Batch   24/26   train_loss = 0.151\n",
      "Epoch 132 Batch    8/26   train_loss = 0.168\n",
      "Epoch 132 Batch   18/26   train_loss = 0.166\n",
      "Epoch 133 Batch    2/26   train_loss = 0.170\n",
      "Epoch 133 Batch   12/26   train_loss = 0.158\n",
      "Epoch 133 Batch   22/26   train_loss = 0.173\n",
      "Epoch 134 Batch    6/26   train_loss = 0.165\n",
      "Epoch 134 Batch   16/26   train_loss = 0.182\n",
      "Epoch 135 Batch    0/26   train_loss = 0.165\n",
      "Epoch 135 Batch   10/26   train_loss = 0.179\n",
      "Epoch 135 Batch   20/26   train_loss = 0.167\n",
      "Epoch 136 Batch    4/26   train_loss = 0.160\n",
      "Epoch 136 Batch   14/26   train_loss = 0.170\n",
      "Epoch 136 Batch   24/26   train_loss = 0.148\n",
      "Epoch 137 Batch    8/26   train_loss = 0.164\n",
      "Epoch 137 Batch   18/26   train_loss = 0.162\n",
      "Epoch 138 Batch    2/26   train_loss = 0.167\n",
      "Epoch 138 Batch   12/26   train_loss = 0.154\n",
      "Epoch 138 Batch   22/26   train_loss = 0.171\n",
      "Epoch 139 Batch    6/26   train_loss = 0.162\n",
      "Epoch 139 Batch   16/26   train_loss = 0.178\n",
      "Epoch 140 Batch    0/26   train_loss = 0.163\n",
      "Epoch 140 Batch   10/26   train_loss = 0.175\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 140 Batch   20/26   train_loss = 0.165\n",
      "Epoch 141 Batch    4/26   train_loss = 0.156\n",
      "Epoch 141 Batch   14/26   train_loss = 0.167\n",
      "Epoch 141 Batch   24/26   train_loss = 0.145\n",
      "Epoch 142 Batch    8/26   train_loss = 0.161\n",
      "Epoch 142 Batch   18/26   train_loss = 0.159\n",
      "Epoch 143 Batch    2/26   train_loss = 0.164\n",
      "Epoch 143 Batch   12/26   train_loss = 0.152\n",
      "Epoch 143 Batch   22/26   train_loss = 0.168\n",
      "Epoch 144 Batch    6/26   train_loss = 0.159\n",
      "Epoch 144 Batch   16/26   train_loss = 0.175\n",
      "Epoch 145 Batch    0/26   train_loss = 0.160\n",
      "Epoch 145 Batch   10/26   train_loss = 0.172\n",
      "Epoch 145 Batch   20/26   train_loss = 0.162\n",
      "Epoch 146 Batch    4/26   train_loss = 0.154\n",
      "Epoch 146 Batch   14/26   train_loss = 0.163\n",
      "Epoch 146 Batch   24/26   train_loss = 0.143\n",
      "Epoch 147 Batch    8/26   train_loss = 0.158\n",
      "Epoch 147 Batch   18/26   train_loss = 0.156\n",
      "Epoch 148 Batch    2/26   train_loss = 0.162\n",
      "Epoch 148 Batch   12/26   train_loss = 0.149\n",
      "Epoch 148 Batch   22/26   train_loss = 0.166\n",
      "Epoch 149 Batch    6/26   train_loss = 0.157\n",
      "Epoch 149 Batch   16/26   train_loss = 0.173\n",
      "Epoch 150 Batch    0/26   train_loss = 0.157\n",
      "Epoch 150 Batch   10/26   train_loss = 0.170\n",
      "Epoch 150 Batch   20/26   train_loss = 0.160\n",
      "Epoch 151 Batch    4/26   train_loss = 0.152\n",
      "Epoch 151 Batch   14/26   train_loss = 0.162\n",
      "Epoch 151 Batch   24/26   train_loss = 0.141\n",
      "Epoch 152 Batch    8/26   train_loss = 0.157\n",
      "Epoch 152 Batch   18/26   train_loss = 0.155\n",
      "Epoch 153 Batch    2/26   train_loss = 0.160\n",
      "Epoch 153 Batch   12/26   train_loss = 0.148\n",
      "Epoch 153 Batch   22/26   train_loss = 0.164\n",
      "Epoch 154 Batch    6/26   train_loss = 0.156\n",
      "Epoch 154 Batch   16/26   train_loss = 0.172\n",
      "Epoch 155 Batch    0/26   train_loss = 0.155\n",
      "Epoch 155 Batch   10/26   train_loss = 0.170\n",
      "Epoch 155 Batch   20/26   train_loss = 0.159\n",
      "Epoch 156 Batch    4/26   train_loss = 0.150\n",
      "Epoch 156 Batch   14/26   train_loss = 0.159\n",
      "Epoch 156 Batch   24/26   train_loss = 0.140\n",
      "Epoch 157 Batch    8/26   train_loss = 0.157\n",
      "Epoch 157 Batch   18/26   train_loss = 0.153\n",
      "Epoch 158 Batch    2/26   train_loss = 0.159\n",
      "Epoch 158 Batch   12/26   train_loss = 0.147\n",
      "Epoch 158 Batch   22/26   train_loss = 0.165\n",
      "Epoch 159 Batch    6/26   train_loss = 0.155\n",
      "Epoch 159 Batch   16/26   train_loss = 0.171\n",
      "Epoch 160 Batch    0/26   train_loss = 0.160\n",
      "Epoch 160 Batch   10/26   train_loss = 0.170\n",
      "Epoch 160 Batch   20/26   train_loss = 0.159\n",
      "Epoch 161 Batch    4/26   train_loss = 0.148\n",
      "Epoch 161 Batch   14/26   train_loss = 0.160\n",
      "Epoch 161 Batch   24/26   train_loss = 0.138\n",
      "Epoch 162 Batch    8/26   train_loss = 0.157\n",
      "Epoch 162 Batch   18/26   train_loss = 0.152\n",
      "Epoch 163 Batch    2/26   train_loss = 0.159\n",
      "Epoch 163 Batch   12/26   train_loss = 0.148\n",
      "Epoch 163 Batch   22/26   train_loss = 0.163\n",
      "Epoch 164 Batch    6/26   train_loss = 0.154\n",
      "Epoch 164 Batch   16/26   train_loss = 0.170\n",
      "Epoch 165 Batch    0/26   train_loss = 0.163\n",
      "Epoch 165 Batch   10/26   train_loss = 0.169\n",
      "Epoch 165 Batch   20/26   train_loss = 0.157\n",
      "Epoch 166 Batch    4/26   train_loss = 0.148\n",
      "Epoch 166 Batch   14/26   train_loss = 0.161\n",
      "Epoch 166 Batch   24/26   train_loss = 0.136\n",
      "Epoch 167 Batch    8/26   train_loss = 0.154\n",
      "Epoch 167 Batch   18/26   train_loss = 0.152\n",
      "Epoch 168 Batch    2/26   train_loss = 0.157\n",
      "Epoch 168 Batch   12/26   train_loss = 0.145\n",
      "Epoch 168 Batch   22/26   train_loss = 0.162\n",
      "Epoch 169 Batch    6/26   train_loss = 0.153\n",
      "Epoch 169 Batch   16/26   train_loss = 0.167\n",
      "Epoch 170 Batch    0/26   train_loss = 0.156\n",
      "Epoch 170 Batch   10/26   train_loss = 0.166\n",
      "Epoch 170 Batch   20/26   train_loss = 0.155\n",
      "Epoch 171 Batch    4/26   train_loss = 0.146\n",
      "Epoch 171 Batch   14/26   train_loss = 0.157\n",
      "Epoch 171 Batch   24/26   train_loss = 0.136\n",
      "Epoch 172 Batch    8/26   train_loss = 0.152\n",
      "Epoch 172 Batch   18/26   train_loss = 0.150\n",
      "Epoch 173 Batch    2/26   train_loss = 0.155\n",
      "Epoch 173 Batch   12/26   train_loss = 0.144\n",
      "Epoch 173 Batch   22/26   train_loss = 0.159\n",
      "Epoch 174 Batch    6/26   train_loss = 0.151\n",
      "Epoch 174 Batch   16/26   train_loss = 0.166\n",
      "Epoch 175 Batch    0/26   train_loss = 0.151\n",
      "Epoch 175 Batch   10/26   train_loss = 0.163\n",
      "Epoch 175 Batch   20/26   train_loss = 0.153\n",
      "Epoch 176 Batch    4/26   train_loss = 0.144\n",
      "Epoch 176 Batch   14/26   train_loss = 0.155\n",
      "Epoch 176 Batch   24/26   train_loss = 0.134\n",
      "Epoch 177 Batch    8/26   train_loss = 0.151\n",
      "Epoch 177 Batch   18/26   train_loss = 0.147\n",
      "Epoch 178 Batch    2/26   train_loss = 0.153\n",
      "Epoch 178 Batch   12/26   train_loss = 0.142\n",
      "Epoch 178 Batch   22/26   train_loss = 0.158\n",
      "Epoch 179 Batch    6/26   train_loss = 0.150\n",
      "Epoch 179 Batch   16/26   train_loss = 0.164\n",
      "Epoch 180 Batch    0/26   train_loss = 0.150\n",
      "Epoch 180 Batch   10/26   train_loss = 0.162\n",
      "Epoch 180 Batch   20/26   train_loss = 0.152\n",
      "Epoch 181 Batch    4/26   train_loss = 0.142\n",
      "Epoch 181 Batch   14/26   train_loss = 0.153\n",
      "Epoch 181 Batch   24/26   train_loss = 0.133\n",
      "Epoch 182 Batch    8/26   train_loss = 0.149\n",
      "Epoch 182 Batch   18/26   train_loss = 0.147\n",
      "Epoch 183 Batch    2/26   train_loss = 0.152\n",
      "Epoch 183 Batch   12/26   train_loss = 0.141\n",
      "Epoch 183 Batch   22/26   train_loss = 0.157\n",
      "Epoch 184 Batch    6/26   train_loss = 0.149\n",
      "Epoch 184 Batch   16/26   train_loss = 0.163\n",
      "Epoch 185 Batch    0/26   train_loss = 0.149\n",
      "Epoch 185 Batch   10/26   train_loss = 0.162\n",
      "Epoch 185 Batch   20/26   train_loss = 0.151\n",
      "Epoch 186 Batch    4/26   train_loss = 0.142\n",
      "Epoch 186 Batch   14/26   train_loss = 0.153\n",
      "Epoch 186 Batch   24/26   train_loss = 0.133\n",
      "Epoch 187 Batch    8/26   train_loss = 0.149\n",
      "Epoch 187 Batch   18/26   train_loss = 0.146\n",
      "Epoch 188 Batch    2/26   train_loss = 0.152\n",
      "Epoch 188 Batch   12/26   train_loss = 0.140\n",
      "Epoch 188 Batch   22/26   train_loss = 0.156\n",
      "Epoch 189 Batch    6/26   train_loss = 0.148\n",
      "Epoch 189 Batch   16/26   train_loss = 0.163\n",
      "Epoch 190 Batch    0/26   train_loss = 0.148\n",
      "Epoch 190 Batch   10/26   train_loss = 0.161\n",
      "Epoch 190 Batch   20/26   train_loss = 0.151\n",
      "Epoch 191 Batch    4/26   train_loss = 0.141\n",
      "Epoch 191 Batch   14/26   train_loss = 0.152\n",
      "Epoch 191 Batch   24/26   train_loss = 0.132\n",
      "Epoch 192 Batch    8/26   train_loss = 0.148\n",
      "Epoch 192 Batch   18/26   train_loss = 0.146\n",
      "Epoch 193 Batch    2/26   train_loss = 0.151\n",
      "Epoch 193 Batch   12/26   train_loss = 0.140\n",
      "Epoch 193 Batch   22/26   train_loss = 0.156\n",
      "Epoch 194 Batch    6/26   train_loss = 0.148\n",
      "Epoch 194 Batch   16/26   train_loss = 0.162\n",
      "Epoch 195 Batch    0/26   train_loss = 0.148\n",
      "Epoch 195 Batch   10/26   train_loss = 0.161\n",
      "Epoch 195 Batch   20/26   train_loss = 0.151\n",
      "Epoch 196 Batch    4/26   train_loss = 0.141\n",
      "Epoch 196 Batch   14/26   train_loss = 0.152\n",
      "Epoch 196 Batch   24/26   train_loss = 0.132\n",
      "Epoch 197 Batch    8/26   train_loss = 0.148\n",
      "Epoch 197 Batch   18/26   train_loss = 0.145\n",
      "Epoch 198 Batch    2/26   train_loss = 0.151\n",
      "Epoch 198 Batch   12/26   train_loss = 0.140\n",
      "Epoch 198 Batch   22/26   train_loss = 0.156\n",
      "Epoch 199 Batch    6/26   train_loss = 0.148\n",
      "Epoch 199 Batch   16/26   train_loss = 0.162\n",
      "Epoch 200 Batch    0/26   train_loss = 0.147\n",
      "Epoch 200 Batch   10/26   train_loss = 0.161\n",
      "Epoch 200 Batch   20/26   train_loss = 0.151\n",
      "Epoch 201 Batch    4/26   train_loss = 0.140\n",
      "Epoch 201 Batch   14/26   train_loss = 0.152\n",
      "Epoch 201 Batch   24/26   train_loss = 0.131\n",
      "Epoch 202 Batch    8/26   train_loss = 0.147\n",
      "Epoch 202 Batch   18/26   train_loss = 0.145\n",
      "Epoch 203 Batch    2/26   train_loss = 0.151\n",
      "Epoch 203 Batch   12/26   train_loss = 0.139\n",
      "Epoch 203 Batch   22/26   train_loss = 0.155\n",
      "Epoch 204 Batch    6/26   train_loss = 0.147\n",
      "Epoch 204 Batch   16/26   train_loss = 0.162\n",
      "Epoch 205 Batch    0/26   train_loss = 0.147\n",
      "Epoch 205 Batch   10/26   train_loss = 0.160\n",
      "Epoch 205 Batch   20/26   train_loss = 0.149\n",
      "Epoch 206 Batch    4/26   train_loss = 0.139\n",
      "Epoch 206 Batch   14/26   train_loss = 0.151\n",
      "Epoch 206 Batch   24/26   train_loss = 0.131\n",
      "Epoch 207 Batch    8/26   train_loss = 0.147\n",
      "Epoch 207 Batch   18/26   train_loss = 0.144\n",
      "Epoch 208 Batch    2/26   train_loss = 0.150\n",
      "Epoch 208 Batch   12/26   train_loss = 0.139\n",
      "Epoch 208 Batch   22/26   train_loss = 0.155\n",
      "Epoch 209 Batch    6/26   train_loss = 0.146\n",
      "Epoch 209 Batch   16/26   train_loss = 0.161\n",
      "Epoch 210 Batch    0/26   train_loss = 0.146\n",
      "Epoch 210 Batch   10/26   train_loss = 0.160\n",
      "Epoch 210 Batch   20/26   train_loss = 0.150\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 211 Batch    4/26   train_loss = 0.141\n",
      "Epoch 211 Batch   14/26   train_loss = 0.153\n",
      "Epoch 211 Batch   24/26   train_loss = 0.136\n",
      "Epoch 212 Batch    8/26   train_loss = 0.155\n",
      "Epoch 212 Batch   18/26   train_loss = 0.164\n",
      "Epoch 213 Batch    2/26   train_loss = 0.210\n",
      "Epoch 213 Batch   12/26   train_loss = 0.285\n",
      "Epoch 213 Batch   22/26   train_loss = 0.534\n",
      "Epoch 214 Batch    6/26   train_loss = 0.697\n",
      "Epoch 214 Batch   16/26   train_loss = 0.801\n",
      "Epoch 215 Batch    0/26   train_loss = 0.747\n",
      "Epoch 215 Batch   10/26   train_loss = 0.697\n",
      "Epoch 215 Batch   20/26   train_loss = 0.607\n",
      "Epoch 216 Batch    4/26   train_loss = 0.529\n",
      "Epoch 216 Batch   14/26   train_loss = 0.452\n",
      "Epoch 216 Batch   24/26   train_loss = 0.396\n",
      "Epoch 217 Batch    8/26   train_loss = 0.359\n",
      "Epoch 217 Batch   18/26   train_loss = 0.328\n",
      "Epoch 218 Batch    2/26   train_loss = 0.298\n",
      "Epoch 218 Batch   12/26   train_loss = 0.243\n",
      "Epoch 218 Batch   22/26   train_loss = 0.260\n",
      "Epoch 219 Batch    6/26   train_loss = 0.228\n",
      "Epoch 219 Batch   16/26   train_loss = 0.240\n",
      "Epoch 220 Batch    0/26   train_loss = 0.215\n",
      "Epoch 220 Batch   10/26   train_loss = 0.207\n",
      "Epoch 220 Batch   20/26   train_loss = 0.200\n",
      "Epoch 221 Batch    4/26   train_loss = 0.185\n",
      "Epoch 221 Batch   14/26   train_loss = 0.187\n",
      "Epoch 221 Batch   24/26   train_loss = 0.165\n",
      "Epoch 222 Batch    8/26   train_loss = 0.178\n",
      "Epoch 222 Batch   18/26   train_loss = 0.172\n",
      "Epoch 223 Batch    2/26   train_loss = 0.175\n",
      "Epoch 223 Batch   12/26   train_loss = 0.162\n",
      "Epoch 223 Batch   22/26   train_loss = 0.178\n",
      "Epoch 224 Batch    6/26   train_loss = 0.166\n",
      "Epoch 224 Batch   16/26   train_loss = 0.182\n",
      "Epoch 225 Batch    0/26   train_loss = 0.166\n",
      "Epoch 225 Batch   10/26   train_loss = 0.177\n",
      "Epoch 225 Batch   20/26   train_loss = 0.166\n",
      "Epoch 226 Batch    4/26   train_loss = 0.157\n",
      "Epoch 226 Batch   14/26   train_loss = 0.168\n",
      "Epoch 226 Batch   24/26   train_loss = 0.145\n",
      "Epoch 227 Batch    8/26   train_loss = 0.160\n",
      "Epoch 227 Batch   18/26   train_loss = 0.158\n",
      "Epoch 228 Batch    2/26   train_loss = 0.164\n",
      "Epoch 228 Batch   12/26   train_loss = 0.151\n",
      "Epoch 228 Batch   22/26   train_loss = 0.166\n",
      "Epoch 229 Batch    6/26   train_loss = 0.158\n",
      "Epoch 229 Batch   16/26   train_loss = 0.172\n",
      "Epoch 230 Batch    0/26   train_loss = 0.158\n",
      "Epoch 230 Batch   10/26   train_loss = 0.170\n",
      "Epoch 230 Batch   20/26   train_loss = 0.159\n",
      "Epoch 231 Batch    4/26   train_loss = 0.150\n",
      "Epoch 231 Batch   14/26   train_loss = 0.161\n",
      "Epoch 231 Batch   24/26   train_loss = 0.141\n",
      "Epoch 232 Batch    8/26   train_loss = 0.155\n",
      "Epoch 232 Batch   18/26   train_loss = 0.153\n",
      "Epoch 233 Batch    2/26   train_loss = 0.159\n",
      "Epoch 233 Batch   12/26   train_loss = 0.148\n",
      "Epoch 233 Batch   22/26   train_loss = 0.163\n",
      "Epoch 234 Batch    6/26   train_loss = 0.154\n",
      "Epoch 234 Batch   16/26   train_loss = 0.169\n",
      "Epoch 235 Batch    0/26   train_loss = 0.154\n",
      "Epoch 235 Batch   10/26   train_loss = 0.167\n",
      "Epoch 235 Batch   20/26   train_loss = 0.156\n",
      "Epoch 236 Batch    4/26   train_loss = 0.147\n",
      "Epoch 236 Batch   14/26   train_loss = 0.158\n",
      "Epoch 236 Batch   24/26   train_loss = 0.138\n",
      "Epoch 237 Batch    8/26   train_loss = 0.152\n",
      "Epoch 237 Batch   18/26   train_loss = 0.150\n",
      "Epoch 238 Batch    2/26   train_loss = 0.157\n",
      "Epoch 238 Batch   12/26   train_loss = 0.145\n",
      "Epoch 238 Batch   22/26   train_loss = 0.160\n",
      "Epoch 239 Batch    6/26   train_loss = 0.152\n",
      "Epoch 239 Batch   16/26   train_loss = 0.166\n",
      "Epoch 240 Batch    0/26   train_loss = 0.152\n",
      "Epoch 240 Batch   10/26   train_loss = 0.164\n",
      "Epoch 240 Batch   20/26   train_loss = 0.154\n",
      "Epoch 241 Batch    4/26   train_loss = 0.145\n",
      "Epoch 241 Batch   14/26   train_loss = 0.156\n",
      "Epoch 241 Batch   24/26   train_loss = 0.136\n",
      "Epoch 242 Batch    8/26   train_loss = 0.151\n",
      "Epoch 242 Batch   18/26   train_loss = 0.149\n",
      "Epoch 243 Batch    2/26   train_loss = 0.155\n",
      "Epoch 243 Batch   12/26   train_loss = 0.143\n",
      "Epoch 243 Batch   22/26   train_loss = 0.159\n",
      "Epoch 244 Batch    6/26   train_loss = 0.150\n",
      "Epoch 244 Batch   16/26   train_loss = 0.165\n",
      "Epoch 245 Batch    0/26   train_loss = 0.150\n",
      "Epoch 245 Batch   10/26   train_loss = 0.163\n",
      "Epoch 245 Batch   20/26   train_loss = 0.153\n",
      "Epoch 246 Batch    4/26   train_loss = 0.143\n",
      "Epoch 246 Batch   14/26   train_loss = 0.155\n",
      "Epoch 246 Batch   24/26   train_loss = 0.134\n",
      "Epoch 247 Batch    8/26   train_loss = 0.150\n",
      "Epoch 247 Batch   18/26   train_loss = 0.147\n",
      "Epoch 248 Batch    2/26   train_loss = 0.153\n",
      "Epoch 248 Batch   12/26   train_loss = 0.142\n",
      "Epoch 248 Batch   22/26   train_loss = 0.158\n",
      "Epoch 249 Batch    6/26   train_loss = 0.149\n",
      "Epoch 249 Batch   16/26   train_loss = 0.163\n",
      "Epoch 250 Batch    0/26   train_loss = 0.149\n",
      "Epoch 250 Batch   10/26   train_loss = 0.162\n",
      "Epoch 250 Batch   20/26   train_loss = 0.152\n",
      "Epoch 251 Batch    4/26   train_loss = 0.142\n",
      "Epoch 251 Batch   14/26   train_loss = 0.153\n",
      "Epoch 251 Batch   24/26   train_loss = 0.134\n",
      "Epoch 252 Batch    8/26   train_loss = 0.149\n",
      "Epoch 252 Batch   18/26   train_loss = 0.146\n",
      "Epoch 253 Batch    2/26   train_loss = 0.152\n",
      "Epoch 253 Batch   12/26   train_loss = 0.141\n",
      "Epoch 253 Batch   22/26   train_loss = 0.157\n",
      "Epoch 254 Batch    6/26   train_loss = 0.148\n",
      "Epoch 254 Batch   16/26   train_loss = 0.162\n",
      "Epoch 255 Batch    0/26   train_loss = 0.148\n",
      "Epoch 255 Batch   10/26   train_loss = 0.161\n",
      "Epoch 255 Batch   20/26   train_loss = 0.151\n",
      "Epoch 256 Batch    4/26   train_loss = 0.141\n",
      "Epoch 256 Batch   14/26   train_loss = 0.153\n",
      "Epoch 256 Batch   24/26   train_loss = 0.133\n",
      "Epoch 257 Batch    8/26   train_loss = 0.148\n",
      "Epoch 257 Batch   18/26   train_loss = 0.146\n",
      "Epoch 258 Batch    2/26   train_loss = 0.151\n",
      "Epoch 258 Batch   12/26   train_loss = 0.140\n",
      "Epoch 258 Batch   22/26   train_loss = 0.156\n",
      "Epoch 259 Batch    6/26   train_loss = 0.148\n",
      "Epoch 259 Batch   16/26   train_loss = 0.162\n",
      "Epoch 260 Batch    0/26   train_loss = 0.147\n",
      "Epoch 260 Batch   10/26   train_loss = 0.161\n",
      "Epoch 260 Batch   20/26   train_loss = 0.150\n",
      "Epoch 261 Batch    4/26   train_loss = 0.140\n",
      "Epoch 261 Batch   14/26   train_loss = 0.152\n",
      "Epoch 261 Batch   24/26   train_loss = 0.132\n",
      "Epoch 262 Batch    8/26   train_loss = 0.147\n",
      "Epoch 262 Batch   18/26   train_loss = 0.145\n",
      "Epoch 263 Batch    2/26   train_loss = 0.151\n",
      "Epoch 263 Batch   12/26   train_loss = 0.140\n",
      "Epoch 263 Batch   22/26   train_loss = 0.155\n",
      "Epoch 264 Batch    6/26   train_loss = 0.147\n",
      "Epoch 264 Batch   16/26   train_loss = 0.161\n",
      "Epoch 265 Batch    0/26   train_loss = 0.147\n",
      "Epoch 265 Batch   10/26   train_loss = 0.160\n",
      "Epoch 265 Batch   20/26   train_loss = 0.150\n",
      "Epoch 266 Batch    4/26   train_loss = 0.139\n",
      "Epoch 266 Batch   14/26   train_loss = 0.151\n",
      "Epoch 266 Batch   24/26   train_loss = 0.132\n",
      "Epoch 267 Batch    8/26   train_loss = 0.147\n",
      "Epoch 267 Batch   18/26   train_loss = 0.144\n",
      "Epoch 268 Batch    2/26   train_loss = 0.150\n",
      "Epoch 268 Batch   12/26   train_loss = 0.139\n",
      "Epoch 268 Batch   22/26   train_loss = 0.155\n",
      "Epoch 269 Batch    6/26   train_loss = 0.147\n",
      "Epoch 269 Batch   16/26   train_loss = 0.161\n",
      "Epoch 270 Batch    0/26   train_loss = 0.146\n",
      "Epoch 270 Batch   10/26   train_loss = 0.160\n",
      "Epoch 270 Batch   20/26   train_loss = 0.149\n",
      "Epoch 271 Batch    4/26   train_loss = 0.139\n",
      "Epoch 271 Batch   14/26   train_loss = 0.151\n",
      "Epoch 271 Batch   24/26   train_loss = 0.131\n",
      "Epoch 272 Batch    8/26   train_loss = 0.146\n",
      "Epoch 272 Batch   18/26   train_loss = 0.144\n",
      "Epoch 273 Batch    2/26   train_loss = 0.150\n",
      "Epoch 273 Batch   12/26   train_loss = 0.139\n",
      "Epoch 273 Batch   22/26   train_loss = 0.154\n",
      "Epoch 274 Batch    6/26   train_loss = 0.146\n",
      "Epoch 274 Batch   16/26   train_loss = 0.160\n",
      "Epoch 275 Batch    0/26   train_loss = 0.146\n",
      "Epoch 275 Batch   10/26   train_loss = 0.159\n",
      "Epoch 275 Batch   20/26   train_loss = 0.149\n",
      "Epoch 276 Batch    4/26   train_loss = 0.138\n",
      "Epoch 276 Batch   14/26   train_loss = 0.150\n",
      "Epoch 276 Batch   24/26   train_loss = 0.131\n",
      "Epoch 277 Batch    8/26   train_loss = 0.146\n",
      "Epoch 277 Batch   18/26   train_loss = 0.143\n",
      "Epoch 278 Batch    2/26   train_loss = 0.149\n",
      "Epoch 278 Batch   12/26   train_loss = 0.138\n",
      "Epoch 278 Batch   22/26   train_loss = 0.154\n",
      "Epoch 279 Batch    6/26   train_loss = 0.146\n",
      "Epoch 279 Batch   16/26   train_loss = 0.160\n",
      "Epoch 280 Batch    0/26   train_loss = 0.145\n",
      "Epoch 280 Batch   10/26   train_loss = 0.159\n",
      "Epoch 280 Batch   20/26   train_loss = 0.148\n",
      "Epoch 281 Batch    4/26   train_loss = 0.138\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 281 Batch   14/26   train_loss = 0.150\n",
      "Epoch 281 Batch   24/26   train_loss = 0.130\n",
      "Epoch 282 Batch    8/26   train_loss = 0.146\n",
      "Epoch 282 Batch   18/26   train_loss = 0.143\n",
      "Epoch 283 Batch    2/26   train_loss = 0.149\n",
      "Epoch 283 Batch   12/26   train_loss = 0.138\n",
      "Epoch 283 Batch   22/26   train_loss = 0.154\n",
      "Epoch 284 Batch    6/26   train_loss = 0.145\n",
      "Epoch 284 Batch   16/26   train_loss = 0.159\n",
      "Epoch 285 Batch    0/26   train_loss = 0.145\n",
      "Epoch 285 Batch   10/26   train_loss = 0.158\n",
      "Epoch 285 Batch   20/26   train_loss = 0.148\n",
      "Epoch 286 Batch    4/26   train_loss = 0.138\n",
      "Epoch 286 Batch   14/26   train_loss = 0.149\n",
      "Epoch 286 Batch   24/26   train_loss = 0.130\n",
      "Epoch 287 Batch    8/26   train_loss = 0.145\n",
      "Epoch 287 Batch   18/26   train_loss = 0.143\n",
      "Epoch 288 Batch    2/26   train_loss = 0.149\n",
      "Epoch 288 Batch   12/26   train_loss = 0.137\n",
      "Epoch 288 Batch   22/26   train_loss = 0.153\n",
      "Epoch 289 Batch    6/26   train_loss = 0.145\n",
      "Epoch 289 Batch   16/26   train_loss = 0.159\n",
      "Epoch 290 Batch    0/26   train_loss = 0.145\n",
      "Epoch 290 Batch   10/26   train_loss = 0.158\n",
      "Epoch 290 Batch   20/26   train_loss = 0.148\n",
      "Epoch 291 Batch    4/26   train_loss = 0.137\n",
      "Epoch 291 Batch   14/26   train_loss = 0.150\n",
      "Epoch 291 Batch   24/26   train_loss = 0.130\n",
      "Epoch 292 Batch    8/26   train_loss = 0.145\n",
      "Epoch 292 Batch   18/26   train_loss = 0.143\n",
      "Epoch 293 Batch    2/26   train_loss = 0.148\n",
      "Epoch 293 Batch   12/26   train_loss = 0.137\n",
      "Epoch 293 Batch   22/26   train_loss = 0.153\n",
      "Epoch 294 Batch    6/26   train_loss = 0.145\n",
      "Epoch 294 Batch   16/26   train_loss = 0.159\n",
      "Epoch 295 Batch    0/26   train_loss = 0.144\n",
      "Epoch 295 Batch   10/26   train_loss = 0.158\n",
      "Epoch 295 Batch   20/26   train_loss = 0.148\n",
      "Epoch 296 Batch    4/26   train_loss = 0.137\n",
      "Epoch 296 Batch   14/26   train_loss = 0.149\n",
      "Epoch 296 Batch   24/26   train_loss = 0.130\n",
      "Epoch 297 Batch    8/26   train_loss = 0.145\n",
      "Epoch 297 Batch   18/26   train_loss = 0.142\n",
      "Epoch 298 Batch    2/26   train_loss = 0.148\n",
      "Epoch 298 Batch   12/26   train_loss = 0.137\n",
      "Epoch 298 Batch   22/26   train_loss = 0.153\n",
      "Epoch 299 Batch    6/26   train_loss = 0.145\n",
      "Epoch 299 Batch   16/26   train_loss = 0.159\n",
      "Model Trained and Saved\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "batches = get_batches(int_text, batch_size, seq_length)\n",
    "\n",
    "with tf.Session(graph=train_graph) as sess:\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    for epoch_i in range(num_epochs):\n",
    "        state = sess.run(initial_state, {input_text: batches[0][0]})\n",
    "\n",
    "        for batch_i, (x, y) in enumerate(batches):\n",
    "            feed = {\n",
    "                input_text: x,\n",
    "                targets: y,\n",
    "                initial_state: state,\n",
    "                lr: learning_rate}\n",
    "            train_loss, state, _ = sess.run([cost, final_state, train_op], feed)\n",
    "\n",
    "            # Show every <show_every_n_batches> batches\n",
    "            if (epoch_i * len(batches) + batch_i) % show_every_n_batches == 0:\n",
    "                print('Epoch {:>3} Batch {:>4}/{}   train_loss = {:.3f}'.format(\n",
    "                    epoch_i,\n",
    "                    batch_i,\n",
    "                    len(batches),\n",
    "                    train_loss))\n",
    "\n",
    "    # Save Model\n",
    "    saver = tf.train.Saver()\n",
    "    saver.save(sess, save_dir)\n",
    "    print('Model Trained and Saved')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 储存参数\n",
    "储存 `seq_length` 和 `save_dir` 来生成新的电视剧剧本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "# Save parameters for checkpoint\n",
    "helper.save_params((seq_length, save_dir))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 检查点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import helper\n",
    "import problem_unittests as tests\n",
    "\n",
    "_, vocab_to_int, int_to_vocab, token_dict = helper.load_preprocess()\n",
    "seq_length, load_dir = helper.load_params()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实现生成函数\n",
    "### 获取 Tensors\n",
    "使用 [`get_tensor_by_name()`](https://www.tensorflow.org/api_docs/python/tf/Graph#get_tensor_by_name)函数从 `loaded_graph` 中获取 tensor。  使用下面的名称获取 tensor：\n",
    "\n",
    "- \"input:0\"\n",
    "- \"initial_state:0\"\n",
    "- \"final_state:0\"\n",
    "- \"probs:0\"\n",
    "\n",
    "返回下列元组中的 tensor `(InputTensor, InitialStateTensor, FinalStateTensor, ProbsTensor)`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def get_tensors(loaded_graph):\n",
    "    \"\"\"\n",
    "    Get input, initial state, final state, and probabilities tensor from <loaded_graph>\n",
    "    :param loaded_graph: TensorFlow graph loaded from file\n",
    "    :return: Tuple (InputTensor, InitialStateTensor, FinalStateTensor, ProbsTensor)\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    InputTensor = loaded_graph.get_tensor_by_name('input:0')\n",
    "    InitialStateTensor = loaded_graph.get_tensor_by_name('initial_state:0')\n",
    "    FinalStateTensor = loaded_graph.get_tensor_by_name('final_state:0')\n",
    "    ProbsTensor = loaded_graph.get_tensor_by_name('probs:0')\n",
    "    return (InputTensor, InitialStateTensor, FinalStateTensor, ProbsTensor)\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_get_tensors(get_tensors)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 选择词汇\n",
    "实现 `pick_word()` 函数来使用 `probabilities` 选择下一个词汇。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def pick_word(probabilities, int_to_vocab):\n",
    "    \"\"\"\n",
    "    Pick the next word in the generated text\n",
    "    :param probabilities: Probabilites of the next word\n",
    "    :param int_to_vocab: Dictionary of word ids as the keys and words as the values\n",
    "    :return: String of the predicted word\n",
    "    \"\"\"\n",
    "    #print(probabilities)\n",
    "    # TODO: Implement Function\n",
    "    #index = np.random.choice(len(probabilities), 1, p=probabilities)  \n",
    "    #print(int_to_vocab[index])\n",
    "    return np.random.choice(list(int_to_vocab.values()),1,p=probabilities)[0]\n",
    "    \n",
    "    \n",
    "    \n",
    "    \n",
    "   \n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_pick_word(pick_word)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 生成电视剧剧本\n",
    "这将为你生成一个电视剧剧本。通过设置 `gen_length` 来调整你想生成的剧本长度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "moe_szyslak:(cheery) hey everybody! homer's back!\n",
      "moe_szyslak:(\" back and\"!\n",
      "moe_szyslak:(exasperated sigh) all right, look, i-i gotta go.\n",
      "moe_szyslak: oh i'm afraid lenny's dead.\n",
      "doreen:(quiet shock) aw, yeah, rumor has it, who are you, nuts. i play jack handle, a round of it.\n",
      "barney_gumble: no, i was.....(indignant) i might be this drink to the uh, in kid.\n",
      "marge_simpson:(torn) i guess it is time to move on... no kneeling....\n",
      "homer_simpson: so you didn't rip out?\n",
      "moe_szyslak:(embarrassed) whoa, tha...(laughs) that, that's a project to be no people.\n",
      "homer_simpson: yes. eventually, i became the most i gotta do or two nickels to school\n",
      "\n",
      "\n",
      "carl_carlson: huh?\n",
      "barney_gumble: no. i guess carl? or passports to a\" flaming moe\" your man.\n",
      "homer_simpson: lenny, you mean like my.\n",
      "marge_simpson: homer, you're supposed to be a children's story, don't you can't inspire terror in your fellow and give me and kill.\n",
      "homer_simpson: it's your own fault. you gotta do!\n",
      "homer_simpson:(tough) just get the back of the game.\n",
      "lenny_leonard: oh, homer. i want a thing with skinner.\n",
      "homer_simpson: i'm not ashamed.\n",
      "moe_szyslak: hey, homer. if you're havin' a banquet for marge, i'd like to help.\n",
      "larry:(to moe, out) i could say that he could never forget your kids' names.\n",
      "lenny_leonard:(surprised) really? you're kiddin' me.\n",
      "lenny_leonard: homer, don't be that. y-you made a mistake, but i guess...(quickly points to items) shotgun, you're looking for three, moe.\n",
      "dr. _julius_hibbert:(looking around) oh, i'm sorry. i thought you didn't.\n",
      "carl_carlson: wait, the frog in the bar, but uh... hunky dory there and spread that message for me.\n",
      "homer_simpson: guys, i... hello to me.\n",
      "homer_simpson:(gasps) gr-aargh! oh my god! go! i've been replaced by a cooler!\n",
      "moe_szyslak: and who are you, crazy? oh, man, you got any idea what direction mecca's in?\n",
      "homer_simpson: listen, moe. if it's like that... inclination...\n",
      "moe_szyslak: sorry, harv. the.\n",
      "barney_gumble:(toasting) to marge! my favorite aggie!\n",
      "homer_simpson:(small laugh) yeah, if you tip.\n",
      "moe_szyslak: well, i gotta tell ya car, that would really make some good friends, we gotta have express.\n",
      "moe_szyslak: uh, listen up. there, lenny.\n",
      "moe_szyslak: oh no, i'm not!\n",
      "moe_szyslak: homer, this was taken. i know a thing or two about tow-joes. i married three of these days.\n",
      "homer_simpson: and i gave that man directions, even though i didn't know the way. 'cause that's the kind of adventure you treat a lot of 'cause he says he left of my friend!\n",
      "moe_szyslak:\" no acronyms!\" homer,\" like\" pian-ee!\"\n",
      "moe_szyslak: and she trashed my bar!(looks around) oh no, wait. she thinks i'm selfish, she thinks i don't spend enough time with the midnight train to drown it.\n",
      "homer_simpson: hey, moe, send him magazine subscriptions he don't wish?\n",
      "moe_szyslak:(sympathetic) aw, that's a shame.\n",
      "carl_carlson:(excited) homer, that ain't welcome here...(looking at thumb) although i do like the occasional beer...(quickly points to items) shotgun, thanks, bret.\"\n",
      "c. _montgomery_burns: yes. i say, i really that\" the guy i was a drink again, then he doesn't stand a huge settlement at him.\n",
      "\n",
      "\n",
      "homer_simpson: and, uh, usin' the phone's a four-drink minimum.\n",
      "moe_szyslak:(aside to skinner.) well i'll just get it, moe.\n",
      "moe_szyslak: hey guys, you want me into a ring... well\n"
     ]
    }
   ],
   "source": [
    "gen_length = 800\n",
    "# homer_simpson, moe_szyslak, or Barney_Gumble\n",
    "prime_word = 'moe_szyslak'\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "loaded_graph = tf.Graph()\n",
    "with tf.Session(graph=loaded_graph) as sess:\n",
    "    # Load saved model\n",
    "    loader = tf.train.import_meta_graph(load_dir + '.meta')\n",
    "    loader.restore(sess, load_dir)\n",
    "\n",
    "    # Get Tensors from loaded model\n",
    "    input_text, initial_state, final_state, probs = get_tensors(loaded_graph)\n",
    "\n",
    "    # Sentences generation setup\n",
    "    gen_sentences = [prime_word + ':']\n",
    "    prev_state = sess.run(initial_state, {input_text: np.array([[1]])})\n",
    "\n",
    "    # Generate sentences\n",
    "    for n in range(gen_length):\n",
    "        # Dynamic Input\n",
    "        dyn_input = [[vocab_to_int[word] for word in gen_sentences[-seq_length:]]]\n",
    "        dyn_seq_length = len(dyn_input[0])\n",
    "\n",
    "        # Get Prediction\n",
    "        probabilities, prev_state = sess.run(\n",
    "            [probs, final_state],\n",
    "            {input_text: dyn_input, initial_state: prev_state})\n",
    "        \n",
    "        pred_word = pick_word(probabilities[dyn_seq_length-1], int_to_vocab)\n",
    "\n",
    "        gen_sentences.append(pred_word)\n",
    "    \n",
    "    # Remove tokens\n",
    "    tv_script = ' '.join(gen_sentences)\n",
    "    for key, token in token_dict.items():\n",
    "        ending = ' ' if key in ['\\n', '(', '\"'] else ''\n",
    "        tv_script = tv_script.replace(' ' + token.lower(), key)\n",
    "    tv_script = tv_script.replace('\\n ', '\\n')\n",
    "    tv_script = tv_script.replace('( ', '(')\n",
    "        \n",
    "    print(tv_script)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 这个电视剧剧本是无意义的\n",
    "如果这个电视剧剧本毫无意义，那也没有关系。我们的训练文本不到一兆字节。为了获得更好的结果，你需要使用更小的词汇范围或是更多数据。幸运的是，我们的确拥有更多数据！在本项目开始之初我们也曾提过，这是[另一个数据集](https://www.kaggle.com/wcukierski/the-simpsons-by-the-data)的子集。我们并没有让你基于所有数据进行训练，因为这将耗费大量时间。然而，你可以随意使用这些数据训练你的神经网络。当然，是在完成本项目之后。\n",
    "# 提交项目\n",
    "在提交项目时，请确保你在保存 notebook 前运行了所有的单元格代码。请将 notebook 文件保存为 \"dlnd_tv_script_generation.ipynb\"，并将它作为 HTML 文件保存在 \"File\" -> \"Download as\" 中。请将 \"helper.py\" 和 \"problem_unittests.py\" 文件一并提交。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:tv-script]",
   "language": "python",
   "name": "conda-env-tv-script-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}