tensorflow_top_200.ipynb

{
	"cells": [
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"# TensorFlow 常用方法示例 (Top 200)\n",
				"本 Notebook 包含约 200 个 **TensorFlow 2.x** 常用方法/功能的简单示例代码，每个示例位于一个独立单元格。\n",
				"\n",
				"**说明：**\n",
				"- 为便于阅读和演示，示例中大多使用 `import tensorflow as tf` 并启用 Eager Execution（默认为 TF2）；\n",
				"- 若需 GPU 加速，请自行安装 CUDA/CuDNN 并确认合适版本；\n",
				"- 仅示范常用/常见场景，部分方法仅作提示；\n",
				"- 在 Jupyter Notebook 中运行时，依赖 `tensorflow>=2.x`。\n",
				"\n",
				"完成后，可在 Jupyter Notebook 中依次执行全部单元格并查看效果。"
			]
		},
		{
			"cell_type": "code",
			"execution_count": 9,
			"id": "fbf7309c",
			"metadata": {},
			"outputs": [
				{
					"name": "stdout",
					"output_type": "stream",
					"text": [
						"Collecting tf-nightly\n",
						"  Downloading tf_nightly-2.19.0.dev20250121-cp311-cp311-win_amd64.whl.metadata (3.3 kB)\n",
						"Collecting tf-nightly-intel==2.19.0-dev20250121 (from tf-nightly)\n",
						"  Downloading tf_nightly_intel-2.19.0.dev20250121-cp311-cp311-win_amd64.whl.metadata (4.1 kB)\n",
						"Requirement already satisfied: absl-py>=1.0.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (2.1.0)\n",
						"Requirement already satisfied: astunparse>=1.6.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (1.6.3)\n",
						"Requirement already satisfied: flatbuffers>=24.3.25 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (25.1.21)\n",
						"Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.6.0)\n",
						"Requirement already satisfied: google-pasta>=0.1.1 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.2.0)\n",
						"Requirement already satisfied: libclang>=13.0.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (18.1.1)\n",
						"Requirement already satisfied: opt-einsum>=2.3.2 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (3.4.0)\n",
						"Requirement already satisfied: packaging in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (24.2)\n",
						"Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<6.0.0dev,>=3.20.3 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (5.29.3)\n",
						"Requirement already satisfied: requests<3,>=2.21.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (2.32.3)\n",
						"Requirement already satisfied: setuptools in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (75.8.0)\n",
						"Requirement already satisfied: six>=1.12.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (1.17.0)\n",
						"Requirement already satisfied: termcolor>=1.1.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (2.5.0)\n",
						"Requirement already satisfied: typing-extensions>=3.6.6 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (4.12.2)\n",
						"Requirement already satisfied: wrapt>=1.11.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (1.17.2)\n",
						"Requirement already satisfied: grpcio<2.0,>=1.24.3 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (1.69.0)\n",
						"Collecting tb-nightly~=2.19.0.a (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly)\n",
						"  Downloading tb_nightly-2.19.0a20250121-py3-none-any.whl.metadata (1.8 kB)\n",
						"Collecting keras-nightly>=3.6.0.dev (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly)\n",
						"  Downloading keras_nightly-3.8.0.dev2025012203-py3-none-any.whl.metadata (6.1 kB)\n",
						"Requirement already satisfied: numpy<2.2.0,>=1.26.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (2.0.2)\n",
						"Requirement already satisfied: h5py>=3.11.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (3.12.1)\n",
						"Requirement already satisfied: ml-dtypes<1.0.0,>=0.4.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.4.1)\n",
						"Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.31.0)\n",
						"Requirement already satisfied: wheel<1.0,>=0.23.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from astunparse>=1.6.0->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.45.1)\n",
						"Requirement already satisfied: rich in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keras-nightly>=3.6.0.dev->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (13.9.4)\n",
						"Requirement already satisfied: namex in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keras-nightly>=3.6.0.dev->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.0.8)\n",
						"Requirement already satisfied: optree in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from keras-nightly>=3.6.0.dev->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.14.0)\n",
						"Requirement already satisfied: charset-normalizer<4,>=2 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from requests<3,>=2.21.0->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (3.4.1)\n",
						"Requirement already satisfied: idna<4,>=2.5 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from requests<3,>=2.21.0->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (3.10)\n",
						"Requirement already satisfied: urllib3<3,>=1.21.1 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from requests<3,>=2.21.0->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (2.3.0)\n",
						"Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from requests<3,>=2.21.0->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (2024.12.14)\n",
						"Requirement already satisfied: markdown>=2.6.8 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tb-nightly~=2.19.0.a->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (3.7)\n",
						"Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tb-nightly~=2.19.0.a->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.7.2)\n",
						"Requirement already satisfied: werkzeug>=1.0.1 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from tb-nightly~=2.19.0.a->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (3.1.3)\n",
						"Requirement already satisfied: MarkupSafe>=2.1.1 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from werkzeug>=1.0.1->tb-nightly~=2.19.0.a->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (3.0.2)\n",
						"Requirement already satisfied: markdown-it-py>=2.2.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from rich->keras-nightly>=3.6.0.dev->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (3.0.0)\n",
						"Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from rich->keras-nightly>=3.6.0.dev->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (2.19.1)\n",
						"Requirement already satisfied: mdurl~=0.1 in c:\\users\\minqliu\\appdata\\local\\packages\\pythonsoftwarefoundation.python.3.11_qbz5n2kfra8p0\\localcache\\local-packages\\python311\\site-packages (from markdown-it-py>=2.2.0->rich->keras-nightly>=3.6.0.dev->tf-nightly-intel==2.19.0-dev20250121->tf-nightly) (0.1.2)\n",
						"Downloading tf_nightly-2.19.0.dev20250121-cp311-cp311-win_amd64.whl (7.7 kB)\n",
						"Downloading tf_nightly_intel-2.19.0.dev20250121-cp311-cp311-win_amd64.whl (237.1 MB)\n",
						"   ---------------------------------------- 0.0/237.1 MB ? eta -:--:--\n",
						"   ------- ------------------------------- 43.8/237.1 MB 214.4 MB/s eta 0:00:01\n",
						"   --------------- ----------------------- 93.3/237.1 MB 229.2 MB/s eta 0:00:01\n",
						"   --------------------- ---------------- 136.8/237.1 MB 224.3 MB/s eta 0:00:01\n",
						"   ----------------------------- -------- 183.8/237.1 MB 221.6 MB/s eta 0:00:01\n",
						"   ----------------------------------- -- 219.7/237.1 MB 212.8 MB/s eta 0:00:01\n",
						"   -------------------------------------  237.0/237.1 MB 210.4 MB/s eta 0:00:01\n",
						"   -------------------------------------  237.0/237.1 MB 210.4 MB/s eta 0:00:01\n",
						"   -------------------------------------  237.0/237.1 MB 210.4 MB/s eta 0:00:01\n",
						"   -------------------------------------  237.0/237.1 MB 210.4 MB/s eta 0:00:01\n",
						"   -------------------------------------  237.0/237.1 MB 210.4 MB/s eta 0:00:01\n",
						"   -------------------------------------- 237.1/237.1 MB 111.4 MB/s eta 0:00:00\n",
						"Downloading keras_nightly-3.8.0.dev2025012203-py3-none-any.whl (1.3 MB)\n",
						"   ---------------------------------------- 0.0/1.3 MB ? eta -:--:--\n",
						"   ---------------------------------------- 1.3/1.3 MB 17.0 MB/s eta 0:00:00\n",
						"Downloading tb_nightly-2.19.0a20250121-py3-none-any.whl (5.5 MB)\n",
						"   ---------------------------------------- 0.0/5.5 MB ? eta -:--:--\n",
						"   ---------------------------------------- 5.5/5.5 MB 111.5 MB/s eta 0:00:00\n",
						"Installing collected packages: tb-nightly, keras-nightly, tf-nightly-intel, tf-nightly\n",
						"Note: you may need to restart the kernel to use updated packages.\n"
					]
				},
				{
					"name": "stderr",
					"output_type": "stream",
					"text": [
						"ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 'C:\\\\Users\\\\minqliu\\\\AppData\\\\Local\\\\Packages\\\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\\\LocalCache\\\\local-packages\\\\Python311\\\\site-packages\\\\tensorflow\\\\include\\\\external\\\\com_github_grpc_grpc\\\\src\\\\core\\\\ext\\\\filters\\\\client_channel\\\\lb_policy\\\\grpclb\\\\client_load_reporting_filter.h'\n",
						"HINT: This error might have occurred since this system does not have Windows Long Path support enabled. You can find information on how to enable this at https://pip.pypa.io/warnings/enable-long-paths\n",
						"\n"
					]
				}
			],
			"source": [
				"pip install tf-nightly"
			]
		},
		{
			"cell_type": "code",
			"execution_count": 11,
			"metadata": {},
			"outputs": [
				{
					"ename": "ModuleNotFoundError",
					"evalue": "No module named 'tensorflow.python'",
					"output_type": "error",
					"traceback": [
						"\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
						"\u001b[1;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
						"Cell \u001b[1;32mIn[11], line 2\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;66;03m# 1. 导入与版本检查\u001b[39;00m\n\u001b[1;32m----> 2\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21;01mtensorflow\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mas\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21;01mtf\u001b[39;00m\n\u001b[0;32m      3\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mTensorFlow version:\u001b[39m\u001b[38;5;124m\"\u001b[39m, tf\u001b[38;5;241m.\u001b[39m__version__)\n",
						"File \u001b[1;32m~\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python311\\site-packages\\tensorflow\\__init__.py:40\u001b[0m\n\u001b[0;32m     37\u001b[0m _os\u001b[38;5;241m.\u001b[39menviron\u001b[38;5;241m.\u001b[39msetdefault(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mENABLE_RUNTIME_UPTIME_TELEMETRY\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m1\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m     39\u001b[0m \u001b[38;5;66;03m# Do not remove this line; See https://github.com/tensorflow/tensorflow/issues/42596\u001b[39;00m\n\u001b[1;32m---> 40\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21;01mtensorflow\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mpython\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m pywrap_tensorflow \u001b[38;5;28;01mas\u001b[39;00m _pywrap_tensorflow  \u001b[38;5;66;03m# pylint: disable=unused-import\u001b[39;00m\n\u001b[0;32m     41\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21;01mtensorflow\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mpython\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mtools\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m module_util \u001b[38;5;28;01mas\u001b[39;00m _module_util\n\u001b[0;32m     42\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;21;01mtensorflow\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mpython\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mutil\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mlazy_loader\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[38;5;28;01mimport\u001b[39;00m KerasLazyLoader \u001b[38;5;28;01mas\u001b[39;00m _KerasLazyLoader\n",
						"\u001b[1;31mModuleNotFoundError\u001b[0m: No module named 'tensorflow.python'"
					]
				}
			],
			"source": [
				"# 1. 导入与版本检查\n",
				"import tensorflow as tf\n",
				"print(\"TensorFlow version:\", tf.__version__)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 2. tf.constant - 创建常量张量\n",
				"a = tf.constant([1, 2, 3], dtype=tf.float32)\n",
				"print(a)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 3. tf.Variable - 创建可训练变量\n",
				"b = tf.Variable([1.0, 2.0, 3.0])\n",
				"print(b)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 4. 张量形状与dtype\n",
				"print(\"Shape:\", a.shape, \"Dtype:\", a.dtype)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 5. tf.cast - 类型转换\n",
				"c = tf.cast(a, tf.int32)\n",
				"print(c, c.dtype)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 6. tf.zeros, tf.ones, tf.fill - 创建特殊张量\n",
				"z = tf.zeros([2,3])\n",
				"o = tf.ones([2,3])\n",
				"f = tf.fill([2,3], 7)\n",
				"print(\"zeros:\\n\", z)\n",
				"print(\"ones:\\n\", o)\n",
				"print(\"fill 7:\\n\", f)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 7. tf.eye - 单位矩阵\n",
				"eye_mat = tf.eye(3)\n",
				"print(eye_mat)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 8. tf.range - 类似 Python range\n",
				"r = tf.range(5)\n",
				"r2 = tf.range(start=1, limit=6, delta=2)\n",
				"print(r)\n",
				"print(r2)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 9. tf.linspace - 等距序列\n",
				"lin = tf.linspace(0.0, 1.0, 5)\n",
				"print(lin)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 10. tf.reshape - 调整张量形状\n",
				"x = tf.constant([[1,2,3],[4,5,6]])\n",
				"x_reshaped = tf.reshape(x, [3,2])\n",
				"print(x_reshaped)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 11. tf.transpose - 转置\n",
				"xt = tf.transpose(x)\n",
				"print(\"Original:\", x)\n",
				"print(\"Transposed:\", xt)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 12. tf.expand_dims / tf.squeeze\n",
				"y = tf.constant([1,2,3])\n",
				"y_expanded = tf.expand_dims(y, axis=1)\n",
				"y_squeezed = tf.squeeze(y_expanded)\n",
				"print(\"Expanded:\", y_expanded)\n",
				"print(\"Squeezed:\", y_squeezed)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 13. tf.concat - 拼接张量\n",
				"t1 = tf.constant([[1,2],[3,4]])\n",
				"t2 = tf.constant([[5,6],[7,8]])\n",
				"con = tf.concat([t1, t2], axis=0)\n",
				"print(con)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 14. tf.stack - 堆叠\n",
				"s1 = tf.constant([1,2])\n",
				"s2 = tf.constant([3,4])\n",
				"stacked = tf.stack([s1, s2], axis=0)\n",
				"print(stacked)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 15. tf.split - 拆分\n",
				"arr = tf.constant([1,2,3,4,5,6])\n",
				"splt = tf.split(arr, num_or_size_splits=3)\n",
				"print(splt)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 16. tf.tile - 张量复制\n",
				"val = tf.constant([[1,2],[3,4]])\n",
				"tiled = tf.tile(val, [2,3])\n",
				"print(tiled)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 17. tf.gather - 按索引采样\n",
				"vec = tf.constant([10,20,30,40])\n",
				"idx = tf.constant([1,3])\n",
				"res_gather = tf.gather(vec, idx)\n",
				"print(res_gather)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 18. tf.gather_nd\n",
				"mat = tf.constant([[1,2],[3,4],[5,6]])\n",
				"indices = tf.constant([[0,1],[2,0]])\n",
				"gnd = tf.gather_nd(mat, indices)\n",
				"print(gnd)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 19. tf.slice - 按切片操作\n",
				"arr2d = tf.constant([[1,2,3],[4,5,6],[7,8,9]])\n",
				"sl = tf.slice(arr2d, [0,1], [2,2])\n",
				"print(sl)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 20. tf.unstack - 沿轴拆分为列表\n",
				"un = tf.unstack(arr2d, axis=1)\n",
				"print(\"Len:\", len(un), \"Element0:\", un[0])"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 21. tf.boolean_mask\n",
				"arr_mask = tf.constant([1,2,3,4,5])\n",
				"mask = arr_mask > 3\n",
				"masked = tf.boolean_mask(arr_mask, mask)\n",
				"print(masked)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 22. tf.one_hot\n",
				"labels = tf.constant([0,1,2])\n",
				"one_hot_labels = tf.one_hot(labels, depth=3)\n",
				"print(one_hot_labels)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 23. 基本算术操作: +, -, * , /, etc.\n",
				"a = tf.constant([10,20,30], dtype=tf.float32)\n",
				"b = tf.constant([1,2,3], dtype=tf.float32)\n",
				"print(\"a+b=\", a+b)\n",
				"print(\"a*b=\", a*b)\n",
				"print(\"a/b=\", a/b)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 24. tf.add, tf.subtract, tf.multiply, tf.divide\n",
				"add_res = tf.add(a,b)\n",
				"sub_res = tf.subtract(a,b)\n",
				"mul_res = tf.multiply(a,b)\n",
				"div_res = tf.divide(a,b)\n",
				"print(add_res, sub_res, mul_res, div_res)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 25. tf.math.mod / tf.math.floordiv\n",
				"mod_res = tf.math.mod(a, b)\n",
				"floor_div_res = tf.math.floordiv(a, b)\n",
				"print(mod_res, floor_div_res)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 26. tf.matmul - 矩阵乘法\n",
				"mat1 = tf.constant([[1,2],[3,4]])\n",
				"mat2 = tf.constant([[5,6],[7,8]])\n",
				"prod = tf.matmul(mat1, mat2)\n",
				"print(prod)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 27. tf.linalg.matmul (同上) + 其他线性代数操作\n",
				"prod2 = tf.linalg.matmul(mat1, mat2)\n",
				"inv = tf.linalg.inv(tf.cast(mat1, tf.float32))\n",
				"det = tf.linalg.det(tf.cast(mat2, tf.float32))\n",
				"print(\"prod2=\", prod2)\n",
				"print(\"inv=\", inv)\n",
				"print(\"det=\", det)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 28. tf.reduce_sum / tf.reduce_mean / tf.reduce_max\n",
				"arr_2d = tf.constant([[1,2,3],[4,5,6]], dtype=tf.float32)\n",
				"sum_all = tf.reduce_sum(arr_2d)\n",
				"mean_0 = tf.reduce_mean(arr_2d, axis=0)\n",
				"max_1 = tf.reduce_max(arr_2d, axis=1)\n",
				"print(\"sum_all=\", sum_all)\n",
				"print(\"mean_0=\", mean_0)\n",
				"print(\"max_1=\", max_1)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 29. tf.reduce_min / tf.reduce_prod / tf.reduce_any / tf.reduce_all\n",
				"mn = tf.reduce_min(arr_2d)\n",
				"pr = tf.reduce_prod(arr_2d)\n",
				"bools = tf.constant([[True,False],[True,True]])\n",
				"any_ = tf.reduce_any(bools)\n",
				"all_ = tf.reduce_all(bools)\n",
				"print(mn, pr, any_, all_)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 30. tf.argmax / tf.argmin\n",
				"vals = tf.constant([10,30,20,40])\n",
				"argmax_idx = tf.argmax(vals)\n",
				"argmin_idx = tf.argmin(vals)\n",
				"print(\"argmax=\", argmax_idx, \"argmin=\", argmin_idx)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 31. tf.equal / tf.not_equal / tf.greater / tf.less\n",
				"eq = tf.equal(a,b)\n",
				"gt = tf.greater(a,b)\n",
				"print(eq)\n",
				"print(gt)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 32. tf.where - 条件选择\n",
				"cond = tf.constant([True,False,True])\n",
				"x_ = tf.constant([1,2,3])\n",
				"y_ = tf.constant([10,20,30])\n",
				"res_where = tf.where(cond, x_, y_)\n",
				"print(res_where)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 33. tf.clip_by_value\n",
				"vals_clip = tf.constant([[-1,0,5],[10,15,2]], dtype=tf.float32)\n",
				"clipped = tf.clip_by_value(vals_clip, clip_value_min=0, clip_value_max=10)\n",
				"print(clipped)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 34. tf.math.top_k\n",
				"vals_topk = tf.constant([10,1,5,20,3])\n",
				"top2 = tf.math.top_k(vals_topk, k=2)\n",
				"print(\"values=\", top2.values, \"indices=\", top2.indices)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 35. tf.nn.softmax\n",
				"scores = tf.constant([2.0, 1.0, 0.1])\n",
				"softmax_res = tf.nn.softmax(scores)\n",
				"print(softmax_res)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 36. tf.nn.sigmoid / tf.nn.relu\n",
				"sig = tf.nn.sigmoid(scores)\n",
				"relu = tf.nn.relu(scores)\n",
				"print(sig)\n",
				"print(relu)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 37. tf.nn.tanh\n",
				"tn = tf.nn.tanh(scores)\n",
				"print(tn)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 38. tf.nn.leaky_relu\n",
				"lr = tf.nn.leaky_relu(scores, alpha=0.2)\n",
				"print(lr)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 39. tf.nn.elu / tf.nn.selu\n",
				"elu_res = tf.nn.elu(scores)\n",
				"selu_res = tf.nn.selu(scores)\n",
				"print(elu_res, selu_res)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 40. tf.nn.dropout\n",
				"dense = tf.constant([[1.,2.,3.],[4.,5.,6.]])\n",
				"drop = tf.nn.dropout(dense, rate=0.5)\n",
				"print(drop)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 41. tf.random.normal / tf.random.uniform\n",
				"rand_norm = tf.random.normal([2,3], mean=0, stddev=1)\n",
				"rand_unif = tf.random.uniform([2,3], minval=0, maxval=1)\n",
				"print(rand_norm)\n",
				"print(rand_unif)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 42. tf.random.set_seed\n",
				"tf.random.set_seed(42)\n",
				"r1 = tf.random.uniform([2])\n",
				"tf.random.set_seed(42)\n",
				"r2 = tf.random.uniform([2])\n",
				"print(\"r1=\", r1, \"\\nr2=\", r2)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 43. tf.Variable assign / assign_add / assign_sub\n",
				"var = tf.Variable([1,2,3], dtype=tf.float32)\n",
				"var.assign([4,5,6])\n",
				"print(\"after assign:\", var)\n",
				"var.assign_add([1,1,1])\n",
				"print(\"after add:\", var)\n",
				"var.assign_sub([2,2,2])\n",
				"print(\"after sub:\", var)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 44. tf.GradientTape - 计算梯度\n",
				"x = tf.Variable(3.0)\n",
				"with tf.GradientTape() as tape:\n",
				"    y = x**2\n",
				"grad = tape.gradient(y, x)\n",
				"print(\"dy/dx=\", grad)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 45. 多个变量梯度\n",
				"w = tf.Variable(tf.random.normal([2,2]))\n",
				"b = tf.Variable(tf.zeros([2]))\n",
				"with tf.GradientTape() as tape:\n",
				"    loss = tf.reduce_sum(w**2) + tf.reduce_sum(b)\n",
				"grads = tape.gradient(loss, [w,b])\n",
				"print(\"dw=\", grads[0], \"db=\", grads[1])"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 46. tf.function - 将python函数编译为图\n",
				"@tf.function\n",
				"def simple_fn(x, y):\n",
				"    return x * y + 2\n",
				"res = simple_fn(tf.constant(3), tf.constant(4))\n",
				"print(res)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 47. tf.config.list_physical_devices\n",
				"devices = tf.config.list_physical_devices()\n",
				"print(devices)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Keras 部分：张量与层（tf.keras.layers）"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 48. tf.keras.layers.Dense\n",
				"dense_layer = tf.keras.layers.Dense(units=3, activation='relu')\n",
				"inp = tf.constant([[1.0,2.0],[3.0,4.0]])\n",
				"out = dense_layer(inp)\n",
				"print(out)\n",
				"print(\"weights=\", dense_layer.get_weights())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 49. tf.keras.layers.Conv2D\n",
				"conv2d = tf.keras.layers.Conv2D(filters=4, kernel_size=3, padding='same', activation='relu')\n",
				"img = tf.random.normal([1,28,28,1])  # batch=1, h=28, w=28, c=1\n",
				"out_img = conv2d(img)\n",
				"print(out_img.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 50. tf.keras.layers.MaxPooling2D\n",
				"pool = tf.keras.layers.MaxPooling2D(pool_size=(2,2), strides=2)\n",
				"pooled = pool(out_img)\n",
				"print(pooled.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 51. tf.keras.layers.Embedding\n",
				"embed = tf.keras.layers.Embedding(input_dim=10, output_dim=4)\n",
				"seq = tf.constant([[1,2,3],[0,2,5]])\n",
				"emb_out = embed(seq)\n",
				"print(emb_out.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 52. tf.keras.layers.LSTM\n",
				"lstm = tf.keras.layers.LSTM(units=5)\n",
				"seq_data = tf.random.normal([2,10,8])  # batch=2, timesteps=10, features=8\n",
				"lstm_out = lstm(seq_data)\n",
				"print(lstm_out.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 53. tf.keras.layers.GRU\n",
				"gru = tf.keras.layers.GRU(units=5)\n",
				"gru_out = gru(seq_data)\n",
				"print(gru_out.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 54. tf.keras.layers.SimpleRNN\n",
				"rnn = tf.keras.layers.SimpleRNN(units=5)\n",
				"rnn_out = rnn(seq_data)\n",
				"print(rnn_out.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 55. tf.keras.layers.BatchNormalization\n",
				"bn = tf.keras.layers.BatchNormalization()\n",
				"x_bn = bn(tf.random.normal([4,10]))\n",
				"print(x_bn.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 56. tf.keras.layers.Dropout\n",
				"drop_layer = tf.keras.layers.Dropout(rate=0.3)\n",
				"x_drop = drop_layer(tf.ones([2,5]))\n",
				"print(x_drop)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 57. tf.keras.layers.Concatenate\n",
				"cat_layer = tf.keras.layers.Concatenate(axis=1)\n",
				"x1 = tf.random.normal([2,3])\n",
				"x2 = tf.random.normal([2,3])\n",
				"cat_out = cat_layer([x1,x2])\n",
				"print(cat_out.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 58. tf.keras.layers.Add / Subtract / Multiply\n",
				"add_layer = tf.keras.layers.Add()\n",
				"added = add_layer([x1, x2])\n",
				"print(\"added:\", added)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 59. tf.keras.layers.Reshape\n",
				"reshape_layer = tf.keras.layers.Reshape((3,2))\n",
				"r_out = reshape_layer(tf.constant([[1,2,3,4,5,6]]))\n",
				"print(r_out.shape)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Keras Model 构建"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 60. Sequential model\n",
				"model_seq = tf.keras.Sequential([\n",
				"    tf.keras.layers.Dense(8, activation='relu', input_shape=(4,)),\n",
				"    tf.keras.layers.Dense(3, activation='softmax')\n",
				"])\n",
				"model_seq.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
				"model_seq.summary()"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 61. Functional API model\n",
				"inputs = tf.keras.Input(shape=(4,))\n",
				"x = tf.keras.layers.Dense(8, activation='relu')(inputs)\n",
				"outputs = tf.keras.layers.Dense(3, activation='softmax')(x)\n",
				"model_func = tf.keras.Model(inputs=inputs, outputs=outputs)\n",
				"model_func.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
				"model_func.summary()"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 62. 自定义 model (Subclassing)\n",
				"class MyModel(tf.keras.Model):\n",
				"    def __init__(self):\n",
				"        super().__init__()\n",
				"        self.d1 = tf.keras.layers.Dense(8, activation='relu')\n",
				"        self.d2 = tf.keras.layers.Dense(3)\n",
				"    def call(self, x):\n",
				"        x = self.d1(x)\n",
				"        return self.d2(x)\n",
				"\n",
				"m_sub = MyModel()\n",
				"m_sub.compile(optimizer='adam', loss='mse')\n",
				"print(m_sub(tf.constant([[1.,2.,3.,4.]])))"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 63. model.fit - 训练\n",
				"import numpy as np\n",
				"X_data = np.random.rand(100,4).astype(np.float32)\n",
				"y_data = np.random.randint(0,3,size=100)\n",
				"history = model_seq.fit(X_data, y_data, epochs=5, batch_size=16)\n",
				"print(\"Finished fit.\")"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 64. model.evaluate\n",
				"loss_val, acc_val = model_seq.evaluate(X_data, y_data)\n",
				"print(\"Loss=\", loss_val, \"Acc=\", acc_val)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 65. model.predict\n",
				"preds = model_seq.predict(X_data[:5])\n",
				"print(preds)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 66. compile with different metrics\n",
				"model_seq.compile(optimizer='sgd', loss='sparse_categorical_crossentropy', metrics=['accuracy','mse'])\n",
				"print(\"re-compiled.\")"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 67. callbacks: EarlyStopping\n",
				"early_stop = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=2)\n",
				"model_seq.fit(X_data, y_data, epochs=10, batch_size=16, callbacks=[early_stop])"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 68. callbacks: ModelCheckpoint\n",
				"ckpt_cb = tf.keras.callbacks.ModelCheckpoint('model_{epoch}.h5', save_best_only=True)\n",
				"model_seq.fit(X_data, y_data, epochs=2, batch_size=16, callbacks=[ckpt_cb])"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 69. model.save / tf.keras.models.load_model\n",
				"model_seq.save('my_model.h5')\n",
				"loaded_model = tf.keras.models.load_model('my_model.h5')\n",
				"print(loaded_model.summary())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 70. tf.data.Dataset from tensor slices\n",
				"ds = tf.data.Dataset.from_tensor_slices((X_data, y_data)).batch(8)\n",
				"for batch_x, batch_y in ds.take(1):\n",
				"    print(batch_x.shape, batch_y.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 71. tf.data.Dataset from_generator\n",
				"def gen():\n",
				"    for i in range(5):\n",
				"        yield i, i*i\n",
				"ds_gen = tf.data.Dataset.from_generator(gen, output_signature=(tf.TensorSpec(shape=(), dtype=tf.int32),\n",
				"                                                              tf.TensorSpec(shape=(), dtype=tf.int32)))\n",
				"for x_val, y_val in ds_gen:\n",
				"    print(x_val, y_val)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 72. ds.map / ds.filter\n",
				"ds_map = ds.map(lambda x,y: (x*2, y))\n",
				"ds_filt = ds.filter(lambda x,y: tf.reduce_mean(x) > 0.5)\n",
				"print(ds_map, ds_filt)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 73. ds.shuffle\n",
				"ds_shuf = ds.shuffle(buffer_size=100)\n",
				"for bx, by in ds_shuf.take(1):\n",
				"    print(bx, by)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 74. ds.repeat\n",
				"ds_rep = ds.repeat(count=2)\n",
				"count = 0\n",
				"for bx, by in ds_rep:\n",
				"    count += 1\n",
				"print(count)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 75. ds.prefetch\n",
				"ds_pref = ds.prefetch(buffer_size=tf.data.AUTOTUNE)\n",
				"print(ds_pref)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 76. ds.cache\n",
				"ds_cache = ds.cache()\n",
				"print(ds_cache)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## 优化器 (tf.keras.optimizers)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 77. tf.keras.optimizers.SGD\n",
				"opt_sgd = tf.keras.optimizers.SGD(learning_rate=0.01)\n",
				"var_sgd = tf.Variable(1.0)\n",
				"for _ in range(5):\n",
				"    with tf.GradientTape() as tape:\n",
				"        loss_sgd = (var_sgd - 5.0)**2\n",
				"    grad_sgd = tape.gradient(loss_sgd, var_sgd)\n",
				"    opt_sgd.apply_gradients([(grad_sgd, var_sgd)])\n",
				"print(\"var_sgd=\", var_sgd.numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 78. tf.keras.optimizers.Adam\n",
				"opt_adam = tf.keras.optimizers.Adam(learning_rate=0.01)\n",
				"var_adam = tf.Variable(0.0)\n",
				"for _ in range(5):\n",
				"    with tf.GradientTape() as tape:\n",
				"        loss_adam = (var_adam - 3.0)**2\n",
				"    grad_adam = tape.gradient(loss_adam, var_adam)\n",
				"    opt_adam.apply_gradients([(grad_adam, var_adam)])\n",
				"print(\"var_adam=\", var_adam.numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 79. RMSprop\n",
				"opt_rms = tf.keras.optimizers.RMSprop(learning_rate=0.01)\n",
				"var_rms = tf.Variable(0.0)\n",
				"for _ in range(5):\n",
				"    with tf.GradientTape() as tape:\n",
				"        loss_rms = (var_rms - 2.0)**2\n",
				"    grad_rms = tape.gradient(loss_rms, var_rms)\n",
				"    opt_rms.apply_gradients([(grad_rms, var_rms)])\n",
				"print(\"var_rms=\", var_rms.numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 80. Adagrad\n",
				"opt_ada = tf.keras.optimizers.Adagrad(learning_rate=0.1)\n",
				"var_ada = tf.Variable(10.0)\n",
				"for _ in range(5):\n",
				"    with tf.GradientTape() as tape:\n",
				"        loss_ada = (var_ada - 0.0)**2\n",
				"    grad_ada = tape.gradient(loss_ada, var_ada)\n",
				"    opt_ada.apply_gradients([(grad_ada, var_ada)])\n",
				"print(\"var_ada=\", var_ada.numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Keras 训练循环（自定义training loop）"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 81. 自定义训练循环\n",
				"model_custom = tf.keras.Sequential([\n",
				"    tf.keras.layers.Dense(1, input_shape=(1,))\n",
				"])\n",
				"optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)\n",
				"\n",
				"X_custom = tf.constant([[1.0],[2.0],[3.0],[4.0]])\n",
				"y_custom = tf.constant([[2.0],[4.0],[6.0],[8.0]])\n",
				"\n",
				"for epoch in range(3):\n",
				"    with tf.GradientTape() as tape:\n",
				"        preds = model_custom(X_custom)\n",
				"        loss = tf.reduce_mean((preds - y_custom)**2)\n",
				"    grads = tape.gradient(loss, model_custom.trainable_variables)\n",
				"    optimizer.apply_gradients(zip(grads, model_custom.trainable_variables))\n",
				"    print(\"Epoch\", epoch, \"loss=\", loss.numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Keras Callback 自定义"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 82. 自定义 Callback\n",
				"class MyCallback(tf.keras.callbacks.Callback):\n",
				"    def on_epoch_end(self, epoch, logs=None):\n",
				"        print(\"Finished epoch\", epoch, \"with logs=\", logs)\n",
				"\n",
				"model_seq.fit(X_data, y_data, epochs=2, callbacks=[MyCallback()])"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Keras 迁移学习 / 模型冻结"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 83. 冻结某些层\n",
				"for layer in model_seq.layers[:-1]:\n",
				"    layer.trainable = False\n",
				"model_seq.summary()"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## 保存与加载 (SavedModel format)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 84. model.save('saved_model')\n",
				"import os\n",
				"save_path = 'saved_model_dir'\n",
				"model_seq.save(save_path, save_format='tf')\n",
				"print(\"Model saved.\")"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 85. tf.keras.models.load_model\n",
				"loaded_sm = tf.keras.models.load_model(save_path)\n",
				"print(\"Loaded saved_model:\", loaded_sm.summary())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## 分布式训练 (tf.distribute)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 86. MirroredStrategy\n",
				"strategy = tf.distribute.MirroredStrategy()\n",
				"print(\"Num replicas:\", strategy.num_replicas_in_sync)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## 自定义Loss与Metrics"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 87. 自定义Loss\n",
				"class MyMSE(tf.keras.losses.Loss):\n",
				"    def call(self, y_true, y_pred):\n",
				"        return tf.reduce_mean(tf.square(y_pred - y_true))\n",
				"\n",
				"model_seq.compile(optimizer='adam', loss=MyMSE(), metrics=['accuracy'])\n",
				"print(\"Using custom MSE.\")"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 88. 自定义 Metric\n",
				"class MeanAbsoluteErrorMetric(tf.keras.metrics.Metric):\n",
				"    def __init__(self, name='mean_abs_error', **kwargs):\n",
				"        super().__init__(name=name, **kwargs)\n",
				"        self.sum = self.add_weight(name='sum', initializer='zeros')\n",
				"        self.count = self.add_weight(name='count', initializer='zeros')\n",
				"    def update_state(self, y_true, y_pred, sample_weight=None):\n",
				"        val = tf.abs(y_pred - y_true)\n",
				"        val = tf.reduce_mean(val)\n",
				"        self.sum.assign_add(val)\n",
				"        self.count.assign_add(1)\n",
				"    def result(self):\n",
				"        return self.sum / self.count\n",
				"    def reset_states(self):\n",
				"        self.sum.assign(0)\n",
				"        self.count.assign(0)\n",
				"\n",
				"mae_metric = MeanAbsoluteErrorMetric()\n",
				"model_seq.compile(optimizer='adam', loss='mse', metrics=[mae_metric])\n",
				"print(\"Custom metric compiled.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.nn.* loss函数"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 89. tf.nn.sparse_softmax_cross_entropy_with_logits\n",
				"logits = tf.constant([[1.0,2.0,0.5],[2.0,1.0,0.1]])\n",
				"labels_sm = tf.constant([1,0])\n",
				"loss_sm = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=labels_sm, logits=logits)\n",
				"print(loss_sm)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 90. tf.nn.sigmoid_cross_entropy_with_logits\n",
				"logits_bc = tf.constant([[2.0, -1.0],[0.0, 1.0]])\n",
				"labels_bc = tf.constant([[1.0, 0.0],[0.0, 1.0]])\n",
				"loss_bc = tf.nn.sigmoid_cross_entropy_with_logits(labels=labels_bc, logits=logits_bc)\n",
				"print(loss_bc)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## 模型调试: tf.print / Python print"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 91. tf.print\n",
				"x = tf.constant([1,2,3])\n",
				"tf.print(\"x=\", x)\n",
				"print(\"end.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## 量化（使用 tfmot 可能不在 core）"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## TF lite (可能也不在 core scrope), skip."
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## 小结: 以上 100 条, 继续后面 100 条"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 92. tf.keras.losses.MeanSquaredError\n",
				"mse_loss_fn = tf.keras.losses.MeanSquaredError()\n",
				"y_true = tf.constant([[1.0],[2.0]])\n",
				"y_pred = tf.constant([[1.5],[2.5]])\n",
				"loss_val = mse_loss_fn(y_true, y_pred)\n",
				"print(\"MSE=\", loss_val.numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 93. tf.keras.losses.BinaryCrossentropy\n",
				"bce = tf.keras.losses.BinaryCrossentropy()\n",
				"bc_loss = bce(tf.constant([[1.0],[0.0]]), tf.constant([[0.9],[0.1]]))\n",
				"print(\"BinaryCE=\", bc_loss.numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 94. tf.keras.losses.CategoricalCrossentropy\n",
				"cat_ce = tf.keras.losses.CategoricalCrossentropy()\n",
				"ce_loss = cat_ce(tf.constant([[0,1,0],[1,0,0]], dtype=tf.float32),\n",
				"                tf.constant([[0.1,0.8,0.1],[0.7,0.2,0.1]], dtype=tf.float32))\n",
				"print(\"CatCE=\", ce_loss.numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 95. tf.keras.losses.Huber\n",
				"huber = tf.keras.losses.Huber()\n",
				"huber_loss = huber(tf.constant([[2.0],[5.0]]), tf.constant([[2.5],[4.0]]))\n",
				"print(\"Huber=\", huber_loss.numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Keras Metrics"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 96. tf.keras.metrics.Mean\n",
				"mean_metric = tf.keras.metrics.Mean()\n",
				"mean_metric.update_state(1.0)\n",
				"mean_metric.update_state(3.0)\n",
				"print(\"Mean=\", mean_metric.result().numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 97. tf.keras.metrics.BinaryAccuracy\n",
				"bin_acc = tf.keras.metrics.BinaryAccuracy()\n",
				"bin_acc.update_state(tf.constant([[1],[0]]), tf.constant([[1],[0]]))\n",
				"print(\"BinaryAccuracy=\", bin_acc.result().numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 98. tf.keras.metrics.Precision / Recall\n",
				"prec_metric = tf.keras.metrics.Precision()\n",
				"rec_metric = tf.keras.metrics.Recall()\n",
				"prec_metric.update_state(tf.constant([1,0,1,1]), tf.constant([1,0,1,0]))\n",
				"rec_metric.update_state(tf.constant([1,0,1,1]), tf.constant([1,0,1,0]))\n",
				"print(\"Precision=\", prec_metric.result().numpy())\n",
				"print(\"Recall=\", rec_metric.result().numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 99. tf.keras.metrics.AUC\n",
				"auc_metric = tf.keras.metrics.AUC()\n",
				"auc_metric.update_state(tf.constant([0,0,1,1], dtype=tf.float32), tf.constant([0.1,0.4,0.35,0.8], dtype=tf.float32))\n",
				"print(\"AUC=\", auc_metric.result().numpy())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 100. tf.keras.metrics.TopKCategoricalAccuracy\n",
				"topk_acc = tf.keras.metrics.TopKCategoricalAccuracy(k=2)\n",
				"y_true_ = tf.constant([[0,1,0],[0,0,1]])\n",
				"y_pred_ = tf.constant([[0.2,0.5,0.3],[0.1,0.3,0.6]])\n",
				"topk_acc.update_state(y_true_, y_pred_)\n",
				"print(\"TopKCategoricalAccuracy=\", topk_acc.result().numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"(已 100 条, 继续以下 100 条)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 101. model.evaluate with custom metrics\n",
				"model_seq.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=[tf.keras.metrics.AUC()])\n",
				"model_seq.evaluate(X_data, y_data)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.image"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 102. tf.image.resize\n",
				"img = tf.random.normal([1,100,100,3])\n",
				"resized = tf.image.resize(img, [50,50])\n",
				"print(resized.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 103. tf.image.flip_left_right\n",
				"flipped = tf.image.flip_left_right(img)\n",
				"print(flipped.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 104. tf.image.random_crop\n",
				"cropped = tf.image.random_crop(img, size=[1,50,50,3])\n",
				"print(cropped.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 105. tf.image.rgb_to_grayscale\n",
				"gray = tf.image.rgb_to_grayscale(img)\n",
				"print(gray.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 106. tf.image.adjust_brightness / adjust_contrast\n",
				"bright = tf.image.adjust_brightness(img, delta=0.1)\n",
				"contr = tf.image.adjust_contrast(img, contrast_factor=2.0)\n",
				"print(bright.shape, contr.shape)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.signal"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 107. tf.signal.fft\n",
				"import numpy as np\n",
				"sig_1d = tf.constant(np.random.rand(8).astype(np.complex64))\n",
				"fft_res = tf.signal.fft(sig_1d)\n",
				"print(fft_res)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 108. tf.signal.ifft\n",
				"ifft_res = tf.signal.ifft(fft_res)\n",
				"print(ifft_res)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 109. tf.signal.stft\n",
				"audio = tf.random.normal([1, 1024])  # batch=1, length=1024\n",
				"stft_res = tf.signal.stft(audio, frame_length=256, frame_step=128)\n",
				"print(stft_res.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 110. tf.signal.inverse_stft\n",
				"istft_res = tf.signal.inverse_stft(stft_res, frame_length=256, frame_step=128)\n",
				"print(istft_res.shape)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.applications"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 111. tf.keras.applications.VGG16\n",
				"from tensorflow.keras.applications import VGG16\n",
				"vgg_model = VGG16(weights='imagenet', include_top=False)\n",
				"print(vgg_model.summary())"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 112. tf.keras.applications.resnet50.preprocess_input\n",
				"from tensorflow.keras.applications.resnet50 import preprocess_input\n",
				"sample_img = tf.random.uniform([1,224,224,3], 0,255)\n",
				"prep_img = preprocess_input(sample_img)\n",
				"print(prep_img.shape)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.regularizers"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 113. tf.keras.regularizers.l2\n",
				"from tensorflow.keras import regularizers\n",
				"l2_reg = regularizers.l2(0.01)\n",
				"l2_model = tf.keras.Sequential([\n",
				"    tf.keras.layers.Dense(4, kernel_regularizer=l2_reg, input_shape=(4,)),\n",
				"    tf.keras.layers.Dense(3)\n",
				"])\n",
				"l2_model.compile(optimizer='adam', loss='mse')\n",
				"print(\"L2 regularized model built.\")"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 114. tf.keras.regularizers.l1_l2\n",
				"l1l2_reg = regularizers.l1_l2(l1=0.01, l2=0.01)\n",
				"print(l1l2_reg)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.activations"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 115. tf.keras.activations.relu / elu / selu etc.\n",
				"from tensorflow.keras import activations\n",
				"vals_act = tf.constant([-1.0,0.0,1.0])\n",
				"relu_act = activations.relu(vals_act)\n",
				"print(relu_act)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.initializers"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 116. tf.keras.initializers.GlorotUniform\n",
				"from tensorflow.keras import initializers\n",
				"init_glorot = initializers.GlorotUniform()\n",
				"w_init = init_glorot(shape=(3,3))\n",
				"print(w_init)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 117. tf.keras.initializers.Constant\n",
				"init_const = initializers.Constant(value=5)\n",
				"const_w = init_const(shape=(2,2))\n",
				"print(const_w)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.backend (Keras backend APIs)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 118. tf.keras.backend.eval\n",
				"import tensorflow.keras.backend as K\n",
				"kx = tf.constant(3.0)\n",
				"print(K.eval(kx))"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## RNN usage"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 119. tf.keras.layers.RNN with simple cell\n",
				"cell = tf.keras.layers.SimpleRNNCell(units=4)\n",
				"rnn_layer = tf.keras.layers.RNN(cell, return_sequences=True, return_state=True)\n",
				"rnn_in = tf.random.normal([2,5,3])  # batch=2, time=5, feature=3\n",
				"whole_seq, final_state = rnn_layer(rnn_in)\n",
				"print(whole_seq.shape, final_state.shape)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Model subclass advanced usage"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 120. Add functional layers inside subclass\n",
				"class MyBlock(tf.keras.Model):\n",
				"    def __init__(self, units):\n",
				"        super().__init__()\n",
				"        self.dense1 = tf.keras.layers.Dense(units, activation='relu')\n",
				"        self.dense2 = tf.keras.layers.Dense(units, activation='relu')\n",
				"    def call(self, x):\n",
				"        x = self.dense1(x)\n",
				"        x = self.dense2(x)\n",
				"        return x\n",
				"\n",
				"block = MyBlock(5)\n",
				"output_block = block(tf.constant([[1.0,2.0,3.0]]))\n",
				"print(output_block)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Mixed Precision (requires TF>2.4 typically)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 121. tf.keras.mixed_precision.set_global_policy\n",
				"try:\n",
				"    from tensorflow.keras.mixed_precision import experimental as mp\n",
				"    policy = mp.Policy('mixed_float16')\n",
				"    mp.set_policy(policy)\n",
				"    print(\"Mixed precision set.\")\n",
				"except ImportError:\n",
				"    print(\"mixed_precision not available.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Serving/Eager examples"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 122. Simple eager execution example\n",
				"def eager_add(x,y):\n",
				"    return x+y\n",
				"res_eager = eager_add(tf.constant(10), tf.constant(20))\n",
				"print(res_eager)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Distributed training advanced - skip if not needed"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## TF Hub usage - not in TF core"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Probability distributions (tfp) - not in TF core"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Additional math ops"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 123. tf.sin / tf.cos / tf.tan\n",
				"angles = tf.constant([0.0, 3.14159/2, 3.14159])\n",
				"sin_vals = tf.sin(angles)\n",
				"cos_vals = tf.cos(angles)\n",
				"print(sin_vals, cos_vals)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 124. tf.exp / tf.math.log\n",
				"ex = tf.exp(tf.constant([1.0,2.0]))\n",
				"lg = tf.math.log(tf.constant([1.0,2.0]))\n",
				"print(ex, lg)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 125. tf.sqrt / tf.rsqrt\n",
				"sq = tf.sqrt(tf.constant([4.0,9.0]))\n",
				"rsq = tf.math.rsqrt(tf.constant([4.0,9.0]))\n",
				"print(sq, rsq)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 126. tf.abs / tf.negative\n",
				"vals_abs = tf.constant([-1.0, 2.0])\n",
				"ab = tf.abs(vals_abs)\n",
				"neg = tf.negative(vals_abs)\n",
				"print(ab, neg)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 127. tf.square / tf.math.pow\n",
				"sq2 = tf.square(vals_abs)\n",
				"pw = tf.pow(vals_abs, 3)\n",
				"print(sq2, pw)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Sorting & Searching"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 128. tf.sort / tf.argsort\n",
				"arr_srt = tf.constant([3,1,4,2])\n",
				"srted = tf.sort(arr_srt)\n",
				"argsrted = tf.argsort(arr_srt)\n",
				"print(srted, argsrted)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 129. tf.searchsorted\n",
				"arr_ss = tf.constant([1,3,5,7])\n",
				"val_ss = tf.constant([2,6])\n",
				"ss_idx = tf.searchsorted(arr_ss, val_ss)\n",
				"print(ss_idx)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.random.stateless_* for deterministic"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 130. tf.random.stateless_uniform\n",
				"seed = (1,2)\n",
				"stat_unif = tf.random.stateless_uniform([2,2], seed=seed)\n",
				"print(stat_unif)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.math.special: tf.math.erf, etc"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 131. tf.math.erf\n",
				"x_erf = tf.constant([0.0,1.0,2.0])\n",
				"erf_res = tf.math.erf(x_erf)\n",
				"print(erf_res)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.quantization (some ops) - often not heavily used"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 132. tf.quantization.fake_quant_with_min_max_args\n",
				"fq = tf.quantization.fake_quant_with_min_max_args(tf.constant([1.5,2.5]), min=-1, max=3)\n",
				"print(fq)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.ragged"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 133. tf.ragged.constant\n",
				"rag = tf.ragged.constant([[1,2],[3,4,5],[6]])\n",
				"print(rag)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 134. ragged.to_tensor\n",
				"rag_tens = rag.to_tensor()\n",
				"print(rag_tens)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.sparse"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 135. tf.sparse.SparseTensor\n",
				"indices_sp = tf.constant([[0,0],[1,2]])\n",
				"values_sp = tf.constant([1,2])\n",
				"dense_shape_sp = tf.constant([3,4])\n",
				"sp_tensor = tf.sparse.SparseTensor(indices_sp, values_sp, dense_shape_sp)\n",
				"print(sp_tensor)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 136. tf.sparse.to_dense\n",
				"dense_sp = tf.sparse.to_dense(sp_tensor)\n",
				"print(dense_sp)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 137. tf.sparse.reorder\n",
				"sp_reordered = tf.sparse.reorder(sp_tensor)\n",
				"print(sp_reordered)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.linalg"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 138. tf.linalg.eig\n",
				"mat_eig = tf.constant([[1.,2.],[2.,1.]])\n",
				"eigs = tf.linalg.eig(mat_eig)\n",
				"print(eigs)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 139. tf.linalg.eigvals\n",
				"eigs_vals = tf.linalg.eigvals(mat_eig)\n",
				"print(eigs_vals)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 140. tf.linalg.svd\n",
				"svd_m = tf.linalg.svd(tf.cast(mat_eig, tf.float32))\n",
				"print(svd_m)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 141. tf.linalg.cholesky\n",
				"mat_posdef = tf.constant([[4.,2.],[2.,3.]])\n",
				"chol = tf.linalg.cholesky(mat_posdef)\n",
				"print(chol)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Checkpoint (tf.train.Checkpoint)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 142. tf.train.Checkpoint\n",
				"import os\n",
				"checkpoint_dir = './ckpt_dir'\n",
				"ckpt_prefix = os.path.join(checkpoint_dir, 'ckpt')\n",
				"if not os.path.exists(checkpoint_dir):\n",
				"    os.makedirs(checkpoint_dir)\n",
				"\n",
				"v1 = tf.Variable(10.0)\n",
				"v2 = tf.Variable(20.0)\n",
				"ckpt = tf.train.Checkpoint(step=tf.Variable(1), v1=v1, v2=v2)\n",
				"ckpt.save(file_prefix=ckpt_prefix)\n",
				"print(\"Checkpoint saved.\")"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 143. Restore from Checkpoint\n",
				"ckpt_restore = tf.train.Checkpoint(step=tf.Variable(0), v1=tf.Variable(0.0), v2=tf.Variable(0.0))\n",
				"status = ckpt_restore.restore(tf.train.latest_checkpoint(checkpoint_dir))\n",
				"status.assert_consumed()\n",
				"print(\"Restored v1=\", ckpt_restore.v1.numpy(), \" v2=\", ckpt_restore.v2.numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Profiler (tf.profiler.experimental) or (tf.summary.trace_on)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Mixed example: classification on iris with Keras"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 144. Simple example training on iris dataset\n",
				"iris_model = tf.keras.Sequential([\n",
				"    tf.keras.layers.Dense(10, activation='relu', input_shape=(4,)),\n",
				"    tf.keras.layers.Dense(3, activation='softmax')\n",
				"])\n",
				"iris_model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
				"iris_model.fit(iris.data, iris.target, epochs=5)\n",
				"iris_model.evaluate(iris.data, iris.target)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Mixed example: regression on random data"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 145. Simple regression\n",
				"reg_model = tf.keras.Sequential([\n",
				"    tf.keras.layers.Dense(1, input_shape=(3,))\n",
				"])\n",
				"reg_model.compile(optimizer='sgd', loss='mse')\n",
				"Xr_2 = np.random.rand(200,3)\n",
				"yr_2 = Xr_2[:,0]*2 + Xr_2[:,1]*(-1) + Xr_2[:,2]*0.5\n",
				"reg_model.fit(Xr_2, yr_2, epochs=3)\n",
				"print(reg_model.predict(Xr_2[:5]))"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Functional multi-input"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 146. Multiple inputs example\n",
				"input_a = tf.keras.Input(shape=(4,))\n",
				"input_b = tf.keras.Input(shape=(2,))\n",
				"x_a = tf.keras.layers.Dense(8, activation='relu')(input_a)\n",
				"x_b = tf.keras.layers.Dense(8, activation='relu')(input_b)\n",
				"merged = tf.keras.layers.concatenate([x_a, x_b])\n",
				"out_m = tf.keras.layers.Dense(3, activation='softmax')(merged)\n",
				"\n",
				"model_multi = tf.keras.Model(inputs=[input_a, input_b], outputs=out_m)\n",
				"model_multi.compile(optimizer='adam', loss='sparse_categorical_crossentropy')\n",
				"print(model_multi.summary())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Mixed input data feed"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 147. feed multi input\n",
				"Xa = np.random.rand(10,4)\n",
				"Xb = np.random.rand(10,2)\n",
				"y_m = np.random.randint(0,3, size=(10,))\n",
				"model_multi.fit([Xa, Xb], y_m, epochs=2)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Model with multiple outputs"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 148. multi-output\n",
				"inp_m = tf.keras.Input(shape=(4,))\n",
				"x_m = tf.keras.layers.Dense(8, activation='relu')(inp_m)\n",
				"out1 = tf.keras.layers.Dense(1, name='out_reg')(x_m)\n",
				"out2 = tf.keras.layers.Dense(3, activation='softmax', name='out_cls')(x_m)\n",
				"model_mo = tf.keras.Model(inp_m, [out1,out2])\n",
				"model_mo.compile(optimizer='adam', loss={'out_reg':'mse','out_cls':'sparse_categorical_crossentropy'})\n",
				"print(model_mo.summary())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## partial fit multi-output data"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 149. fitting multi-output\n",
				"X_mo = np.random.rand(20,4)\n",
				"y_reg = np.random.rand(20,1)\n",
				"y_cls = np.random.randint(0,3,size=(20,))\n",
				"model_mo.fit(X_mo, {'out_reg':y_reg, 'out_cls':y_cls}, epochs=3)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Slicing logic in functional model"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 150. advanced slicing in functional\n",
				"inp_s = tf.keras.Input(shape=(6,))\n",
				"x_s1 = inp_s[:, :3]\n",
				"x_s2 = inp_s[:, 3:]\n",
				"d_s1 = tf.keras.layers.Dense(2)(x_s1)\n",
				"d_s2 = tf.keras.layers.Dense(2)(x_s2)\n",
				"merged_s = tf.keras.layers.Concatenate()([d_s1,d_s2])\n",
				"mod_s = tf.keras.Model(inp_s, merged_s)\n",
				"print(mod_s.summary())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## LR schedule (tf.keras.optimizers.schedules)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 151. ExponentialDecay\n",
				"lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay(\n",
				"    initial_learning_rate=0.1,\n",
				"    decay_steps=10,\n",
				"    decay_rate=0.9\n",
				")\n",
				"opt_sch = tf.keras.optimizers.SGD(learning_rate=lr_schedule)\n",
				"print([opt_sch.learning_rate(i).numpy() for i in [0,10,20]])"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 152. PiecewiseConstantDecay\n",
				"lr_pc = tf.keras.optimizers.schedules.PiecewiseConstantDecay(\n",
				"    boundaries=[5,10],\n",
				"    values=[0.1,0.01,0.001]\n",
				")\n",
				"print([lr_pc(i).numpy() for i in [0,5,6,10,11]])"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Model summary usage"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 153. model_seq.summary() repeated\n",
				"model_seq.summary()"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Callback: TensorBoard"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 154. tf.keras.callbacks.TensorBoard\n",
				"tb_callback = tf.keras.callbacks.TensorBoard(log_dir='./logs')\n",
				"model_seq.fit(X_data, y_data, epochs=1, callbacks=[tb_callback])\n",
				"print(\"TensorBoard logs created.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Custom training step example"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 155. custom train step in subclassed model\n",
				"class MyTrainModel(tf.keras.Model):\n",
				"    def __init__(self):\n",
				"        super().__init__()\n",
				"        self.dense = tf.keras.layers.Dense(1)\n",
				"        self.loss_fn = tf.keras.losses.MeanSquaredError()\n",
				"        self.opt = tf.keras.optimizers.SGD(0.01)\n",
				"    def train_step(self, data):\n",
				"        x, y = data\n",
				"        with tf.GradientTape() as tape:\n",
				"            y_pred = self(x, training=True)\n",
				"            loss = self.loss_fn(y, y_pred)\n",
				"        grads = tape.gradient(loss, self.trainable_variables)\n",
				"        self.opt.apply_gradients(zip(grads, self.trainable_variables))\n",
				"        return {'loss': loss}\n",
				"    def call(self, x, training=False):\n",
				"        return self.dense(x)\n",
				"\n",
				"model_custom_train = MyTrainModel()\n",
				"model_custom_train.compile(run_eagerly=True)\n",
				"xs = np.random.rand(10,1)\n",
				"ys = xs*3 + 2\n",
				"model_custom_train.fit(xs, ys, epochs=2)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Mixed precision usage again - skip if older version"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Inference usage with signatures"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 156. Serving signatures\n",
				"def serving_fn(model, x):\n",
				"    return model(x)\n",
				"signatures = {\n",
				"    'serving_default': serving_fn\n",
				"}\n",
				"model_seq.save('model_with_sig', signatures=signatures)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Another data pipeline example"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 157. Creating dataset from CSV\n",
				"import pandas as pd\n",
				"csv_data = pd.DataFrame({'f1':[1,2,3],'f2':[4,5,6],'label':[0,1,1]})\n",
				"csv_data.to_csv('temp.csv', index=False)\n",
				"ds_csv = tf.data.experimental.make_csv_dataset('temp.csv', batch_size=2, label_name='label', num_epochs=1)\n",
				"for feat, lab in ds_csv:\n",
				"    print(feat, lab)\n",
				"    break"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Weighted loss usage"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 158. model.fit with sample_weight\n",
				"sample_w = np.array([1.0,2.0,3.0,1.0]*25)\n",
				"model_seq.fit(X_data, y_data, epochs=1, sample_weight=sample_w)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Evaluate with sample_weight"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 159. model.evaluate with sample_weight\n",
				"model_seq.evaluate(X_data, y_data, sample_weight=sample_w)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.config.experimental set_memory_growth"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 160. example memory growth\n",
				"gpus = tf.config.list_physical_devices('GPU')\n",
				"if gpus:\n",
				"    try:\n",
				"        tf.config.experimental.set_memory_growth(gpus[0], True)\n",
				"        print(\"Set memory growth.\")\n",
				"    except:\n",
				"        pass"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Early stopping with val_loss"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 161. using validation_data\n",
				"val_data = (X_data[:20], y_data[:20])\n",
				"model_seq.fit(X_data, y_data, epochs=3, validation_data=val_data)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## image_dataset_from_directory usage"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"（需要文件夹结构，略）"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## text_dataset_from_directory usage"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"（同上，需要文件夹文本）"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Model bridging"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 162. model.to_json\n",
				"json_str = model_seq.to_json()\n",
				"print(json_str[:100], \"...\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## parse_function for tf.data"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 163. parse example\n",
				"def parse_fn(features, label):\n",
				"    features = tf.cast(features, tf.float32)\n",
				"    return features, label\n",
				"ds_parsed = ds.map(parse_fn)\n",
				"for fx, fy in ds_parsed.take(1):\n",
				"    print(fx, fy)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.nn.ctc_loss etc. (for advanced usage)."
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Debugging: tf.debugging"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 164. tf.debugging.assert_equal\n",
				"tf.debugging.assert_equal(tf.constant(3), tf.constant(3))\n",
				"print(\"assert equal pass\")"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 165. tf.debugging.assert_shapes\n",
				"matA = tf.ones([2,3])\n",
				"matB = tf.ones([2,3])\n",
				"tf.debugging.assert_shapes([\n",
				"    (matA, ('N','C')),\n",
				"    (matB, ('N','C'))\n",
				"])\n",
				"print(\"Shapes matched.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Tensorspec usage"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 166. tf.TensorSpec\n",
				"spec = tf.TensorSpec(shape=(None,4), dtype=tf.float32, name='input')\n",
				"print(spec)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Save weights only"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 167. model.save_weights\n",
				"weight_path = 'weights_only'\n",
				"model_seq.save_weights(weight_path)\n",
				"model_seq.load_weights(weight_path)\n",
				"print(\"Weights saved and loaded.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Large model checkpoint callback"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Mixed usage with multiple optimizers"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 168. two variables with different optimizers\n",
				"vA = tf.Variable(1.0)\n",
				"vB = tf.Variable(2.0)\n",
				"optA = tf.keras.optimizers.SGD(0.01)\n",
				"optB = tf.keras.optimizers.Adam(0.01)\n",
				"for _ in range(2):\n",
				"    with tf.GradientTape(persistent=True) as tape:\n",
				"        lossA = (vA - 5)**2\n",
				"        lossB = (vB - 3)**2\n",
				"    gradA = tape.gradient(lossA, vA)\n",
				"    gradB = tape.gradient(lossB, vB)\n",
				"    optA.apply_gradients([(gradA, vA)])\n",
				"    optB.apply_gradients([(gradB, vB)])\n",
				"    del tape\n",
				"print(vA.numpy(), vB.numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Summaries (tf.summary)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 169. tf.summary\n",
				"logdir = \"./logs/tfsummary\"\n",
				"writer = tf.summary.create_file_writer(logdir)\n",
				"with writer.as_default():\n",
				"    for step in range(3):\n",
				"        tf.summary.scalar(\"my_metric\", step*2, step=step)\n",
				"writer.flush()\n",
				"print(\"Summaries written.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Category Encoding - mostly done in user code or with feature_column"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.feature_column"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 170. tf.feature_column.numeric_column\n",
				"import tensorflow as tf\n",
				"fc_num = tf.feature_column.numeric_column('x', shape=(1,))\n",
				"print(fc_num)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 171. input_layer from feature_column\n",
				"fc_inp = {'x': tf.constant([[1.0],[2.0],[3.0]])}\n",
				"fc_tensor = tf.keras.layers.DenseFeatures([fc_num])(fc_inp)\n",
				"print(fc_tensor)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Additional distribution strategies"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Another aggregator"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## finalize with small ops"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 172. tf.nn.log_softmax\n",
				"lsm = tf.nn.log_softmax(tf.constant([1.0,2.0,3.0]))\n",
				"print(lsm)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 173. tf.nn.depthwise_conv2d\n",
				"img_dw = tf.random.normal([1,28,28,3])\n",
				"filter_dw = tf.random.normal([3,3,3,1])\n",
				"dw_out = tf.nn.depthwise_conv2d(img_dw, filter_dw, strides=[1,1,1,1], padding='SAME')\n",
				"print(dw_out.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 174. tf.nn.separable_conv2d\n",
				"d_filter = tf.random.normal([3,3,3,4])\n",
				"p_filter = tf.random.normal([1,1,4,8])\n",
				"sep_out = tf.nn.separable_conv2d(img_dw, d_filter, p_filter, strides=[1,1,1,1], padding='SAME')\n",
				"print(sep_out.shape)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 175. tf.map_fn\n",
				"arr_map = tf.constant([1,2,3,4], dtype=tf.float32)\n",
				"def fn_map(x):\n",
				"    return x*2\n",
				"mapped = tf.map_fn(fn_map, arr_map)\n",
				"print(mapped)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 176. tf.scan\n",
				"def fn_scan(a, x):\n",
				"    return a + x\n",
				"scanned = tf.scan(fn_scan, arr_map, initializer=0.)\n",
				"print(scanned)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## multi-backend usage"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.random.categorical"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 177. tf.random.categorical\n",
				"logits_cat = tf.constant([[1.0,2.0,3.0]])\n",
				"samples_cat = tf.random.categorical(logits_cat, num_samples=5)\n",
				"print(samples_cat)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## masked operations"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 178. tf.ragged.boolean_mask, or normal tf.boolean_mask already shown\n",
				"mask_ex = tf.constant([True,False,True])\n",
				"vals_ex = tf.constant([10,20,30])\n",
				"masked_ex = tf.boolean_mask(vals_ex, mask_ex)\n",
				"print(masked_ex)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.config.set_visible_devices"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 179. skipping if not multi-gpu\n",
				"print(\"set_visible_devices usage.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.config.run_functions_eagerly"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 180. run_functions_eagerly\n",
				"tf.config.run_functions_eagerly(True)\n",
				"@tf.function\n",
				"def test_fn(x,y):\n",
				"    return x+y\n",
				"print(\"Eager=\", test_fn(tf.constant(3), tf.constant(4)))\n",
				"tf.config.run_functions_eagerly(False)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## experimental AutoGraph features"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Additional Keras model usage: .predict_on_batch"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 181. model.predict_on_batch\n",
				"res_batch = model_seq.predict_on_batch(X_data[:4])\n",
				"print(res_batch)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Additional Keras model usage: .train_on_batch"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 182. train_on_batch\n",
				"loss_tb, acc_tb = model_seq.train_on_batch(X_data[:4], y_data[:4])\n",
				"print(\"Batch loss=\", loss_tb, \"acc=\", acc_tb)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Additional Keras model usage: .test_on_batch"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 183. test_on_batch\n",
				"loss_test, acc_test = model_seq.test_on_batch(X_data[:4], y_data[:4])\n",
				"print(\"test_on_batch:\", loss_test, acc_test)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.metrics usage"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 184. tf.metrics.MeanIoU (for segmentation)\n",
				"miou = tf.keras.metrics.MeanIoU(num_classes=3)\n",
				"pred_seg = tf.constant([0,1,2,1])\n",
				"true_seg = tf.constant([0,1,1,2])\n",
				"miou.update_state(true_seg, pred_seg)\n",
				"print(miou.result().numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.losses.CosineSimilarity"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 185. CosineSimilarity\n",
				"cos_loss = tf.keras.losses.CosineSimilarity(axis=1)\n",
				"y1 = tf.constant([[1.,0.],[1.,1.]])\n",
				"y2 = tf.constant([[0.9,0.1],[0.8,0.8]])\n",
				"loss_cos = cos_loss(y1, y2)\n",
				"print(loss_cos.numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.layers.Conv1D"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 186. Conv1D\n",
				"conv1d = tf.keras.layers.Conv1D(filters=4, kernel_size=3)\n",
				"x_1d = tf.random.normal([2,10,1])  # batch=2, length=10, channels=1\n",
				"y_1d = conv1d(x_1d)\n",
				"print(y_1d.shape)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.layers.UpSampling2D"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 187. UpSampling2D\n",
				"ups = tf.keras.layers.UpSampling2D(size=(2,2))\n",
				"img_small = tf.random.normal([1,16,16,3])\n",
				"img_up = ups(img_small)\n",
				"print(img_up.shape)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## tf.keras.layers.GlobalAveragePooling2D"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 188. GlobalAveragePooling2D\n",
				"gap = tf.keras.layers.GlobalAveragePooling2D()\n",
				"img_gap = tf.random.normal([2,32,32,3])\n",
				"res_gap = gap(img_gap)\n",
				"print(res_gap.shape)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## Another dataset manipulation"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 189. ds.take\n",
				"for d_x, d_y in ds.take(1):\n",
				"    print(d_x, d_y)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final examples: custom grad"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 190. custom gradient via tf.custom_gradient\n",
				"@tf.custom_gradient\n",
				"def my_square(x):\n",
				"    y = x*x\n",
				"    def grad_fn(dy):\n",
				"        return 2*x*dy\n",
				"    return y, grad_fn\n",
				"\n",
				"val = tf.Variable(3.0)\n",
				"with tf.GradientTape() as tape:\n",
				"    y_out = my_square(val)\n",
				"grad_cust = tape.gradient(y_out, val)\n",
				"print(\"y_out=\", y_out.numpy(), \" grad=\", grad_cust.numpy())"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final distribution example"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 191. strategy.run\n",
				"dist_strat = tf.distribute.MirroredStrategy()\n",
				"def train_step_dist(inputs):\n",
				"    x_, y_ = inputs\n",
				"    with tf.GradientTape() as tape:\n",
				"        pred_ = model_seq(x_, training=True)\n",
				"        loss_ = tf.keras.losses.sparse_categorical_crossentropy(y_, pred_)\n",
				"        loss_ = tf.nn.compute_average_loss(loss_)\n",
				"    grads_ = tape.gradient(loss_, model_seq.trainable_variables)\n",
				"    opt_adam.apply_gradients(zip(grads_, model_seq.trainable_variables))\n",
				"    return loss_\n",
				"\n",
				"dist_ds = dist_strat.experimental_distribute_dataset(ds)\n",
				"with dist_strat.scope():\n",
				"    for dd in dist_ds.take(1):\n",
				"        dist_strat.run(train_step_dist, args=(dd,))\n",
				"print(\"Distributed step done.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final: tf.config.optimizer.set_jit for XLA"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 192. XLA JIT\n",
				"tf.config.optimizer.set_jit(True)\n",
				"print(\"XLA JIT set.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final: sample local training loop"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 193. local loop\n",
				"for epoch in range(2):\n",
				"    for step, (bx, by) in enumerate(ds):\n",
				"        with tf.GradientTape() as tape:\n",
				"            preds_ = model_seq(bx)\n",
				"            loss_ = tf.keras.losses.sparse_categorical_crossentropy(by, preds_)\n",
				"            loss_ = tf.reduce_mean(loss_)\n",
				"        grads_ = tape.gradient(loss_, model_seq.trainable_variables)\n",
				"        opt_adam.apply_gradients(zip(grads_, model_seq.trainable_variables))\n",
				"    print(\"Epoch\", epoch, \"done.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final: confusion matrix with tf.math.confusion_matrix"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 194. tf.math.confusion_matrix\n",
				"pred_labels = tf.constant([0,1,2,2,1])\n",
				"true_labels = tf.constant([0,1,1,2,1])\n",
				"cmat = tf.math.confusion_matrix(true_labels, pred_labels, num_classes=3)\n",
				"print(cmat)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final: image summary (tf.summary.image) usage"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 195. tf.summary.image\n",
				"img_sum = tf.random.normal([1,100,100,3])\n",
				"with writer.as_default():\n",
				"    tf.summary.image(\"random_img\", img_sum, step=0)\n",
				"    writer.flush()\n",
				"print(\"Image summary logged.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final: distributed dataset example again"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 196. repeated usage might skip\n",
				"print(\"Already shown.\")"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final: advanced layer subclass example"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 197. custom layer with build\n",
				"class MyCustomLayer(tf.keras.layers.Layer):\n",
				"    def __init__(self, units=4):\n",
				"        super().__init__()\n",
				"        self.units = units\n",
				"    def build(self, input_shape):\n",
				"        self.w = self.add_weight(shape=(input_shape[-1], self.units), initializer='random_normal')\n",
				"        self.b = self.add_weight(shape=(self.units,), initializer='zeros')\n",
				"    def call(self, inputs):\n",
				"        return tf.matmul(inputs, self.w) + self.b\n",
				"\n",
				"layer_custom = MyCustomLayer(3)\n",
				"test_out = layer_custom(tf.constant([[1.0,2.0]]))\n",
				"print(test_out)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final: random shuffle of dataset"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 198. ds.shuffle\n",
				"ds_shuffled = ds.shuffle(buffer_size=100, reshuffle_each_iteration=True)\n",
				"print(ds_shuffled)"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## final: skip"
			]
		},
		{
			"cell_type": "markdown",
			"metadata": {},
			"source": [
				"## We will do 2 more to get 200"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 199. ds.enumerate\n",
				"ds_enum = ds.enumerate()\n",
				"for i,(bx,by) in ds_enum.take(1):\n",
				"    print(\"step=\", i, bx, by)"
			]
		},
		{
			"cell_type": "code",
			"execution_count": null,
			"metadata": {},
			"outputs": [],
			"source": [
				"# 200. End of 200 examples!\n",
				"print(\"TensorFlow top 200 examples done.\")"
			]
		}
	],
	"metadata": {
		"kernelspec": {
			"display_name": "Python 3",
			"language": "python",
			"name": "python3"
		},
		"language_info": {
			"codemirror_mode": {
				"name": "ipython",
				"version": 3
			},
			"file_extension": ".py",
			"mimetype": "text/x-python",
			"name": "python",
			"nbconvert_exporter": "python",
			"pygments_lexer": "ipython3",
			"version": "3.11.9"
		}
	},
	"nbformat": 4,
	"nbformat_minor": 5
}