update materi

3. Visualisasi Bobot/3. Visualisasi Bobot.ipynb

@@ -0,0 +1 @@
+{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"Copy of 3. Visualisasi Bobot.ipynb","provenance":[{"file_id":"1aDdt6aQPVSYjEgTDJEIxmHKdgkTxIH4a","timestamp":1572423788077}],"collapsed_sections":[],"toc_visible":true},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"id":"385pu3XhZxUD","colab_type":"text"},"source":["![alt text](https://anvaqta.id/headerai.jpg)"]},{"cell_type":"markdown","metadata":{"id":"cqe9AvV89rND","colab_type":"text"},"source":["# Visualisasi Bobot Klasifikasi Linier"]},{"cell_type":"markdown","metadata":{"id":"dEKiHyYF9vCB","colab_type":"text"},"source":["## *Deep Representations*\n","\n","Seperti yang telah dijelaskan sebelumnya, Salah satu intuisi alasan mengapa Jaringan yang dalam mampu mengatasi permasalahan lebih baik adalah dikarenakan Jaringan tersebut mampu membuat representasi permasalahan dengan lebih detil dan lebih baik\n","\n","Sebagai contoh, misalkan kita membangun Jaringan Saraf Tiruan untuk mengenali identitas seseorang berdasarkan citra wajah. \n","\n","<img src=\"https://image.ibb.co/iOG8MK/deep_representation.jpg\" alt=\"deep_representation\"/>\n","\n","Layer pertama pada Jaringan yang telah dilatih dengan ratusan dan ribuan citra mungkin akan bertindak lebih sebagai suatu pengenalan fitur atau *Edge Detection*, di mana setiap neuron mencoba mengenali orientasi garis yang berbeda. Kemudian terdapat layer berikutnya yang dilatih terhadap data garis hasil ekstraksi layer pertama dan menggabungkannya, dan mungkin membentuk suatu pengenalan fitur baru misalnya membentuk neuron yang mengenali fitur mata, hidung, dan lain sebagainya. Dan hal ini terus dilakukan untuk layer-layer berikutnya sehingga Jaringan dengan layer yang banyak dapat membentuk representasi fitur yang lebih detil.\n","\n","Namun apakah benar demikian?\n","Mari kita selidiki"]},{"cell_type":"code","metadata":{"id":"Rrx9qJhL7RUk","colab_type":"code","colab":{}},"source":["import numpy as np\n","import matplotlib.pyplot as plt\n","%matplotlib inline"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"yFk18QJs90yd","colab_type":"text"},"source":["---\n","## Load MNIST Dataset"]},{"cell_type":"code","metadata":{"id":"HRgIf8uq8E-2","colab_type":"code","colab":{}},"source":["!pip install tensorflow-gpu==2.0.0"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"XO_W7UtG8HaH","colab_type":"code","colab":{}},"source":["import tensorflow as tf\n","tf.__version__"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"id":"c-wtNcb37JWh","colab_type":"code","colab":{}},"source":["(X_train, y_train), (X_test, y_test) = tf.keras.datasets.mnist.load_data()\n","\n","print('X_train.shape =',X_train.shape)\n","print('y_train.shape =',y_train.shape)\n","print('X_test.shape  =',X_test.shape)\n","print('y_test.shape  =',y_test.shape)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"yKzBbFZk96PV","colab_type":"text"},"source":["Tampilkan 20 gambar pertama\n"]},{"cell_type":"code","metadata":{"id":"6wDv411G7Oh4","colab_type":"code","colab":{}},"source":["fig, ax = plt.subplots(2,10,figsize=(15,4.5))\n","fig.subplots_adjust(hspace=0.1, wspace=0.1)\n","for j in range(0,2):\n","    for i in range(0, 10):\n","        ax[j,i].imshow(X_train[i+j*10], cmap='gray')\n","        ax[j,i].set_title(y_train[i+j*10])\n","        ax[j,i].axis('off')\n","plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"V8Z2xi8u-FnI","colab_type":"text"},"source":["Sebelumnya kita sudah mengklasifikasi Mnist Digit, pada tahap awal jaringan kita melakukan Flatten, mari kita lakukan menggunakan numpy\n"]},{"cell_type":"code","metadata":{"id":"82UlFbty7UuV","colab_type":"code","colab":{}},"source":["# reshape X_train menjadi 2 dimensi berukuran 60000 x (28*28) menggunakan np.reshape\n","X_train_reshape = np.reshape(X_train, (60000, (28*28)))\n","\n","# reshape X_test menjadi 2 dimensi berukuran 10000 x (28*28) menggunakan np.reshape\n","X_test_reshape = np.reshape(X_test, (10000, (28*28)))\n","\n","# reshape y_train menjadi 1 dimensi dengan fungsi np.ravel() \n","y_train_reshape = np.ravel(y_train)\n","# reshape y_test menjadi 1 dimensi dengan fungsi np.ravel()\n","y_test_reshape = np.ravel(y_test)\n","\n","print('X_train:', X_train_reshape.shape)\n","print('X_test:', X_test_reshape.shape)\n","\n","print('y_train:', y_train_reshape.shape)\n","print('y_test:', y_test_reshape.shape)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Lqa4DrTE-eBm","colab_type":"text"},"source":["---\n","## Latih model menggunakan Scikit-Learn"]},{"cell_type":"code","metadata":{"id":"dfQ85fU37nfz","colab_type":"code","colab":{}},"source":["from sklearn.linear_model import SGDClassifier\n","clf = SGDClassifier(loss='log', max_iter=2000, verbose=2, n_jobs=2,learning_rate='invscaling', eta0=1e-7)\n","clf.fit(X_train_reshape, y_train_reshape)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"kqUhXjKy-tC5","colab_type":"text"},"source":["Hitung akurasinya"]},{"cell_type":"code","metadata":{"id":"kVsldEoa7rBC","colab_type":"code","colab":{}},"source":["y_pred = clf.predict(X_test_reshape)\n","\n","from sklearn.metrics import accuracy_score\n","print('akurasi =',accuracy_score(y_test_reshape, y_pred)*100,'%')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"uzIeDOkU-vTz","colab_type":"text"},"source":["---\n","## Ambil nilai bobot\n","Untuk memvisualisasikan bobot dari jaringan, kita ambil nilai bobotnya terlebih dahulu"]},{"cell_type":"code","metadata":{"id":"iWo0Nswr8w-x","colab_type":"code","colab":{}},"source":["from sklearn.preprocessing import MinMaxScaler\n","weights = clf.coef_\n","class_id = 0\n","w0 = weights[class_id]\n","scaler = MinMaxScaler((0.1,.9))\n","w0 = scaler.fit_transform(w0.reshape(-1, 1))\n","print(w0.shape)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"jONRXBCZ-56P","colab_type":"text"},"source":["Visualisaikan bobot"]},{"cell_type":"code","metadata":{"id":"XJKetTkB70kp","colab_type":"code","colab":{}},"source":["fig, ax = plt.subplots(1,10,figsize=(18,5))\n","fig.subplots_adjust(hspace=0.1, wspace=0.1)\n","for i in range(0, 10):\n","    w = weights[i]\n","    \n","    w = scaler.fit_transform(w.reshape(-1, 1))\n","    \n","    w = w.reshape((28,28))\n","    \n","    ax[i].imshow(w, cmap='gray')\n","    ax[i].axis('off')\n","    ax[i].set_title(i)\n","plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"3NxbdBnu9T1f","colab_type":"text"},"source":["---\n","# Visualisasi Bobot Cifar-10"]},{"cell_type":"markdown","metadata":{"id":"MhR71frJ9ZG2","colab_type":"text"},"source":["---\n","## Load Cifar-10"]},{"cell_type":"code","metadata":{"id":"lxGDVLu29XFW","colab_type":"code","colab":{}},"source":["(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()\n","\n","\n","classes = ['plane', 'car', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']\n","\n","print('X_train.shape =',X_train.shape)\n","print('y_train.shape =',y_train.shape)\n","print('X_test.shape  =',X_test.shape)\n","print('y_test.shape  =',y_test.shape)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"nPZQccURdAow"},"source":["Tampilkan 20 data pertama beserta kelasnya dari 50000 data latih"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"EbSOLMhsBef2","colab":{}},"source":["fig, ax = plt.subplots(2,10,figsize=(15,4.5))\n","fig.subplots_adjust(hspace=0.1, wspace=0.1)\n","for j in range(0,2):\n","    for i in range(0, 10):\n","        ax[j,i].imshow(X_train[i+j*10])\n","        ax[j,i].set_title(classes[y_train[i+j*10,0]])\n","        ax[j,i].axis('off')\n","plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"cGaNEcgsdHfR"},"source":["Selanjutnya kita akan mengklasifikasikan 10 kelas data cifar-10 menggunakan Klasifikasi Linier\n","\n","Untuk itu, pertama kali kita preprocessing data dengan melakukan *zero-mean centering*"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"NDU4d9a7QCHW","colab":{}},"source":["X_train = X_train.astype('float32')\n","X_test = X_test.astype('float32')\n","\n","mean_image = np.mean(X_train, axis = 0)\n","X_train -= mean_image\n","X_test -= mean_image"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"iyXreV3VdSWK"},"source":["Pada Klasifikasi Linier, input harus diubah menjadi satu dimensi, \n","\n","\n","<img src=\"https://i.ibb.co/HDTHkp9/linear-algebra.png\" alt=\"klasifikasi linier\"/>\n","\n","\n","karena itu, lakukan reshape"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"wBsMqLlRRf1T","colab":{}},"source":["# reshape X_train menjadi 2 dimensi berukuran 50000 x (32*32*3)\n","X_train_reshape = np.reshape(50000, (32*32*3))\n","\n","# reshape X_test menjadi 2 dimensi berukuran 10000 x (32*32*3)\n","X_test_reshape = np.reshape(10000, (32*32*3))\n","\n","# reshape y_train menjadi 1 dimensi dengan fungsi ravel()\n","y_train_reshape = np.ravel(y_train)\n","# reshape y_test menjadi 1 dimensi dengan fungsi ravel()\n","y_test_reshape = np.ravel(y_test)\n","\n","print('X_train:', X_train_reshape.shape)\n","print('X_test:', X_test_reshape.shape)\n","\n","print('y_train:', y_train_reshape.shape)\n","print('y_test:', y_test_reshape.shape)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"pubLr8DYamOg"},"source":["**EXPECTED OUTPUT**\n","<pre>\n"," X_train: (50000, 3072)\n"," X_test: (10000, 3072)\n"," y_train: (50000,)\n"," y_test: (10000,)"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"sSXpgqkzeKfx"},"source":["---\n","## Klasifikasi dengan Sci-Kit Learn\n","Selanjutnya kita lakukan pelatihan Klasifikasi Linier menggunakan library sklearn SGDClassifier"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"-TPSsCHzS_rq","colab":{}},"source":["from sklearn.linear_model import SGDClassifier\n","clf = SGDClassifier(loss='log', max_iter=2000, verbose=2, n_jobs=2,learning_rate='invscaling', eta0=1e-7)\n","clf.fit(X_train_reshape, y_train_reshape)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"EHVkZ1KKeV3s"},"source":["Kita bisa mengecek akurasi dari model yang telah dilatih. \n","\n","Dapat dilihat bahwa akurasi masih kurang bagus dikarenakan kita hanya menggunakan klasifikasi linier atau Jaringan Saraf Tiruan satu layer"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"aPHCEWq1S_ox","colab":{}},"source":["y_pred = clf.predict(X_test_reshape)\n","\n","from sklearn.metrics import accuracy_score\n","print('akurasi =',accuracy_score(y_test_reshape, y_pred)*100,'%')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"XHT_oCihejs1"},"source":["---\n","## Visualisasi Bobot\n","\n","Selanjutnya mari kita ambil bobot yang telah dilatih dari model.\n","\n","Perhatikan bahwa ukuran bobot adalah (10 x 3072)"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"S-OjFBu7BWHa","colab":{}},"source":["from sklearn.preprocessing import MinMaxScaler\n","weights = clf.coef_\n","class_id = 0\n","w0 = weights[class_id]\n","scaler = MinMaxScaler((0.1,.9))\n","w0 = scaler.fit_transform(w0.reshape(-1, 1))\n","print(w0.shape)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"GvAT1CMufGIm"},"source":["Sekarang, mari kita tampilkan visualisasi keseluruhan bobot"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"xHrtdRSPlTjT","colab":{}},"source":["fig, ax = plt.subplots(1,10,figsize=(18,5))\n","fig.subplots_adjust(hspace=0.1, wspace=0.1)\n","for i in range(0, 10):\n","    w = weights[i]\n","    \n","    w = scaler.fit_transform(w.reshape(-1, 1))\n","    \n","    w = w.reshape((32,32,3))\n","    \n","    ax[i].imshow(w)\n","    ax[i].axis('off')\n","    ax[i].set_title(classes[i])\n","plt.show()"],"execution_count":0,"outputs":[]}]}