yeger/Master-DataScience-Notes
1
0
Fork 0
mirror of https://github.com/Andreaierardi/Master-DataScience-Notes.git synced 2025-01-28 20:27:37 +01:00
Code Issues Projects Releases Wiki Activity
Master-DataScience-Notes/1year/3trimester/Machine Learning, Statistical Learning, Deep Learning and Artificial Intelligence/Machine Learning/Project/project_code.ipynb

4414 lines
1.1 MiB
Plaintext
Raw Normal View History

Add files via upload
2020-08-29 16:11:34 +02:00
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "CNN.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/github/Andreaierardi/Fruits-Neural-Networks/blob/master/project_code.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HtU6p0pB-N0A",
"colab_type": "text"
},
"source": [
"\n",
"# Image classification with Machine Learning\n",
"---\n",
"\n",
"## University of Milan \n",
"### DataScience and Economics - Machine Learning Module\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LGE0Fo3e-3Wk",
"colab_type": "text"
},
"source": [
"**Authors** : Andrea Ierardi, Emanuele Morales, Gregorio Luigi Saporito\n",
"\n",
"<br>\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "F5Ew02Va-6DV",
"colab_type": "text"
},
"source": [
"How to load the dataset: \n",
"\n",
"\n",
"\n",
"```python\n",
"from google.colab import drive\n",
"drive.mount('/content/gdrive')\n",
"\n",
"\n",
"import os\n",
"os.environ['KAGGLE_CONFIG_DIR'] = \"/content/gdrive/My Drive/Kaggle\"\n",
"\n",
"\n",
"%cd /content/gdrive/My Drive/Kaggle\n",
"\n",
"\n",
"!kaggle datasets download --force -d moltean/fruits\n",
"\n",
"!unzip fruits.zip\n",
"```\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tb4yPcdji5Ai",
"colab_type": "text"
},
"source": [
"# Image classification with Neural Networks\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pcRWLxstjHvN",
"colab_type": "text"
},
"source": [
"## 1. The dataset\n",
"### 1.1 Libraries\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "iXeBOWLtcdMv",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "317f98fa-192a-4556-e97b-9e2d8da88fb0"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import os\n",
"from tqdm import tqdm\n",
"import random\n",
"import pandas as pd \n",
"\n",
"from plotnine import *\n",
"from sklearn.decomposition import PCA\n",
"\n",
"from sklearn.datasets import load_files\n",
"from keras.preprocessing.image import array_to_img, img_to_array, load_img\n",
"from sklearn import preprocessing\n",
"\n",
"from keras.utils import np_utils\n",
"from sklearn.utils import shuffle\n",
"import numpy as np\n",
"\n",
"import matplotlib.pyplot as plt\n",
"\n",
"\n",
"from keras.models import Sequential\n",
"from keras.layers import Conv2D,MaxPooling2D\n",
"from keras.layers import Activation, Dense, Flatten, Dropout\n",
"from keras.preprocessing.image import ImageDataGenerator\n",
"from keras.callbacks import ModelCheckpoint\n",
"from keras import backend as K\n",
"\n",
"from keras.applications import MobileNetV2\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
],
"name": "stderr"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QXB2MCkujQIi",
"colab_type": "text"
},
"source": [
"# 1.1 Data Loading"
]
},
{
"cell_type": "code",
"metadata": {
"id": "2w9LPk2Tc8Y7",
"colab_type": "code",
"colab": {}
},
"source": [
"DATADIR = \"fruits-360/Training\"\n",
"DATADIR_test = \"fruits-360/Test\"\n",
"\n",
"TYPES = [\"Apple\", \"Banana\", \"Plum\", \"Pepper\", \"Cherry\", \"Grape\", \"Tomato\", \"Potato\", \"Pear\", \"Peach\"]\n",
"fruits = {}\n",
"def load_dataset(dire):\n",
" fruits = {}\n",
" images_as_array = []\n",
" labels =[]\n",
" for category in tqdm(os.listdir(dire)):\n",
" for typ in TYPES:\n",
" if(category.split()[0] == typ):\n",
" fruits[category]= typ\n",
" path = os.path.join(dire,category)\n",
" class_num =TYPES.index(fruits[category])\n",
"\n",
" class_name = fruits[category]\n",
" for img in tqdm(os.listdir(path)):\n",
" file = os.path.join(path,img)\n",
" images_as_array.append(img_to_array(load_img(file,target_size=(32, 32))))\n",
" labels.append(class_num)\n",
" images_as_array = np.array(images_as_array)\n",
" labels = np.array(labels)\n",
" return images_as_array, labels"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WHUwPqnPjoPp",
"colab_type": "text"
},
"source": [
"### Split in test and training sets\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "qY-O9bmQdDnq",
"colab_type": "code",
"colab": {}
},
"source": [
"train = load_dataset(DATADIR)\n",
"test = load_dataset(DATADIR_test)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "oEFfdWThdNjj",
"colab_type": "code",
"colab": {}
},
"source": [
"x_train, y_train= train"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "3x9fYmsWhoKe",
"colab_type": "code",
"colab": {}
},
"source": [
"x_test, y_test = test"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "oepGscO8jxgv",
"colab_type": "text"
},
"source": [
"### Train and test shape\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "hUqYyOVThpX6",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 124
},
"outputId": "f475c7c4-1acb-4345-ff7a-c5e437a5942d"
},
"source": [
"print('Train shape:')\n",
"print('X: ',x_train.shape)\n",
"print('y: ',y_train.shape)\n",
"\n",
"print('Test shape')\n",
"print('X: ',x_test.shape)\n",
"print('y: ',y_test.shape)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Train shape:\n",
"X: (32607, 32, 32, 3)\n",
"y: (32607,)\n",
"Test shape\n",
"X: (10906, 32, 32, 3)\n",
"y: (10906,)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nNf0OLzBj0Ot",
"colab_type": "text"
},
"source": [
"# 1.2 Pre-processing"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ENOcYiDEj4aD",
"colab_type": "text"
},
"source": [
"### Pre-process the labels and the images\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "JnU3AsJ3hqyw",
"colab_type": "code",
"colab": {}
},
"source": [
"x_train = x_train.astype('float32')/255\n",
"x_test = x_test.astype('float32')/255\n",
"\n",
"no_of_classes = len(np.unique(y_train))\n",
"y_train = np_utils.to_categorical(y_train,no_of_classes)\n",
"y_test = np_utils.to_categorical(y_test,no_of_classes)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "zklIYBIxhtug",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 214
},
"outputId": "c39ab752-199f-4775-c2d2-932e637a6b71"
},
"source": [
"print(y_train[0:10])\n",
"print(\"Number of classes: \",no_of_classes)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"[[1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]\n",
" [1. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]\n",
"Number of classes: 10\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RQiX-6kfkCRl",
"colab_type": "text"
},
"source": [
"### Visualisation of the first 10 images\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "FmmJABIHhwIh",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 324
},
"outputId": "becfb7a0-905a-4795-9ef5-2e373a3e6971"
},
"source": [
"\n",
"fig = plt.figure(figsize =(30,5))\n",
"for i in range(10):\n",
" ax = fig.add_subplot(2,5,i+1,xticks=[],yticks=[])\n",
" ax.imshow(np.squeeze(x_train[i]))\n",
" ax.set_title(\"{}\".format(TYPES[np.argmax(y_train[i])]),color=(\"green\"),fontdict= {'fontsize': '25'})"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABfYAAAE6CAYAAABUGAPxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeZhkaVnn/fucE1sutVf1Si92tyAwMqO2gPKyKKIoghujoAMNgzBuICL6DooUiSyCgKiMOMhiy9IojOOCo8MILyKISrXgxm7TTTe91JqVS0Sc9Xn/qCLPc/+iK6uyqazK6Pl+rouLePqcOHEiu+OOJ56I87uTEIIBAAAAAAAAAIDpkJ7vEwAAAAAAAAAAAGeOhX0AAAAAAAAAAKYIC/sAAAAAAAAAAEwRFvYBAAAAAAAAAJgiLOwDAAAAAAAAADBFWNgHAAAAAAAAAGCKsLC/iZKF5IPJQhKSheTF5/tcAODeiDoLAJuPWgsAm4s6CwCb695aZzvn+wQ2IllIdpnZ7WY2OPmP7hv2h8+dx1MCgHsV6iwAbD5qLQBsLuosAGwu6uzWMG2/2P8Ra/+DMTP7z+frRADgXoo6CwCbj1oLAJuLOgsAm4s6uwVM28L+M07+/2+e/P/rkoUkO18nAwD3QtRZANh81FoA2FzUWQDYXNTZLWBqFvaTheTrzew/mNmimf28mX3BzC42s+86n+cFAPcW1FkA2HzUWgDYXNRZANhc1NmtY5oy9r/8TdDvh/1hnCwkv2dm++3EpR5/end3SBaS3zWz68zsejN7upn9l5P7f83JXf7JzH4r7A/vPMX9P2hmjzSzBTN7uZk9z8x+2MyuNrPCzA6Y2WvD/vDn9/RJJQvJvzOz55jZt5jZpWbWmNlNJ5/Tr4X94fA9PTYAbBB1FgA2H7UWADYXdRYANhd1douYil/sJwvJwE78yzIz+73o/4OZfXeykFx4Boe5wczeYGbfYGaVmc2b2cPM7B3JQvKWZCFJ1rlvz8z+0sxeYWb3txP/wew0s28zs/91TzsqJwvJz5vZP5rZM83smpPPp2tmX2tmv2Bm/5QsJF93T44NABtBnQWAzUetBYDNRZ0FgM1Fnd1apmJh38x+wE78S/p82B/+xsws7A83mdmH7cRVB089zf2/18x+0Mx+ycx2hf1ht5ldaGavP7n96Wb27HXu/xNm9mAz+zEz2xb2h11mdrmZvefk9v3JQvKEjTyhZCF5hpm90syGZvaLZnZx2B/mzGzWzK41sw/YictY/iRZSOY3cmwAuAeoswCw+ai1ALC5qLMAsLmos1vItCzsf/kSj9+Tf/7l8ek6L+8ws5eG/eGlYX9YMjML+8OhsD8828zefnKf/Se/dTrV/X8i7A//PewP45P3v9XMfsjMPnRyn5ef2VMxSxaSbWb26pPDJ4b94eVhf7jz5HHrsD/caGbfYWY3mtl9zOxHz/TYAHAPUWcBYPNRawFgc1FnAWBzUWe3kC2/sJ8sJFeZ2aPsxCUQb5PNf2BmIzP7mmQh+eZ1DjOy9l+SesnJ/99tZo85xT63mtlb9R+G/aExs5eeHD4wWUi+dp1ziH35262Ph/3hf9/dDmF/qOzEpSlmJ/4DAoBNQZ01M+osgE1GrTUzai2ATUSdNTPqLIBNRJ01sy1WZ6ehee7TzSwxsw+F/eHmeEPYH5aSheSPzOzJduIbo785xTEOfPlbIBX2h88lC8ltduJbl2vt7ps8fDDsD+EUx/5rO5EH1Tl5/39e/+mY2YncKDOz+ycLyZ3r7Ddz8v+vOINjAsA9RZ2lzgLYfNRaai2AzUWdpc4C2FzU2S1WZ7f0L/aThSQ1s6edHOolHl92/cn//8F1co6+dJqH+vL2CzZ6/5OXfRw5zf3VJSf/f2AncqRO9b/tJ/ebPcPjAsCGUGepswA2H7WWWgtgc1FnqbMANhd1dmvW2a3+i/3vsBPf0piZvSlZSN60zr7zdqL5wls2/ay+ctnJ///9sD886byeCYD/21FnAWDzUWsBYHNRZwFgc1Fnt6At/Yt9axsyfKX7X3qa+315+8GN3j9ZSPpmtuc091dfvrRjS12+AeD/StRZANh81FoA2FzUWQDYXNTZLWjLLuwnC8k+M3vCyeETzWzbOv978Mn9vjlZSO53N4e79lSXgCQLyTXWfuN04BSn88hkIUlOse3h1l75cKr7q4+c/P9vSBaSi8/wPgBwVlFnAWDzUWsBYHNRZwFgc1Fnt66tHMXzFDPrmtlxM/vTsD8U6+z7sWQh+bSZfY2d+Ebo52X7jJk938xefDf3feHJ/z9qZv/nFMe/3MyuM7Pfjf/hyXypXzg5/GTYH86kKYOZ2bvN7FV2ouvya5OF5IdP1fjh5GNsD/vD4hkeGwDOFHXWqLMANh211qi1ADYVddaoswA2FXXWtmad3bK/2Lf2ko0/Ps1/MF/27pP//9RkIdEvLI6b2S8lC8kLkoVkm5lZspDsTRaSX7cT/zGYmf3yySYLd+e4mb0hWUiemSwkg5P3v8zMbjCzbzm5zwtPcd8JJ/8DeO7J4ZPM7M+SheQhJ/8DsWQhSZOF5P7JQvKzZvavZvbdZ3psANgA6ix1FsDmo9ZSawFsLuosdRbA5qLObtE6uyUX9pOF5KFm9oCTw3evt2/ky/tdaGaPk21/dHL7y83sWLKQHLUTWUvPObn998zsN9Y59m/ZiUs43mhmSyfv/0U70QjCzOylYX/4n2d4nmZmFvaH683sx82sMLPvNLO/NbNhspAcNrOxmX3SzF5tJ77huttvigDgnqLOUmcBbD5qLbUWwOaizlJnAWwu6uzWrrNbcmHf2m+CjpvZ+87kDicvsfiU3D/2ZDP7CTP7uJ2IIFo1s4+a2VPD/nBd2B+adQ5fmNmj7cQlHZ8xs/7Jc3u/mT0u7A+/dCbneDfn/Ntmdj878R/HP5pZbicu/VixE/+R/qaZPcZOfOsEAGcTdZY6C2DzUWuptQA2F3WWOgtgc1Fnt3CdTcLdxwbdKyQLye/aics4rg/7w9Puwf0/aGaPNLOFsD+8+GyeGwDcG1BnAWDzUWsBYHNRZwFgc1FnN8dW/cU+AAAAAAAAAAC4GyzsAwAAAAAAAAAwRVjYBwAAAAAAAABgirCwDwAAAAAAAADAFLlXN88FAAAAAAAAAODeprORnffu3RuuvPLKs/TQQUaJG/vRRo/m7z+5bb29z4L4y5LkLB/7HLn55pvt8OHD03nywBQ7XZ3d5Op1z230xPRL5Q3Xyvj+G7tvkJNNzuNf8cYbbzwcQth33k4A+L/QZtbZyfqizl+90R/zJKetu9H+QfY9zV2pswDO5ZxWa44ebcvMl+/OhtYO1v9R5uT0+uw98/X+wqwdAOfHiTp7RfRPzvxluOGP7xvc/2yamFducF66qc7iH+Z0hzrVnHZDC/tXXnmlfezAgVNu39j5VzLy984sW/fY+oRrGadNe7tJ/d5p7R87Tf1jhcYnFOn7YQiNbPc71EV7NsnAHzvV/wDl3bfR/2BTORd/74n9zdpzSybe2DPZVXZI26Nfe+21BuDcO1Fn/35tnNS+BpTyMu6s82EgMV+rJurkxOxfaqHUl7Q59RtoU/ujp+lpkt4aOZtM3o4mir7f3y1QJf6+k+8X/nlU8thdrY0T76j+iGGdpzbxPrhOnT1x6OSWUx8NwGa48sor7e8P/N3aOK19DaikJGRRvTndIkllpRt3pCqE00y9k6CVOjqZiSnkqeeAJ47ln0he5m7c7fTdONWJY9qeS1P5B08zfb/w5z1RZ7OeP3YtP/BJT/3+8pV+0KTOAufHlVdeaQeiOa3JZ+xS5lPdaM7USK1NfXmzJvM1ppAa1DH9DC7jRA7ot66zzSxM1GlPP3OHWmqzzJHrql2bSDuyBiJ1uWn8sSaWFuTUOok8l2yd5zYxZ/XjUqptN/obs3YAnB9XXnmFfezA366Nk9B122upEVlUj6rMb+xM7OzrjV9FNUu0Vup6QLbeDM7fN0ht07WEUh69W+q5yuRdS10879TzOl2Ijexey/OcmD5LHXdOU2crORl9L
"text/plain": [
"<Figure size 2160x360 with 10 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "y_E6_3LzkFsR",
"colab_type": "text"
},
"source": [
"### Suffle of the data\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "tyKinSP8h5KH",
"colab_type": "code",
"colab": {}
},
"source": [
"x_train,y_train = shuffle(x_train, y_train)\n",
"x_test,y_test = shuffle(x_test, y_test)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "NIVoRrD7kHvO",
"colab_type": "text"
},
"source": [
"### Visualisation of the first 10 images shuffled\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "9a4KNtsoh6n7",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 324
},
"outputId": "ae70bb27-0b73-4f45-da52-b4103cdab909"
},
"source": [
"\n",
"fig = plt.figure(figsize =(30,5))\n",
"for i in range(10):\n",
" ax = fig.add_subplot(2,5,i+1,xticks=[],yticks=[])\n",
" ax.imshow(np.squeeze(x_train[i]))\n",
" ax.set_title(\"{}\".format(TYPES[np.argmax(y_train[i])]),color=(\"green\"),fontdict= {'fontsize': '25'})\n",
"\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABfYAAAE6CAYAAABUGAPxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd5gkV3k9/nOrqrunJ+3OJmVplSUkcjAWMiKaHyba2ICwMRgM2MYGbNIXMG4aAyYZZMCBDAKBZMAES9giiAzCSIgglLVapc1hYk+Hqrq/P6p3qs47OzM72jDTu+fzPHq2363q6upezdu371Sf67z3EBERERERERERERGR3hAs9QmIiIiIiIiIiIiIiMi+08S+iIiIiIiIiIiIiEgP0cS+iIiIiIiIiIiIiEgP0cS+iIiIiIiIiIiIiEgP0cS+iIiIiIiIiIiIiEgP0cS+iIiIiIiIiIiIiEgP0cS+iIiIiMgy4Oruha7uvKu7jUt9LiIicui4uvtut/+/ZanPRUREeke01CewP7pverW9bGoB2AHg5wA+C+ALvub9ITw1EZFl61D3Tld36wG8EAB8zb9lf4+3VI8hIrIYru5CAM8C8FQAjwSwDkA/gFEAtwD4AYBLfM1fv2QnKSJymHJ1NwJgE4C+7l+d4Wv+1iU8JRGRw57maQ+9w+mK/a2F/zyA4wA8DcBlAK5wdVdZwnMTEVmuDkXvXI/szX1vb/AHyqF4DBGRfeLq7pEAbkDWS58P4HRkk/oTAFYDeBSA/wfg167uvuTqrrxU5yoicpj6Y+ST+gDwoqU6ERGRI5TmaQ+Bw2Zi39f80Xv+AzAA4FwA3+xufjKAty3ZyYmILFPqnSIiB5aru6cB+C6AMwDsBPAGZFeKln3NrwZQBvBwAO8EMA7gD5BN+ouIyIHz4u6fH+z++YLuN6lEROQQ0FzDoXHYTOwX+ZpPfc3/BsDTAdzW/euXubrr6eghEZGDSb1TRGT/uLo7HdnXiyvIrth/kK/5dxbjH3zNJ77mr/E1/wYAJwP46tKcrYjI4cnV3UMAPAhZ9NnrANwB4BgAv7eU5yUicqTSXMPBc1i/gL7mm67uvoDsSqkhAGcBuB4AXN0NAXg5gGcAOBPZb4+2AfgRgH/xNf+TvR3T1d25AP4QwKMBnATgWAAdALcDuALARb7md8xx343d+/wZgC8iG2T8PrIIiUEAJ/ua37h/z1pEZP8s0DtPBfAaAE8AcDyy/ncbgK8g63/jxWMV+t6e2ubofdrX/Au720rIfnP/VAAPQfZVvdXIPpRdB+BTAC61WXyLeYzCPot6HiIi++htAIYBNAH8vq/5e+bb2df8LgDPdHXn9rbd1d1DAbwewO8AWAXgXmR96h99ze+e67j3ZZzbXavkjm55MoCw+9hPRDbe3exrfr2ru8cA+E73/J2ruwcj66cXADiq+xh/AeDG7rF+y9f8/81zrp8B8CcAvudr/jFz7Scisgh7rta/rDuuvRhZXOOLAPz33u7g6u5TAF4A4NPIPq+/rLv/Wd1dfgXg33zNf26O+38XWR+sA3gHgL8D8DwApwJoA7gGwPt8zf/PfX1S3bmIVwB4LLJxcgpgQ/c5vX+ueQgRkeVC87QH3mF5xb5R/EA1DACu7h6E7Cqqf0K2mNkwsoUcjgfwHAA/cnX3hjmOdzmyQcFjkf3P0kD2j/0gAG8C8AtXd2cucE6rAVwL4M3IvqYdL/pZiYgcXHvrnc8G8BtkEzanIXuzLAN4MLIPMde7ujvbHGc7gOLk01bz31hh26OQXbn6EgAPRRZN0QSwFsDvAvgcgMtc3dn3rsU8xn19HiIi83J1dxSyDxVAtijuLft6370tHubq7nkAfgLgjwBUkV2QczKAvwXwA1d3g3Ocx/6Mc/c4D8AvkPXjdcj65N4e61kAfops8moI3TGtr/mbAHyvu9tL53qQ7uKWe16zjyxwTiIiC3J114esJwHAxYU/PYCndnv1Qj4P4N+RjUdjZJ/3HwXgEld3n5jrl7FdZQDfQtaDz0Y2qb8S2cUkX+8uLLloru5eB+CXyPryad3nUwJwfwBvBPCr7i9aRUSWO83THkBHwsT++sLtXa7ujgFwJbL/Of4LwMMAVH3NDyO7yugfASQA3uHq7pl7Od73ALwQwEm+5qvdrNQ+ZG/U/4fsN+d7/S1+wVuQ/U/6+wAGfc2PADgB2W+iRESWg/WF27u6X2neEy/xIwAP6PbNfmRfp9uMrI/9d3Gyydf8w5HlR++pjzb/vbLwOA0AH0Z2degKX/Mruo+xGsArkWVR/xGAvy6e6GIe474+DxGRffBY5GPrL+/nsdYC+ASyK0dP9DW/EtnE+V8jm2Q/B9kVReQAjHP3+DCyX4A+3Nf8gK/5QWS/YLU+hSwr9exuz64im3QCskkxAHhu9wqsvfkTZOPonQC+NM/5iIjsq2chm0i/zdf8jwHA1/wGAD9E9gvSP13g/s8E8GxkkzsjvuZXIeufH+pu/zMAfzPP/f8KwCOQXUAy1P2sfyKyK0EBoObq7umLeUKu7l4M4F3IxspvAnCMr/kBZOPXhwG4ClnU0Nc0fhWRHrC+cFvztPvpsI7icXU3DOCPu+UuALcA+CiyK48+52v+j4v7+5rfBuAfXN3tBvA+ZP+wXzH7vMA+jq/5NoBvu7p7PLIoh4e4ujvf1/wP5zi1KoBH+5q/rnCMeb+qLSJyqMzRO69AdlXQbQB+19d8A8iy8pBNgt+D7E3zVGQfZN672MftRjXMimvoRlV8wNXdJgBfQPYV5A8s9vhdbz/Yz0NEjljnFG5fN+de+6YfWYzYnklydPvVv7q6OwVZxMOFAP7B3O9t2I9xbsFOAE/wNT9ZuP/evoFwA4Cn+5pPCvvtWU/gv5B9GFqH7OrZD+/l/nue36d9zbfmOBcRkcXYE8Nzsfn7i5HFmr0IwHvmuf8KZHFnM4s6+prfDuBvXN2tRPYLyZqru4/4mm/Ocf8X+5r/ROH+d7u6ew6yGLNHI4vq+dq+PJnuL0b3jEf/0Nf8lYXjJgCudXX3JABXI/uGwZ8DuGhfji0icqhpnvbAOyyv2Hd1t7L7j3cVsq9hAMC/IPta3J6v5b1rnkPsGQQ8cB+/qgcA6H742fO14/Pn2fV/i/+ziIgsB/P0zmEAT+rW79kzGV7U7Wn/1S0vPEineEX3z1Nd3R292Dt3P4wth+chIoen1YXbuw7A8d42x9/vWWz3NFd3/Xv+0sRP7O8490PFSf15vKc4qV/ka74D4OPdclYcj6u7RyKLkAAUwyMiB0D3F5+PQRZT8xmz+T8BTAM4y9XdefMcZhpzX9jx1u6fq5B9w3Rv7gbwSfuX3YtI9vT1c1zd3d/uM4c930C4rjipb44dI4sPAvKxrojIsqF52oPnsLlify+LJRZ9FtlVmo9E9nUMAPiGq88XjTfjJGQZzcXHeiqA5wN4OLKvhfTv5X7Hz3PMH+3LA4uIHGz72DsvALCnYX5rnv2/ieyryw9wdVfqTuos9nyGkF0p/1RkuaQrkV1hbx0PYMsiD/8QHKLnISKyn3b5mr9tjm2bCrdHkEUzANmVmvs9zu3a17HqQvt9BNkCvA9xdfcQX/M/L2zbc7X+93zN37yPjyciMp8/QzbW+75d7NDX/Liru68gu3DjxQB+PMcxrvE1P763Db7mb+1+u/N4ZFERe1uI97t7Wzel6wfIcpuj7v1/Pf/TAZBl+wPA2a7u5hv7Vrt/nrQPxxQROeg0T3toHDYT++B/1BaAHci+Bn2Jr/nvAICru2ML++zrb3iKV0IFyP7nK17FGSNbtLHdrVcg+59yYJ5jLvuMJhE5YuxL71xX2OfeeY6156tqEbIrmfY2WTQnV3dnAPg2+A23AWAUQNqt9/Tu+XrsXA7J8xCRI9bOwu1V4An4xZqYZ1txMa/iLz73a5xr7OtYdd79fM1vdHV3JYAnI7tq/y+Ama9hP6e7294iekREFqX7Wf2F3dLG8OzxaWSf5Z/t6u6Vc3wzab4x4p7tx4PHlft0f1/zTVd3O5H16Lnub+3p7X3IJ7/mM1dfF
"text/plain": [
"<Figure size 2160x360 with 10 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aI-v_vkFkK6A",
"colab_type": "text"
},
"source": [
"### Split in validation and test set\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "chR6UKO9h9Az",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 53
},
"outputId": "4ea738a5-6b55-46a7-9d3e-97c6c86c17b9"
},
"source": [
"# Using 80-20 rule\n",
"split = len(x_test)*80//100\n",
"\n",
"print('Test len before split: ',len(x_test))\n",
"print('Validation split len:', split)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Test len before split: 10906\n",
"Validation split len: 8724\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "GTktl3TSh-zx",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 142
},
"outputId": "2bd7432b-98b1-4715-f527-f399dda3a2bc"
},
"source": [
"# Now, we have to divide the validation set into test and validation set\n",
"x_test,x_valid = x_test[split:],x_test[:split]\n",
"y_test,y_valid = y_test[split:],y_test[:split]\n",
"print('Train X : ',x_train.shape)\n",
"print('Train y :',y_train.shape)\n",
"\n",
"print('1st training image shape ',x_train[0].shape)\n",
"\n",
"print('Validation X : ',x_valid.shape)\n",
"print('Validation y :',y_valid.shape)\n",
"print('Test X : ',x_test.shape)\n",
"print('Test y : ',y_test.shape)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Train X : (32607, 32, 32, 3)\n",
"Train y : (32607, 10)\n",
"1st training image shape (32, 32, 3)\n",
"Validation X : (8724, 32, 32, 3)\n",
"Validation y : (8724, 10)\n",
"Test X : (2182, 32, 32, 3)\n",
"Test y : (2182, 10)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wtCzXW5-EmnM",
"colab_type": "text"
},
"source": [
"### Definition of zero-one loss function "
]
},
{
"cell_type": "code",
"metadata": {
"id": "atod5RB6ErJA",
"colab_type": "code",
"colab": {}
},
"source": [
"def zero_one(prediz,test):\n",
" y_hat = []\n",
" y_t = []\n",
" for i in range(len(prediz)):\n",
" y_hat.append(np.argmax(prediz[i]))\n",
" y_t.append(np.argmax(test[i]))\n",
"\n",
" \n",
" loss = []\n",
" for i in range(len(prediz)):\n",
" if(y_hat[i] == y_t[i]):\n",
" loss.append(0)\n",
" else:\n",
" loss.append(1)\n",
"\n",
"\n",
" return np.mean(loss)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "W4E_e-iXB-BS",
"colab_type": "text"
},
"source": [
"\n",
"# 1.3 PCA and feed-forward NN\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "hj6pgmhVCH1q",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 284
},
"outputId": "db0fb3e0-bff7-45f5-a7e5-e18e5ff88b0e"
},
"source": [
"plt.imshow(x_train[0])"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<matplotlib.image.AxesImage at 0x7f72992944a8>"
]
},
"metadata": {
"tags": []
},
"execution_count": 25
},
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAX6ElEQVR4nO3de5DdZXkH8O9zbnvNPZsLSSCAVMtYRbtNURm1WB2qTsGxw+BMHZwyxGmlI622pXRGcKbtoKOgpa1tEGpSkYAiJeMwKlKm0VGQBSEEEIG4SEIuu0k2e7+cc57+cU6cBd/vu5s9t4T3+5nJZPf37O/83v2d85zf7u/Z93nN3SEir32ZVg9ARJpDyS6SCCW7SCKU7CKJULKLJELJLpKIXC07m9lFAL4MIAvgq+5+Q+zrV65c6Rs3bqzlkCIS0d/fj8HBQQvFFpzsZpYF8G8A3gtgL4BHzGyHuz/N9tm4cSP6+voWekgRmUNvby+N1fJj/CYAz7v7HnefBrAdwMU1PJ6INFAtyb4OwEuzPt9b3SYiJ6GG36Azs81m1mdmfQMDA40+nIgQtST7PgAbZn2+vrrtFdx9i7v3untvT09PDYcTkVrUkuyPADjHzM40swKAywDsqM+wRKTeFnw33t2LZnYVgO+hUnq7zd2fqtvIRKSuaqqzu/t9AO6r01hEpIH0F3QiiVCyiyRCyS6SCCW7SCKU7CKJULKLJELJLpIIJbtIIpTsIolQsoskQskukgglu0gilOwiiVCyiyRCyS6SCCW7SCKU7CKJULKLJELJLpIIJbtIIpTsIolQsoskQskukgglu0gilOwiiahpRRgz6wcwAqAEoOjufCX4KmePxQIAShbenl3APpX9ijzo/JRMkbfGPEp0n0x5OnKsyDeQzfLdym08xrZH3tazRT5+5Mo0VCxHHjQTHn8WM3SXMvixMuDfc+y1w3fixypFroH8WTn51ZTsVX/g7oN1eBwRaSD9GC+SiFqT3QF838weNbPN9RiQiDRGrT/GX+Du+8xsFYD7zezn7r5z9hdU3wQ2A8Dpp59e4+FEZKFqurK7+77q/4cA3ANgU+Brtrh7r7v39vT01HI4EanBgpPdzLrMbNHxjwG8D8Dueg1MROqrlh/jVwO4x8yOP8433P27c+1kpMpTyvJSSJa8J3mkvBZ7Fxsx/m3/1w2fpbFVhaPB7dnMBN2nLcPrQp7jsY7l7TQ2ZrwMNT4WLm0tXbKe7jO1/Dwa+8CF76WxQqTmRUOep/tEXgLRmpcbL+exHS1WXouV8iKvuZPdgpPd3fcAeHMdxyIiDaTSm0gilOwiiVCyiyRCyS6SCCW7SCLqMRHmhDipr2QiM57AZpWVeRnn9q99iz/ci9+joSXFcHkNALKj48HtuRwfx3jpCI2tOfsMGvOjIzTWOVOgsdL4aHB7e3aM7nPGksM09rPv7qSxqezraCy/9Ozg9t7fu4Dvk5misfh8M37+eaUs8nqLzIhrQcrUja7sIolQsoskQskukgglu0gilOwiiWjyrUWH0/5v/A6zkVuqt2/fQfcZfPCrNNYdmcCRb+P96bLkrvvQ+DG6z1RuksY6Bg/wWAcf47LCEhqbGTkY3N6+nN9hfvmX/C54Wwd/XqbKe2nMx54Ibn9k+CG6z2TuN2ZI/9qF73onjVnszjptvsevc7HeerlTuAmdruwiiVCyiyRCyS6SCCW7SCKU7CKJULKLJKLJpTdDhpTYYlMP7r3nB8Htz9+5le5zRgcvoXUVOmnsV0+9QGN+5tLg9myeL/F0ZraLxqan+BinynwizJEy36+cD9eGDh8cpvssWcPLfDND3TS2Zt1qGhsYOhTcPjnGy5Q96/bR2E8e4L1Ms5181bHffVu4nJeNvOJymdfmNfC1+V2JyG9QsoskQskukgglu0gilOwiiVCyiyRiztKbmd0G4IMADrn7G6vblgO4E8BGAP0ALnV33rxtFlbwmI4sq/P0tm3B7W3T4RleADB8iJenjpbDZSEAyJf4QMb7w+WrPHjpaiiylmV+cD+NLVm1ksYyXbznWom8fU9Nh3vTAUC+cw2Nta3lZcpDQy/R2OJC+KXVtiTSr2+EL2vV3vELGusY58/n7oeeDW7/nfM/SveJTWwrOS/ZZYxfOw18iTC6JFZkmTKPLVFFzOfK/jUAF71q2zUAHnD3cwA8UP1cRE5icyZ7db31V7dIvRjA8b9o2QrgkjqPS0TqbKG/s6929+M/gx5AZUVXETmJ1XyDzt0d4L+0mtlmM+szs76BgYFaDyciC7TQZD9oZmsBoPo/vUPi7lvcvdfde3t6InerRKShFprsOwBcXv34cgD31mc4ItIo8ym93QHg3QBWmtleANcBuAHAXWZ2BYAXAVw63wM6qWz97Z9dTfdpHwzPhioe4b8WdI3P8Mcr8rIcZniJJE+WBWrLR0okR3jJa9lpvMhTauPjf+HZXTTWvSzcjHI6z0tGixatpbGxMV5RXbGaz+hzC5/j8jRvblnAczS2ePlZNJbv5CW74ni49PbQ/22n+/z+u/jLOVeO1IiNP2fwDh4iDxmbCRoZBTVnsrv7R0joPQs4noi0iP6CTiQRSnaRRCjZRRKhZBdJhJJdJBFNbTg5MHgUt95ydzBmz/6c7jcyRNYvG+SNHifGh2hsdKpEY+Y8tqY7XNYq5HlZZewgbxw5U+QFlKPDvEGkRUp9JbKWXtf6FXSftjZe5ClP8XLS2Et8rbpj5JSsaTuDjyPXTmPFw7xcOjrOG1W2l8OzB1cW+Cy6e7f/K419+LIraeyHP3yMxm6+5UYau2PbncHtmci1uByZYcfoyi6SCCW7SCKU7CKJULKLJELJLpIIJbtIIppaehsbGcNPdv4kGMsMvbrz1SzD4fLP2MQ43aVzgpeMjFfX0NnOGywOj02GxzHM1y9bumwxjZVHeelwYOgwjZ1+Du8LcOilF4Pbezp56ar/OV6m7IqMv3sRL5VlSbPEkRFeipzs4rMAl3fwc5yf4WXFyZk9we3Zycgadjlelnv8O7xEnC3zcRzo5805b/zC14Pb//rqj9F9MgvIXF3ZRRKhZBdJhJJdJBFKdpFEKNlFEtHUu/HFYhFHj4TvMmeHee83nwnfPs9N8Nvq5UgDr+4M71k2VeR3hLPlcMfsxRm+pNHYDL8LXoiMMZ/hk2sm+Q1trNwQ7tXWkeGTbgb3RaoJK3h/uoOHeQWlo6MQ3D5e5j3oli3jPe2GbTmNFQp80hCbbFQs8306s/zc52b4eTTw6tDOnT+msRmShlmM0X3KZX6uGF3ZRRKhZBdJhJJdJBFKdpFEKNlFEqFkF0nEfJZ/ug3ABwEccvc3VrddD+BKAMfXX7rW3e+b82hmKGbDh/QcXQgWmeyy4PZyhpfrikVe1xrN8GN1R97+JknprRApk2GUl5pKiyK95Mq8BNgxxUt9h385GNze1cPLjcvXn0Zj48fCk38AIBNZdml0MjwRadVSPrHm2AF+rNGpl2ksx08VcmTZq3IhXBoEgBnw56w4yifQ9I/zpb7Oj6Qaj/AxZhZwmZ7PLl8DcFFg+03ufl7139yJLiItNWeyu/tOAJH5pyJyKqjld/arzGyXmd1mZuGfs0XkpLHQZP8KgLMBnAdgP4Avsi80s81m1mdmfdPT/HcyEWmsBSW7ux9095K7lwHcAmBT5Gu3uHuvu/cWCryziYg01oKS3cxmz474EIDd9RmOiDTKfEpvdwB4N4CVZrYXwHUA3m1m5wFwAP0APj6fg2WyGSzqDvd4OxLpC5cdC8/KysXKZOA96AqRb7tUjsxSQ3iQxRL/9aQNvMzXORZZhirLS4dDB/n90iUrwj89TQ7yMR7Jhct1ADA9xMdR6uax7mXhpbL2TQ8EtwNA0fk41na8nsYmIjPRitPh8mz7Il4uPVzk4/jVKJ9t9vf//AMaA1mWCwBg4ddICbzEGqk2UnMmu7t/JLD51gUcS0RaSH9BJ5IIJbtIIpTsIolQsoskQskukoimNpzcuGEDbrn5S8HYH7/9HXS/yaO/DAeKvJzUHSlblEu8ZDQNHstkw
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "8wJ0sLn7CLb_",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 284
},
"outputId": "ea683341-8d6d-4ae7-e06d-30a537c65f5c"
},
"source": [
"x_train[0].shape\n",
"type(x_train[1])\n",
"rgb_weights = [0.2989, 0.5870, 0.1140]\n",
"image_test = x_train[0]\n",
"image_grey = np.dot(image_test[...,:3], rgb_weights)\n",
"plt.imshow(image_grey, cmap=plt.get_cmap(\"gray\"))"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<matplotlib.image.AxesImage at 0x7f729920a198>"
]
},
"metadata": {
"tags": []
},
"execution_count": 26
},
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAATaElEQVR4nO3de4xd1XXH8e/KYBuDDdiegUyM8SQGqUJRA2hEKEXIJAqiKBIgVQgUIf4gcVIFCaL0D0SlxpWqKKkbUP5AVBNAcVAKcZvwEDIpLkKCRImTSQrm1RaDDNj4MQYc2yHB2Kz+cY/VsXXWmutzX+Ps30eyfGfvu+9Zc2bW3HvPuntvc3dE5E/fRwYdgIj0h5JdpBBKdpFCKNlFCqFkFymEkl2kECd0MtjMrgC+CwwB97j7t7L7Dw8P+9jYWCeHFJHEli1b2L17t9X1NU52MxsC7gI+B2wFfm1mj7r7S9GYsbExJicnmx5SRGYwPj4e9nXyMv5CYLO7v+buB4AHgas6eDwR6aFOkn0p8Oa0r7dWbSIyC/X8Ap2ZrTKzSTObnJqa6vXhRCTQSbJvA5ZN+/rMqu0I7j7h7uPuPj4yMtLB4USkE50k+6+Bc8zs42Y2F7gOeLQ7YYlItzW+Gu/uB83sZuA/aJXe7nP3F7sWmYh0VUd1dndfD6zvUiwi0kP6BJ1IIZTsIoVQsosUQskuUgglu0ghlOwihVCyixRCyS5SCCW7SCGU7CKFULKLFELJLlIIJbtIIZTsIoVQsosUQskuUgglu0ghlOwihVCyixRCyS5SCCW7SCGU7CKFULKLFELJLlIIJbtIITraEcbMtgD7gEPAQXePd4KfRdy9q+M+8pH4b+aHH37Y6FjZYzaJ38zCvizGpt9bdLwsjuz7ysZJezpK9spl7r67C48jIj2kl/Eiheg02R14wsx+Y2aruhGQiPRGpy/jL3H3bWZ2OrDBzP7b3Z+efofqj8AqgLPOOqvDw4lIUx09s7v7tur/XcBDwIU195lw93F3Hx8ZGenkcCLSgcbJbmYnm9nCw7eBy4EXuhWYiHRXJy/jzwAeqkoiJwD/6u4/nWlQt8tGTWQlozVr1oR98+fPr23P4stKV0NDQ2HfwoULw77sHL733nu17YsWLQrHnHLKKWHf5ZdfHvZl33fUl8Wu0ltvNU52d38N+FQXYxGRHlLpTaQQSnaRQijZRQqhZBcphJJdpBDdmAhzTLpZQslKNWvXrg373nrrrbDvgw8+CPv27t1b2z5nzpxwzIEDB8K+sbGxsC8qoQEcPHgw7Pv9739f237iiSeGY0477bSwb/369WFf9n2feuqpte2f/vSnwzEqr/WWntlFCqFkFymEkl2kEEp2kUIo2UUK0fer8d2cCPPAAw+EY375y1+GfSecEH/b2RXmefPm1bZHV+kh/37ffvvtsC+7en7SSSeFfX/84x9r27Mr+G+88UajOLLH3L9/f237E088EY6ZO3du2HfZZZeFfd126NChsC+bvDTb6ZldpBBKdpFCKNlFCqFkFymEkl2kEEp2kULMmokwWYnq4Ycfrm3PJmlka7hl5bVXX3017Fu6dGlte9O15JpMaIF8Db0olqmpqXDM8PBw2BeV0ABGR0fDvnfffbe2Pfu+PvrRj4Z9GzZsCPuycxxNvMkm3RzP5bWMntlFCqFkFymEkl2kEEp2kUIo2UUKoWQXKcSMpTczuw/4PLDL3T9ZtS0GfgSMAVuAa929vtZylKjElpXeHnnkkdr2rHS1Y8eOsC+b1ZSVtXbu3Bn2RZYsWRL2ZVtDZeOyslEUfzQbDuL14iBfn67JrL2sTJbFmM2Iy8p5v/jFL2rbL7744nBMpukWVbNh27N2ntm/D1xxVNttwJPufg7wZPW1iMxiMyZ7td/6O0c1XwUcXr51LXB1l+MSkS5r+p79DHffXt3eQWtHVxGZxTq+QOetNyPhGxIzW2Vmk2Y2mX1kU0R6q2my7zSzUYDq/13RHd19wt3H3X18ZGSk4eFEpFNNk/1R4Mbq9o1A/eVyEZk12im9PQCsBIbNbCvwDeBbwDozuwl4Hbi23QNG5YRVq1aFY3bv3l3bvmfPnnBMVpbLZOOi2VDZLLqsLJS90skWldy8eXPYF5XRsjJOVl7LznEWf3Qes+21srJnFmN2rqJZe0899VQ4ZuXKlWFfU90uozUp5c2Y7O5+fdD12WM+mogMjD5BJ1IIJbtIIZTsIoVQsosUQskuUoi+Lji5e/du7rnnntq+bdu2heN+97vf1ba/99574ZhsBlVW/slE5Z9sP7R33jl6WkF7cezbty/sy2bLRaWm008/PRyTzSjLzmM2szAqRy5fvvyYx0C+n172exCVS7P9/rI9BK+/PipOwTPPPBP23XXXXcd8vKxc16SUp2d2kUIo2UUKoWQXKYSSXaQQSnaRQijZRQrR19Lb/v37w/JEVF6DuLSSlaey2WvZjKGsjBaVtbLYFy9eHPa9//77YV+0VxrAihUrwr6tW7fWtmdlrawEmM02yxaPjMqD2c8sO/fz5s0L+7KfZzTrMCtfZmW5xx57LOzLZu29+eabYd8dd9xR237rrbeGY5rsR6dndpFCKNlFCqFkFymEkl2kEEp2kUL09Wr8wYMHw4kh2RXt6Mp6Nkkju0KbXZnOruJHV3Czq7fZFfdsG6rsams28ePMM8+sbc9ijNb4AxgeHg77sqXBoyvrTa/8Z/FnE3mi42U/56ZX/rOr8T//+c8bPWY3x+iZXaQQSnaRQijZRQqhZBcphJJdpBBKdpFCtLP9033A54Fd7v7Jqm018CXgcO3ldndf384Bo7WzsjW1olJZNiYrgzQteUXlmqwslG3/dPLJJ4d9WWnlwIEDYd8bb7xR275o0aJwzOjoaNiXTVzJSphRWTQrr2UlwKyEmf3Mop9N03Jp9juXbZWV6fbWUJF2ntm/D1xR036nu59X/Wsr0UVkcGZMdnd/GoiXSBWR40In79lvNrNNZnafmcWvEUVkVmia7HcDK4DzgO3Ad6I7mtkqM5s0s8nsvaaI9FajZHf3ne5+yN0/BL4HXJjcd8Ldx919PPsMs4j0VqNkN7Ppl2+vAV7oTjgi0ivtlN4eAFYCw2a2FfgGsNLMzgMc2AJ8uZ2DDQ0NsWDBgtq+bBZSk5f/2eNlmhwrm32XrXX2hz/8odG4bLZZVGLLZhVmpau333477MteqZ166qm17dlabNnPrMnMtqwvK3tmvwPZeVyzZk3Yl5VS+1V6mzHZ3b1uc6t7exCLiPSQPkEnUgglu0ghlOwihVCyixRCyS5SiL4uOHnWWWdx99131/Zdeuml4bio3JGVXLJyRjbrLSuRRCWq7PGy2VVZjE1mckFcljvppJPCMVnpcNmyZWFfNuttx44dte3ZApbZ9k979+4N+7IZjtHvTrRNFuTnfmJiIuzL9Ku8ltEzu0ghlOwihVCyixRCyS5SCCW7SCGU7CKF6GvpbWhoiFNOOaW2r8nssGxM07JWNvMq6sseLxPNDIO8HJb1RWW5bH+47DxmfVlZbv/+/bXt0c8fms8ezMqlURwf+9jHwjFZefB4pmd2kUIo2UUKoWQXKYSSXaQQSnaRQvT1ajzEV06XLFkSjnnrrbdq27NterIrtNkV9+wqfnSlO5uIkcX4zjvx3htNJ05Ea/xlV7OzLarmz58f9kVbTUE8ISeLI6sYZGvhNd1GK/KFL3wh7Mt+1k0rBrNp+ycR+ROgZBcphJJdpBBKdpFCKNlFCqFkFylEO9s/LQN+AJxBa7unCXf/rpktBn4EjNHaAupad3+3aSCPP/542HfBBRfUtmfriDXd/ikrg0SP2XTSTbZ2XfaYWRknesx58+aFY7JyUjSRBPJyaTSp5fXXXw/HZCW0LMbt27eHfaOjo7Xt3/zmN8MxTSdYzXbtPLMfBL7u7ucCFwFfNbNzgduAJ939HODJ6msRmaVmTHZ33+7uv61u7wNeBpYCVwFrq7utBa7uVZAi0rljes9uZmPA+cBG4Ax3P/z6aQetl/kiMku1nexmtgD4MXCrux+xiLe33
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "F7WMKP77CN5l",
"colab_type": "code",
"colab": {}
},
"source": [
"# transform np.ndarray from rgb to grey\n",
"x_train_grey = np.ndarray(shape=(x_train.shape[0], 32, 32))\n",
"for i in range(x_train.shape[0]):\n",
" image_convert = x_train[i]\n",
" x_train_grey[i] = np.dot(image_convert[...,:3], rgb_weights)\n",
"\n",
"x_valid_grey = np.ndarray(shape=(x_valid.shape[0], 32, 32))\n",
"for i in range(x_valid.shape[0]):\n",
" image_convert = x_valid[i]\n",
" x_valid_grey[i] = np.dot(image_convert[...,:3], rgb_weights)\n",
" \n",
"x_test_grey = np.ndarray(shape=(x_test.shape[0], 32, 32))\n",
"for i in range(x_test.shape[0]):\n",
" image_convert = x_test[i]\n",
" x_test_grey[i] = np.dot(image_convert[...,:3], rgb_weights)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "kQp1VlmNCTrg",
"colab_type": "code",
"colab": {}
},
"source": [
"# flatten 32x32 images by concatenating them into a vector, each column of the matrix will be an image\n",
"x_train_flat = np.ndarray(shape=(1024, x_train_grey.shape[0]))\n",
"for i in range(x_train_grey.shape[0]):\n",
" x_train_flat[:,i] = np.concatenate(x_train_grey[i])\n",
" \n",
"x_valid_flat = np.ndarray(shape=(1024, x_valid_grey.shape[0]))\n",
"for i in range(x_valid_grey.shape[0]):\n",
" x_valid_flat[:,i] = np.concatenate(x_valid_grey[i])\n",
" \n",
"x_test_flat = np.ndarray(shape=(1024, x_test_grey.shape[0]))\n",
"for i in range(x_test_grey.shape[0]):\n",
" x_test_flat[:,i] = np.concatenate(x_test_grey[i])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "EDytYvesCVeH",
"colab_type": "code",
"colab": {}
},
"source": [
"standard_scaler = preprocessing.StandardScaler()\n",
"x_train_flat_T = standard_scaler.fit_transform(x_train_flat.T)\n",
"x_valid_flat_T = standard_scaler.transform(x_valid_flat.T)\n",
"x_test_flat_T = standard_scaler.transform(x_test_flat.T)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ZGtU-XWnCWvV",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "1922c5a6-2567-4bf3-fe73-9b52459a5b41"
},
"source": [
"x_train_flat_T.shape"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(32607, 1024)"
]
},
"metadata": {
"tags": []
},
"execution_count": 30
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "uP7JKLlfCtyz",
"colab_type": "code",
"colab": {}
},
"source": [
"x_train_flat = x_train_flat_T.T\n",
"x_valid_flat = x_valid_flat_T.T\n",
"x_test_flat = x_test_flat_T.T"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "thz9QZ8vCwqv",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "711ec999-f6f6-43eb-f2a0-27b722610167"
},
"source": [
"x_train_flat.shape"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(1024, 32607)"
]
},
"metadata": {
"tags": []
},
"execution_count": 32
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "GJdRuaA8C6i1",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "49eb7a01-aacf-473f-8535-e3e46f941c4d"
},
"source": [
"a = np.cov(x_train_flat)\n",
"b = np.linalg.eig(a)\n",
"b[0].shape"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(1024,)"
]
},
"metadata": {
"tags": []
},
"execution_count": 33
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "vlld5mxVC8jK",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 285
},
"outputId": "a4622afa-ad6d-4a8f-c13d-976f75310bbd"
},
"source": [
"b"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(array([384.41591328, 86.57192039, 60.39849467, ..., 0. ,\n",
" 0. , 0. ]),\n",
" array([[-2.77148730e-03, 1.99671963e-02, -7.00245270e-03, ...,\n",
" 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
" [ 1.06770166e-04, 5.26867240e-03, -3.60037909e-03, ...,\n",
" 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
" [-1.31497722e-04, 4.57673250e-03, -2.40477865e-03, ...,\n",
" 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
" ...,\n",
" [ 9.24994377e-04, 3.61739014e-03, 2.95348978e-03, ...,\n",
" 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
" [ 1.13335613e-03, 5.13559532e-03, 3.96259018e-03, ...,\n",
" 0.00000000e+00, 0.00000000e+00, 0.00000000e+00],\n",
" [ 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, ...,\n",
" 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]]))"
]
},
"metadata": {
"tags": []
},
"execution_count": 34
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4h8Gn0NrZMoB",
"colab_type": "text"
},
"source": [
"### PCA explained variance ratio and \"Eigenfruits\""
]
},
{
"cell_type": "code",
"metadata": {
"id": "CLkaQNHKDAAR",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 446
},
"outputId": "74dbcbf3-c122-4bab-e5e3-c59b223a2221"
},
"source": [
"pca = PCA().fit(x_train_flat)\n",
"plt.figure(figsize=(18, 7))\n",
"plt.plot(pca.explained_variance_ratio_.cumsum(), lw=3)"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[<matplotlib.lines.Line2D at 0x7f72991f9b38>]"
]
},
"metadata": {
"tags": []
},
"execution_count": 35
},
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABBEAAAGbCAYAAACS6AhyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZSkV30f/O/tffZFM9JIGkkjyRJCAgRizGoMNosB+wWSYAdOwHiJyebEiZ3kYL9v7Nd2nBw7eePYiUPA2MbxMRBsYyMbYWIIBsJmBAihFW1IGmlGs0izT+/3/aOqq6tn0VTP9Ex1d30+58yp597nqapft0p15n7n3vuUWmsAAAAATqev2wUAAAAAS4MQAQAAAOiIEAEAAADoiBABAAAA6IgQAQAAAOjIQLfeeNOmTXXbtm3densAAADgJL761a/urbVuPtm5roUI27Zty6233tqttwcAAABOopTy8KnOWc4AAAAAdESIAAAAAHREiAAAAAB0RIgAAAAAdESIAAAAAHREiAAAAAB0RIgAAAAAdESIAAAAAHREiAAAAAB0RIgAAAAAdESIAAAAAHREiAAAAAB05LQhQinld0spu0spd5zifCml/GYp5f5Syu2llJsWvkwAAACg2zqZifD+JK99mvOvS3JN8887k7z77MsCAAAAFpuB011Qa/1sKWXb01zyxiT/o9Zak3yplLK+lHJxrXXnAtUIAACwKNVaM12Tqema6VpTazJVG8fT041z7cdTreOTnGv1n3hupr82j0/1OF2TpP05jRrr07Uz2z89XVOT2Xbr2pn3OUk7mVNz6tx2rbO/p+nm9Y33mm13+rMd/35zf5bmz39cu/V+zfecaaftfWf6mr++Oe3a/B01fo7Z/+71hOc3nnz869Wa/PrfvTHfe91F5+dDeY6dNkTowKVJHm1r72j2CREAAOA8mp6umao1U9PNP81B4exxMjk9nenptK6bbr/+FM9pXDudqenMec7TP/ck9bQNkqemZwd2U9Ptg74Tz9XafO3max5/3cy52lbXzAD1xHOzrzHVNgCdmm4GAMcP8E9yrj0omBlUwtMZn5zudgkLZiFChI6VUt6ZxpKHXH755efzrQEAWEZqrZlsDkonp2umpmomp6dn263H6UxO10xO1ePOTR/33JP0t85PH/eac1977vPb+o97/ZMNvKebP8fsYHt24D3d9n5PP2ifDQSAxWk5hU0LESI8luSytvbWZt8Jaq3vTfLeJNm+ffsy+jUCACxO09M1E9PTmZxqDKRnjiemZgbX05loDsAnphrtyenm+fb+1vlTXzv7PtOZaL52o3/u+8x5/5lB99TswLiTgb7xMotJX0n6SklfX0lfSfpLSV8pKSXp7ytzzvWVmfYprmueax03z5U0+mceG88rKW3vX5qv1f4+aR03r0lS2tvlxPacxyR9fce1m9eXtutO1u4rpfn7abb72t9/5jkntmd+lr7S/Hnbanza2tt/tr6T/6wzNbX/nCUzj43+zGmXVv/MdU/3/Jzweo3nrxo+r/9+f04txE9yc5KfLKV8KMkLkxywHwIAsFzN/Av4+OR0JqamM94cHE+0tRvn6uz549ozz51oPre9PT45nfHmtbN9tXWu/XmtAXnbwHziuEG8wXZvKTMD2L6S/lKaA9NkoL8vfaWkv6/t/Myf1nXNx76S/pIM9PU1Brpt52aunb1u9rkDM319J9bQek5bu1Xr0wy2+487d8Jge+a6vpNc13qt2YH4Gb3G8TX1nXgOeslpQ4RSygeTvCLJplLKjiS/kGQwSWqt/z3JLUlen+T+JEeT/Oi5KhYA6D211kxM1YxNTmV8cjpjk9PHPc72jx3XPtV1rfbUdMYmpuc8Hj+4b/U1g4DxqeWzrnWpG2gORluP/X1z263HZn//Kfpbzy/p7+s7yfOb/Sc8/7jr+0/ef8JgvXnc15dmDbOD05nHgb7jBunHP2fmX7KPG+Ab0ALnWid3Z3jrac7XJP9kwSoCABadmYH86ORURiemMjYxndGJqYxOTLf6Rlt9UxmdnM7YxHH9k1MnDNhPPrg/MQzg7Az2NwbMA/0lg/2Nwe1gf6M997jvNNc2z8857stgcwA/0F8y2Hxue//c1zzx9Tsa5PfP7Z+Z1gzA+bV8FmYAQI+ptWZssjFAPzo+lWMTUzk23n48mWPtA/7J6TmD+rHJ4wb+rUCgLQCYnD1vWvysmYH1YH/J0EB/hvpLBgf6mn19GeovGWprD/b3ZWigNM/1ZXCg+dhf2s639Q30zV7bPDfYX1rPnX3/4wfus8czA/TG1GuDbQAWhhABAM6RmUH+0fGpHB2fnB3sj0/l6MRURpsD/vbjmcF/eyhwrO15M8eN15rsmYH9zAB6eLC/+dg393GgP0MDfRkeaA7GBxp9w219w239J2sP9c9eN9g2wJ8JA2YG8oN9faaMA9CzhAgA0DQ+OZ2j45M5Mj6VI2OTOTLWGMwfHpts9I81+8encnRsMkfm9LVdOzbVai+nW64N9JWMDPZnZLAx8B4Z7Gu2m8cDjePhmf451zQeh59mEN8KAJrhwHB/fyskMGgHgMVBiADAkjY5NZ3DY5M5NNr4c3hsMofHJua0D41O5HBb+4RgYLwx8F+KG+YN9fdlZLAvK4cGsnKoMYhfOdSfFUP9WdE+wJ85Hmj8a/7xA//2wf7wCX2N5w3093X7xwUAukyIAEBX1FozOjGdg6MTOXBsIodGJ3JwdLJtsN8Y+B+cCQZGJ3Oo2XeoGRocHm2s+V/Mhvr7snK4PysH+zMy1BzgD/ZnxdBAVg42B/vNvpXHHTcCgePCgeOuM7AHAM4nIQIAZ2xquubw6GQOHGsEATOBwMET2pMnnD94bHLR/ct/f1/JqqH+rBpuDNxXDw9k5dBAVg0PZNVwY0C/uvm4arhx3aqhtmuHB1rPXzU0kJXD/Rk0yAcAlhEhAgCZmJrOU0fHs//oRJ46Mp6njk5k/9HZx/1HJ7L/2PgJgcCh0clul55SktXDA1k7MpjVwwNZPTKQNSMDWT3ceFzT7J/pWz08GwrMDPZnQoPhgT672AMAPA0hAsAyUmvN4bHJPHVkIk8dHc9TRxsD/+ODgVZg0Hw8PNadMGCovy9rVwxm7YpGCLCmGQCsGR7M6jlBwGwYsHpkIGtHBrK6ec2qoX4DfwCA80SIALDIHRufyt7DY3nyyHj2HRnL3sPjjePDY9l3eDz7mv0zx+OT53eJwJrhgWYQMJh1zTBgXas9czzQeDzunH/5BwBYWoQIAOfZ5NR0njwynt2HxhoBQDMg2Hu47bit/+j4ud84sK8k61cOZf3KwWxYOZQNKwezfs5j49zcIKAxO6DfrfcAAHqGEAFggYxOTGX3wbHsOTya3QfHsvvQWHYfGs2eQ83jZt+TR8YyXc9dHSODfdm4cqgx+F81GwZsmOlrHc8EBkNZMzKQPmEAAACnIUQAOI3Riak8cXA0Ow+MZteB0ew+NBsS7GkGBbsPjZ2zTQaH+vuycdVQLlg9lAtWD+eCVUONPzPHq+cerxzy1Q4AwLnhb5pATzs8NpldB45l14Gx7DxwLLsOjGbnwUZYsPPAaJ44OJonj4wv+PtuXDWUzauHs2nNUDauagQAm1Y3j1fPPV4zPGDfAAAAFgUhArBsTU5NZ+eB0Ty2/1h2PHUsjz11LI/tP9qaUbDrwGgOLeBdCQb6SjavGc6Fa4azec1I6/jCtcO5cM1Is384m1YPZ2igb8HeFwAAzhchArBkjU1OZef+mZDgaB57qhEW7NjfCAx2HRzN1AJsPtDfV7Jl7UguWjucLetGctHamYBgZE5IsH7FoH0FAABY1oQIwKJVa82eQ2N5+MmjeXjf0Ty870geefJoIyh46mh2HxpLPcuMYGigLxevG8mWtSO5eN1ILlo3kovXjmTLuhW5eF2j74LVw+5AAAAAESIAXTaz5ODhfUfz7WZI8PC+I3l439E88uTRs7694eY1w9m6YUUuXb8iWzeszKUbVuSSdSPZsm4kF69bkQ0rB+03AAAAHRIiAOdcrTW7D43lg
"text/plain": [
"<Figure size 1296x504 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "oDjU6YEtDCCg",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 300
},
"outputId": "441a7dcc-7cbc-46ce-8a62-e0aefa9a491a"
},
"source": [
"# try to plot some of the eigenvectors, the so called \"eigenfruits\"\n",
"fig = plt.figure(figsize =(30,5))\n",
"for i in range(10):\n",
" ax = fig.add_subplot(2,5,i+1,xticks=[],yticks=[])\n",
" ax.imshow(np.squeeze(b[1][:,i].reshape(32,32)))\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAABfYAAAEhCAYAAAA9CfGpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdWawkWX7f9xNbrnerW3vv68z0LBwOR54ZjkaiZQsEaIrwJlgQYOvR9oNhA5QhwYANwzBsw/CT9CDbsGAYAmzJsC2ClG2QFi2TIKmByOHsw+6Znp7unq7qquqqW3XX3CPCD4RunN8vujLrit11O8nv5yn/HXEjI7M6/3HyZObvJHVdBwAAAAAAAAAAsB7S8z4BAAAAAAAAAADw6JjYBwAAAAAAAABgjTCxDwAAAAAAAADAGmFiHwAAAAAAAACANcLEPgAAAAAAAAAAa4SJfQAAAAAAAAAA1kh+lp03d/P68pPd07qfzGV7bfv3k+Shx1rY3nWtdRn0b6e1nmpl2xd1JvUs2n+8KHTfSj/PqGo9Vl0//LxDCCFJ9FyztNI62t7L9DnqpgupB+lU/9ael649h3XrWVZZ9FlNZftO7DneLwdSX8xOTm/fvFGG+/er5U8EgA9cvjWoiys7p3U1y5bsbZa3h9VWveJ9e9rcYZLqnafWF1Prm167Onhf1u1x3y5L+4zaW5f39FJLP5XWJeCP0AmHGxOpR4c9qafv3rhX1/Xlf/Z7AHBW2XBYFzu7zX9Y8RpfOixc2S9WNGZrX2mmvTOPah9TdhJtZmmif+v81HwsPa9sLF01Y+nJwsbhq/qsP+xVfbRcssOqa5tt9yNNb9JngfOQbQzr/OKF5j+s6pdnGceecWzn91VnSxqHjWl9SsPnAlx7+/IxbazyMa3v+0cd65/l7b0/bp3GCHXRnMxi70Eoj0+YOwAes2xjWOe78Zh2+aAoiXpf3RrLnfHOWz3d5kmtzw7y2entnXwk2zrWYHxeNLNGXFkjnduA2ueP4/nhw7m+H5+XOv5t9WEbo/pwu/U+IV1yDTjjmNbNbrz/mPZME/uXn+yG//KXXjmtP9W5Ldt9Mv6VQifUYw8qneyY2T/Mgb25eH2u5z6p9Nh3F1tS35g1g4jv7D+h+55sSD2a2sT/Qu87tX+YotD/6TZ7Ojm/3W0e2ytb+hy90L8r9ed7b+nf2kT/C/YcTmq7oprttH96e1TNZNsPF/p/4C8d/JTUf2Xnd09v/ys/f2/p/QD4cBRXdsJz/82/fVqPb2zK9taFJLpgJosVH0queHNg179Q+0XJf+O12UwydQc64TSwvti3vtkvdP/UTmbmE0x2wR3Pmt54cKgfUlYT3TexOjvWB5JpqwxlR2t/3FX+8CuuP8df+NL3pf76//sJqV//T/7q2w89GIAPRbGzG579d37xtK4KfeHWrdd8VNiLvOos75Or+mjS09mo4fZY6iubx6e3P779nmx7orsv9UamY2vnb5Im1vRvT7elfnvUvFF8fU/H4aNRV+rSPoSu/U1QXtl2fSKSY+vb0ZtNn0xKVrwR9T78xl+nzwLnIb94IVz7j/6D5j90tA8kNp6qZ1FfaH8SqX87t7Gcje3yiR6g7Nok0LZ9EtBt7iDt6LZOT5tQnut2n8gvMt1e2ZcKS5sFirePjrS31v4Fn2Ufgoaw8oOAdKLnkkbvHfx65e8Luvf0b8dPNM/L7f/qbyw/LwAfinx3N1z/a02frX1capPrxWbzHn0+tvla7y8+Y229znt4Zr1zY6jj0p+8evP09r988Ruy7YVC5x83bfC3meq5jGz++HapvfNHsytSvzNvxrT/z51PyrZ393UueXykE//hUJthcWRfFLencbFl14hF8r63Q2g/xalP99qY983/8P3HtETxAAAAAAAAAACwRs70jf1usggvFs03znv2E2D/xv7cv/IUGST66fO79snFt6dPSv2b+/otx6/feUrq/Vv6KUux3xw/P1r+0wn/Zo//zsAeZutTlX37IP3eRnPAVy/p4+hf1G9ifeyyflr007s/kvpz/bekHto3+jdT/arpTtp8s2s71RMb2gP585vflTqLHhe/owPORzXLwvhm86ui1qe2bklGRL3iW6h1V3tCMdRv0Q8H2m92Btq/NjvN9o6daJ4uj4RwQ/va/Gahn/Bn9lWtvkVSxO5M9Xrw1vGu1P6p/GhPv/Gfjuybo3N9jtP5ozfLr37rZam7D9kPwONVxd9gWvENx3jc6N/KCTYG9HgH/0Z+b0P76qXNE6mf2DiQ+sle863853p7sm03P5a6ZxGZhX3b6ajqa13aN5KsQcXxkb1Mj3XjeEfq+yM9tn+jvxUrN1/RPKPnvPUN/bNdXgCclyTot/R9HOrfDo3qdGrfLJ/qvrmlvvj79cVw+Ri4OLDohU6zQzm0SAfrQWVHt3d72nvHU/3pp8f8Lub2C6f49sx/dbo8brL1PmFVDNqSXzx5r61skmS+Zdc3eVzMHgDnJn4tetaX9dky6j+dwcPfT4fQjurxuYEnt3XM+qXdN6X+wuANqXtpc3/vzC/Ktl8/1nnRO3N9v36y0HFl3+YOBlZfKQ6lfrq4f3r73336N2XbO9d0ruB395+X+nt3r0l9dKhj3nq+IkItSoTxeQX/xZf34dYvfx+Cb+wDAAAAAAAAALBGmNgHAAAAAAAAAGCNMLEPAAAAAAAAAMAaOVPGfhrqMIjyOrsWpTa1PKeDqsk5ul9pXtxbiwtS/6MDzVT6h+98XOrRG9tSD27qZxJX39PAzc5xE7SXTi1jL7PcIov9bFkVA5pbHl10/NmWPsXTC5oV9b2Lm1J/+6quHfCxZz4m9VcuaU7VT/R/LPVONjq9/XSm2au+knQII6nKJXGzAB6T2lZLtxdjarlscaRe1bNe19O+2NvR3PrLW9ojrg01i+5SV7OfPUe/iu68svDSyhqn/+3A1gcp0nLpdq83sybv/3Ku593b1LzA+UXtw69f05y8rx89I/Wr969KfeemXq/y+83xfM0Wl43sefF8bgDnbtXrWL4G4/HEKzL1OwPtXVsD7cP9XPtVZVnM+/NmDZA37Ps49/INqbuWqe99eGIN6LjUvNJZpb3yeNHkRO/PNE/Uz7Ms9dw8l7WeWG70zDJGF0sG277mwYoM6SVLzwB4nJJg/dNevJYnn46bnYsjy9j3KGh7nXsf7z5Y/n7flxiJj1f62DvX/9Dpaq/t5FoXXT2Z2ULvvCz0sS2i/jm352jRtRPf1z6ezux7mr4kif25r0WwjD+nVcfXLZDVAR79wAAem8TWK6nypikUQx2jejz/haGur/fyzl2pPzG8LbWv3fS3b/9Zqd86aLLs9490XDkfa29LRsunqutcG1RifTe3Ph2vhXJpQ+c4LnR1XvRCRx/3n7r2jtSv9y9LffdQx+OTAx1f1/E1xDLz09nyQWv1iDP2fGMfAAAAAAAAAIA1wsQ+AAAAAAAAAABrhIl9AAAAAAAAAADWyJkz9ntR2Fon0Twgz9h/t2yyOb850Qzj/+XmF6R+89XrUm+9roFw19/RQLjuvmaUJgu972zcZCgl8xVhcolnVlvOkWfy55YjmnmGclN3jvRxDO5a3um2bh/f1TymH97V5+3GiztS33tW85x+cthk7nc6+rifSjQrarAkzolPfIDzE2daJpWHB1sZ5V1WW5old/XavtQvbO9Jfb13IPVGPpXa85jHpdb78yYb79Dzly3kM7UT71mu9DCzDP1Ce/x2rv0rFq8tEkIIF1PNzbuc6+P8dEefhy8Ofij1b2/oGi+/3v+E1D+4deX0dn1LswQ9JzpdeI50APAREMcYe5+tLeNYUoQ9zthe47XVlWXPj2bLF9rIUs0ITaNz8b7qmfjeZ+d2sgsLWz6JxukhhHA81zHo8aypp6X+7eFIe990qo+r8mxUz8kvrVd65n687pMPSj3K2Y/NIBb46IhfzNZrvQ9kk6YudBmoVt57PrL3/jpsbGXozzd9TTz9+zJep8oy9ZNkeZ2lPsbVwd5OT8ewqf39tGz65Whua6FMtC9PC31/P+/o9
"text/plain": [
"<Figure size 2160x360 with 10 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bmppOi2NDOey",
"colab_type": "text"
},
"source": [
"\n",
"### Reduce image noise with PCA\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "oSbd6LrbDNjj",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "c9101411-6870-4b18-a05f-c16d40770250"
},
"source": [
"x_train_flat.shape, x_valid_flat.shape, x_test_flat.shape"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"((1024, 32607), (1024, 8724), (1024, 2182))"
]
},
"metadata": {
"tags": []
},
"execution_count": 37
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "V-msFuA7DWSH",
"colab_type": "code",
"colab": {}
},
"source": [
"\n",
"def PCA_iter(x_all,start, end, step):\n",
" lis =[]\n",
" for i in range(start, end, step):\n",
" print(\"\\n\\n===== Component: \",i,\"=====\\n\")\n",
" \n",
" (train,valid, test) = x_all\n",
" pca = PCA(n_components=i)\n",
" print(\"original shape: \", train.shape)\n",
"\n",
" \n",
" pca.fit_transform(train)\n",
" \n",
" train_PCA = pca.transform(train)\n",
" train_new = pca.inverse_transform(train_PCA)\n",
"\n",
" valid_PCA = pca.transform(valid) \n",
" valid_new = pca.inverse_transform(valid_PCA)\n",
" \n",
" \n",
" test_PCA = pca.transform(test)\n",
" test_new = pca.inverse_transform(test_PCA)\n",
"\n",
" \n",
" print(\"transformed shape:\", train_PCA.shape)\n",
" print(\"final shape:\", train_new.shape)\n",
"\n",
" tupla = (x_train_PCA, x_valid_PCA, x_test_PCA) =train_new,valid_new,test_new\n",
" \n",
" lis.append(tupla)\n",
" return lis"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "CHoTcijWDddc",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "a39f8b54-12c3-4e8a-eaee-e8c8ee2aa339"
},
"source": [
"lis_PCA = PCA_iter((x_train_flat_T,x_valid_flat_T, x_test_flat_T),10,211,20)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"\n",
"\n",
"===== Component: 10 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 10)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 30 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 30)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 50 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 50)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 70 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 70)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 90 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 90)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 110 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 110)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 130 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 130)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 150 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 150)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 170 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 170)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 190 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 190)\n",
"final shape: (32607, 1024)\n",
"\n",
"\n",
"===== Component: 210 =====\n",
"\n",
"original shape: (32607, 1024)\n",
"transformed shape: (32607, 210)\n",
"final shape: (32607, 1024)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "1W9YHNj7DpS_",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 284
},
"outputId": "4e0f78b6-47ca-4bda-acbd-8eca4b2289bc"
},
"source": [
"# 10 components example of the same image\n",
"tr,va,te = lis_PCA[1] \n",
"plt.imshow(tr[2,:].reshape(32,32), cmap=plt.get_cmap(\"gray\"))"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<matplotlib.image.AxesImage at 0x7f7296f20a20>"
]
},
"metadata": {
"tags": []
},
"execution_count": 40
},
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAWyklEQVR4nO2da4xd1XXH/8uDH4Nn/BgbzPgh7FDEQxExMDJUQRFNlIiiSDxUIfwB8QHFURWkIqUfEJUKlYpEqgLiQ0VlihWnUB4NQVgVakNRJBTxiMcufmFqjGVgzPhtj8dvPLP64R6rY+us/53Z995zHe//T7J8Z6+7z15nn7PuY//vWtvcHUKIi59J7XZACFENCnYhMkHBLkQmKNiFyAQFuxCZoGAXIhMuaaSzmd0B4DkAHQD+xd2fYs/v7Oz0GTNmlDtySezKpEnlr0nTpk0L+5w5cybJxqTIkZGRpo518uTJCY9Vj2geJ0+ePOE+9fpNmTJlwjY2VkdHR2hj83H69OnQNjo6Gtoi2D3Ajsf6mdmEbSl9Dhw4gOHh4VJjcrCbWQeAfwLwQwADANaa2Rp3/yTqM2PGDCxfvrzU1tPTE47V3d1d2n7NNdeEffbv3x/aDh48GNpOnDgR2o4cOVLafuDAgbDPvn37Qttnn30W2oaGhkIbu+Euv/zy0vbe3t6wz5w5c0Lb/Pnzk2yLFy8ubZ89e3bYh9nYfAwMDIS248ePl7azOWQvLNHxAP7izV4Yp06dOuE+UbA/+eSTYZ9GPsYvA7Dd3Xe4+2kArwK4q4HjCSFaSCPBvgDAV2P+HijahBAXIC1foDOzFWbWb2b97COyEKK1NBLsuwAsGvP3wqLtHNx9pbv3uXtfZ2dnA8MJIRqhkWBfC+BqM1tiZlMA3A9gTXPcEkI0m+TVeHc/Y2YPA/gv1KS3Ve6+hfXp6OgIV9aZzPDNN9+Utp86dSrsc+mll4Y2tqK6bdu20PbJJ+VCw/bt28M+zEcmNV122WWhbcmSJaEtkjZnzZoV9omuCcBXhI8dOxbaPv/889L26dOnh32YYnDFFVeEthtuuCG0RV8dmYLClBw2H0xmZav/kbTMpLzovmLjNKSzu/vbAN5u5BhCiGrQL+iEyAQFuxCZoGAXIhMU7EJkgoJdiExoaDU+hUhiY9lQkdzBEg9Y4sT69etD26ZNm0Lb4OBgaTvLumIJPiyRhElvLHGlq6urtJ1JkUwCZFlvTC6NZKjh4eGwD5ONjh49GtrYfETzz6Q8ds7svmL3AbNFEhtLyInmPpLxAL2zC5ENCnYhMkHBLkQmKNiFyAQFuxCZUOlqvJmFK+usnly0Us8SWrZu3Rravvrqq9AWlZ4CYh8XLlwY9okSUwCeFMISV6IVdyBedWcrzKn16ejKb2BjK8wsaYiVEmOqTGRj88tSsdk5M6WB3d9Roher/5CybZve2YXIBAW7EJmgYBciExTsQmSCgl2ITFCwC5EJlUpvkyZNCqUhlqgRyTVsJ5Cvv/46tDGJZO7cuaEtksqiHT0ALuMwOYb1Y7ZmS2+pSTIRKZIRwOu7sVp4UQIKk+vYzjSsBl3KPQzE88iSZ5hMGaF3diEyQcEuRCYo2IXIBAW7EJmgYBciExTsQmRCQ9Kbme0EMAxgBMAZd++r8/ykrLdIZmD1wFj2GpNPmB+R9MYkKHa8VBvLeotkOeYjy+RKld6iY7I6c1H2F8Alu5RjMvmVnRebezZX7NyifizjMDoem6dm6Ox/5u7x5lhCiAsCfYwXIhMaDXYH8FszW2dmK5rhkBCiNTT6Mf42d99lZpcDeMfMPnX398Y+oXgRWAHwnyEKIVpLQ+/s7r6r+H8vgDcBLCt5zkp373P3Pra4IYRoLcnBbmbTzaz77GMAPwKwuVmOCSGaSyMf4+cBeLPYhuYSAP/m7v/JOrCsN/auH8kkhw8fDvswGYRJbyyjLMpuY2OxjDgmr7EMKjZX0bkxGYf5n9ovsrHsL5bZxraaYn5E47Fijuy+YvPB7h3WL8puYxIgkxtDHybco8DddwD4Tmp/IUS1SHoTIhMU7EJkgoJdiExQsAuRCQp2ITLhgik4yeSTQ4cOlbazooGpEgmTwyIZjclrbD83JqGxfkyWi3xh88FsTPJKsbHrzLLvmGTHbFEWGJOuWDFHlk2ZOscR7LxS0Du7EJmgYBciExTsQmSCgl2ITFCwC5EJla7Gd3R0YObMmaW2ffv2hf1Y0kJESrIIkLYaz1bHUxNaUreGilZ92Uo3s7EV9xTYWKlJNyn16ZiCwnxkteSYjRFda3bfR33Y9dI7uxCZoGAXIhMU7EJkgoJdiExQsAuRCQp2ITKh8kSYSPZiSS1RQsCMGTMm3AfgsguT5SJpqBVbPLFzY/6nJFwwWMJIiuRFpaEKJUBW747NIZsPJpUx6TDyhd2LUX06SW9CCAW7ELmgYBciExTsQmSCgl2ITFCwC5EJdXUaM1sF4McA9rr7t4u2HgCvAVgMYCeA+9y9vFDcGNw9lBmYVBbJDCyjjB0vdbujyJZ6PCahpdoi+YpJMik13FL7sT6p0ltKPyZrsfll0huT89h5R/cIq0MY1cmj2Y2h5f/5JYA7zmt7FMC77n41gHeLv4UQFzB1g73Yb/3gec13AVhdPF4N4O4m+yWEaDKp39nnuftg8Xg3aju6CiEuYBpeoPPal7Pwi52ZrTCzfjPrHxoaanQ4IUQiqcG+x8x6AaD4f2/0RHdf6e597t4XlaQSQrSe1GBfA+DB4vGDAN5qjjtCiFYxHuntFQC3A5hrZgMAHgfwFIDXzewhAF8AuG88g42OjiYVj4wkNlbgj0kQTCpjtkiuYfJaJBvW61dl9hojVfJKGY+NxaQyJg9G1zN17pkfTF5LmQ+WFcnu04i6d5S7Lw9MP5jwaEKItqFf0AmRCQp2ITJBwS5EJijYhcgEBbsQmVBpwUl3x+nTp0ttLMPn+PHjpe2pElrqnmKRjMaOx+ST1Ey0lGwzJv2kFpVkWV4p2Y3Mj5RCoEAsD7LrzK5LqiyXcq+ymOjp6Sltp3MRWoQQFxUKdiEyQcEuRCYo2IXIBAW7EJmgYBciEyqV3syMyhOsXxksoywlE6qeLYL5waQmlrWXmtF3oez1FpGajcjmOGVPNOYHOy9mS537qMBlyl6AbA71zi5EJijYhcgEBbsQmaBgFyITFOxCZEKlq/GTJk0K68mlrDCnrqimrLgDfFugCLaqzmA+shXylC2qUrd4YrYIlmRCV5ITFYjOzs7S9hRVCIi3XQK4jynbPzEForu7e8I+6J1diExQsAuRCQp2ITJBwS5EJijYhcgEBbsQmTCe7Z9WAfgxgL3u/u2i7QkAPwGwr3jaY+7+dr1jMemNOpkgTbD6aEziSa0ZF5EiTwHpWytF46UmaaTKctEcs/Ni85u6DVVkY9c55XgAwvqK9fpF14bFSjT3jUpvvwRwR0n7s+6+tPhXN9CFEO2lbrC7+3sADlbgixCihTTynf1hM9toZqvMbHbTPBJCtITUYH8ewFUAlgIYBPB09EQzW2Fm/WbWf/jw4cThhBCNkhTs7r7H3UfcfRTACwCWkeeudPc+d++bNWtWqp9CiAZJCnYz6x3z5z0ANjfHHSFEqxiP9PYKgNsBzDWzAQCPA7jdzJYCcAA7Afx0PIOxGnQpNddSt+lh2WupGXERqTXXmOSVspUTkyJTasnV65dyTHY9UyVANlcRTJZr9nUB4vsxytgD0qS3usHu7stLml+s108IcWGhX9AJkQkKdiEyQcEuRCYo2IXIBAW7EJlQ+fZPkdzEpLKoT0oWWr2xGJHskipdpWa2MaJ+zd7GqR4p1yZ1PlIkQNaH+d7V1RXaUn2MpDcmEUdSNc0ODC1CiIsKBbsQmaBgFyITFOxCZIKCXYhMULALkQmVSm/uHsoTTDKICksyqYbRbImH9WHnlVqMko2XkiGYmm3GSPGDZQGm7gMX2djxWCFTtkcc2weOXbOosCTLvmNZjBF6ZxciExTsQmSCgl2ITFCwC5EJCnYhMqHy1fgTJ06UO0KSUyIb65OaSJKyM
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "v1zpZzo_DgPr",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 284
},
"outputId": "23384fb3-d61f-4670-f741-39e6717355c8"
},
"source": [
"# 210 components example of an image\n",
"tr,va,te = lis_PCA[len(lis_PCA)-1]\n",
"plt.imshow(tr[2,:].reshape(32,32), cmap=plt.get_cmap(\"gray\"))"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<matplotlib.image.AxesImage at 0x7f7296e8c1d0>"
]
},
"metadata": {
"tags": []
},
"execution_count": 41
},
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD5CAYAAADhukOtAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAXXElEQVR4nO2dXYhdVZbH/6vyWUlVJalUPsokbWIMiMh0lCI4tDRON904IqgwSHwQH6TTDC2M2PMgDowK82APo+LD4BDH0OnBMTrtV4ihbSe0aCvYKR2NiZmZjiGiManKRyVV+TCaypqHe8JU5Kx/Ve1777np7P8PQm7tdfc56+571v3Y/7vWMneHEOLSp63VDgghqkHBLkQmKNiFyAQFuxCZoGAXIhMU7EJkwtR6JpvZTQCeBDAFwL+6+6Ps/p2dnd7T01Nqa2uLX3ciedDMJupqQ+alHO/EiROh7eTJk0nnmzFjRmibPn166fjUqfFTzfyfNm1aaGPHnDJlSun4uXPnwjmjo6OhjUnEKfIxe8zMRzYv9ZgR0XMJAF9//XXp+ODgIIaHh0sdSQ52M5sC4J8B/AjAFwC2m9lmd/8kmtPT04NHHnmk1MYu4G+++WbSc9iLR2qwRxc3O9e7774b2rZv357kx8qVK0Pbd77zndLx7u7ucA67qBYsWBDaohduAJg7d27pOHuBY7boGgCAr776KrRFLzrsOUs5HsBfGM+cORPaoherpUuXhnP2799fOn7//feHc+r5GL8GwB533+vuXwPYBODWOo4nhGgi9QT7EgCfj/n7i2JMCHER0vQNOjNbZ2b9ZtY/MjLS7NMJIQLqCfb9AJaN+XtpMXYB7r7e3fvcva+zs7OO0wkh6qGeYN8OYJWZrTCz6QDWAtjcGLeEEI0meTfe3c+a2b0AXkdNetvg7rvYnLa2NsyePbvUdvr06dhJIvEQ/0LbnDlzQtvMmTND25YtW0rHP/kkFCDQ3t4e2lasWBHaOjo6QtusWbNCWyRfDQ8Ph3PYzvSRI0dCW/RcAvFOfW9vbzhn4cKFoY09n2z3PJI+2c45kwCZYpAqb0ZqyNGjRyd9PHaeunR2d98KYGs9xxBCVIN+QSdEJijYhcgEBbsQmaBgFyITFOxCZEJdu/GNhMkWkTTBMomYhPbpp5+Gttdeey20RZIXS1hgMhlLqmD+M8krWkeW7MLWMVXyGhgYKB1nWYCnTp0KbUyyS0nIGRoaCuewtWLJVyzZhUl9UQZbSqYfm6N3diEyQcEuRCYo2IXIBAW7EJmgYBciEyrdjW9rawsTQ1gyRmRju9l79uwJbW+++WZoi8o6AXGiBlMSWDkl9pjZjjt73NEx2blSEo0A4OzZs6Et2hVmO/iDg4OhjSVKsQSayHbZZZeFc1ii1MGDB0Mbe15YLYeU5yxaD+3GCyEU7ELkgoJdiExQsAuRCQp2ITJBwS5EJlQqvbl7KNcw+SeawxI4du/eHdr6+vpCG0tciWQ0Jq+ltGoC+HowSSY6HzseS9JgNiY1Rc8Zk+uYjSXQsJpxkUSVkjwDAJdffnloY7Ickw6j55PJlJE0S2XZ0CKEuKRQsAuRCQp2ITJBwS5EJijYhcgEBbsQmVCX9GZm+wCMABgFcNbdY02rdv+wPQ3L1pk/f37p+NatcTMaVheONZiM6oEBsdTHpDAmr7FWPWw92DEjW6rMx+YxWS56bEyCYi2q2HowCTY6JjseqyXHpNnu7u6kY0bXHFurqF4fW4tG6Ox/4e6HG3AcIUQT0cd4ITKh3mB3AL81s/fNbF0jHBJCNId6P8bf4O77zWwhgDfM7L/d/a2xdyheBNYBwIIFC+o8nRAilbre2d19f/H/IICXAawpuc96d+9z9z5W7kcI0VySg93MZptZ5/nbAH4MYGejHBNCNJZ6PsYvAvByIbFMBfDv7v6b8SZFkgyTw/bt21c6zjLKUj9FpBRmZHOYPMVkEjaPtYaK1oQdL0XKA9IKX7I5TA5LkUSZ7ciRI+Eclm3W0dER2tg1zLLsDh8uF7PY8SJZjsm5ycHu7nsBfDd1vhCiWiS9CZEJCnYhMkHBLkQmKNiFyAQFuxCZUGnBSSCWZFIyl1LlpBSpBoh9Z36wIoos24zJa0z+iWxM8kqR0IC0ophMLmWFI5mNyU3RPCbznTx5MrSl9LcDuHQYZXV+/vnn4Zwoi45ev6FFCHFJoWAXIhMU7EJkgoJdiExQsAuRCZXuxptZuHPN2vtE7ZVYPTC268vqgbEd5mjXOnWnmO24R+19AGDhwoVJ8yLYDjN7bKmqRkR7e3toYy22mP+RKsBUBnYuBquhx+rJRdcxa0MVXVdq/ySEULALkQsKdiEyQcEuRCYo2IXIBAW7EJlQeSIMk3IiIimkt7c3nMOSEppRMy6CSSFMemNltxcvXjzpY7J1j1oJATwphNVqi+Qw9rwwH5mUmipvRqQmUaUm8gwODpaOr1ixIpxz7Nix0Bahd3YhMkHBLkQmKNiFyAQFuxCZoGAXIhMU7EJkwrjSm5ltAHALgEF3v6YY6wbwPIDlAPYBuMPdhyZwrFDWYBlDUVYQk2NGRkZCG6sHxqQyll0VwXxk7X26urpCW3d3d2iLst5YJhd7zCxDkMlJkSyXWsON2Zj0Fs1jklxqRhxbK5b1FsmbTNqMJGK2FhN5Z/8lgJu+NfYAgG3uvgrAtuJvIcRFzLjBXvRbP/qt4VsBbCxubwRwW4P9EkI0mNTv7Ivc/UBx+yBqHV2FEBcxdW/Qee1LUfiFyszWmVm/mfWn/MRPCNEYUoN9wMx6AaD4v/zHvQDcfb2797l7HyuzI4RoLqnBvhnA3cXtuwG82hh3hBDNYiLS23MAbgTQY2ZfAHgIwKMAXjCzewB8BuCOiZysra0tlJsOHToUzos+EaS2XWJSU0phQ1akkklvLLuK2VLaLrG1YjYm5TAJM5KN2Bx2LuYjI3rOUq8PJtkx/5n0Fp2PSW+RNMt8HzfY3f3OwPTD8eYKIS4e9As6ITJBwS5EJijYhcgEBbsQmaBgFyITKi04OXXqVMybN6/UliJRsawrJkEwWEHBSNZikguzMZmP9b4bGooTDCOJh52L/bKRnYtlFkZFLNn6srViMDksug5SMhgBnn3H/GB9CaM1Ydd3FEcsY0/v7EJkgoJdiExQsAuRCQp2ITJBwS5EJijYhciESqU3MwvliZRMNCbjpEpvLNsskoYanZEF8IwnVpwzkt5Yttnx48dDG5PXWIHFRvfFY5l+7e3toS263lL7uTEf2XXA1iqysXOlyMB6ZxciExTsQmSCgl2ITFCwC5EJCnYhMqHy3Xj2Q/3JwnZN2c4oS2Zgu7Qpu+7s8bLd+CiRBACOHDky6fOx3WC288/8YIka0a4w2y1ma88SpVgCSjQvapMFcB9Ta9cx/6P1ZwpEZNNuvBBCwS5ELijYhcgEBbsQmaBgFyITFOxCZMJE2j9tAHALgEF3v6YYexjATwCc79n0oLtvrccRJodFEluqjMfmMfkn8pHJHSmtmgAu8bDklChpiMlrLEmGkZKcktryislazBYdk0lh7HhMwmTXcIpsy9Yj5dqfyIxfAripZPwJd19d/Ksr0IUQzWfcYHf3twAcrcAXIUQTqec7+71mtsPMNphZeV1bIcRFQ2qwPwVgJYDVAA4AeCy6o5mtM7N+M+tnP/MUQjSXpGB39wF3H3X3cwCeBrCG3He9u/e5e9/8+fNT/RRC1ElSsJtZ75g/bwewszHuCCGaxUSkt+cA3Aigx8y+APAQgBvNbDUAB7APwE8ncjJ3D2U0JjNE0hCTflhGVqpkF/meUiusHhvL9osed0pbK4DLiky+imypWYWpz1n03LBrh2XRMXmNtdhK8Z+tb8rxxg12d7+zZPiZSZ9JCNFS9As6ITJBwS5EJijYhcgEBbsQmaBgFyITKi042dbWlpQNFckdTD5JlUFYtlk0L6XwIsBlHFo4MCFrj81JlSmZRBXZmLyWmtnGjhnZUjPK2LnYtZPyfM6aNSuck5KBqXd2ITJBwS5EJijYhcgEBbsQmaBgFyITFOxCZ
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6-rpOqx0Du8Y",
"colab_type": "text"
},
"source": [
"\n",
"## Train feed-forward NN with reduced images\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D1mWo4zKDulG",
"colab_type": "code",
"colab": {}
},
"source": [
"def FW_iter(lis_PCA, ep, bs):\n",
" lis_FW = []\n",
" epochs = ep\n",
" batch_size = bs\n",
" for itr in range(len(lis_PCA)):\n",
" x_train_PCA, x_valid_PCA, x_test_PCA = lis_PCA[itr]\n",
" \n",
" \n",
" print(\"FW- components: \",(itr+1)*20-10)\n",
" #feed forward neural network\n",
" model = tf.keras.Sequential([\n",
" tf.keras.layers.Input(shape = (1024)),\n",
" tf.keras.layers.Dense(32, activation = \"relu\"),\n",
" tf.keras.layers.Dense(10, activation='softmax')\n",
" ])\n",
" model.compile(optimizer = \"adam\", loss='categorical_crossentropy', metrics=['accuracy'])\n",
" \n",
" \n",
" history = model.fit(x_train_PCA, y_train,\n",
" batch_size = bs,\n",
" epochs = epochs,\n",
" validation_data=(x_valid_PCA, y_valid),\n",
" verbose = 2\n",
"\n",
" )\n",
" \n",
" y_pred = model.predict(x_test_PCA).round()\n",
" \n",
" zo_loss = zero_one(y_pred,y_test)\n",
" print(\"Zero-one loss: \",zo_loss)\n",
" tupla = (history, model, zo_loss)\n",
" lis_FW.append(tupla)\n",
" return lis_FW"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "sPVnWMi8Du2S",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "8ab75f95-e34a-4179-bd88-7673124174c1"
},
"source": [
"epochs = 10\n",
"batch_size = 32\n",
"res = FW_iter(lis_PCA, epochs, batch_size)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"FW- components: 10\n",
"Epoch 1/10\n",
"WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.175393). Check your callbacks.\n",
"1019/1019 - 5s - loss: 0.8917 - accuracy: 0.6877 - val_loss: 1.3535 - val_accuracy: 0.5962\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.6398 - accuracy: 0.7802 - val_loss: 1.3279 - val_accuracy: 0.6405\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.5631 - accuracy: 0.8033 - val_loss: 1.2432 - val_accuracy: 0.6723\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.5202 - accuracy: 0.8203 - val_loss: 1.2590 - val_accuracy: 0.6750\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.4846 - accuracy: 0.8326 - val_loss: 1.3413 - val_accuracy: 0.6714\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.4573 - accuracy: 0.8445 - val_loss: 1.2513 - val_accuracy: 0.6985\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.4322 - accuracy: 0.8521 - val_loss: 1.2741 - val_accuracy: 0.6983\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.4193 - accuracy: 0.8566 - val_loss: 1.3041 - val_accuracy: 0.7063\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.4008 - accuracy: 0.8640 - val_loss: 1.3032 - val_accuracy: 0.7003\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.3982 - accuracy: 0.8641 - val_loss: 1.3579 - val_accuracy: 0.6978\n",
"Zero-one loss: 0.28689275893675525\n",
"FW- components: 30\n",
"Epoch 1/10\n",
"WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.261349). Check your callbacks.\n",
"1019/1019 - 6s - loss: 0.6436 - accuracy: 0.7845 - val_loss: 1.0769 - val_accuracy: 0.6961\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.3235 - accuracy: 0.8948 - val_loss: 1.0283 - val_accuracy: 0.7299\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.2424 - accuracy: 0.9211 - val_loss: 1.0095 - val_accuracy: 0.7387\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.1909 - accuracy: 0.9384 - val_loss: 0.9939 - val_accuracy: 0.7633\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.1558 - accuracy: 0.9502 - val_loss: 1.1439 - val_accuracy: 0.7466\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.1404 - accuracy: 0.9538 - val_loss: 1.0946 - val_accuracy: 0.7713\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.1227 - accuracy: 0.9603 - val_loss: 1.1294 - val_accuracy: 0.7809\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.1098 - accuracy: 0.9649 - val_loss: 1.1915 - val_accuracy: 0.7785\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.1003 - accuracy: 0.9675 - val_loss: 1.1726 - val_accuracy: 0.7894\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.0947 - accuracy: 0.9702 - val_loss: 1.2234 - val_accuracy: 0.7822\n",
"Zero-one loss: 0.19798350137488543\n",
"FW- components: 50\n",
"Epoch 1/10\n",
"1019/1019 - 5s - loss: 0.5459 - accuracy: 0.8254 - val_loss: 0.8101 - val_accuracy: 0.7436\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.2140 - accuracy: 0.9335 - val_loss: 0.8365 - val_accuracy: 0.7656\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.1433 - accuracy: 0.9549 - val_loss: 0.8540 - val_accuracy: 0.7918\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.1039 - accuracy: 0.9682 - val_loss: 0.8805 - val_accuracy: 0.7915\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0848 - accuracy: 0.9738 - val_loss: 0.9716 - val_accuracy: 0.8034\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0715 - accuracy: 0.9778 - val_loss: 0.9149 - val_accuracy: 0.8155\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0664 - accuracy: 0.9790 - val_loss: 1.1081 - val_accuracy: 0.8093\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0539 - accuracy: 0.9832 - val_loss: 1.1118 - val_accuracy: 0.7929\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.0512 - accuracy: 0.9835 - val_loss: 1.0935 - val_accuracy: 0.8323\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.0465 - accuracy: 0.9845 - val_loss: 1.0836 - val_accuracy: 0.8343\n",
"Zero-one loss: 0.15902841429880843\n",
"FW- components: 70\n",
"Epoch 1/10\n",
"WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.200094). Check your callbacks.\n",
"1019/1019 - 5s - loss: 0.4945 - accuracy: 0.8395 - val_loss: 0.8700 - val_accuracy: 0.7372\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.1802 - accuracy: 0.9462 - val_loss: 0.7953 - val_accuracy: 0.7839\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.1150 - accuracy: 0.9651 - val_loss: 0.8178 - val_accuracy: 0.8015\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.0828 - accuracy: 0.9752 - val_loss: 0.8297 - val_accuracy: 0.7953\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0659 - accuracy: 0.9796 - val_loss: 0.7400 - val_accuracy: 0.8329\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0544 - accuracy: 0.9829 - val_loss: 0.8350 - val_accuracy: 0.8300\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0438 - accuracy: 0.9867 - val_loss: 0.8506 - val_accuracy: 0.8344\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0406 - accuracy: 0.9879 - val_loss: 0.8914 - val_accuracy: 0.8357\n",
"Epoch 9/10\n",
"1019/1019 - 6s - loss: 0.0387 - accuracy: 0.9884 - val_loss: 0.8949 - val_accuracy: 0.8410\n",
"Epoch 10/10\n",
"1019/1019 - 6s - loss: 0.0312 - accuracy: 0.9904 - val_loss: 0.9068 - val_accuracy: 0.8454\n",
"Zero-one loss: 0.1457378551787351\n",
"FW- components: 90\n",
"Epoch 1/10\n",
"1019/1019 - 5s - loss: 0.5080 - accuracy: 0.8416 - val_loss: 0.7913 - val_accuracy: 0.7591\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.1627 - accuracy: 0.9520 - val_loss: 0.7948 - val_accuracy: 0.7869\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.0948 - accuracy: 0.9742 - val_loss: 0.7662 - val_accuracy: 0.8137\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.0640 - accuracy: 0.9823 - val_loss: 0.7883 - val_accuracy: 0.8290\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0502 - accuracy: 0.9860 - val_loss: 0.7893 - val_accuracy: 0.8356\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0395 - accuracy: 0.9891 - val_loss: 0.8706 - val_accuracy: 0.8399\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0400 - accuracy: 0.9881 - val_loss: 1.0676 - val_accuracy: 0.8155\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0282 - accuracy: 0.9915 - val_loss: 1.0146 - val_accuracy: 0.8299\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.0238 - accuracy: 0.9928 - val_loss: 0.9587 - val_accuracy: 0.8548\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.0224 - accuracy: 0.9929 - val_loss: 1.0540 - val_accuracy: 0.8471\n",
"Zero-one loss: 0.15627864344637946\n",
"FW- components: 110\n",
"Epoch 1/10\n",
"1019/1019 - 5s - loss: 0.4724 - accuracy: 0.8546 - val_loss: 0.8186 - val_accuracy: 0.7603\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.1382 - accuracy: 0.9634 - val_loss: 0.7725 - val_accuracy: 0.8089\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.0794 - accuracy: 0.9783 - val_loss: 0.8203 - val_accuracy: 0.8191\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.0528 - accuracy: 0.9850 - val_loss: 0.9337 - val_accuracy: 0.8204\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0417 - accuracy: 0.9879 - val_loss: 0.9542 - val_accuracy: 0.8313\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0369 - accuracy: 0.9895 - val_loss: 0.9346 - val_accuracy: 0.8412\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0267 - accuracy: 0.9927 - val_loss: 1.0282 - val_accuracy: 0.8474\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0211 - accuracy: 0.9939 - val_loss: 1.1628 - val_accuracy: 0.8286\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.0264 - accuracy: 0.9914 - val_loss: 1.1593 - val_accuracy: 0.8408\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.0205 - accuracy: 0.9938 - val_loss: 1.2966 - val_accuracy: 0.8436\n",
"Zero-one loss: 0.14848762603116408\n",
"FW- components: 130\n",
"Epoch 1/10\n",
"WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.148986). Check your callbacks.\n",
"1019/1019 - 5s - loss: 0.5150 - accuracy: 0.8395 - val_loss: 0.7711 - val_accuracy: 0.7811\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.1556 - accuracy: 0.9550 - val_loss: 0.7043 - val_accuracy: 0.8064\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.0860 - accuracy: 0.9749 - val_loss: 0.7319 - val_accuracy: 0.8285\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.0547 - accuracy: 0.9850 - val_loss: 0.8580 - val_accuracy: 0.8292\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0448 - accuracy: 0.9874 - val_loss: 0.9114 - val_accuracy: 0.8353\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0326 - accuracy: 0.9913 - val_loss: 0.9489 - val_accuracy: 0.8368\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0302 - accuracy: 0.9910 - val_loss: 0.8936 - val_accuracy: 0.8608\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0265 - accuracy: 0.9921 - val_loss: 0.9696 - val_accuracy: 0.8584\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.0264 - accuracy: 0.9920 - val_loss: 1.0818 - val_accuracy: 0.8422\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.0166 - accuracy: 0.9955 - val_loss: 1.1559 - val_accuracy: 0.8453\n",
"Zero-one loss: 0.152153987167736\n",
"FW- components: 150\n",
"Epoch 1/10\n",
"1019/1019 - 5s - loss: 0.4667 - accuracy: 0.8549 - val_loss: 0.7300 - val_accuracy: 0.7691\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.1365 - accuracy: 0.9635 - val_loss: 0.6678 - val_accuracy: 0.8116\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.0762 - accuracy: 0.9800 - val_loss: 0.6308 - val_accuracy: 0.8370\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.0458 - accuracy: 0.9884 - val_loss: 0.6961 - val_accuracy: 0.8396\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0426 - accuracy: 0.9883 - val_loss: 0.8004 - val_accuracy: 0.8357\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0267 - accuracy: 0.9929 - val_loss: 0.7318 - val_accuracy: 0.8495\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0267 - accuracy: 0.9921 - val_loss: 0.7954 - val_accuracy: 0.8433\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0232 - accuracy: 0.9931 - val_loss: 1.0402 - val_accuracy: 0.8232\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.0190 - accuracy: 0.9945 - val_loss: 0.9076 - val_accuracy: 0.8549\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.0184 - accuracy: 0.9949 - val_loss: 0.9122 - val_accuracy: 0.8505\n",
"Zero-one loss: 0.14115490375802017\n",
"FW- components: 170\n",
"Epoch 1/10\n",
"1019/1019 - 5s - loss: 0.5013 - accuracy: 0.8487 - val_loss: 0.7488 - val_accuracy: 0.7780\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.1291 - accuracy: 0.9651 - val_loss: 0.7400 - val_accuracy: 0.8058\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.0664 - accuracy: 0.9830 - val_loss: 0.6988 - val_accuracy: 0.8416\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.0447 - accuracy: 0.9883 - val_loss: 0.7572 - val_accuracy: 0.8386\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0339 - accuracy: 0.9907 - val_loss: 0.8077 - val_accuracy: 0.8454\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0254 - accuracy: 0.9930 - val_loss: 0.7187 - val_accuracy: 0.8663\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0213 - accuracy: 0.9939 - val_loss: 0.7830 - val_accuracy: 0.8504\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0189 - accuracy: 0.9946 - val_loss: 0.7266 - val_accuracy: 0.8778\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.0124 - accuracy: 0.9968 - val_loss: 0.8611 - val_accuracy: 0.8610\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.0178 - accuracy: 0.9946 - val_loss: 0.8310 - val_accuracy: 0.8737\n",
"Zero-one loss: 0.12465627864344637\n",
"FW- components: 190\n",
"Epoch 1/10\n",
"1019/1019 - 5s - loss: 0.4545 - accuracy: 0.8607 - val_loss: 0.7529 - val_accuracy: 0.7865\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.1264 - accuracy: 0.9652 - val_loss: 0.7090 - val_accuracy: 0.8212\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.0705 - accuracy: 0.9815 - val_loss: 0.6724 - val_accuracy: 0.8435\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.0461 - accuracy: 0.9876 - val_loss: 0.7778 - val_accuracy: 0.8344\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0383 - accuracy: 0.9895 - val_loss: 0.7378 - val_accuracy: 0.8549\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0264 - accuracy: 0.9932 - val_loss: 0.9976 - val_accuracy: 0.8267\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0244 - accuracy: 0.9929 - val_loss: 0.9187 - val_accuracy: 0.8483\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0181 - accuracy: 0.9950 - val_loss: 0.9842 - val_accuracy: 0.8478\n",
"Epoch 9/10\n",
"1019/1019 - 6s - loss: 0.0234 - accuracy: 0.9936 - val_loss: 0.9568 - val_accuracy: 0.8530\n",
"Epoch 10/10\n",
"1019/1019 - 6s - loss: 0.0140 - accuracy: 0.9960 - val_loss: 0.9279 - val_accuracy: 0.8657\n",
"Zero-one loss: 0.12557286892758937\n",
"FW- components: 210\n",
"Epoch 1/10\n",
"1019/1019 - 5s - loss: 0.4542 - accuracy: 0.8597 - val_loss: 0.7646 - val_accuracy: 0.7775\n",
"Epoch 2/10\n",
"1019/1019 - 5s - loss: 0.1189 - accuracy: 0.9685 - val_loss: 0.7023 - val_accuracy: 0.8039\n",
"Epoch 3/10\n",
"1019/1019 - 5s - loss: 0.0641 - accuracy: 0.9839 - val_loss: 0.7556 - val_accuracy: 0.8193\n",
"Epoch 4/10\n",
"1019/1019 - 5s - loss: 0.0412 - accuracy: 0.9896 - val_loss: 0.8243 - val_accuracy: 0.8222\n",
"Epoch 5/10\n",
"1019/1019 - 5s - loss: 0.0262 - accuracy: 0.9936 - val_loss: 0.7042 - val_accuracy: 0.8571\n",
"Epoch 6/10\n",
"1019/1019 - 5s - loss: 0.0262 - accuracy: 0.9922 - val_loss: 0.7838 - val_accuracy: 0.8453\n",
"Epoch 7/10\n",
"1019/1019 - 5s - loss: 0.0157 - accuracy: 0.9956 - val_loss: 0.8158 - val_accuracy: 0.8592\n",
"Epoch 8/10\n",
"1019/1019 - 5s - loss: 0.0245 - accuracy: 0.9926 - val_loss: 0.9040 - val_accuracy: 0.8639\n",
"Epoch 9/10\n",
"1019/1019 - 5s - loss: 0.0147 - accuracy: 0.9961 - val_loss: 0.9469 - val_accuracy: 0.8372\n",
"Epoch 10/10\n",
"1019/1019 - 5s - loss: 0.0174 - accuracy: 0.9947 - val_loss: 1.0241 - val_accuracy: 0.8486\n",
"Zero-one loss: 0.1457378551787351\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "kZIGTbP0EUl-",
"colab_type": "code",
"colab": {}
},
"source": [
"df = pd.DataFrame(\n",
"columns = ['epochs', 'valid', 'components', 'accuracy', 'value']\n",
")\n",
"\n",
"for itr in range(len(res)):\n",
" \n",
" time = [i for i in range(1,epochs+1)]\n",
" valids = [0 for i in range(1,epochs+1)]\n",
" components = [(itr+1)*20-10 for i in range(1,epochs+1)]\n",
" \n",
"\n",
" accur = [1 for i in range(1,epochs+1)]\n",
" acc = res[itr][0].history['accuracy']\n",
" \n",
" df1= pd.DataFrame(data= np.vstack((time,valids,components,accur,acc)).T, columns = ['epochs', 'valid', 'components', 'accuracy', 'value'])\n",
"\n",
" loss= res[itr][0].history['loss']\n",
" accur = [0 for i in range(1,epochs+1)]\n",
" df2= pd.DataFrame(data= np.vstack((time,valids,components,accur,loss)).T, columns = ['epochs', 'valid', 'components', 'accuracy', 'value'])\n",
"\n",
" valids = [1 for i in range(1,epochs+1)]\n",
" accur = [1 for i in range(1,epochs+1)]\n",
" val_acc = res[itr][0].history['val_accuracy']\n",
" \n",
" df3= pd.DataFrame(data= np.vstack((time,valids,components,accur,val_acc)).T, columns = ['epochs', 'valid', 'components', 'accuracy', 'value'])\n",
"\n",
" accur = [0 for i in range(1,epochs+1)]\n",
"\n",
" val_loss = res[itr][0].history['val_loss']\n",
" df4= pd.DataFrame(data= np.vstack((time,valids,components,accur,val_loss)).T, columns = ['epochs', 'valid', 'components', 'accuracy', 'value'])\n",
" \n",
" df = df.append(df1.append(df2).append(df3).append(df4))"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ekgc7EAAEUwF",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "14e77fc7-eca6-48b8-8e6a-71ca0e4d1728"
},
"source": [
"df['components'] = df['components'].astype('category')\n",
"df = df.assign(accuracy = ['accuracy' if accuracy == 1. else 'loss' for accuracy in df['accuracy']])\n",
"df = df.assign(valid = ['validation' if valid == 1. else 'training' for valid in df['valid']])\n",
"df['accuracy'].unique()"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array(['accuracy', 'loss'], dtype=object)"
]
},
"metadata": {
"tags": []
},
"execution_count": 45
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "FSQbrnQMEUy0",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 504
},
"outputId": "0a9a0092-d80b-4d54-e785-aa24a661d724"
},
"source": [
"ggplot(df, aes(x='epochs', y='value',color='components')) + \\\n",
" geom_line() + \\\n",
" facet_wrap(['accuracy','valid'],scales='free') + theme_bw(base_size=12)"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAHVCAYAAADl4K3UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd7gc1WH///ecmdm+e3tXu6DeEB0VhEQHmWYRxXEwdhIZ2ziuIcHG+ZJgkvixHSfGjgv8sE0I3WCaASEsEBJImKKCCur99rZ3++6U8/tj967uRQU1dIV0XnrmmbKzM2dWO3c/e+bMWU1KKVEURVEURVEUZb/EYBdAURRFURRFUU5kKjAriqIoiqIoykGowKwoiqIoiqIoB6ECs6IoiqIoiqIchArMiqIoiqIoinIQKjAriqIoiqIoykGowKwoiqIoiqIoB2EMdgEGWzQaJZVKDXYxFEU5BgKBAKWlpft9TJ3rinJyOdj5rijH2ikdmKPRKP/zP/+DbduDXRRFUY4BwzD4+7//+30+RNW5rignnwOd74rycTilA3MqlcK2bc4880xCodBgF0dRlKOQSCRYuXIlqVRqnw9Qda4rysnlYOe7onwcTunA3CcUCqkTTlFOAepcVxRFUY6EuulPURRFURRFUQ5CBWZFURRFURRFOQgVmBVFURRFURTlIFRgVhRFURRFUZSDUIFZOWU5joPruoNdDEVRjgN1viuKcjRULxnKcffCCy/w0ksv0dHRgdfrZcqUKcyfP59IJALAokWLeO6552hra8M0TWbOnMkXv/hFAHbv3s3//u//smnTJnK5HEOGDOH222+nqqqK+fPnM2/ePC6//PLivq699lq+//3vM2XKFNasWcP3vvc9brvtNh555BHa29v5zW9+w7p163jqqadobW1FCMG4ceOYP38+tbW1xe288847PP744zQ1NSGEYNKkSXznO9/hv/7rv9B1nW984xvFdTdv3sztt9/Ob3/7W9Ujg3LKU+e7oignAxWYleOurKyM7373u9TV1dHZ2cmPfvQj7r33Xv7xH/+RhQsX8uCDD3LbbbcxadIkstks27ZtA6Cnp4fvfOc7XHHFFXz729/G6/Wybds2vF7vYe3/jTfe4Ec/+hF+vx8hBH6/n69//esMGzaMZDLJPffcw09+8hN+/OMfA7By5Up++MMf8s1vfpPzzz8fKSUffPABAFdddRV33nkn8+fPJxgMAvDyyy8zdepU9eGpKKjzXVGUk4NqkqEcd9OmTaOhoQEhBNXV1cydO5dVq1YB8NxzzzF37lymTJmCrusEAgEmTpwIwOLFiykrK+Pmm28mEAig6zqjRo0q1lQdqptvvplIJIJpmui6ztlnn01jYyO6rhOJRPjsZz/Lxo0biz+j/Pzzz3PppZcyY8YMTNPE4/FwxhlnADBu3Djq6+tZvHgxkP+BjCVLlnDllVceo1dLUT7Z1PmuKMrJQNUwK8fdsmXLeOaZZ2hpaSGXyyGlJJPJ4DgObW1tNDQ07Pd5B3vscNTU1AyYX7t2LY899hi7d+8mk8kUl/f29hIIBGhra+Oss8464PauuuoqXnzxRebMmcPixYupqKhg0qRJR11ORTkZqPNdUZSTgaphVo6rvkuyc+bM4Te/+Q2PP/443/rWt4qPV1dX09TUtN/nVldX09zcfMBt+/3+AR+AXV1d+11PiL1ve8uyuPvuuznrrLP41a9+xeOPP85//Md/ACClBPIfuAcqE8BFF11Ee3s7GzZs4OWXX1a1TYpSoM53RVFOFiowK8dVJpPBdV0ikQgej4fm5maefPLJ4uPXXHMNf/jDH1i9ejWO45BKpVi7di0As2fPpquri4ceeohUKoXjOGzevJlYLAbAyJEjWbJkCYlEglQqxYMPPviR5bFtm2w2SygUIhAI0NXVxcMPPzxgnWuuuYZXXnmFZcuWYVkWuVyO1atXFx/3+/3MmjWLe++9l6amJi6++OJj8VIpyieeOt8VRTlZqCYZynE1ZMgQbr75Zn7605+SyWQYPnw4s2bNYvPmzQBceeWVCCG4//77aW9vx+v1MnPmTCZOnEhZWRk/+MEPeOCBB/jiF7+I67rFu+YBbrrpJu655x7+7u/+jtLSUr7whS/w2muvHbQ8fr+fr33tazz66KPcf//91NbWct111/Huu+8W1znzzDO57bbbeOKJJ/jZz36GrutMnjy52K4R9l6mnT17NuFw+GN45RTlk0ed74qinCw02Xcd6hTU3NzMfffdx4UXXqjucFaOSiKR4POf/zx3330348ePH+zinJKi0ShLly7llltuob6+fsBj6lxXjiV1vg++g53vivJxUE0yFOUoOY7DU089RWNjo/rwVJSTnDrfFeXUpAKzohyFHTt28JnPfIbly5fz1a9+dbCLoyjKx0id74py6lKBWTlpPPHEE9x5552HvP66deuYN28ejuMc8T5HjBjB73//e37961/T2Nh4xNtRFCXvpz/9KT/5yU+K81/96ldZtGjRAddfuHAh8+fPP+r9zps3jzVr1hx0HXW+K8qpS930pwy6O+64g/Hjx3PTTTcd1XbmzZvHvHnzDnn9CRMm8MQTTxzVPhVF+Xj94he/OKbbW7RoEQ899BC/+93vBixXfwsURTkYVcOsnPAsyxrsIiiKoiiKcgpTNczKoPrlL3/J+vXr2bBhA8899xwAX/rSl3jooYe48cYbefrpp4nH4zz++OO88MILvPTSS3R0dOD1epkyZQrz588v/lTuI488wurVq/nhD38I5GuuGxsb6e3t5d133yUQCHDjjTdy9dVXA7BmzRq+973v8fTTT6PrevH5U6ZMYcGCBViWxfTp0/nyl7+MrusAbNq0iV//+tc0NTXR0NDAzJkz+e1vf1ssu6Kc6hYsWMAf/vAH7r33XjRNA/Jfev/mb/6GW2+9le3bt7N06VJ6enoIBoOcf/75fOELX8Dr9e53e/Pnz2fevHlcfvnlAKxYsYLf/e53tLW1MWrUKCZMmDBg/TfeeIOnnnqK1tZWhBCMGzeO+fPnU1tby7p16/jlL3+JbdvFq1G33nors2bN4tprr+X73/8+U6ZMAeCdd97hkUceoaWlhdLSUq644gquu+664g+hXHvttdxyyy0sWbKEHTt2UFNTw1e+8hV1I6CinKRUDbMyqG699VbGjx/Ppz/9aZ544oniZdGenh62b9/Oz3/+8+IPEpSVlfHd736XRx99lP/8z/+kubmZe++996DbX7RoEZdddhmPPPII8+fP57777jvor4dt2rQJn8/H/fffz49//GPefPPNYt+uiUSCu+66i/POO4+HHnqIb3/72yxYsOAYvRKKcnKYOXMm0Wi0+AMkAG+99RZCCM477zzq6+u5++67eeyxx/jXf/1X3nvvPR577LFD2nZrayv/9m//xpw5c3jkkUe46aabePHFFwes4/f7+frXv85DDz3Er371K6SUxTbREyZM4NZbb6W8vLz492bWrFn77Gfz5s384Ac/YO7cuTz88MPcdtttPPvsszz//PMD1lu4cCHf/OY3efTRR5k8efKAtteKopxcVGBWTlhf/OIX8fv9xZqnadOm0dDQgBCC6upq5s6dy6pVqw66jalTp3LGGWcghGDatGmEQiG2bt16wPUrKyu54YYbME2ThoYGJk+ezKZNm4B8jZOu68ybNw/TNBkyZAif+tSnjt0BK8pJIBAIMH36dBYuXFhctnDhQmbPno1hGMyePZuqqio0TWPYsGHMmTPnI8/jPkuWLGH48OFceeWVGIbBuHHjmD179oB1zj77bBobG9F1nUgkwmc/+1k2btxIKpU65GNYuHAh55xzDjNmzEDXdUaOHMkNN9ywzxfk66+/nvr6enRd54orrqCjo4Oenp5D3o+iKJ8cqkmGckIqLS3d5xLtsmXLeOaZZ2hpaSGXyyGlJJPJ4DhOscnEh1VUVAyY9/l8pNPpA+63vLz8gOt3d3dTWVlZvCQLUFNTc1jHpSingssvv5z/9//+H4lEgmQyyZo1a/jyl78M5JtsLFiwgI6ODhzHw
"text/plain": [
"<Figure size 640x480 with 4 Axes>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<ggplot: (8758028328796)>"
]
},
"metadata": {
"tags": []
},
"execution_count": 46
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "E7xC_l3SEU8I",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 504
},
"outputId": "43050193-0c47-4bbc-9709-8f42376f48fd"
},
"source": [
"ggplot(df, aes(x='epochs', y='value',color='components')) + \\\n",
" geom_line() + \\\n",
" facet_wrap(['accuracy','valid']) + \\\n",
" theme_bw(base_size=12)"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAHVCAYAAADl4K3UAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeZwcVb3//9c5VdXb9PTMJJlkshNkS4AYQIlhDYuEGNnEmyv8FOEaIvsFxS+IyiJeuYDcC6JieKCgsgsCIkuCLAICahS4gEJCIBBmskyS2Xup5ZzfH93TM5NlMiGTTJL5PHk0VXWquqpOZU7Pe06frlbWWosQQgghhBBig/RAn4AQQgghhBDbMwnMQgghhBBC9EICsxBCCCGEEL2QwCyEEEIIIUQvJDALIYQQQgjRCwnMQgghhBBC9EICsxBCCCGEEL1wB/oEBlJzczPZbHagT0MI0U9SqRTV1dUbXCftXYidR29tXYitYdAG5ubmZn7yk58QhuFAn4oQop+4rst555233i9Sae9C7Fw21taF2FoGbWDOZrOEYch+++1HOp0e6NMRQmyh9vZ2Xn31VbLZ7Hq/RKW9C7Hz6K2tC7G1DNrA3CmdTkuDE2KQkPYuhBDi45AP/QkhhBBCCNELCcxCCCGEEEL0QgKzEEIIIYQQvZDALIQQQgghRC8kMItBK4oijDEDfRpCiG1A2rsQYksM+rtkiG3vscce44knnqCxsZF4PM6UKVOYM2cOmUwGgKeffprf//73rFy5Es/zOOywwzjzzDMBWLZsGb/61a9YtGgRvu8zZswYLrnkEmpra5kzZw6zZ8/mmGOOKR/r+OOP5/vf/z5TpkzhjTfe4Dvf+Q4XX3wxd999N6tWreIXv/gFb731Fg8++CArVqxAa83EiROZM2cOdXV15f387W9/47777qO+vh6tNfvuuy+XXnop//M//4PjOPznf/5nedvFixdzySWX8Mtf/lLuyCAGPWnvQoidgQRmsc3V1NTw7W9/m5EjR7J69Wquu+465s2bx7e+9S0WLFjAr3/9ay6++GL23XdfCoUC7733HgBNTU1ceumlzJgxg2984xvE43Hee+894vH4Zh3/xRdf5LrrriOZTKK1JplMcsEFFzBu3Dg6Ojq46aabuOGGG7j++usBePXVV7n22mu58MILmTp1KtZa/vWvfwEwc+ZMLr/8cubMmUNFRQUA8+fPZ9q0afLLUwikvQshdg4yJENscwcddBCjR49Ga83w4cM5+eSTee211wD4/e9/z8knn8yUKVNwHIdUKsU+++wDwHPPPUdNTQ2nnXYaqVQKx3HYfffdyz1VfXXaaaeRyWTwPA/HcTjggAOYMGECjuOQyWQ49dRTeeedd8pfo/zoo49y9NFHc8ghh+B5HrFYjE9+8pMATJw4kVGjRvHcc88BxS/IeP755zn22GP76WoJsWOT9i6E2BlID7PY5l566SUefvhhli9fju/7WGvJ5/NEUcTKlSsZPXr0Bp/X27rNMWLEiB7Lb775Jvfeey/Lli0jn8+Xy1taWkilUqxcuZL9999/o/ubOXMmjz/+OLNmzeK5555j6NCh7Lvvvlt8nkLsDKS9CyF2BtLDLLapzrdkZ82axS9+8Qvuu+8+LrroovL64cOHU19fv8HnDh8+nIaGho3uO5lM9vgFuGbNmg1up3XXj30QBFx99dXsv//+3HLLLdx333388Ic/BMBaCxR/4W7snAAOP/xwVq1axdtvv838+fOlt0mIEmnvQoidhQRmsU3l83mMMWQyGWKxGA0NDTzwwAPl9ccddxy/+93veP3114miiGw2y5tvvgnAEUccwZo1a7jzzjvJZrNEUcTixYtpbW0FYLfdduP555+nvb2dbDbLr3/9602eTxiGFAoF0uk0qVSKNWvWcNddd/XY5rjjjuOpp57ipZdeIggCfN/n9ddfL69PJpNMnz6defPmUV9fz5FHHtkfl0qIHZ60dyHEzkKGZIhtasyYMZx22mnceOON5PN5xo8fz/Tp01m8eDEAxx57LFprbrvtNlatWkU8Huewww5jn332oaamhmuuuYY77riDM888E2NM+VPzAF/+8pe56aab+NrXvkZ1dTWnn346zz77bK/nk0wmOf/887nnnnu47bbbqKur44QTTmDhwoXlbfbbbz8uvvhi7r//fn784x/jOA6TJ08uj2uErrdpjzjiCCorK7fClRNixyPtXQixs1C2832oQaahoYFbb72VQw89VD7dLLZYe3s7X/3qV7n66quZNGnSQJ/OoNTc3MwLL7zA3LlzGTVqVI910t5Ff5L2PrB6a+tCbC0yJEOILRRFEQ8++CATJkyQX55C7OSkvQsxOElgFmILLF26lC996Uu8/PLLnHvuuQN9OkKIrUjauxCDlwRmsdO4//77ufzyy/u8/VtvvcXs2bOJouhjH3OXXXbht7/9LT//+c+ZMGHCx96PEAJuvPFGbrjhhvLyueeey9NPP73R7RcsWMCcOXO2+LizZ8/mjTfe2OR20t6FGLzkQ39iwF122WVMmjSJL3/5y1u0n9mzZzN79uw+b7/33ntz//33b9ExhRBbz09/+tN+3d/TTz/NnXfeye23396jXF4HhBCbIj3MYrsXBMFAn4IQQgghBjHpYRYD6mc/+xn//Oc/efvtt/n9738PwNe//nXuvPNOvvjFL/LQQw/R1tbGfffdx2OPPcYTTzxBY2Mj8XicKVOmMGfOnPJX5d599928/vrrXHvttUCx53rChAm0tLSwcOFCUqkUX/ziF/nc5z4HwBtvvMF3vvMdHnroIRzHKT9/ypQpPPnkkwRBwMEHH8xZZ52F4zgALFq0iJ///OfU19czevRoDjvsMH75y1+Wz12IwezJJ5/kd7/7HfPmzUMpBRT/4D3jjDM455xzeP/993nhhRdoamqioqKCqVOncvrppxOPxze4vzlz5jB79myOOeYYAP7xj39w++23s3LlSnbffXf23nvvHtu/+OKLPPjgg6xYsQKtNRMnTmTOnDnU1dXx1ltv8bOf/YwwDMvvRJ1zzjlMnz6d448/nu9///tMmTIFgL/97W/cfffdLF++nOrqambMmMEJJ5xQ/hKU448/nrlz5/L888+zdOlSRowYwdlnny0fAhRiJyY9zGJAnXPOOUyaNIkvfOEL3H///eW3Rpuamnj//fe5+eaby19IUFNTw7e//W3uuecefvSjH9HQ0MC8efN63f/TTz/NZz/7We6++27mzJnDrbfe2uu3hy1atIhEIsFtt93G9ddfz5///OfyvV3b29u56qqrOPDAA7nzzjv5xje+wZNPPtlPV0KIHd9hhx1Gc3Nz+ctHAF555RW01hx44IGMGjWKq6++mnvvvZcrr7ySv//979x777192veKFSv4wQ9+wKxZs7j77rv58pe/zOOPP95jm2QyyQUXXMCdd97JLbfcgrW2PCZ677335pxzzmHIkCHl15rp06evd5zFixdzzTXXcPLJJ3PXXXdx8cUX88gjj/Doo4/22G7BggVceOGF3HPPPUyePLnH2GshxM5HArPYbp155pkkk8ly79NBBx3E6NGj0VozfPhwTj75ZF577bVe9zFt2jQ++clPorXmoIMOIp1Os2TJko1uP2zYME466SQ8z2P06NFMnjyZRYsWAcVeJ8dxmD17Np7nMWbMGD7/+c/3X4WF2MGlUikOPvhgFixYUC5bsGABRxxxBK7rcsQRR1BbW4tSinHjxjFr1qxNtuFOzz//POPHj+fYY4/FdV0mTpzIEUcc0WObAw44gAkTJuA4DplMhlNPPZV33nmHbDbb5zosWLCAT33qUxxyyCE4jsNuu+3GSSedtN4fxyeeeCKjRo3CcRxmzJhBY2MjTU1NfT6OEGLHIkMyxHapurp6vbdpX3rpJR5++GGWL1+O7/tYa8nn80RRVB4ysa6hQ4f2WE4kEuRyuY0ed8iQIRvdfu3atQwbNqz8tizAiBEjNqteQuzsjjnmGL73ve/R3t5OR0cHb7zxBmeddRZQHLLx5JNP0tjYSBRFhGFYH
"text/plain": [
"<Figure size 640x480 with 4 Axes>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<ggplot: (-9223363278826112789)>"
]
},
"metadata": {
"tags": []
},
"execution_count": 47
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LyBlhJEEmfq_",
"colab_type": "text"
},
"source": [
"## Feed-Forward Zero-One Loss"
]
},
{
"cell_type": "code",
"metadata": {
"id": "yY9zTezLFQjo",
"colab_type": "code",
"colab": {}
},
"source": [
"df_loss = pd.DataFrame(\n",
"columns = ['components', 'zero_one']\n",
")\n",
"\n",
"losses =[]\n",
"\n",
"for i in range(len(res)):\n",
" losses.append(res[i][2])\n",
"components = [i for i in range(10,211,20)]"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "UMZyeh0zFR8n",
"colab_type": "code",
"colab": {}
},
"source": [
"df_loss = pd.DataFrame( data = [components,losses], index = ['components', 'zero_one']).T"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "rPvWBDt4FTWN",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 394
},
"outputId": "ba5c37bf-cc8e-4d08-f35a-16970c011046"
},
"source": [
"df_loss"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>components</th>\n",
" <th>zero_one</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>10.0</td>\n",
" <td>0.286893</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>30.0</td>\n",
" <td>0.197984</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>50.0</td>\n",
" <td>0.159028</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>70.0</td>\n",
" <td>0.145738</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>90.0</td>\n",
" <td>0.156279</td>\n",
" </tr>\n",
" <tr>\n",
" <th>5</th>\n",
" <td>110.0</td>\n",
" <td>0.148488</td>\n",
" </tr>\n",
" <tr>\n",
" <th>6</th>\n",
" <td>130.0</td>\n",
" <td>0.152154</td>\n",
" </tr>\n",
" <tr>\n",
" <th>7</th>\n",
" <td>150.0</td>\n",
" <td>0.141155</td>\n",
" </tr>\n",
" <tr>\n",
" <th>8</th>\n",
" <td>170.0</td>\n",
" <td>0.124656</td>\n",
" </tr>\n",
" <tr>\n",
" <th>9</th>\n",
" <td>190.0</td>\n",
" <td>0.125573</td>\n",
" </tr>\n",
" <tr>\n",
" <th>10</th>\n",
" <td>210.0</td>\n",
" <td>0.145738</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" components zero_one\n",
"0 10.0 0.286893\n",
"1 30.0 0.197984\n",
"2 50.0 0.159028\n",
"3 70.0 0.145738\n",
"4 90.0 0.156279\n",
"5 110.0 0.148488\n",
"6 130.0 0.152154\n",
"7 150.0 0.141155\n",
"8 170.0 0.124656\n",
"9 190.0 0.125573\n",
"10 210.0 0.145738"
]
},
"metadata": {
"tags": []
},
"execution_count": 50
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "rGwbQWP6FUOw",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 498
},
"outputId": "9f216196-38eb-4a5f-dc50-aca3fba25438"
},
"source": [
"ggplot(df_loss, aes(x='components', y='zero_one')) + \\\n",
" geom_line() + \\\n",
" geom_point() + \\\n",
" theme_bw(base_size=12) + ggtitle(\"Zero-one loss FW\") + ylab(\"loss\")"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAkIAAAHPCAYAAAC/YbWbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeVxU5eIG8GeGAUEEQWU1BYUERUzcFZhRcwvKNfUmai6lkZqmuafgmmma5m6aLaRelyxMuxii4r7hgqRogeUCgogbqMjM+/vDy/zkAgqynJkzz/fz4XMvZ86ceY4vA0/nvHOOQgghQERERGSClFIHICIiIpIKixARERGZLBYhIiIiMlksQkRERGSyWISIiIjIZLEIERERkcliESIiIiKTxSJEREREJotFiIiIiEwWixAR0X/t27cPCoUC4eHhUkchogrCIkRkANzd3aFQKIr1NWjQIKnjkgHJK2/P+7py5Qq0Wi3s7Oxgbm6O+/fvF9hOSkqKfv3169cX+loajQYKhQKxsbHlvVtEFUYldQAiAsaMGYM7d+4U+Xh2djYWLVoErVaLhg0bVmAyMhZubm5FlmQ7OzuYmZmhbdu2+OWXXxAbG4vg4OB86+zZswcAoFAoEBMTg8GDB+d7PDs7G0ePHoW1tTVat25dLvtAJAUWISIDMGbMmCIfE0KgT58+0Gq1ePvttzFu3LgKTEbGwt3d/YWn9Nq3b49ffvkFMTExBYpQTEwMrKys0L59e+zdu7fAcw8ePIicnBy0b98e5ubmZRmdSFI8NUZk4KZPn46tW7fCz88P3333HRQKRb7Hc3NzsWLFCrRq1Qq2traoXLky/Pz8sGzZMuh0unzrXrlyRX967dKlS+jbty8cHR2hVCqxb98+AIBOp8OqVavQvHlzVKlSBdbW1mjevDlWrlxZYHvFtWfPHnTp0gXVqlVDpUqVUK9ePUyaNAl3794tsG7btm2hUCiQm5uLuXPn4tVXX0WlSpVQq1YtTJw4ETk5OYW+xsWLFzFo0CDUqlULFhYWcHJyQr9+/ZCYmPhSmf/X5cuXMXDgQNSsWRMWFhZwdXXFwIEDcfny5QLr3r9/H7NmzULDhg1ha2sLGxsbeHh4oG/fvjh16lS+dSMjI/H666/DxcUFlSpVgqurKzQaDVasWFEmuZ/1+uuvA3haev5XTEwM/P390blzZ1y/fr3Av1vec/K2QSQbgogM1oYNGwQA4ezsLK5evVrg8ZycHNG5c2cBQHh5eYnhw4eL0aNHi0aNGgkAon///vnWT05OFgBEQECAsLOzEy1atBBjxowRw4cPF6dOnRJCCNGvXz8BQNSqVUuMHj1ajBkzRri5uQkAol+/fiXeh1WrVgmFQiGqVKkiBg8eLCZOnChatmwpAIgGDRqIzMzMfOtrNBoBQPTu3Vs4OzuLwYMHi9GjR4tXX31VABCDBg0q8Bq//fabsLKyEiqVSvTo0UOMHz9evPPOO6JSpUrC1tZWv28vsnfvXgFAhIWF5Vt+/PhxYWtrKxQKhejWrZuYPHmy6NGjh1AoFMLW1lYcP35cv65OpxNt2rQRAETr1q3Fxx9/rM/j7Owsli5dql939erV+vF9//33xeTJk8XQoUNF8+bNRbNmzUqUWaPRFGt9JycnoVAoxK1bt/TL/vzzTwFAzJ07V8THxwsAYvny5fme16xZMwFAxMXFFet1iIwFixCRgTp27JiwtLQUlpaW4ujRo4WuExYWJgCIkSNHitzcXP3y3NxcMWTIEAFA/Pzzz/rleUUIgJg8eXKB7eUVLz8/P3H//n398gcPHoimTZsKAOLHH38s9j5cuXJFWFhYCBsbG3HhwoV8j4WGhgoA4v3338+3PK8INWnSRGRkZOTL4OHhIZRKpUhJSdEvv337trCzsxPVq1cXCQkJ+bYVHx8vrK2thZ+fX7HyFlaEdDqd8Pb2FgBEREREvvU3bdqkL6FarVYIIcS5c+cEANG9e/cC29dqteL27dv675s0aSIsLCzEzZs3C6ybnp5eosxubm4iLCyswNfevXvzrf/OO+8IAGLLli36ZWvWrBEAxJEjR4ROpxMODg6iV69e+sfv3LkjzMzMRPXq1YVOpytWLiJjwSJEZICuXr0qXFxcCv3jm0er1Ypq1aoJZ2dn8eTJkwKPZ2ZmCoVCIXr37q1flleEnJycxKNHjwo8p0OHDgKAiIqKKvBYdHS0ACDatWtX7P2YPXt2kaXr9u3bwsbGRlhaWubLkleEfv/99wLPmT59ugAgduzYoV+2ePFiAUAsW7as0AxjxowRAAqUpMIUVoQOHjyoP7pTmICAAAFA7N+/Xwjx/0XonXfeeeHrNWnSRFSuXDlfOSqpvMxFff3v0a21a9cKACI0NFS/7F//+pewsbHR/xz17t07X+n5+eefBQDx9ttvv3ROIkPFydJEBiY7OxvdunVDSkoKJk+ejJCQkELXu3TpEm7fvo1XX30Vs2fPLnQdKysrXLhwocDy1157DZUqVSqwPC4uDkqlEm3bti3wmEajgZmZGU6fPq1ftm/fPv3cojzu7u76Ty/FxcUBeDpJ93/Z29vDz88PsbGxuHjxIl577bV8jzdr1qzAc2rVqgUAyMzM1C87cuQIAODs2bOFTha+dOkSAODChQto0KBBgcdf5Hn7kLf84MGDOH36NNRqNRo0aIDGjRtj48aN+Pvvv9GtWzcEBASgWbNmsLCwyPfckJAQjBs3Dg0aNMC//vUvaDQa+Pv7w8HBocQ5NRpNgbEoKi+Qf57Q3r17ERgYCJXq6Z+Edu3aYcuWLTh79iwaN27M+UEkayxCRAZECIF3330XcXFx6N69O+bMmVPkuhkZGQCeTuKdMWNGkes9ePCgwDJnZ+dC17179y6qVatW4A82AKhUKtSoUQNpaWn6Zfv27Svw2hqNRl+E8iZDu7i4FPp6ecsLu3SAnZ1doRkAQKvV6pfl/Tt8/fXXhb5GnsL+HYqjpPtgZmaGmJgYzJw5E1u3bsXEiRMBADY2Nnj33Xfx2WefoUqVKgCAsWPHokaNGlixYgW++uorLF68GAqFAhqNBgsWLCi0DJZWnTp1UKdOHSQmJuL69evIzMzEzZs38xW9vCK8Z88eNG7cWP/R+g4dOpR5HiKp8VNjRAYkLCwMW7duRaNGjRAREVHgE2LPqlq1KgCgR48eEE9Pcxf6lZycXOC5RW23atWquH37Np48eVLgsdzcXNy6dQu2trb6ZeHh4QVe79mjEnkZU1NTC329lJSUfOu9jLznnj179rn/Du+++26ptl+SfbC3t8eXX36Jq1ev4vLly1i7di28vb2xbNkyhIaG5nv+wIEDcfToUWRkZGDnzp0YOnQoYmNj0blzZ6Snp79U5hd59qhQ3tGedu3a6R+vX78+nJ2dERMTg7S0NCQkJKB27drw9PQslzxEUmIRIjIQmzZtwqxZs+Do6IjIyEhYW1s/d31vb2/Y2dnh6NGjhRaXl+Hn5wedTlfolYNjY2Oh1WrRpEmTEm0PQKGnbO7cuYMzZ87A0tIS9evXf+nMrVq1AgAcOHDgpbfxPM/bBwD6a+4U9e/i6emJoUOHYv/+/ahSpQp++eWXQtezs7NDUFAQvv76awwaNAi3b98utys4P/sx+piYGNjb26Nx48b51mnbti0OHDiAqKiofM8hkhsWISIDcPz4cQwePBgWFhbYvn073NzcXvgclUqFUaNGISUlBR999BEePnxYYJ2UlBT88ccfxc4xZMgQAMDkyZORnZ2tX56dnY1JkyYBAIYOHVrs7fXv3x/m5uZYunQp/vzzz3yPTZs2Dffu3UP//v0Lna9UXIMHD4adnR1mzJiB48ePF3hcp9MVa+5MUfz9/eHl5YWDBw9i69at+R7bunUrDhw4gHr16iEgIAAAkJycjKSkpALbyczMxOPHj2FlZaVftnfvXgghCqybd/qxcuXKL537efKOCEVHR2P//v3QaDRQKvP/OWjXrh3u37+PBQsWAGARIvniHCEiid2/fx/du3fHo0eP0Lx5c+zevRu7d+8ucv1nJyNPmzYNZ8+ex
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<ggplot: (-9223363278721333202)>"
]
},
"metadata": {
"tags": []
},
"execution_count": 51
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4DJIeuMIkUON",
"colab_type": "text"
},
"source": [
"# 1.4 Convolutional Neural Newtworks"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XPAm0p36kVVh",
"colab_type": "text"
},
"source": [
"## One VGG block CNN\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "CHXZ0nRciBW2",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 732
},
"outputId": "d10bce0d-8fa6-4f43-c97a-6f6728eaa7e0"
},
"source": [
"\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", activation='relu', input_shape = (32, 32, 3)),\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(128, activation = 'relu'),\n",
" tf.keras.layers.Dense(10, activation = 'softmax')\n",
"])\n",
"\n",
"model.compile(optimizer = \"adam\", loss='categorical_crossentropy', metrics=['accuracy'])\n",
"\n",
"model.summary()"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Model: \"sequential_11\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d (Conv2D) (None, 32, 32, 32) 896 \n",
"_________________________________________________________________\n",
"conv2d_1 (Conv2D) (None, 32, 32, 32) 9248 \n",
"_________________________________________________________________\n",
"max_pooling2d (MaxPooling2D) (None, 16, 16, 32) 0 \n",
"_________________________________________________________________\n",
"flatten (Flatten) (None, 8192) 0 \n",
"_________________________________________________________________\n",
"dense_22 (Dense) (None, 128) 1048704 \n",
"_________________________________________________________________\n",
"dense_23 (Dense) (None, 10) 1290 \n",
"=================================================================\n",
"Total params: 1,060,138\n",
"Trainable params: 1,060,138\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n",
"Model: \"sequential_11\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d (Conv2D) (None, 32, 32, 32) 896 \n",
"_________________________________________________________________\n",
"conv2d_1 (Conv2D) (None, 32, 32, 32) 9248 \n",
"_________________________________________________________________\n",
"max_pooling2d (MaxPooling2D) (None, 16, 16, 32) 0 \n",
"_________________________________________________________________\n",
"flatten (Flatten) (None, 8192) 0 \n",
"_________________________________________________________________\n",
"dense_22 (Dense) (None, 128) 1048704 \n",
"_________________________________________________________________\n",
"dense_23 (Dense) (None, 10) 1290 \n",
"=================================================================\n",
"Total params: 1,060,138\n",
"Trainable params: 1,060,138\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "cwz11kP4iEnV",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 374
},
"outputId": "0a397b6b-86af-4ae0-f32b-8d0e6941d4c3"
},
"source": [
"history = model.fit(x_train,y_train,\n",
" batch_size = 32,\n",
" epochs=10,\n",
" validation_data=(x_valid, y_valid),\n",
" verbose=2\n",
" )"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1019/1019 - 12s - loss: 0.2258 - accuracy: 0.9251 - val_loss: 0.1457 - val_accuracy: 0.9591\n",
"Epoch 2/10\n",
"1019/1019 - 11s - loss: 0.0122 - accuracy: 0.9964 - val_loss: 0.1096 - val_accuracy: 0.9716\n",
"Epoch 3/10\n",
"1019/1019 - 11s - loss: 0.0083 - accuracy: 0.9981 - val_loss: 0.0620 - val_accuracy: 0.9841\n",
"Epoch 4/10\n",
"1019/1019 - 11s - loss: 0.0058 - accuracy: 0.9989 - val_loss: 0.1105 - val_accuracy: 0.9822\n",
"Epoch 5/10\n",
"1019/1019 - 11s - loss: 0.0071 - accuracy: 0.9986 - val_loss: 0.1179 - val_accuracy: 0.9836\n",
"Epoch 6/10\n",
"1019/1019 - 11s - loss: 0.0080 - accuracy: 0.9989 - val_loss: 0.1146 - val_accuracy: 0.9782\n",
"Epoch 7/10\n",
"1019/1019 - 11s - loss: 0.0039 - accuracy: 0.9994 - val_loss: 0.1395 - val_accuracy: 0.9790\n",
"Epoch 8/10\n",
"1019/1019 - 11s - loss: 0.0020 - accuracy: 0.9995 - val_loss: 0.1240 - val_accuracy: 0.9827\n",
"Epoch 9/10\n",
"1019/1019 - 11s - loss: 0.0045 - accuracy: 0.9995 - val_loss: 0.1455 - val_accuracy: 0.9733\n",
"Epoch 10/10\n",
"1019/1019 - 11s - loss: 0.0029 - accuracy: 0.9994 - val_loss: 0.1433 - val_accuracy: 0.9771\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "wva4Qa2uiGZH",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "90606185-813d-40de-81dc-058250c8da2f"
},
"source": [
"y_pred = model.predict(x_test)\n",
"\n",
"cnn_loss = [] \n",
"zol = zero_one(y_pred, y_test)\n",
"\n",
"print(\"Zero-one Loss: \", zol)\n",
"cnn_loss.append(zol)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Zero-one Loss: 0.02153987167736022\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "GG5q04ECiIUc",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 540
},
"outputId": "e4785a50-1e97-4640-c27d-d599680e60c2"
},
"source": [
"# plot a random sample of test images, their predicted labels, and ground truth\n",
"fig = plt.figure(figsize=(16, 9))\n",
"for i, idx in enumerate(np.random.choice(x_test.shape[0], size=16, replace=False)):\n",
" ax = fig.add_subplot(4, 4, i + 1, xticks=[], yticks=[])\n",
" ax.imshow(np.squeeze(x_test[idx]))\n",
" pred_idx = np.argmax(y_pred[idx])\n",
" true_idx = np.argmax(y_test[idx])\n",
" ax.set_title(\"{} ({})\".format(TYPES[pred_idx], TYPES[true_idx]),\n",
" color=(\"green\" if pred_idx == true_idx else \"red\"))\n",
"\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAILCAYAAAA63AmGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd7glRbX23+re4aTJAZhhYMiSRIGL4FVBQb2oKAa4YvZeVMwJI+pm+wnqFRXlyhUDCpIFREVQQEERleSQk8AMTM7hpJ266/uj+3StVefskzgzw+a8v+eZZ2rvXt1dvU+tCl1vrTLWWhBCCCGEEEJIKxNs7wwQQgghhBBCyDOFAxtCCCGEEEJIy8OBDSGEEEIIIaTl4cCGEEIIIYQQ0vJwYEMIIYQQQghpeTiwIYQQQgghhLQ822xgY8rmdFM2F22r+40GUzYfMGVz9hjsbzFlc/LWzNMI97/DlM3+2+v+hGxLTNkUTdk8ZMpmp+2dl7FAPyWTCVM2t5myeeH2zsd4MGXzbVM2H9ze+SBkazEZ29HcBGfkbQA+BeB5ALoB3APgDFuyf53I+0wEpmwKAL4E4HDvuy8CeDuAeQDWAvgTgK/akl2yHbLpcxaArwJ48/bOCGkdTNksAbADgAhAL4DrAXzElmzP9szXKHg/gL/Ykl0JAKZsfg7gbQBq6b+7AXzUluwj2y2HQ0M/JSNiykb6XweAKhIfBYAP2JK9eBvkwQLYy5bs4+M8/zgA3bZkF5my+SGAd6SHCgAMkmcCgFttyR77jDM8cn5+DmCZLdkvjfKUswDcYcrmp7Zka1svZ6TVYTu6zRl3OzphMzambD4F4GwAZyL54+8C4FwAb5ioe4h75bzPxpTNWJ/lDQAesSW7XHx3JYDXI/mjTwNwEJI/+tHPILuDGCq//jM14TcAXm7KZseJzA+ZFBxnS7YLwMEADkUyqH9WMEzZPwXAL7zv/id9jp0BrAHw862YtTEhnoN+SkbElmzXwD8ATyP10fTfVh/UTBCZj9qSPUU8z5kALhfPs9UHNeMh7ew9gqTdJ2Qk2I5uZSaiHZ2QGRtTNtOQjKzea0v2anHot+m/AQqmbC4E8EYkFfm7bcnelV5jHoBzALwMQA+A79qS/X567HQABwCoIKmAPmXK5h0AbgNwFJJC9hVTNm+1JXuIyNenABxpS3aowdWxAP4sbI8B8EoAe9uSXZp+vRnAD7zzdjVlcxuA5wP4O4C32ZJdl17jcADfAbAfgKcAfNyW7C3psVu8/B5oyuZfAD4C4BMAcqZsrgNQsSX7aZGv3wC42Zbsd23JVkzZ3A3g1QAuGOKZCBkWW7LLTdlcj8SfRlNm/45kYP88ADcj8fENpmwWAlgM4AMATkfydvbbtmTPSs8NAHwWwPsATAfwRwCneOeeDKAEYAkSv88wZbMLgN0B3N7kOfpM2VwC4PLUfjT1RwTgNQD+lT7HvenxJQDOA/BOADsBuAbAB23JVtLjrwPwNQALATyUPsd94tz/QzLLu48pm076KXkmmLIpAvgmgBPTr64A8DlbslVTNkcBuAjA9wGciqRMfxDJm9ezAcwGcJYt2TPTax0G4HsA9gXQD+AqAJ+yJVszZfOX9Pr3pjM3/21L9nJTNu8D8DkAMwH8FUl5XzFEPgsAXoGkDhjpmV4P4OsA5iNRcnzQluzD6bElSNrZdwLYA8BlSJQTPwfwEiR1wAm2ZDem9r8E8FIA7QDuTa/1oCmb9yPxQ2vK5hNI2s3jTNnsi8RHXwBgOYAv2JL9jcjeLQBei+TFJiEjwnb02d2OTtSMzREA2gD8agS71yOptKYjGY39L5D98X6LpJKaj6QAfMKUzavFuW9AUvFMBzDwNuudSKbZpiCp6HdLKzGI4xc2ycuBAB4Vn48BcIcY1DTjbQDeC2Aukun2U9NnmA/gd0j+cDPT768yZTPHy89Afp9KvzsewIuQOMQFAE4amM0xZTM7zdcl4hoPI5lJImTMmLJZgKRSWjTKMvsuAP+FpKJqIPEzycsB7AXgVQA+l74gAICPIinbRyKRdW7E4JcERyLpcL0agzkQwJO2ZBtNnqMLSSW4aAz1xy/T57wEwDWmbPLi+NvTfOwBYG+kb+LStQPnI2l4ZiGpuH+Tdj4HOAlJx2i6yC/9lIyX05BIpF+ApAwdBv1meEck7e18AF8B8GMkErBDkHT4v2zKZrfUNgLwSSQDniOQ+MaHAMCW7EAn6KB0VuVyUzavQDIAORGJzz+FpM0eir0AxLZklw33MKZs9gZwKZIXeHMAXAfgt+nAaIA3I32xCOA4JDKfL6b2AYCPCdvr03vPBfBPpP0BW7I/StP/kz7PcamP/xbADan9RwFcbMpmH3E9+ioZE2xHn93t6EQNbGYBWNfsxxP81ZbsdbZkIyRTYwMZ/jcAc2zJftWWbM2W7JNIKuu3inP/bkv2GluysS3Z/vS7n9uSfdCWbMOWbBXJqPMdAJAuOloI4NomeZmOZB2QfIaVo3jWn9mSfSzNwxVIGh+k970ufb7YluyNAO5CUvgHkPmtp9993ZbsBluy/bZk70AySzQgfXsrgFtsya4W1+hO807IWLjGlM0mJG9g/4xEKjKaMvsLW7IP2JLtBfBlACeasgnF8bIt2V5bsvcD+BmSyglIpr9PsyW7LPXN0wG8xZsuPz09tx+D8f1zgFPT53gcQBeA92B09cfdtmSvTP3uO0g6hoeL4/9rS3apLdkNAM4Qz/F+AOfZkr3dlmxkS/YCJOsG5LnfT8+Vz0E/JePl7UjWda6xJbsWQBnJS7EB6kjWrtaRDDpmA/ieLdluW7IPInkbehAA2JK925bsP9I2ZwmSDsWRI9z7fFuy/0z99gsAjkjfDvs081Gf/wTwO1uyN6Z5PgvJbMuLhc05tmRX20QafiuA223JLkrf9v4KQBacwJbs+emzDtQrB6WqkaE4HEk98Y20bvgTkj7BScKGvkpGC9vRFmhHJyp4wHoAs03Z5EYY3KwS6T4AbekfaFcA89IfeoAQSQU3wFAzKf53FwC41JTNl5A0BFekhWEoNiKZOZHPsPcweR/Af4auNL0rgBPSxZQD5JFMOzbL71DfXYDEUW5M//+ed3wKgE0gZGwcb0v2JvmFKZuxltmn0uOzhzl+YJreFcCvTNnE4niEZP3dUOf6+P45wFnWWxhsyuYQjKH+sCUbm7JZhuQNWLPnGDi2K4B3m7L5qDheGObcAeinZLzMg5vRB3R5BID16ctBIJGXAYB8+dWPtF1KZ0u+g2Q9QAeSNv/uEe79z4EPtmR7TNmsR/IGd4ln28xHh7pm9jyp/y1NrzmAn/9mzxMi6TCdgGQ2Z6B+mY3kpeBQ915qS1bWQ09596avktHCdjTl2dyOTtTA5u9IRl/HY3w61aUAFtuS3WsYGzvSd7Zk/2HKpoZkOv5t6b9m3Ac9kLkJwMdN2ew80tR6E5YiGZW/bxibEZ8BiX76AVM2ByGZXrzGO75vakPIM2U0ZXaBSO+C5G3xOvH9AiSLbweOD2jxlwL4L1uyt/kXFG9/h/KHAe5DIi0d6WXJwL1Gqj+y50in3HcWeVXHMfg5zrAle8Yw1x7qOeinZLysQNIReDD9LMvjWPk/AIsAnGRLtjtde/KWUdwbAGDKphOJmmH5ELaPAzCmbOZbHYRnqGsOdNRgysYg8bfhzmnG25DIYY5BMtCahqTzZtLjvi+uALDAlE0gBje7AHhM2OyLRH5DyHhgOzrEcWzHdnRCBja2ZDebsvkKgB+Ysmkg0bPWkVQ+L7cl+9kRLnEHgG5TNp9Doj2sIXmgdluyd44xOxciWbtTt8OHmb4OyTTfGekz3GTK5kYko+NTkFR07Uim5mu2ZM8f4b4XAbgz1SPehGREfjiAx8cyULIlu8yUzZ1IpHpXyWk5UzZtSHTU7x7t9QgZhtGU2XeYJODHEiQBQq60JRuZ8kA/Al9OFxvvhmTt2UC41x8COMOUzbttyT6V6o1fbEv216PJWOoHjyNZX/C3EcxHU38cYsrmTUjW9n0MyYuYf4hrfNiUzbVIZmFPQ
"text/plain": [
"<Figure size 1152x648 with 16 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "kajPVWLGiKXn",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "6bbc10dd-f36a-4412-c125-612eecd03838"
},
"source": [
"#Loss and accuracy visualisation\n",
"\n",
"plt.figure(1) \n",
" \n",
" \n",
"plt.subplot(211) \n",
"plt.plot(history.history['accuracy']) \n",
"plt.plot(history.history['val_accuracy']) \n",
"plt.title('model accuracy') \n",
"plt.ylabel('accuracy') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
" \n",
" \n",
"plt.subplot(212) \n",
"plt.plot(history.history['loss']) \n",
"plt.plot(history.history['val_loss']) \n",
"plt.title('model loss') \n",
"plt.ylabel('loss') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
"plt.show()"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU1d348c93lqwEAmEnbCqguLELVdzQCrjbFtFqaze02l/tU+ujdtG2z2P1edrHWltra93a2oK70hZ3sdq6QEBUEAVUIGENSyALWWbm+/vj3EkmIcskzGQmyff9es1r7nLuvWduJvc755x7zhVVxRhjjImXL9UZMMYY07VY4DDGGNMuFjiMMca0iwUOY4wx7WKBwxhjTLtY4DDGGNMuFjiMaYWIPCQi/x1n2o0ickay82RMqlngMMYY0y4WOIzpAUQkkOo8mO7DAofp8rwqoutF5D0RqRSR+0VkkIg8KyLlIvKSiPSNSX+eiKwRkTIReVVEjopZN1FEVnrbPQJkNTnWOSKyytv2DRE5Ls48ni0i74jIfhEpFpEfN1l/kre/Mm/9Fd7ybBH5PxHZJCL7RORf3rJTRaSkmfNwhjf9YxF5XEQeFpH9wBUiMk1E3vSOsU1EfiMiGTHbHy0iL4rIHhHZISLfF5HBIlIlIgUx6SaJSKmIBOP57Kb7scBhuovPAWcCY4FzgWeB7wMDcN/zbwOIyFhgIfAdb90S4G8ikuFdRJ8G/gz0Ax7z9ou37UTgAeBKoAD4PbBYRDLjyF8l8CUgHzgb+KaIXODtd6SX3197eZoArPK2+wUwGfiMl6f/BCJxnpPzgce9Y/4FCAP/AfQHZgCzgKu9POQBLwHPAUOBI4CXVXU78CowL2a/lwOLVLUuznyYbsYCh+kufq2qO1R1C/A68LaqvqOq1cBTwEQv3cXAP1T1Re/C9wsgG3dhng4EgTtVtU5VHweWxxxjAfB7VX1bVcOq+kegxtuuVar6qqq+r6oRVX0PF7xO8VZfCrykqgu94+5W1VUi4gO+Clyrqlu8Y76hqjVxnpM3VfVp75gHVHWFqr6lqiFV3YgLfNE8nANsV9X/U9VqVS1X1be9dX8ELgMQET9wCS64mh7KAofpLnbETB9oZr6XNz0U2BRdoaoRoBgY5q3boo1H/twUMz0SuM6r6ikTkTJguLddq0TkBBFZ6lXx7AOuwv3yx9vHx81s1h9XVdbcungUN8nDWBH5u4hs96qvfhZHHgCeAcaLyGhcqW6fqi7rYJ5MN2CBw/Q0W3EBAAAREdxFcwuwDRjmLYsaETNdDNyqqvkxrxxVXRjHcf8KLAaGq2of4HdA9DjFwOHNbLMLqG5hXSWQE/M5/LhqrlhNh76+B/gQGKOqvXFVebF5OKy5jHultkdxpY7LsdJGj2eBw/Q0jwJni8gsr3H3Olx10xvAm0AI+LaIBEXkImBazLZ/AK7ySg8iIrleo3deHMfNA/aoarWITMNVT0X9BThDROaJSEBECkRkglcaegC4Q0SGiohfRGZ4bSrrgCzv+EHgh0BbbS15wH6gQkSOBL4Zs+7vwBAR+Y6IZIpInoicELP+T8AVwHlY4OjxLHCYHkVVP8L9cv417hf9ucC5qlqrqrXARbgL5B5ce8iTMdsWAd8AfgPsBTZ4aeNxNfBTESkHbsYFsOh+NwNzcUFsD65h/Hhv9feA93FtLXuA/wF8qrrP2+d9uNJSJdDoLqtmfA8XsMpxQfCRmDyU46qhzgW2A+uB02LW/xvXKL9SVWOr70wPJPYgJ2NMPETkFeCvqnpfqvNiUssChzGmTSIyFXgR10ZTnur8mNSyqipjTKtE5I+4Ph7fsaBhwEocxhhj2slKHMYYY9qlRwx81r9/fx01alSqs2GMMV3KihUrdqlq0/5ByQ0cIvIAbiiDnap6TDPrBfgV7lbEKuAKVV3prfsy7t50gP/2hndARCYDD+GGiViCG46h1fq2UaNGUVRUlJDPZIwxPYWINHvrdbKrqh4CZreyfg4wxnstwPVsRUT6AbcAJ+A6YN0SM7rpPbh76aPbtbZ/Y4wxCZbUwKGqr+E6LbXkfOBP6rwF5IvIEOAs4EVV3aOqe3G3Ac721vX2BmpTXG/WC5L5GYwxxjSW6jaOYTQeiK3EW9ba8pJmlh9ERBbgSjGMGDGiuSSmC1FVasMRakIR6kIRfCL4RBAf3jT1y+qnfdL2jtOEqhJRCEeUiLpXOKJEIhD25iMRJRyzPKJuPhJRQhG33L1HCIVj5lUJh2PTRGLSeu/hSJN9uPdI7D4bpT94eTiiaMxnib5HvJpk9xlAaViv3vr6+UbpovtoSKdKQ5rovDaebyp25DFBml3u1sVu0/J3p9H+DtrHwfuX+nlpmG9tnbdtdF8HHaO5tPX7a7yNAL+5dBLD++WQSKkOHEmjqvcC9wJMmTLloK9TXV0dJSUlVFdXd3reOlNWVhaFhYUEgx1/5k4kolTWhiivDlFdF3YX8LoIteEItSH3qgmFqfGmm66vCYXr00XX1TTZtrnt3LqG7ToiNqCIgN/XMO0T8ebdP2Oj4ONrsp0cvJ0I9RfM6IVelZiLuxcMmlz01QsQ0eXRC2c68vvcZw00evc1zPvdu7/+nETPJ975cpcyX/08MWl89ee51XS4v0dDmoa/mUjMPNLoIgo0CiSNppuM/9h4Ha2sazlh7Gy02VXr5xvSNF1H/TqtT6vNLGtuf6DNrGu8f38SfkClOnBswY1MGlXoLdsCnNpk+ave8sJm0rdbSUkJeXl5jBo1qtVfF12ZqrJ79242by4mb+BQyqtD7D9QR3m1CwL7q+sor65j/4EQ5dV19cv2N0pXR3lNqNlfcvHyCWQG/GQEfGQEfGR67xl+N50Z8JOTESC/yboMb13sdpkBd9EC6n/NRi/W9b9MoxdlbfjFGvGW109rw4U99hdvOHLwdmFVNx9p+GUb3c7vlWyiF06fF4hil0cvjtGgFQ1WviYXXDfdsNxXvw8a0tTv00vrpQv6m1zQvbw0zPsaXegPWu4T/P6G5bH5MqapVAeOxcC3RGQRriF8n6puE5HngZ/FNIh/FrhJVfeIe/TmdOBt3BPVft2RA1dXV3eZoBFRJRSOeL9uY37RakMVQewv2rB3kXPVCgG279jLgj+sbXH/PoFemQF6ZwfJywrSOytAYd8cemdFlwXonRWkV1aA7KC/4aIfPPgCn9lMcAj4rbuQMd1Jsm/HXYgrOfT3no98C+4Ja6jq73C3087FjTJaBXzFW7dHRP6Lhqev/VRVo43sV9NwO+6z3quj+evopp1GVVm/o4KaULjFNL4mv0QDPh9+v/ul6/cJ1dlB/uv8o+uDgAsOXkDIDpKb4e8S58IYkx6SGjhU9ZI21itwTQvrHsA9i6Dp8iLgoD4h3VVFTYiaUJgBeZnkZATqqy18MYHC18ZFvywryOUTR3VOho0x3Z7VIaRIWVkZv/3tb9tMt6eyloDPx6DeWfTJDjLvovOpqaogM+An4Pe1GTSMMSbRLHCkSEuBIxQK1U/XhSPsrw6RnxOsDxBLliwhPz+/0/JpjDFNpbpxvMe68cYb+fjjj5kwYQLBYJCsrCz69u3Lhx9+yLp167jgggv4dNNmKqoO8N3vXMs137wKaBg+paKigjlz5nDSSSfxxhtvMGzYMJ555hmys7NT/MmMMd2dBQ7gJ39bwwdb9yd0n+OH9uaWc49ucf3tt9/O6tWrWbVqFa+++ipnn302q1evZvTo0QDcf//97KrLIFRTzcVnn8b8eV+goKCg0T7Wr1/PwoUL+cMf/sC8efN44oknuOyyyxL6OYwxpikLHGli2rRp9UED4P9++SueePJJgn4fxcXFrF+//qDAMXr0aCZMmADA5MmT2bhxY2dm2RjTQ1nggFZLBp0lNze3fvrVV1/lxZde4i+LX2TSYYM5/fTTmu3hnpmZWT/t9/s5cOBAp+TVGNOzWeN4iuTl5VFe3vxTOPfs3UtuXm+GFPRh3bqPeOuttzo5d8YY0zIrcaRIQUEBJ554IscccwzZ2dkMGjSoft0JJ88id
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "BoQoU4BZ6tUW",
"colab_type": "code",
"colab": {}
},
"source": [
""
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "NPxdIIMiFuNo",
"colab_type": "text"
},
"source": [
"\n",
"## Two VGG blocks\n",
"\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Eg1bEN-WFz8b",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 482
},
"outputId": "747bb61a-0bcf-48e9-817f-a0655534e483"
},
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", activation='relu', input_shape = (32, 32, 3)),\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Conv2D(64, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.Conv2D(64, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(128, activation = 'relu'),\n",
" tf.keras.layers.Dense(10, activation = 'softmax')\n",
"])\n",
"\n",
"model.compile(optimizer = \"adam\", loss='categorical_crossentropy', metrics=['accuracy'])\n",
"\n",
"model.summary()\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Model: \"sequential_12\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d_2 (Conv2D) (None, 32, 32, 32) 896 \n",
"_________________________________________________________________\n",
"conv2d_3 (Conv2D) (None, 32, 32, 32) 9248 \n",
"_________________________________________________________________\n",
"max_pooling2d_1 (MaxPooling2 (None, 16, 16, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_4 (Conv2D) (None, 16, 16, 64) 18496 \n",
"_________________________________________________________________\n",
"conv2d_5 (Conv2D) (None, 16, 16, 64) 36928 \n",
"_________________________________________________________________\n",
"max_pooling2d_2 (MaxPooling2 (None, 8, 8, 64) 0 \n",
"_________________________________________________________________\n",
"flatten_1 (Flatten) (None, 4096) 0 \n",
"_________________________________________________________________\n",
"dense_24 (Dense) (None, 128) 524416 \n",
"_________________________________________________________________\n",
"dense_25 (Dense) (None, 10) 1290 \n",
"=================================================================\n",
"Total params: 591,274\n",
"Trainable params: 591,274\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "vDTNqcq_t43S",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 392
},
"outputId": "1e99e209-4b95-4b30-f1b6-9e180972565b"
},
"source": [
"history2 = model.fit(x_train,y_train,\n",
" batch_size = 32,\n",
" epochs=10,\n",
" validation_data=(x_valid, y_valid),\n",
" verbose=2\n",
"\n",
" )"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.210605). Check your callbacks.\n",
"1019/1019 - 14s - loss: 0.2516 - accuracy: 0.9158 - val_loss: 0.1399 - val_accuracy: 0.9559\n",
"Epoch 2/10\n",
"1019/1019 - 13s - loss: 0.0240 - accuracy: 0.9931 - val_loss: 0.0467 - val_accuracy: 0.9812\n",
"Epoch 3/10\n",
"1019/1019 - 13s - loss: 0.0077 - accuracy: 0.9973 - val_loss: 0.0338 - val_accuracy: 0.9900\n",
"Epoch 4/10\n",
"1019/1019 - 13s - loss: 3.4333e-05 - accuracy: 1.0000 - val_loss: 0.0311 - val_accuracy: 0.9900\n",
"Epoch 5/10\n",
"1019/1019 - 13s - loss: 1.1592e-05 - accuracy: 1.0000 - val_loss: 0.0367 - val_accuracy: 0.9900\n",
"Epoch 6/10\n",
"1019/1019 - 13s - loss: 5.8665e-06 - accuracy: 1.0000 - val_loss: 0.0342 - val_accuracy: 0.9903\n",
"Epoch 7/10\n",
"1019/1019 - 13s - loss: 2.9794e-06 - accuracy: 1.0000 - val_loss: 0.0304 - val_accuracy: 0.9903\n",
"Epoch 8/10\n",
"1019/1019 - 13s - loss: 1.6845e-06 - accuracy: 1.0000 - val_loss: 0.0328 - val_accuracy: 0.9906\n",
"Epoch 9/10\n",
"1019/1019 - 13s - loss: 9.3372e-07 - accuracy: 1.0000 - val_loss: 0.0308 - val_accuracy: 0.9905\n",
"Epoch 10/10\n",
"1019/1019 - 14s - loss: 5.2326e-07 - accuracy: 1.0000 - val_loss: 0.0337 - val_accuracy: 0.9907\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "tQgAdE38uDae",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "614a589a-a58d-4df1-8e8c-9593c4c7789f"
},
"source": [
"# evaluate zero-one loss\n",
"y_pred = model.predict(x_test)\n",
"zol = zero_one(y_pred, y_test)\n",
"\n",
"print(\"Zero-one Loss: \", zol)\n",
"\n",
"cnn_loss.append(zol)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Zero-one Loss: 0.00916590284142988\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8yLD8Y1wF8KK",
"colab_type": "text"
},
"source": [
"## Three VGG blocks"
]
},
{
"cell_type": "code",
"metadata": {
"id": "wEwt6F37F4_D",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 589
},
"outputId": "5860601b-ab4d-43ab-8a5a-5dfb6ece43ad"
},
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", activation='relu', input_shape = (32, 32, 3)),\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Conv2D(64, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.Conv2D(64, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Conv2D(128, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.Conv2D(128, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(128, activation = 'relu'),\n",
" tf.keras.layers.Dense(10, activation = 'softmax')\n",
"])\n",
"\n",
"model.compile(optimizer = \"adam\", loss='categorical_crossentropy', metrics=['accuracy'])\n",
"\n",
"model.summary()\n",
"\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Model: \"sequential_13\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d_6 (Conv2D) (None, 32, 32, 32) 896 \n",
"_________________________________________________________________\n",
"conv2d_7 (Conv2D) (None, 32, 32, 32) 9248 \n",
"_________________________________________________________________\n",
"max_pooling2d_3 (MaxPooling2 (None, 16, 16, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_8 (Conv2D) (None, 16, 16, 64) 18496 \n",
"_________________________________________________________________\n",
"conv2d_9 (Conv2D) (None, 16, 16, 64) 36928 \n",
"_________________________________________________________________\n",
"max_pooling2d_4 (MaxPooling2 (None, 8, 8, 64) 0 \n",
"_________________________________________________________________\n",
"conv2d_10 (Conv2D) (None, 8, 8, 128) 73856 \n",
"_________________________________________________________________\n",
"conv2d_11 (Conv2D) (None, 8, 8, 128) 147584 \n",
"_________________________________________________________________\n",
"max_pooling2d_5 (MaxPooling2 (None, 4, 4, 128) 0 \n",
"_________________________________________________________________\n",
"flatten_2 (Flatten) (None, 2048) 0 \n",
"_________________________________________________________________\n",
"dense_26 (Dense) (None, 128) 262272 \n",
"_________________________________________________________________\n",
"dense_27 (Dense) (None, 10) 1290 \n",
"=================================================================\n",
"Total params: 550,570\n",
"Trainable params: 550,570\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "OAywgc-Gt7EC",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 374
},
"outputId": "d7593dd0-543e-4417-e0e4-1e0be19ba341"
},
"source": [
"history3 = model.fit(x_train,y_train,\n",
" batch_size = 32,\n",
" epochs=10,\n",
" validation_data=(x_valid, y_valid),\n",
" verbose=2 \n",
"\n",
" )"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1019/1019 - 19s - loss: 0.3207 - accuracy: 0.8886 - val_loss: 0.0736 - val_accuracy: 0.9760\n",
"Epoch 2/10\n",
"1019/1019 - 18s - loss: 0.0340 - accuracy: 0.9895 - val_loss: 0.2608 - val_accuracy: 0.9379\n",
"Epoch 3/10\n",
"1019/1019 - 18s - loss: 0.0234 - accuracy: 0.9932 - val_loss: 0.0518 - val_accuracy: 0.9861\n",
"Epoch 4/10\n",
"1019/1019 - 18s - loss: 8.6958e-05 - accuracy: 1.0000 - val_loss: 0.0193 - val_accuracy: 0.9917\n",
"Epoch 5/10\n",
"1019/1019 - 18s - loss: 9.9077e-06 - accuracy: 1.0000 - val_loss: 0.0171 - val_accuracy: 0.9928\n",
"Epoch 6/10\n",
"1019/1019 - 18s - loss: 4.4342e-06 - accuracy: 1.0000 - val_loss: 0.0164 - val_accuracy: 0.9925\n",
"Epoch 7/10\n",
"1019/1019 - 18s - loss: 2.2970e-06 - accuracy: 1.0000 - val_loss: 0.0168 - val_accuracy: 0.9925\n",
"Epoch 8/10\n",
"1019/1019 - 18s - loss: 1.2628e-06 - accuracy: 1.0000 - val_loss: 0.0164 - val_accuracy: 0.9932\n",
"Epoch 9/10\n",
"1019/1019 - 18s - loss: 6.9922e-07 - accuracy: 1.0000 - val_loss: 0.0156 - val_accuracy: 0.9940\n",
"Epoch 10/10\n",
"1019/1019 - 18s - loss: 3.8787e-07 - accuracy: 1.0000 - val_loss: 0.0166 - val_accuracy: 0.9938\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "I8inLgtsuGdz",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "83821c38-f6d0-4bb9-eaa8-53df6ae416b8"
},
"source": [
"# evaluate zero-one loss\n",
"y_pred = model.predict(x_test)\n",
"\n",
"zol = zero_one(y_pred, y_test)\n",
"\n",
"print(\"Zero-one Loss: \", zol)\n",
"\n",
"cnn_loss.append(zol)\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Zero-one Loss: 0.00916590284142988\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pRX7_710GB43",
"colab_type": "text"
},
"source": [
"## Three VGG blocks with Dropout\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "78uoa45HGGHK",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 732
},
"outputId": "b292579e-dfa9-4d19-8787-2a25edd23832"
},
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", activation='relu', input_shape = (32, 32, 3)),\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Dropout(0.2),\n",
" tf.keras.layers.Conv2D(64, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.Conv2D(64, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Dropout(0.2),\n",
" tf.keras.layers.Conv2D(128, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.Conv2D(128, (3, 3), padding = \"same\", activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Dropout(0.2),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(128, activation = 'relu'),\n",
" tf.keras.layers.Dropout(0.2),\n",
" tf.keras.layers.Dense(10, activation = 'softmax')\n",
"])\n",
"\n",
"model.compile(optimizer = \"adam\", loss='categorical_crossentropy', metrics=['accuracy'])\n",
"\n",
"\n",
"model.summary()\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Model: \"sequential_14\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d_12 (Conv2D) (None, 32, 32, 32) 896 \n",
"_________________________________________________________________\n",
"conv2d_13 (Conv2D) (None, 32, 32, 32) 9248 \n",
"_________________________________________________________________\n",
"max_pooling2d_6 (MaxPooling2 (None, 16, 16, 32) 0 \n",
"_________________________________________________________________\n",
"dropout (Dropout) (None, 16, 16, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_14 (Conv2D) (None, 16, 16, 64) 18496 \n",
"_________________________________________________________________\n",
"conv2d_15 (Conv2D) (None, 16, 16, 64) 36928 \n",
"_________________________________________________________________\n",
"max_pooling2d_7 (MaxPooling2 (None, 8, 8, 64) 0 \n",
"_________________________________________________________________\n",
"dropout_1 (Dropout) (None, 8, 8, 64) 0 \n",
"_________________________________________________________________\n",
"conv2d_16 (Conv2D) (None, 8, 8, 128) 73856 \n",
"_________________________________________________________________\n",
"conv2d_17 (Conv2D) (None, 8, 8, 128) 147584 \n",
"_________________________________________________________________\n",
"max_pooling2d_8 (MaxPooling2 (None, 4, 4, 128) 0 \n",
"_________________________________________________________________\n",
"dropout_2 (Dropout) (None, 4, 4, 128) 0 \n",
"_________________________________________________________________\n",
"flatten_3 (Flatten) (None, 2048) 0 \n",
"_________________________________________________________________\n",
"dense_28 (Dense) (None, 128) 262272 \n",
"_________________________________________________________________\n",
"dropout_3 (Dropout) (None, 128) 0 \n",
"_________________________________________________________________\n",
"dense_29 (Dense) (None, 10) 1290 \n",
"=================================================================\n",
"Total params: 550,570\n",
"Trainable params: 550,570\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "TEQPF4Hmt9JA",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 392
},
"outputId": "ccd411fe-3171-43d6-9d56-8eeb97a599ec"
},
"source": [
"history4 = model.fit(x_train,y_train,\n",
" batch_size = 32,\n",
" epochs=10,\n",
" validation_data=(x_valid, y_valid),\n",
" verbose=2\n",
" )"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.151422). Check your callbacks.\n",
"1019/1019 - 20s - loss: 0.4977 - accuracy: 0.8243 - val_loss: 0.1280 - val_accuracy: 0.9546\n",
"Epoch 2/10\n",
"1019/1019 - 19s - loss: 0.0688 - accuracy: 0.9775 - val_loss: 0.1614 - val_accuracy: 0.9452\n",
"Epoch 3/10\n",
"1019/1019 - 19s - loss: 0.0458 - accuracy: 0.9857 - val_loss: 0.0644 - val_accuracy: 0.9795\n",
"Epoch 4/10\n",
"1019/1019 - 19s - loss: 0.0364 - accuracy: 0.9886 - val_loss: 0.0873 - val_accuracy: 0.9727\n",
"Epoch 5/10\n",
"1019/1019 - 19s - loss: 0.0243 - accuracy: 0.9931 - val_loss: 0.1079 - val_accuracy: 0.9783\n",
"Epoch 6/10\n",
"1019/1019 - 19s - loss: 0.0328 - accuracy: 0.9902 - val_loss: 0.0799 - val_accuracy: 0.9782\n",
"Epoch 7/10\n",
"1019/1019 - 20s - loss: 0.0262 - accuracy: 0.9922 - val_loss: 0.2361 - val_accuracy: 0.9506\n",
"Epoch 8/10\n",
"1019/1019 - 19s - loss: 0.0189 - accuracy: 0.9949 - val_loss: 0.1377 - val_accuracy: 0.9713\n",
"Epoch 9/10\n",
"1019/1019 - 19s - loss: 0.0246 - accuracy: 0.9928 - val_loss: 0.0759 - val_accuracy: 0.9793\n",
"Epoch 10/10\n",
"1019/1019 - 19s - loss: 0.0190 - accuracy: 0.9952 - val_loss: 0.3236 - val_accuracy: 0.9211\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "hq0aCW3iuIJV",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "44cd30ee-ed2c-44ed-d3b0-7a31cb5de5b8"
},
"source": [
"# evaluate zero-one loss\n",
"y_pred = model.predict(x_test)\n",
"zol = zero_one(y_pred, y_test)\n",
"\n",
"print(\"Zero-one Loss: \", zol)\n",
"\n",
"cnn_loss.append(zol)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Zero-one Loss: 0.07561869844179651\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zKK0HS_eGLZy",
"colab_type": "text"
},
"source": [
"## Three VGG blocks with Dropout and Batch Normalization\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "fpUsbm4dGLmF",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "1a2d68e2-2727-4805-fdd7-96b2aa58957c"
},
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", use_bias=False, input_shape = (32, 32, 3)),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation('relu'),\n",
" tf.keras.layers.Conv2D(32, (3, 3), padding = \"same\", use_bias=False),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation('relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Dropout(0.2),\n",
" tf.keras.layers.Conv2D(64, (3, 3), padding = \"same\", use_bias=False),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation('relu'),\n",
" tf.keras.layers.Conv2D(64, (3, 3), padding = \"same\", use_bias=False),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation('relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Dropout(0.3),\n",
" tf.keras.layers.Conv2D(128, (3, 3), padding = \"same\", use_bias=False),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation('relu'),\n",
" tf.keras.layers.Conv2D(128, (3, 3), padding = \"same\", use_bias=False),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation('relu'),\n",
" tf.keras.layers.MaxPooling2D(pool_size = (2, 2)),\n",
" tf.keras.layers.Dropout(0.4),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(128, use_bias=False),\n",
" tf.keras.layers.BatchNormalization(),\n",
" tf.keras.layers.Activation('relu'),\n",
" tf.keras.layers.Dropout(0.5),\n",
" tf.keras.layers.Dense(10, activation = 'softmax')\n",
"])\n",
"\n",
"model.compile(optimizer = \"adam\", loss='categorical_crossentropy', metrics=['accuracy'])\n",
"\n",
"model.summary()\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Model: \"sequential_15\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d_18 (Conv2D) (None, 32, 32, 32) 864 \n",
"_________________________________________________________________\n",
"batch_normalization (BatchNo (None, 32, 32, 32) 128 \n",
"_________________________________________________________________\n",
"activation (Activation) (None, 32, 32, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_19 (Conv2D) (None, 32, 32, 32) 9216 \n",
"_________________________________________________________________\n",
"batch_normalization_1 (Batch (None, 32, 32, 32) 128 \n",
"_________________________________________________________________\n",
"activation_1 (Activation) (None, 32, 32, 32) 0 \n",
"_________________________________________________________________\n",
"max_pooling2d_9 (MaxPooling2 (None, 16, 16, 32) 0 \n",
"_________________________________________________________________\n",
"dropout_4 (Dropout) (None, 16, 16, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_20 (Conv2D) (None, 16, 16, 64) 18432 \n",
"_________________________________________________________________\n",
"batch_normalization_2 (Batch (None, 16, 16, 64) 256 \n",
"_________________________________________________________________\n",
"activation_2 (Activation) (None, 16, 16, 64) 0 \n",
"_________________________________________________________________\n",
"conv2d_21 (Conv2D) (None, 16, 16, 64) 36864 \n",
"_________________________________________________________________\n",
"batch_normalization_3 (Batch (None, 16, 16, 64) 256 \n",
"_________________________________________________________________\n",
"activation_3 (Activation) (None, 16, 16, 64) 0 \n",
"_________________________________________________________________\n",
"max_pooling2d_10 (MaxPooling (None, 8, 8, 64) 0 \n",
"_________________________________________________________________\n",
"dropout_5 (Dropout) (None, 8, 8, 64) 0 \n",
"_________________________________________________________________\n",
"conv2d_22 (Conv2D) (None, 8, 8, 128) 73728 \n",
"_________________________________________________________________\n",
"batch_normalization_4 (Batch (None, 8, 8, 128) 512 \n",
"_________________________________________________________________\n",
"activation_4 (Activation) (None, 8, 8, 128) 0 \n",
"_________________________________________________________________\n",
"conv2d_23 (Conv2D) (None, 8, 8, 128) 147456 \n",
"_________________________________________________________________\n",
"batch_normalization_5 (Batch (None, 8, 8, 128) 512 \n",
"_________________________________________________________________\n",
"activation_5 (Activation) (None, 8, 8, 128) 0 \n",
"_________________________________________________________________\n",
"max_pooling2d_11 (MaxPooling (None, 4, 4, 128) 0 \n",
"_________________________________________________________________\n",
"dropout_6 (Dropout) (None, 4, 4, 128) 0 \n",
"_________________________________________________________________\n",
"flatten_4 (Flatten) (None, 2048) 0 \n",
"_________________________________________________________________\n",
"dense_30 (Dense) (None, 128) 262144 \n",
"_________________________________________________________________\n",
"batch_normalization_6 (Batch (None, 128) 512 \n",
"_________________________________________________________________\n",
"activation_6 (Activation) (None, 128) 0 \n",
"_________________________________________________________________\n",
"dropout_7 (Dropout) (None, 128) 0 \n",
"_________________________________________________________________\n",
"dense_31 (Dense) (None, 10) 1290 \n",
"=================================================================\n",
"Total params: 552,298\n",
"Trainable params: 551,146\n",
"Non-trainable params: 1,152\n",
"_________________________________________________________________\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "5fCia4UgWeLD",
"colab_type": "code",
"colab": {}
},
"source": [
""
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "L99mDAP9t-T1",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 374
},
"outputId": "f75668b9-9120-4db3-aa16-b9b8cdadf17a"
},
"source": [
"history5 = model.fit(x_train,y_train,\n",
" batch_size = 32,\n",
" epochs=10,\n",
" validation_data=(x_valid, y_valid),\n",
" verbose=2\n",
" )"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"1019/1019 - 22s - loss: 0.3911 - accuracy: 0.8740 - val_loss: 0.1592 - val_accuracy: 0.9462\n",
"Epoch 2/10\n",
"1019/1019 - 21s - loss: 0.0870 - accuracy: 0.9735 - val_loss: 0.0608 - val_accuracy: 0.9725\n",
"Epoch 3/10\n",
"1019/1019 - 21s - loss: 0.0421 - accuracy: 0.9875 - val_loss: 0.0188 - val_accuracy: 0.9944\n",
"Epoch 4/10\n",
"1019/1019 - 21s - loss: 0.0428 - accuracy: 0.9876 - val_loss: 0.0137 - val_accuracy: 0.9958\n",
"Epoch 5/10\n",
"1019/1019 - 21s - loss: 0.0259 - accuracy: 0.9927 - val_loss: 0.0061 - val_accuracy: 0.9978\n",
"Epoch 6/10\n",
"1019/1019 - 22s - loss: 0.0252 - accuracy: 0.9924 - val_loss: 0.0168 - val_accuracy: 0.9929\n",
"Epoch 7/10\n",
"1019/1019 - 21s - loss: 0.0234 - accuracy: 0.9928 - val_loss: 0.0430 - val_accuracy: 0.9826\n",
"Epoch 8/10\n",
"1019/1019 - 21s - loss: 0.0239 - accuracy: 0.9925 - val_loss: 0.0387 - val_accuracy: 0.9829\n",
"Epoch 9/10\n",
"1019/1019 - 21s - loss: 0.0169 - accuracy: 0.9948 - val_loss: 0.0207 - val_accuracy: 0.9916\n",
"Epoch 10/10\n",
"1019/1019 - 21s - loss: 0.0140 - accuracy: 0.9956 - val_loss: 0.0163 - val_accuracy: 0.9936\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "MqZKRNxuuKbI",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "7440c448-0d61-4b33-e3fa-a089164dd253"
},
"source": [
"# evaluate zero-one loss\n",
"y_pred = model.predict(x_test)\n",
"zol = zero_one(y_pred, y_test)\n",
"\n",
"print(\"Zero-one Loss: \", zol)\n",
"\n",
"cnn_loss.append(zol)\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Zero-one Loss: 0.005957836846929423\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Gp8Gq3QYl2--",
"colab_type": "text"
},
"source": [
"## VGG CNN results"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nasekeAkl8Lw",
"colab_type": "text"
},
"source": [
"**One VGG block**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "u4nDL4ikucM0",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "fdffd0b1-f5e9-4b69-f75b-00b4e0a7116a"
},
"source": [
"\n",
"plt.figure(1) \n",
" \n",
" \n",
"plt.subplot(211) \n",
"plt.plot(history.history['accuracy']) \n",
"plt.plot(history.history['val_accuracy']) \n",
"plt.title('model accuracy') \n",
"plt.ylabel('accuracy') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
" \n",
" \n",
"plt.subplot(212) \n",
"plt.plot(history.history['loss']) \n",
"plt.plot(history.history['val_loss']) \n",
"plt.title('model loss') \n",
"plt.ylabel('loss') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
"plt.show()\n",
"\n",
"\n",
"\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU1d348c93lqwEAmEnbCqguLELVdzQCrjbFtFqaze02l/tU+ujdtG2z2P1edrHWltra93a2oK70hZ3sdq6QEBUEAVUIGENSyALWWbm+/vj3EkmIcskzGQmyff9es1r7nLuvWduJvc755x7zhVVxRhjjImXL9UZMMYY07VY4DDGGNMuFjiMMca0iwUOY4wx7WKBwxhjTLtY4DDGGNMuFjiMaYWIPCQi/x1n2o0ickay82RMqlngMMYY0y4WOIzpAUQkkOo8mO7DAofp8rwqoutF5D0RqRSR+0VkkIg8KyLlIvKSiPSNSX+eiKwRkTIReVVEjopZN1FEVnrbPQJkNTnWOSKyytv2DRE5Ls48ni0i74jIfhEpFpEfN1l/kre/Mm/9Fd7ybBH5PxHZJCL7RORf3rJTRaSkmfNwhjf9YxF5XEQeFpH9wBUiMk1E3vSOsU1EfiMiGTHbHy0iL4rIHhHZISLfF5HBIlIlIgUx6SaJSKmIBOP57Kb7scBhuovPAWcCY4FzgWeB7wMDcN/zbwOIyFhgIfAdb90S4G8ikuFdRJ8G/gz0Ax7z9ou37UTgAeBKoAD4PbBYRDLjyF8l8CUgHzgb+KaIXODtd6SX3197eZoArPK2+wUwGfiMl6f/BCJxnpPzgce9Y/4FCAP/AfQHZgCzgKu9POQBLwHPAUOBI4CXVXU78CowL2a/lwOLVLUuznyYbsYCh+kufq2qO1R1C/A68LaqvqOq1cBTwEQv3cXAP1T1Re/C9wsgG3dhng4EgTtVtU5VHweWxxxjAfB7VX1bVcOq+kegxtuuVar6qqq+r6oRVX0PF7xO8VZfCrykqgu94+5W1VUi4gO+Clyrqlu8Y76hqjVxnpM3VfVp75gHVHWFqr6lqiFV3YgLfNE8nANsV9X/U9VqVS1X1be9dX8ELgMQET9wCS64mh7KAofpLnbETB9oZr6XNz0U2BRdoaoRoBgY5q3boo1H/twUMz0SuM6r6ikTkTJguLddq0TkBBFZ6lXx7AOuwv3yx9vHx81s1h9XVdbcungUN8nDWBH5u4hs96qvfhZHHgCeAcaLyGhcqW6fqi7rYJ5MN2CBw/Q0W3EBAAAREdxFcwuwDRjmLYsaETNdDNyqqvkxrxxVXRjHcf8KLAaGq2of4HdA9DjFwOHNbLMLqG5hXSWQE/M5/LhqrlhNh76+B/gQGKOqvXFVebF5OKy5jHultkdxpY7LsdJGj2eBw/Q0jwJni8gsr3H3Olx10xvAm0AI+LaIBEXkImBazLZ/AK7ySg8iIrleo3deHMfNA/aoarWITMNVT0X9BThDROaJSEBECkRkglcaegC4Q0SGiohfRGZ4bSrrgCzv+EHgh0BbbS15wH6gQkSOBL4Zs+7vwBAR+Y6IZIpInoicELP+T8AVwHlY4OjxLHCYHkVVP8L9cv417hf9ucC5qlqrqrXARbgL5B5ce8iTMdsWAd8AfgPsBTZ4aeNxNfBTESkHbsYFsOh+NwNzcUFsD65h/Hhv9feA93FtLXuA/wF8qrrP2+d9uNJSJdDoLqtmfA8XsMpxQfCRmDyU46qhzgW2A+uB02LW/xvXKL9SVWOr70wPJPYgJ2NMPETkFeCvqnpfqvNiUssChzGmTSIyFXgR10ZTnur8mNSyqipjTKtE5I+4Ph7fsaBhwEocxhhj2slKHMYYY9qlRwx81r9/fx01alSqs2GMMV3KihUrdqlq0/5ByQ0cIvIAbiiDnap6TDPrBfgV7lbEKuAKVV3prfsy7t50gP/2hndARCYDD+GGiViCG46h1fq2UaNGUVRUlJDPZIwxPYWINHvrdbKrqh4CZreyfg4wxnstwPVsRUT6AbcAJ+A6YN0SM7rpPbh76aPbtbZ/Y4wxCZbUwKGqr+E6LbXkfOBP6rwF5IvIEOAs4EVV3aOqe3G3Ac721vX2BmpTXG/WC5L5GYwxxjSW6jaOYTQeiK3EW9ba8pJmlh9ERBbgSjGMGDGiuSSmC1FVasMRakIR6kIRfCL4RBAf3jT1y+qnfdL2jtOEqhJRCEeUiLpXOKJEIhD25iMRJRyzPKJuPhJRQhG33L1HCIVj5lUJh2PTRGLSeu/hSJN9uPdI7D4bpT94eTiiaMxnib5HvJpk9xlAaViv3vr6+UbpovtoSKdKQ5rovDaebyp25DFBml3u1sVu0/J3p9H+DtrHwfuX+nlpmG9tnbdtdF8HHaO5tPX7a7yNAL+5dBLD++WQSKkOHEmjqvcC9wJMmTLloK9TXV0dJSUlVFdXd3reOlNWVhaFhYUEgx1/5k4kolTWhiivDlFdF3YX8LoIteEItSH3qgmFqfGmm66vCYXr00XX1TTZtrnt3LqG7ToiNqCIgN/XMO0T8ebdP2Oj4ONrsp0cvJ0I9RfM6IVelZiLuxcMmlz01QsQ0eXRC2c68vvcZw00evc1zPvdu7/+nETPJ975cpcyX/08MWl89ee51XS4v0dDmoa/mUjMPNLoIgo0CiSNppuM/9h4Ha2sazlh7Gy02VXr5xvSNF1H/TqtT6vNLGtuf6DNrGu8f38SfkClOnBswY1MGlXoLdsCnNpk+ave8sJm0rdbSUkJeXl5jBo1qtVfF12ZqrJ79242by4mb+BQyqtD7D9QR3m1CwL7q+sor65j/4EQ5dV19cv2N0pXR3lNqNlfcvHyCWQG/GQEfGQEfGR67xl+N50Z8JOTESC/yboMb13sdpkBd9EC6n/NRi/W9b9MoxdlbfjFGvGW109rw4U99hdvOHLwdmFVNx9p+GUb3c7vlWyiF06fF4hil0cvjtGgFQ1WviYXXDfdsNxXvw8a0tTv00vrpQv6m1zQvbw0zPsaXegPWu4T/P6G5bH5MqapVAeOxcC3RGQRriF8n6puE5HngZ/FNIh/FrhJVfeIe/TmdOBt3BPVft2RA1dXV3eZoBFRJRSOeL9uY37RakMVQewv2rB3kXPVCgG279jLgj+sbXH/PoFemQF6ZwfJywrSOytAYd8cemdFlwXonRWkV1aA7KC/4aIfPPgCn9lMcAj4rbuQMd1Jsm/HXYgrOfT3no98C+4Ja6jq73C3087FjTJaBXzFW7dHRP6Lhqev/VRVo43sV9NwO+6z3quj+evopp1GVVm/o4KaULjFNL4mv0QDPh9+v/ul6/cJ1dlB/uv8o+uDgAsOXkDIDpKb4e8S58IYkx6SGjhU9ZI21itwTQvrHsA9i6Dp8iLgoD4h3VVFTYiaUJgBeZnkZATqqy18MYHC18ZFvywryOUTR3VOho0x3Z7VIaRIWVkZv/3tb9tMt6eyloDPx6DeWfTJDjLvovOpqaogM+An4Pe1GTSMMSbRLHCkSEuBIxQK1U/XhSPsrw6RnxOsDxBLliwhPz+/0/JpjDFNpbpxvMe68cYb+fjjj5kwYQLBYJCsrCz69u3Lhx9+yLp167jgggv4dNNmKqoO8N3vXMs137wKaBg+paKigjlz5nDSSSfxxhtvMGzYMJ555hmys7NT/MmMMd2dBQ7gJ39bwwdb9yd0n+OH9uaWc49ucf3tt9/O6tWrWbVqFa+++ipnn302q1evZvTo0QDcf//97KrLIFRTzcVnn8b8eV+goKCg0T7Wr1/PwoUL+cMf/sC8efN44oknuOyyyxL6OYwxpikLHGli2rRp9UED4P9++SueePJJgn4fxcXFrF+//qDAMXr0aCZMmADA5MmT2bhxY2dm2RjTQ1nggFZLBp0lNze3fvrVV1/lxZde4i+LX2TSYYM5/fTTmu3hnpmZWT/t9/s5cOBAp+TVGNOzWeN4iuTl5VFe3vxTOPfs3UtuXm+GFPRh3bqPeOuttzo5d8YY0zIrcaRIQUEBJ554IscccwzZ2dkMGjSoft0JJ88id
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Exhe_XRUmHOZ",
"colab_type": "text"
},
"source": [
"**Two VGG block**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "7o6favwguecD",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "880218d6-c9e9-4826-afd0-a687cd27416e"
},
"source": [
"\n",
"plt.figure(1) \n",
" \n",
" \n",
"plt.subplot(211) \n",
"plt.plot(history2.history['accuracy']) \n",
"plt.plot(history2.history['val_accuracy']) \n",
"plt.title('model accuracy') \n",
"plt.ylabel('accuracy') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
" \n",
" \n",
"plt.subplot(212) \n",
"plt.plot(history2.history['loss']) \n",
"plt.plot(history2.history['val_loss']) \n",
"plt.title('model loss') \n",
"plt.ylabel('loss') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
"plt.show()"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3gU5dn48e+9m92cISQBFAKEIqKoCAU5eKqIVjxrPSse2r7FvtXW9lWqtmpbf6+tfdtaa6tWq9QDilo80YqKKB5aRQREBTmqQBLOCYEcSDbZ3L8/ZhI2IYdN2M1skvtzXXtldmaemXsnydw7zzPzPKKqGGOMMdHyeR2AMcaYrsUShzHGmHaxxGGMMaZdLHEYY4xpF0scxhhj2sUShzHGmHaxxGFMK0TkMRH53yjX3SAip8Q7JmO8ZonDGGNMu1jiMKYHEJEkr2Mw3YclDtPluVVEM0TkUxGpEJFHRaS/iLwqImUiskBE+kSsf46IrBSRUhF5W0QOj1g2RkSWueWeBVKa7OssEVnuln1fREZFGeOZIvKxiOwRkQIR+WWT5ce72yt1l1/jzk8VkT+IyEYR2S0i/3bnnSQihc0ch1Pc6V+KyBwRmSUie4BrRGS8iHzg7mOLiPxFRIIR5Y8QkTdEpEREtonIz0TkIBGpFJGciPW+LiI7RCQQzWc33Y8lDtNdXACcChwKnA28CvwM6Ivzd/4jABE5FJgN/NhdNg/4p4gE3ZPoS8CTQDbwD3e7uGXHADOBa4Ec4CFgrogkRxFfBXAVkAWcCfy3iJznbneIG++f3ZhGA8vdcr8HxgLHujH9FKiL8picC8xx9/kUEAZ+AuQCk4ApwA/cGDKBBcBrwADgEOBNVd0KvA1cHLHdK4FnVLUmyjhMN2OJw3QXf1bVbapaBLwHfKiqH6tqFfAiMMZd7xLgFVV9wz3x/R5IxTkxTwQCwL2qWqOqc4CPIvYxHXhIVT9U1bCqPg5Uu+Vapapvq+pnqlqnqp/iJK9vuIsvBxao6mx3v8WqulxEfMB3gBtUtcjd5/uqWh3lMflAVV9y97lXVZeq6iJVrVXVDTiJrz6Gs4CtqvoHVa1S1TJV/dBd9jgwDUBE/MBlOMnV9FCWOEx3sS1iem8z7zPc6QHAxvoFqloHFAAD3WVF2rjnz40R00OAG92qnlIRKQUGueVaJSITRGShW8WzG/g+zjd/3G180UyxXJyqsuaWRaOgSQyHisi/RGSrW3316yhiAHgZGCkiQ3Gu6nar6uIOxmS6AUscpqfZjJMAABARwTlpFgFbgIHuvHqDI6YLgLtUNSvilaaqs6PY79PAXGCQqvYG/grU76cAGNZMmZ1AVQvLKoC0iM/hx6nmitS06+sHgdXAcFXthVOVFxnD15oL3L1qew7nquNK7Gqjx7PEYXqa54AzRWSK27h7I0510/vAB0At8CMRCYjIt4DxEWX/BnzfvXoQEUl3G70zo9hvJlCiqlUiMh6neqreU8ApInKxiCSJSI6IjHavhmYC94jIABHxi8gkt01lLZDi7j8A3Aa01daSCewBykXkMOC/I5b9CzhYRH4sIskikikiEyKWPwFcA5yDJY4ezxKH6VFUdQ3ON+c/43yjPxs4W1VDqhoCvoVzgizBaQ95IaLsEuB7wF+AXcB6d91o/AC4U0TKgDtwElj9djcBZ+AksRKchvGj3cU3AZ/htLWUAL8FfKq6293mIzhXSxVAo7usmnETTsIqw0mCz0bEUIZTDXU2sBVYB0yOWP4fnEb5ZaoaWX1neiCxgZyMMdEQkbeAp1X1Ea9jMd6yxGGMaZOIHAO8gdNGU+Z1PMZbVlVljGmViDyO84zHjy1pGLArDmOMMe1kVxzGGGPapUd0fJabm6v5+fleh2GMMV3K0qVLd6pq0+eDekbiyM/PZ8mSJV6HYYwxXYqINHvrdVyrqkRkpohsF5EVLSwXEblPRNaL07Pp1yOWXS0i69zX1RHzx4rIZ26Z+5o85WuMMSbO4t3G8RgwtZXlpwPD3dd0nC4REJFs4BfABJwnd38h+7rFfhDnIaz6cq1t3xhjTIzFtapKVd8VkfxWVjkXeMLtVG6RiGSJyMHAScAbqloCICJvAFNF5G2gl6oucuc/AZyH0yW16cJUlfLqWkoqQuwsD1FcXk1xRch9X01xeYjiCudnaWUNdXY3oDFRee7aSeTnpsd0m163cQykcQ+ehe681uYXNjN/PyIyHecqhsGDB++3vKamhsLCQqqqqg4g/MSXkpJCXl4egUDnj7lTVROmuGJfEiiOSAg7y6spiZi3syJEqLb5YSYyk5PIzgiSkx5kUHYao/IC+H1WQ2lMNNKC/phv0+vEETeq+jDwMMC4ceP2+3paWFhIZmYm+fn5dNdmElWluLiYwsJChg4desDbqw3XUVJZf7LfdwVQ/3OnO12fEMqra5vdTjDJR9+MZHIyguRkBDm0fya57nROejLZGUFy053l2elBUgKx/8M3xnSc14mjCKdL63p57rwinOqqyPlvu/Pzmlm/3aqqqrp10gAQEXJyctixY0dU66sqxRUhCkoq2VRSSUFJJQUle9nkvt+yey91zdQQ+X1CdrpzRZCbkczg7DSy3emc9CA59UnCnU4P+rv1cTemu/M6ccwFrheRZ3Aawner6hYReR34dUSD+DeBW1W1RJwxmycCH+IMxfnnju68J5y8mn7GqpqwkxB2VbKpuJJNJXsp2FXZkCwqQ+FG6/fNTGZQn1SOye/DoOyB9MtMdhKBmwRyM4L0Sgngs6ojY3qMuCYOEZmNc+WQKyKFOHdKBQBU9a844z2fgdM9dSXwbXdZiYj8P/YN23lnfUM5TlfSj+EM9/kq1jDeiKpSE1ZC4TpCtXWEwnWUVIS44MH3KSipZHtZ41FHUwN+BmenMSg7lUnDchicnea+TyOvTyppQa+/WxhjEk2876q6rI3lClzXwrKZOIPYNJ2/BDgyJgF6qLS0lKeffpof/OAH7Sp3xhln8OSsWaRl9NqXHGobJ4rI/scECNXWEfALJ43oy6A+aQzOcRLDoD5p5GYEe8SVlzEmduzrpEdKS0t54IEH9ksctbW1JCU1/rWUVobYvbeGULiO3z0ym6JKhcryhuV+nxD0+0gJ+OiVmkTQ7yOY5CPo9xFI8rGmLIVnpo/plM9ljOn+LHF45JZbbuGLL75g9OjRBAIBUlJS6NOnD6tXr2bt2rWcd955FBQUsLeqiouums6lV32blICfkyceyRvv/ofQ3kouPv9cjjv+OBZ98AEDBw7k5ZdfJjU11euPZozp5ixxAL/650o+37wnptscOaAXvzj7iBaX33333axYsYLly5fz9ttvc+aZZ7JixYqG22ZnzpxJdnY2G7bt4pQTj+Xaqy9jQG4//D4hNyOFcmpZv34dzzwzm0cfeYSLL76Y559/nmnTpsX0cxhjTFOWOBLE+PHjGz1rcd999/Hiiy9SXVvHti2b2bThSwYc1K9RmaFDhzJ69GgAxo4dy4YNG+IXYLgGNi+Hmor47aOrUQWt2/eT+vfNvTRi/SavRuWartNcGW1SNhGeoq//fGGoC++Lry7cZF4Y6tyf+y2PLB+5vK5J+frl2vz2ARBoaLurn47iJ0TMi6ZsK+u0+ftu63fd9Pfczr+r+nLXfQR9D43pb9sSB7R6ZdBZ0tP3dQnw9ttvs2DBAl5761127FV+cPm5zT7hnpyc3DDt9/vZu3dvbIMq+Qq+eBPWvwVfvQshG/wtMSXIzQ3ic14+P4jf/SkR0/6I5RE/Gy2XJuXd5UnBJuVb2T4AbjJVZV9ybesn7SvT4rrufJ8PJLDvuIjPjbeZ9zSdH/mi5WUN5VrYrvggLTvmv2pLHB7JzMykrKz5E/Hu3bvp06cPe+v8FHy5miWLP+ycoKrLYcO/3WTxJpR84czPGgxHXQjDJkP6fl3z92z1J7tm/3mb/nO38Q/edH6rJ5P6sgmSNEyPYonDIzk5ORx33HEceeSRpKam0r9//4ZlU6dO5YEHH+SUY8cyYsShTJw4MT5BqMK2FU6S+OJN2LQIwiEIp
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uO42DrCnmJ7o",
"colab_type": "text"
},
"source": [
"**Three VGG block**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "ENoCopkvufEv",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "d5b9cb2e-7217-43eb-8636-908d0dd3e692"
},
"source": [
"\n",
"plt.figure(1) \n",
" \n",
" \n",
"plt.subplot(211) \n",
"plt.plot(history3.history['accuracy']) \n",
"plt.plot(history3.history['val_accuracy']) \n",
"plt.title('model accuracy') \n",
"plt.ylabel('accuracy') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
" \n",
" \n",
"plt.subplot(212) \n",
"plt.plot(history3.history['loss']) \n",
"plt.plot(history3.history['val_loss']) \n",
"plt.title('model loss') \n",
"plt.ylabel('loss') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
"plt.show()"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhU5dn48e89k8keQkgATQIEERVEBUUEl4prwQ211qVCtfYt7U9t9a21amu1te1b+7b1dWlrtZW64oZaqaIiCtZWRAFRQVAWWcIaAtn35P79cU6SSZgkk2QmJ8ncn+uaa86c7blzkpx7zvOc8zyiqhhjjDGt+bwOwBhjTO9kCcIYY0xIliCMMcaEZAnCGGNMSJYgjDHGhGQJwhhjTEiWIIwBRORREflVmOtuFpEzox2TMV6zBGGMMSYkSxDG9CMiEud1DKb/sARh+gy3audmEflERMpF5BERGSoir4lIqYgsEpGMoPUvEJE1IlIkIktEZEzQsgkistLd7lkgsVVZ54nIKnfb90Tk6DBjPFdEPhKREhHZJiI/b7X8ZHd/Re7yq935SSLyBxHZIiLFIvJvd95UEckPcRzOdKd/LiLzRORJESkBrhaRSSKy1C1jp4j8UUTig7Y/UkTeFJF9IrJbRH4iIgeJSIWIZAatd6yIFIhIIJyf3fQ/liBMX/M14CzgMOB84DXgJ8BgnL/nHwCIyGHA08CN7rIFwD9FJN49Wf4DeAIYBDzv7hd32wnAHOC7QCbwEDBfRBLCiK8c+CYwEDgX+H8icqG73xFuvA+4MY0HVrnb/R44DjjRjenHQEOYx2QGMM8t8ymgHvhvIAuYApwBXOvGkAYsAl4HsoFDgbdUdRewBLg0aL+zgGdUtTbMOEw/YwnC9DUPqOpuVd0OvAssU9WPVLUKeAmY4K53GfCqqr7pnuB+DyThnIAnAwHgXlWtVdV5wIdBZcwGHlLVZapar6qPAdXudu1S1SWq+qmqNqjqJzhJ6lR38TeARar6tFtuoaquEhEfcA1wg6pud8t8T1WrwzwmS1X1H26Zlaq6QlXfV9U6Vd2Mk+AaYzgP2KWqf1DVKlUtVdVl7rLHgJkAIuIHrsBJoiZGWYIwfc3uoOnKEJ9T3elsYEvjAlVtALYBOe6y7dqyp8otQdMjgJvcKpoiESkChrnbtUtEThCRxW7VTDHwPZxv8rj72BhisyycKq5Qy8KxrVUMh4nIKyKyy612+p8wYgB4GRgrIiNxrtKKVfWDLsZk+gFLEKa/2oFzogdARATn5Lgd2AnkuPMaDQ+a3gb8WlUHBr2SVfXpMMqdC8wHhqlqOvAXoLGcbcCoENvsBaraWFYOJAf9HH6c6qlgrbtkfhBYB4xW1QE4VXDBMRwSKnD3Kuw5nKuIWdjVQ8yzBGH6q+eAc0XkDLeR9SacaqL3gKVAHfADEQmIyMXApKBt/wp8z70aEBFJcRuf08IoNw3Yp6pVIjIJp1qp0VPAmSJyqYjEiUimiIx3r27mAPeISLaI+EVkitvm8QWQ6JYfAG4HOmoLSQNKgDIROQL4f0HLXgEOFpEbRSRBRNJE5ISg5Y8DVwMXYAki5lmCMP2Sqn6O8034AZxv6OcD56tqjarWABfjnAj34bRXvBi07XLgO8Afgf3ABnfdcFwL3CUipcAdOImqcb9bgXNwktU+nAbqY9zFPwI+xWkL2Qf8FvCparG7z7/hXP2UAy3uagrhRziJqRQn2T0bFEMpTvXR+cAuYD1wWtDy/+A0jq9U1eBqNxODxAYMMsYEE5G3gbmq+jevYzHesgRhjGkiIscDb+K0oZR6HY/xllUxGWMAEJHHcJ6RuNGSgwG7gjDGGNMGu4IwxhgTUr/p2CsrK0vz8vK8DsMYY/qUFStW7FXV1s/WAP0oQeTl5bF8+XKvwzDGmD5FRNq8nTlqVUwiMkdE9ojI6jaWi4jcLyIbxOmd89igZVeJyHr3dVW0YjTGGNO2aLZBPApMa2f5dGC0+5qN0z0AIjIIuBM4Aefp1jslqAtnY4wxPSNqVUyq+i8RyWtnlRnA426Hae+LyEARORiYCrypqvsARORNnEQTTj84pg+qrW9gS2EFG/aUsn53GRsKyli/u4y9ZeF2ZmpMbBuXk86cq4+P+H69bIPIoWUvlPnuvLbmH0BEZuNcfTB8+PADltfW1pKfn09VVVWEQu69EhMTyc3NJRDovWO7VNXWs6mgnPV7Stm4p4z1e8rYsKeML/eWU9fQfLt1zsAkRg9N5ejcdFp0p2eMCSk3I7njlbqgTzdSq+rDwMMAEydOPOCBjvz8fNLS0sjLy0P68ZlGVSksLCQ/P5+RI0d6HQ5l1XVscE/+wclg274KGvOATyAvM4VRQ1I5a+xQRg9N5dDBaYwakkJyfJ/+szSm3/DyP3E7TvfLjXLdedtxqpmC5y/pSgFVVVX9PjkAiAiZmZkUFBT0aLn7y2uaqoOCk8GO4uYrtni/j5FZKYzLSefC8TmMHprK6CFp5GUlkxDn79F4jTGd42WCmA9cLyLP4DRIF6vqThF5A/ifoIbps4HbulpIf08OjaL1c6oqBaXVTdVB6/eUNl0d7C2raVovKeDn0CGpnHBIJocOSWX0kFQOHZLK8EHJxPnteUzTD6hCfS001Lrv9aD1gIAIiPt3Lr6gz9Jq2l0WPN2Lz1FRSxAi8jTOlUCWO+j6nTjDPKKqf8EZI/gcnK6UK4Bvucv2icgvaR4C8q7GBmsTeQ0NSlFlLYVl1RSUVVNYVsOu4qoWyaCkqq5p/bTEOEYPSeWMI4Zy6JBUDh3qJIPs9CR8vt77h95pDfVQVw31NaANHf+Tt/iH9/Xqf/qoUYWGuuZX40k0eN4Br3p3vaDPTdMhtm/ap7tNfV3QCbu21ee65vkHrFvXxja1QTG1Wqb1UTx47fwtHZBkQnzOngBXPh/xqKJ5F9MVHSxX4Lo2ls3BGUClzysqKmLu3Llce+21ndrunHPOYe7cuQwcOLDTZVbV1rPXPdkXllezt7SGveXO58b5e8uqKSyvYV95DfUNB/bHlZUaz6FDUrlgfDajh6Q1XREMTkuI/lVZTQUUbYHyvc4JuulV23zSrq+F+uDpGndZbatt2tou1DZB+9SGyPws7X5zbPUPH863Ta+1SAKtTuaROmZd4YsDXwD8AWfaH3A/xwUtC14nAPHJobfxxbVat/X+gvbR+DtRBdQ5Buq+B39usazxc4j1mj63nu5g/xkjQh6W7rLWwCgrKiriz3/+8wEJoq6ujri4tg//ggULmqZVlfoGpa5BqatX6hoaWk7XO8t2FVfx9TvfoKy6LuQ+U+L9ZKYmkJkaz7BByUwYPpDMlASyUuOb5melJjAkLYGByfGROQChqELZHtj/Jezf7Lz2BU2X7er8Pn1x4E9w/nH98RAXNO0PuMvc+QlprZYHvwLusvjm5eJr5x80eJoOTgyttyH0em2dCHoLX9BJ1+cP+hwXdEJu7+VvPvE2fm7aX1zLffpDlNG0TtAJvDckz37IEkSU3XrrrWzcuJHx48cTCARITEwkIyODdevW8cUXX3DhhReybds2qqqquOa713LFN6+hrkE5cfwY5r3+DqVlpXz3ykuYcPxkVq34gCFDD+a+R54iMSkJAfw+H3F+Ic4nxMf5uHTiMPdE75zsM1MTyExxppPie7BRuLYKira2TALBiaCuMmhlgQE5kJEHh54Jg/IgYySkDoG4xDZO5PEQ5777AuCzdg5jIi1mEsQv/rmGz3aURHSfY7MHcOf5R7a7zt13383q1atZtWoVS5Ys4dxzz2X16tVNt6POmTOHQYMGsa2giNNOnsIJp59DVlYmAPF+ITUhjq1fbuSRx55gwvi/cs2sb7DmP28ya9ZM/D5pUd1TvTeeO84fE9GfsU2qThXQ/s2hk0DpjpbrB5Kdk/6gQ2DU6TBopJMQMvJg4HDnW7sxpleJmQTRW0yaNKnFswr3338/L730EtV1DezeuYO4st0cMW4kcT5heGYKZWXKyJEjOe3ESQCcMOl4tudv7Zk7g+pq3KuAzQcmgf2boaas5
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aVPavynHmM_q",
"colab_type": "text"
},
"source": [
"**Three VGG block with Dropout**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "XQGGTd7nufZ5",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "27cc9afc-37bb-4681-e535-4d33400d0a52"
},
"source": [
"\n",
"plt.figure(1) \n",
" \n",
" \n",
"plt.subplot(211) \n",
"plt.plot(history4.history['accuracy']) \n",
"plt.plot(history4.history['val_accuracy']) \n",
"plt.title('model accuracy') \n",
"plt.ylabel('accuracy') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
" \n",
" \n",
"plt.subplot(212) \n",
"plt.plot(history4.history['loss']) \n",
"plt.plot(history4.history['val_loss']) \n",
"plt.title('model loss') \n",
"plt.ylabel('loss') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
"plt.show()"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU1fn48c8zk0kmG0lIAIGwaUHBDRQR993ihgvuu9XSzdb+qlbbr0trN7/ffttva92tWPcNN6y4obi0ArKIyiaLBZKwGsi+Tub5/XFukkkIJJCZzCR53q/XvObO3eaZyeQ895x77rmiqhhjjDGt+eIdgDHGmMRkCcIYY0ybLEEYY4xpkyUIY4wxbbIEYYwxpk2WIIwxxrTJEoQxgIj8Q0R+28F114rIybGOyZh4swRhjDGmTZYgjOlBRCQp3jGYnsMShOk2vKadm0XkCxGpFJFHRWSAiLwpIuUiMktEciLWnywiS0WkREQ+EJHREcvGicgib7vngWCr9zpTRBZ7234iIgd1MMYzROQzESkTkQIR+VWr5Ud7+yvxll/tzU8VkT+JyDoRKRWRf3nzjheRwja+h5O96V+JyHQReUpEyoCrRWSCiMzx3mOjiNwrIskR2+8vIu+KyDYR2SwivxSRvUSkSkRyI9Y7RES2ikigI5/d9DyWIEx3MwU4BRgFnAW8CfwS6If7Pf8EQERGAc8CP/WWzQReF5Fkr7B8FXgS6Au86O0Xb9txwDTge0Au8BAwQ0RSOhBfJXAlkA2cAfxARM7x9jvMi/dvXkxjgcXedv8LHAoc6cX0cyDcwe/kbGC6955PAw3A/wPygCOAk4AfejFkArOAt4BBwLeA91R1E/ABcGHEfq8AnlPV+g7GYXoYSxCmu/mbqm5W1SLgY2Ceqn6mqjXAK8A4b72LgDdU9V2vgPtfIBVXAE8EAsBfVLVeVacD8yPeYyrwkKrOU9UGVX0cqPW22yVV/UBVv1TVsKp+gUtSx3mLLwVmqeqz3vsWq+piEfEB3wFuUNUi7z0/UdXaDn4nc1T1Ve89q1V1oarOVdWQqq7FJbjGGM4ENqnqn1S1RlXLVXWet+xx4HIAEfEDl+CSqOmlLEGY7mZzxHR1G68zvOlBwLrGBaoaBgqAwd6yIm05UuW6iOlhwI1eE02JiJQAQ7ztdklEDheR2V7TTCnwfdyRPN4+1rSxWR6uiautZR1R0CqGUSLyTxHZ5DU7/b4DMQC8BowRkRG4Wlqpqn66hzGZHsAShOmpNuAKegBERHCFYxGwERjszWs0NGK6APidqmZHPNJU9dkOvO8zwAxgiKpmAQ8Cje9TAOzTxjbfADU7WVYJpEV8Dj+ueSpS6yGZHwBWACNVtQ+uCS4yhr3bCtyrhb2Aq0VcgdUeej1LEKanegE4Q0RO8k6y3ohrJvoEmAOEgJ+ISEBEzgMmRGz7CPB9rzYgIpLunXzO7MD7ZgLbVLVGRCbgmpUaPQ2cLCIXikiSiOSKyFivdjMN+LOIDBIRv4gc4Z3zWAkEvfcPALcB7Z0LyQTKgAoR2Q/4QcSyfwIDReSnIpIiIpkicnjE8ieAq4HJWILo9SxBmB5JVb/CHQn/DXeEfhZwlqrWqWodcB6uINyGO1/xcsS2C4DvAvcC24HV3rod8UPgLhEpB+7AJarG/a4HTsclq224E9QHe4tvAr7EnQvZBvw34FPVUm+ff8fVfiqBFr2a2nATLjGV45Ld8xExlOOaj84CNgGrgBMilv8bd3J8kapGNruZXkjshkHGmEgi8j7wjKr+Pd6xmPiyBGGMaSIihwHv4s6hlMc7HhNf1sRkjAFARB7HXSPxU0sOBqwGYYwxZidiVoMQkWkiskVEluxkuYjIPSKyWtzQCYdELLtKRFZ5j6tiFaMxxpidi1kNQkSOBSqAJ1T1gDaWnw78GNer43Dgr6p6uIj0BRYA43H9uxcCh6rq9l29X15eng4fPjy6H8IYY3q4hQsXfqOqra+tASBmIz+q6kciMnwXq5yNSx4KzBWRbBEZCBwPvKuq2wBE5F1gEm7Igp0aPnw4CxYsiEboxhjTa4jITrszx/Mk9WBaDhFQ6M3b2fwdiMhUEVkgIgu2bt0as0CNMaY36ta9mFT1YVUdr6rj+/Vrs4ZkjDFmD8Xz5iJFuLFxGuV784pwzUyR8z/osqiMMTEVaghTXd9AdV0DVd6j+XWoxbKW0yH3XNdAXUOYZL+PlICflCQfKUk+gk3TfoIBN69xeeSylICPoPe8w7IkHz6ftP8h9vAz19SHqalvoKa++bPVhMLuOXJ+63XrWs6rjli3pq6B/Qb2YdrVh0U97ngmiBnA9SLyHO4kdamqbhSRt4HfS/ONX04FfrEnb1BfX09hYSE1NTXRiTiBBYNB8vPzCQTs3i7tUVVq6sOUVNdRUlVPSVU9pY3T1a1eV9UTViU9JYm0ZL/3cNNtzUtLTiI9Zcd5yUmJXVlXVUJhpTYUpra+wT2HwtSGGqitb56uqW8s3ENtFODNhXyVV+A1FuiRy+saOnqbC8cnkJacRKr3Xad6BXptKEydF2dNU8wN1Dd0ruNNsr8xuTQnlMbkEYyYTgn48QtN30lkAd80r66Bmk7EFAz4SA34CQb8Tc/BgI/UZD/ZqQGC3vexd7/0Tn3mnYlZghCRZ3E1gTzvjlh34sbgR1UfxN3A5XTcODdVwDXesm0i8huax+e/q/GE9e4qLCwkMzOT4cOH03Lgzp5FVSkuLqawsJARI0bEO5wuo6qU14Yo9QrykohCvrRqJwV+dT2l1fXUhXZeSCX7fWSnBchKdQ+fT9hSXuOOdmtdAVhZ10BDuOP/9AG/kBpwSSU12U96ckRySUkiLbCThJOSRHqyn9RkP6o0FdKtC+4W06Gw93rnBX3jOjVN8xvYjY/TggikBrzCO9lPWiCJYLKftICfvfoESPUKMbc8qamQT236rK7ga/zMLdf3k+z37db/b0NYW3zeyOTR9HpXyyK+v5r6HZeVVNc3JdGGsHoFt6uJ5KQnMzDJxd04r0UBn+wnmOQK+GDTev6mRBA5PyVp9z53LMSyF9Ml7SxX4Ec7WTYNN7plp9TU1PT45AAgIuTm5tKdTtSHGsJU1TcXuI1NDZV1IarrGqisDVFRG/IK93pKquoiCnv3urS6fpeFWnqy3xXyaclkpwb4Vv8Mr+BPJjstQHZqoOXrtADZqckEA+3/Y6oqdQ2uaaCyzh1RV9ZGxB8xL/LzNSaXqlo375uKOqq2VbnP7s0L7WFJ7RN2aEqJbDpJCfjokxpoapLZ2TqtpyOPmpOTfDsU6IlQkEXy+8SLLd6RdH89/gbnifTDjaVYfc76hnBTwRZ5BN1YmFc1FnZeYd9mAVnfXCA2FpC7OoJvrU8wiey05Kaj+iF90yIK94Bb5r1uLPCzUgMxbdYREa/Q9JOd1v76u6Mu1Jh4Qs3fdW0Dfp+0auKImE7ykeRP7GYs0/30+ARh2lYXCrOhpJrC7dUUbK+iYFtV0/SWslpXONXuXnuxTyDdayuObC7JTg0wKCvY1D7fsonFm9dG80tGShJ9UgP4Y3DSMJEle0fqWWl2PsnElyWIGCspKeGZZ57hhz/84W5td/rpp/PMM8+QnZ29R+8bagizqayGgm3VFG6vomC7ey7c5pLAprIaIi+i9/uEQdlBhuSkcfjefclISXKFt9cO3Fzgt5yXGtF2nmhNDcaYzrEEEWMlJSXcf//9OySIUChEUtLOv/6ZM2fucr+qSqjBtYPXNbjeHNur6rj0kbkUbK9iY0lNi7ZsERjYJ0h+ThpH7JPLkJw08nNSGdLXPe/VJ2hNFMaYFixBxNitt97KmjVrGDt2LIFAgGAwSE5ODitWrGDlypWcc845FBQUUFNTww033MDUqVMBN3TInHmfUlJaxjmTz+SwiUcyf+5c+u81kHsfexZfcgqtx9Fq7CM9bkgOkw9OJT8njSE5aQzpm8rArNT22+RVYfUsWPk2pOdB5kDoM6j5OTXHZZreRBU0spnN+/zS6tn0DrUVUFoAJevdIzUHDpjSY38HvSZB/Pr1p
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7R_fSnGNmUqm",
"colab_type": "text"
},
"source": [
"**Three VGG block with Dropout and Batch Normalization**"
]
},
{
"cell_type": "code",
"metadata": {
"id": "-7wTMgylufmv",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "a25c02ff-7370-41e2-c141-d544d53d3a72"
},
"source": [
"\n",
"plt.figure(1) \n",
" \n",
" \n",
"plt.subplot(211) \n",
"plt.plot(history5.history['accuracy']) \n",
"plt.plot(history5.history['val_accuracy']) \n",
"plt.title('model accuracy') \n",
"plt.ylabel('accuracy') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
" \n",
" \n",
"plt.subplot(212) \n",
"plt.plot(history5.history['loss']) \n",
"plt.plot(history5.history['val_loss']) \n",
"plt.title('model loss') \n",
"plt.ylabel('loss') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
"plt.show()"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzde3xU9Zn48c8zl8zknpBAIAQhICp4KSjeirQoXvBee9HW2q22W2y1W7vbutXdXrbd7a77a9dtbSveSq9W29Xa2lYrqKBWRA2Iyk25CCQBkhBIyG2SzMzz++OcJJMwIRMyw+TyvF+vec2Zc5tnDuE85/v9nvP9iqpijDHG9OVJdwDGGGOGJ0sQxhhj4rIEYYwxJi5LEMYYY+KyBGGMMSYuSxDGGGPisgRhDCAiPxeR/0hw3Z0icmGqYzIm3SxBGGOMicsShDGjiIj40h2DGT0sQZgRw63auV1E3hKRFhH5qYiUiMjTItIkIs+KSGHM+leJyEYRaRCRVSIyK2bZXBFZ5273WyDY57uuEJH17rarReS0BGO8XETeEJFDIlIpIv/WZ/l57v4a3OU3uvMzReR/RGSXiDSKyN/ceQtFpCrOcbjQnf43EXlMRH4tIoeAG0XkLBF5xf2OvSLyYxHJiNn+ZBFZISIHRKRGRP5FRCaKSKuIFMWsd7qI1ImIP5HfbkYfSxBmpPkIcBFwAnAl8DTwL8B4nL/nLwGIyAnAI8CX3WVPAX8SkQz3ZPkH4FfAOOD/3P3ibjsXWAbcDBQB9wNPikgggfhagL8DCoDLgS+IyIfc/U514/2RG9McYL273feBM4D3uzH9MxBN8JhcDTzmfufDQAT4R6AYOBdYBNzixpALPAv8FSgFjgeeU9V9wCrg2pj9fgp4VFU7E4zDjDKWIMxI8yNVrVHVauAl4FVVfUNVQ8ATwFx3veuAv6jqCvcE930gE+cEfA7gB36gqp2q+hjwesx3LAHuV9VXVTWiqr8A2t3tjkhVV6nq26oaVdW3cJLUB93F1wPPquoj7vfWq+p6EfEAnwFuU9Vq9ztXq2p7gsfkFVX9g/udbaq6VlXXqGpYVXfiJLiuGK4A9qnq/6hqSFWbVPVVd9kvgBsARMQLfAIniZoxyhKEGWlqYqbb4nzOcadLgV1dC1Q1ClQCk91l1dq7p8pdMdNTga+4VTQNItIATHG3OyIROVtEVrpVM43A53Gu5HH3sT3OZsU4VVzxliWisk8MJ4jIn0Vkn1vt9J8JxADwR2C2iJTjlNIaVfW1o4zJjAKWIMxotQfnRA+AiAjOybEa2AtMdud1OS5muhL4rqoWxLyyVPWRBL73N8CTwBRVzQfuA7q+pxKYEWeb/UCon2UtQFbM7/DiVE/F6tsl81JgCzBTVfNwquBiY5geL3C3FPY7nFLEp7DSw5hnCcKMVr8DLheRRW4j61dwqolWA68AYeBLIuIXkQ8DZ8Vs+yDwebc0ICKS7TY+5ybwvbnAAVUNichZONVKXR4GLhSRa0XEJyJFIjLHLd0sA+4WkVIR8YrIuW6bx7tA0P1+P/B1YKC2kFzgENAsIicBX4hZ9mdgkoh8WUQCIpIrImfHLP8lcCNwFZYgxjxLEGZUUtV3cK6Ef4RzhX4lcKWqdqhqB/BhnBPhAZz2it/HbFsBfA74MXAQ2Oaum4hbgO+ISBPwTZxE1bXf3cBlOMnqAE4D9fvcxV8F3sZpCzkA/DfgUdVGd58P4ZR+WoBedzXF8VWcxNSEk+x+GxNDE0710ZXAPmArcH7M8pdxGsfXqWpstZsZg8QGDDLGxBKR54HfqOpD6Y7FpJclCGNMNxE5E1iB04bSlO54THpZFZMxBgAR+QXOMxJftuRgwEoQxhhj+mElCGOMMXGlrGMvEVmG89RmraqeEme5AD/EuaujFbhRVde5yz6NczsfwH+4T7IeUXFxsU6bNi1J0RtjzNiwdu3a/ara99kaIIUJAvg5zm2Cv+xn+aXATPd1Ns7DPWeLyDjgW8A8nAeA1orIk6p68EhfNm3aNCoqKpIUujHGjA0i0u/tzCmrYlLVF3Hu5+7P1cAv1bEGKBCRScAlwApVPeAmhRXA4lTFaYwxJr50tkFMpncfMlXuvP7mH0ZElohIhYhU1NXVpSxQY4wZi0b04CKq+gDwAMC8efPsdiwDqtDZBh0t0NnivHe0Dm3aH4TcSZBX2vOeVwq5pZA3CbIngHdE/1cyJq50/lVX43Se1qXMnVcNLOwzf9XRfEFnZydVVVWEQqGjDHHkCAaDlJWV4fcPw7FdVCHcDpF2573rFWmHcCjOvK5XCDqanZN09wl/oOlWDu+77gg8fsjIgowc8Gc50/5syCqGguOc6c4WOLQXdq2Gpr0QDffeh3ggp6RPEpnUk0C63gOJdOVkxhJVJRJVwtG+71HnPdLP/K7P7vKsgJfTjysc+AsHKZ0J4kngiyLyKE4jdaOq7hWRZ4D/lJ6RwS4G7jyaL6iqqiI3N5dp06bRu+PO0UVVqa+vp6qqivLy8qHt7OAu2P6ccxUee6KOdLgnc/c97sm+vc86Me9DIpCR7Z7As3tP50yIP797us/Jv28i8GUM/PWxolFo3Q+H9jjJovt9LzTtgfrtsPMlCDUevm0gL07y6FMyyR4PHu8Qj5cZjGhUae2M0NIeprk9TEt7mJZ253NLR8+85q55Meu1dUbinrAPO5F3L++Z3zUvGeZMKeAPt85Pyr5ipfI210dwSgLF7pCJ38IZpAVVvQ9nhK/LcDpCawVucpcdEJF/p2cAl++o6pEau/sVCoVGfXIAEBGKiooYUjtMzUZ4+Yfw9mOgkd7LvBngDYAv9hV05vuCzkk2mBfz2Z3Xdx1f0NnWG+hnXp99+zOdk7k/E9Lwb9gRjlLbFCLUGQEEjzjHWsjGEzgBCZ4A48HjEQQnRI8404Rb8LXU4m3ag7d5H96WvXia9uFp3ouneS9StwpprkH6HGv1+JyklzsZ8iYhscljwmx0/EmoeImqElWIqqLue9c8jVnWe7lzMhzU+qo9y6POe/fvFGKOiTOva37vdSTh9URA3GPds07P9l3zVaG5PUxr9wm89wm+e15H7Ek//rqtnRESfV44O8NLdsBHTsBHVsBLlt+H3+sh6Bd8HsHr8Tjv3q7Pfea7n32eKEHaCarzCmiITEJkaAeBaBsZGiIj2k5GtI2MaBu+aAh/NIQ/0oYv0oYvEsIXacMbacMbCRHOPx4YQQlCVT8xwHIFbu1n2TKc7o+HbLQnhy5H/Tt3r4G//S+8+1fnivqcL8C8z0BWUc+J2jP6nqds7QizrzHkvA6F2Bszva/R+Vzf0p7wiePICtzXrF5zPUQpopGJcpCJcoAS933iwYOUHDzARFlLiawgT9q6t2nRIG9GZ7BOZ7IuOpP10RkcJC8ZQY5KQb+HnICP7ICP7Awf2QEvRTkZHFeURU6Gzz3ZOyf9rhO/M+3tnu56z/J78XjEqTJt2gf73oaDO3uqODvd12HTLW67WMx0eLDV3hJT8s10/q9mZEEgC/xFMGFaCo7eCG+kNkdJFbaugL/dDbtfgcxxsPBf4KzPQda4dEc3JKpKY1unc8J3T/Zdr72HQtQ0htjb2MahUPiwbfMz/UzKD1KSF+Tk0jxK8oJMzA+SHfDR1SVN1xV211W2AsRM957fc4Wu0D3daz8x23TFv0ehWkFxtveHW8gO1TCpdQuTmjZwfNPbnNvyJzw4pY+GzOOozT+NuvxTqS14H405xyNef8xVd+8rco8IHk/X555lR1zfndc17FD379TY36JEo7HHouv395ROun77kdbrPl5uiaVnna7vc2LIDnjdk37XSdwbc4L3kZ3hxecd4sVNJAz1W2Hn27DvLScp7NvgVDP25c/qXX3pz3SrQSf2THeViONOd23rVpPGJgJfMC2laEsQKdbQ0MBvfvMbbrnllkFtd9lll/Gb3/yGgoKC5AUTCcPGJ5wSQ+1GyCuDxf8Np3/K+YMc5iJRpb65nb3uFX7Nodj3tu4SQKgz2ms7ESjOCTApP8jUoizOnj6OiflBJroJYFJ+J
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ACRJH30KmYcR",
"colab_type": "text"
},
"source": [
"## VGG Zero-One Loss"
]
},
{
"cell_type": "code",
"metadata": {
"id": "BXTkF0GpY-tS",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 106
},
"outputId": "93c29f81-f9c2-49ae-ebc5-4ee5bcddd79d"
},
"source": [
"cnn_loss"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[0.02153987167736022,\n",
" 0.00916590284142988,\n",
" 0.00916590284142988,\n",
" 0.07561869844179651,\n",
" 0.005957836846929423]"
]
},
"metadata": {
"tags": []
},
"execution_count": 75
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "pyTo4qloNAnp",
"colab_type": "code",
"colab": {}
},
"source": [
"df_cnn_loss = pd.DataFrame(\n",
"columns = ['type', 'zero_one']\n",
")\n",
"\n",
"types = [\"1VGG\",\"2VGG\",\"3VGG\",\"3VGG-drop\", \"3VGG-drop-norm\"]\n",
"df_cnn_loss = pd.DataFrame( data = [types,cnn_loss], index = ['types', 'zero_one']).T"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "DCVd9nicVyDy",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 206
},
"outputId": "baee1afb-af6a-476c-d35a-4e49f3e513a1"
},
"source": [
"df_cnn_loss"
],
"execution_count": null,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>types</th>\n",
" <th>zero_one</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1VGG</td>\n",
" <td>0.0215399</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2VGG</td>\n",
" <td>0.0091659</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3VGG</td>\n",
" <td>0.0091659</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>3VGG-drop</td>\n",
" <td>0.0756187</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>3VGG-drop-norm</td>\n",
" <td>0.00595784</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" types zero_one\n",
"0 1VGG 0.0215399\n",
"1 2VGG 0.0091659\n",
"2 3VGG 0.0091659\n",
"3 3VGG-drop 0.0756187\n",
"4 3VGG-drop-norm 0.00595784"
]
},
"metadata": {
"tags": []
},
"execution_count": 77
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "7W7HR2H_Vz9u",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 498
},
"outputId": "ba53bf43-4ce4-44df-8db0-efbabf8f7124"
},
"source": [
"ggplot(df_cnn_loss, aes(x='types', y='zero_one',group=1)) + \\\n",
" geom_point() + \\\n",
" geom_line() + \\\n",
" theme_bw(base_size=12) + ggtitle(\"Zero-one loss CNN\") + ylab(\"loss\")\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAsoAAAHPCAYAAABKs/xuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd1QU1/8+8GdZehMbgtixY0wsKIoVKzYQCwq7YgsGNcZEP4AFwV5jYjTYY0M0SmyJ2IK9RSxRE0uIoDEogoACgpTd+f3Bj/262VVBylCe1zk5JztzZ+aZyyLvnb1zRyIIggAiIiIiIlKjI3YAIiIiIqLSiIUyEREREZEWLJSJiIiIiLRgoUxEREREpAULZSIiIiIiLVgoExERERFpwUKZiIiIiEgLFspERERERFqwUCYiIiIi0oKFMhER0RtOnz4NiUSCoKAgsaMQkchYKBMRlRL16tWDRCLJ13+jR48WOy6VUidOnICnpyfq168PY2NjGBkZoWHDhpDL5Thy5Iha27wPBRKJBMOHD9e6v4cPH0IikaBTp05Fti1RWaErdgAiIso1depUvHjx4q3r09PTsXLlSigUCrRo0aIEk1FZkJqailGjRuHAgQMwNDSEk5MT3NzcoKenh5iYGISHhyMkJATTpk3DihUrNLbfu3cvLl++DAcHhwIfuzDbEpVmLJSJiEqJqVOnvnWdIAgYPnw4FAoFhg4dimnTppVgMirtlEolhg0bhmPHjqF79+4ICQlBzZo11dpkZmZi3bp1+OuvvzS2t7W1xYMHDzB9+nScP3++QMcuzLZEpR2HXhARlQFz5sxBWFgYWrVqhW3btkEikaitz8nJQXBwMBwcHGBubg5jY2O0atUKa9asgVKpVGub93X46NGj8ddff8Hd3R2WlpbQ0dHB6dOnAeQWXuvWrYO9vT1MTU1hYmICe3t7rF27VmN/+RUREYG+ffuiSpUqMDAwQOPGjeHv74+XL19qtO3WrRskEglycnKwaNEiNGrUCAYGBqhduzb8/PyQlZWl9Rj37t3D6NGjUbt2bejr66NGjRrw8PDA/fv3Pyjzf0VFRWHUqFGwsbGBvr4+atasiVGjRiEqKkqjbWpqKubPn48WLVrA3NwcZmZmsLW1hbu7O65du6bW9tChQ+jRowesra1hYGCAmjVromvXrggODs5Xrl27duHYsWNo2LAhfv75Z40iGQAMDAzwxRdfYOXKlRrrHBwc4OLiggsXLuCnn37KZ28Uflui0o6FMhFRKbdr1y4sWLAAVlZWOHToEIyNjdXWZ2dnY8CAAZg0aRJevHgBDw8PeHt7Q6lU4vPPP4eXl5fW/T548ADt27fHw4cP4enpCW9vb5ibmwMA5HI5fHx88OzZM4wfPx7e3t5ISEjAxIkTIZfLC3wO69evR69evXDhwgW4urriyy+/RJUqVbB06VJ07NjxrUNOPDw8sHr1anTu3Bk+Pj4wMjLCsmXLMGHCBI22R48eRevWrbFz507Y29tj6tSp6NGjB/bt24d27drh+vXrBc79psjISLRt2xYhISGwt7fH9OnT4eDggJCQELRt2xaRkZGqtoIgoG/fvpgzZw7Mzc0xfvx4+Pj4oH379jh79iwuXbqkarthwwa4uLjgzp07GDhwIKZNm4Z+/fohIyMDW7ZsyVe2DRs2AACmT58OExOTd7Y1MDDQunzZsmXQ1dWFv78/srOz83XcotiWqFQTiIio1Prtt98EQ0NDwdDQULh8+bLWNoGBgQIAYfLkyUJOTo5qeU5OjjB27FgBgHDgwAHV8piYGAGAAECYMWOGxv5CQ0MFAEKrVq2E1NRU1fK0tDShTZs2AgBh586d+T6Hhw8fCvr6+oKZmZlw9+5dtXU+Pj4CAOHTTz9VW961a1cBgNC6dWshMTFRLYOtra2go6MjPH36VLU8KSlJsLCwEKpWrSr8+eefavu6ffu2YGJiIrRq1SpfeU+dOiUAEAIDA1XLlEql0LRpUwGAEBISotZ+9+7dAgChSZMmgkKhEARBEG7duiUAEFxdXTX2r1AohKSkJNXr1q1bC/r6+sKzZ8802iYkJLw3b3Z2tqCvry8AEKKiovJ1jnnyztXT01MQBEGYNGmSAEBYtWqVqk3e+8XR0bHItiUqK3hFmYiolPr333/h6uqK169fY9OmTWjfvr1GG6VSidWrV8PKygrffPMNpFKpap1UKsXXX38NiUSCnTt3amxbo0YNBAYGaiz/4YcfAABLliyBqamparmJiQmWLl0KANi0aVO+zyMkJARZWVmYPHkymjZtqrZu4cKFMDMzw44dO5CZmamx7dKlS1GlShW1DJ6enlAqlbh69apq+fbt2/HixQvMnTsXzZs3V9tHixYt8Omnn+LGjRu4c+dOvnO/6eLFi7h37x46dOgAT09PtXXu7u7o1KkT7t+/rzFG18jISGNfOjo6qFy5stoyXV1d6OnpabStVq3ae7MlJSWphqLUqlXrve3fJTAwEObm5pg3b57WITHFtS1RacWb+YiISqH09HS4uLjg6dOnmDFjhkZxluevv/5CUlISGjVqhAULFmhtY2RkhLt372os//jjj7V+DX/9+nXo6OigW7duGuu6du0KqVSKGzduqJadPn1aNbY5T7169VRT2OUNeXByctLYX+XKldGqVSucPXsW9+7dw8cff6y2vm3bthrb1K5dGwCQnJysWpY3lOHmzZta5z/Ou4Ht7t27GoV0frzrHPKWnz9/Hjdu3ECXLl3QvHlzfPLJJ9i1axcePXoEFxcXdOrUCW3btoW+vr7atp6enpg2bRqaN2+OESNGoGvXrnB0dET16tULnLOwqlevDn9/f8ycORMLFy7EsmXLSmRbotKKhTIRUSkjCAK8vLxw/fp1uLq6YuHChW9tm5iYCCD3JrO5c+e+tV1aWprGMisrK61tX758iSpVqmgUdEDulc9q1aohPj5etez06dMax+7atauqUM67umhtba31eHnLtY1TtrCw0JoBABQKhWpZXj9s3LhR6zHyaOuH/CjoOUilUpw8eRLz5s1DWFgY/Pz8AABmZmbw8vLC4sWLVVfrv/rqK1SrVg3BwcH47rvv8O2330IikaBr165Yvny51g8Lb8r7WWVlZSE2Nha2trYfdI55vvzyS6xduxbfffcdJk2aVGLbEpVGHHpBRFTKBAYGIiwsDC1btkRISIjGDBdvqlSpEgBg8ODBEAThrf/FxMRobPu2/VaqVAlJSUlab8rKycnB8+fPVTf9AUBQUJDG8d68wpyXMS4uTuvxnj59qtbuQ+Rte/PmzXf2w9tubMzv/gtyDpUrV8Y333yDx48fIyoqCps2bULTpk2xZs0a+Pj4qG0/atQoXL58GYmJiTh8+DDGjRuHs2fPok+fPkhISHhnNl1dXdX8xRERER90fm8yNDTEggULkJmZiZkzZ5bYtkSlEQtlIqJSZPfu3Zg/fz4sLS1x6NCh985g0LRpU1hYWODy5ctFNttAq1atoFQqcfbsWY11Z8+ehUKhQOvWrQu0PwAawzOA3Cuwv//+OwwNDdGsWbMPzpxXKJ47d+6D9/Eu7zoHADh16hQAvLVfGjZsiHHjxuHMmTMwNTXFwYMHtbazsLBAv379sHHjRowePRpJSUlafw7/5e3tDQBYsWIF0tPT39lW21jw/5LL5WjVqhV27dqlNhY8PwqzLVFpw0KZiKiUuHLlCsaMGQN9fX3s378fdevWfe82urq6+Pzzz/H06VNMmTIFGRkZGm2ePn1aoJvYxo4dCwCYMWOGWtGVnp4Of39/AMC4cePyvT+ZTAY9PT2sXr0af//9t9q6gIAApKSkQCaTvXXasvwYM2YMLCwsMHfuXFy5ckVjvVKpfGuRmx+Ojo5o0qQJzp8/j7CwMLV1YWFhOHfuHBo3bqx6VHNMTAyio6M19pOcnIzMzEy1m/xOnToFQRA02uYNb/nvdIDajBw5En369EFUVJRqbPt/ZWVl4fvvv8/Xw2okEglWrFgBQRAwY8aM97Yvqm2JShuOUSYiKgVSU1NVM1zY29vj+PHjOH78+
"text/plain": [
"<Figure size 640x480 with 1 Axes>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<ggplot: (8757990059065)>"
]
},
"metadata": {
"tags": []
},
"execution_count": 78
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3_kbZU_nNC_i",
"colab_type": "text"
},
"source": [
"# 1.5 LeNet Neural Networks\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "rpN3eUW2NFME",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 446
},
"outputId": "7118b9e2-674a-4061-d739-3c7d70a696d4"
},
"source": [
"\n",
"input_shape = (32,32,3)\n",
"num_classes = 10\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Conv2D(6, kernel_size=(5, 5), strides=(1, 1), activation='tanh', input_shape=input_shape, padding=\"same\"),\n",
" tf.keras.layers.AveragePooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'),\n",
" tf.keras.layers.Conv2D(16, kernel_size=(5, 5), strides=(1, 1), activation='tanh', padding='valid'),\n",
" tf.keras.layers.AveragePooling2D(pool_size=(2, 2), strides=(2, 2), padding='valid'),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(120, activation = 'tanh'),\n",
" tf.keras.layers.Dense(84, activation = 'tanh'),\n",
" tf.keras.layers.Dense(10, activation = 'softmax')\n",
"])\n",
"\n",
"model.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])\n",
"\n",
"model.summary()\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Model: \"sequential_16\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"conv2d_24 (Conv2D) (None, 32, 32, 6) 456 \n",
"_________________________________________________________________\n",
"average_pooling2d (AveragePo (None, 16, 16, 6) 0 \n",
"_________________________________________________________________\n",
"conv2d_25 (Conv2D) (None, 12, 12, 16) 2416 \n",
"_________________________________________________________________\n",
"average_pooling2d_1 (Average (None, 6, 6, 16) 0 \n",
"_________________________________________________________________\n",
"flatten_5 (Flatten) (None, 576) 0 \n",
"_________________________________________________________________\n",
"dense_32 (Dense) (None, 120) 69240 \n",
"_________________________________________________________________\n",
"dense_33 (Dense) (None, 84) 10164 \n",
"_________________________________________________________________\n",
"dense_34 (Dense) (None, 10) 850 \n",
"=================================================================\n",
"Total params: 83,126\n",
"Trainable params: 83,126\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "9-1JbKMPNK8V",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 392
},
"outputId": "25e7c6af-33c0-4469-ed1f-960b015c7db8"
},
"source": [
"history = model.fit(x_train, y_train,\n",
" batch_size = 32,\n",
" epochs = 10,\n",
" validation_data=(x_valid, y_valid),\n",
" verbose = 2\n",
"\n",
" )"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"WARNING:tensorflow:Method (on_train_batch_end) is slow compared to the batch update (0.219765). Check your callbacks.\n",
"1019/1019 - 8s - loss: 0.3274 - accuracy: 0.9028 - val_loss: 0.1608 - val_accuracy: 0.9496\n",
"Epoch 2/10\n",
"1019/1019 - 7s - loss: 0.0138 - accuracy: 0.9977 - val_loss: 0.0819 - val_accuracy: 0.9768\n",
"Epoch 3/10\n",
"1019/1019 - 9s - loss: 0.0014 - accuracy: 1.0000 - val_loss: 0.0839 - val_accuracy: 0.9735\n",
"Epoch 4/10\n",
"1019/1019 - 7s - loss: 6.4137e-04 - accuracy: 1.0000 - val_loss: 0.0796 - val_accuracy: 0.9779\n",
"Epoch 5/10\n",
"1019/1019 - 7s - loss: 3.2755e-04 - accuracy: 1.0000 - val_loss: 0.0749 - val_accuracy: 0.9799\n",
"Epoch 6/10\n",
"1019/1019 - 7s - loss: 1.7176e-04 - accuracy: 1.0000 - val_loss: 0.0822 - val_accuracy: 0.9790\n",
"Epoch 7/10\n",
"1019/1019 - 7s - loss: 9.9440e-05 - accuracy: 1.0000 - val_loss: 0.0775 - val_accuracy: 0.9815\n",
"Epoch 8/10\n",
"1019/1019 - 8s - loss: 5.2685e-05 - accuracy: 1.0000 - val_loss: 0.0819 - val_accuracy: 0.9813\n",
"Epoch 9/10\n",
"1019/1019 - 7s - loss: 2.9746e-05 - accuracy: 1.0000 - val_loss: 0.0737 - val_accuracy: 0.9812\n",
"Epoch 10/10\n",
"1019/1019 - 7s - loss: 1.6986e-05 - accuracy: 1.0000 - val_loss: 0.0801 - val_accuracy: 0.9822\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "brru_mIGNPQs",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 540
},
"outputId": "6cd8216f-5c82-4dea-cf91-06a13d35e9ab"
},
"source": [
"y_pred = model.predict(x_test)\n",
"\n",
"# plot a random sample of test images, their predicted labels, and ground truth\n",
"fig = plt.figure(figsize=(16, 9))\n",
"for i, idx in enumerate(np.random.choice(x_test.shape[0], size=16, replace=False)):\n",
" ax = fig.add_subplot(4, 4, i + 1, xticks=[], yticks=[])\n",
" ax.imshow(np.squeeze(x_test[idx]))\n",
" pred_idx = np.argmax(y_pred[idx])\n",
" true_idx = np.argmax(y_test[idx])\n",
" ax.set_title(\"{} ({})\".format(TYPES[pred_idx], TYPES[true_idx]),\n",
" color=(\"green\" if pred_idx == true_idx else \"red\"))"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAILCAYAAAA63AmGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd5wlRbXHf9V906Td2cgGYJecJEkQnwqK6SGu4DMDIhgI7yEqBgzg5aIg+uApBtRnxAAiogiCTwEBQZAsOcMCu8DGyXNDh3p/dE+fc2rmzs4syy6XOd/Phw91b59bXT1bp6ur63dOGWstFEVRFEVRFEVRWhlvUzdAURRFURRFURTlhaITG0VRFEVRFEVRWh6d2CiKoiiKoiiK0vLoxEZRFEVRFEVRlJZHJzaKoiiKoiiKorQ8OrFRFEVRFEVRFKXlmdITG1Mxc0zFPGQqpm0jn3czUzEPmoopbszzKkqrYSrmH6Zi9tzU7VgfTMWcYyrm+E3dDkV5MVEfVRTCVMxppmJ+tanbwTEVc6ypmG9Nwv46UzEffTHbtI7z32oqZpf1/X1uQzZmfTAVsxTAZgAiAEMA/gzgBFu2gxvh9J8H8HNbttW0LdcB2A9ACKAG4O8A/suW7XMb8qS2bFeYirkWwDEAvrMh61YUADAVw/2nHUAdiY8BwLG2bH+9EdpgAWxny/ax9fz9EgADtmzvMhXzAwBHpIcKAAySawKAG2zZHvSCG7zu9vwcwDJbtqdM8CdnA7jVVMxPbNk2XryWKa2I+uiGR31U2RiYijkMwEkAdgQwAOBfAM6wZXvjJm3YGJiKKQA4BcmzLf/uiwAOB7AAwCoAfwNwui3bpZugmS5nAzgdwLvW58cvlRWbJbZsOwG8EsDeSP4RNhimYkZN4NLVkg8BcGfWJ6Rt2R5AN4Bvvkht+TWAYzdk3Yoygi3bzpH/ADyN1MfS/170B6YNxHEAfgkAtmyPY9dzJoCL2PW86A9M60P6QuQhAO/Y1G1RXnqoj2561EeVyWIq5iQA30LSxzcDsCWA8wAc8iKcK+d8NqZiJvvcfgiAh2zZLmff/Q5Jnz8MwHQAuwO4A8AbX0BzRzFWe8d6Hh+DywC8wVTMvPU57yZfseHYsl1uKubPAF4BAKZi9gPwPwB2BvAUgE/Ysr0uPXY0gM8B2BzJbPPrtmx/mB57PZIJy3cAfArAVQA+6JzuVQB6bdkua9KWtaZiLgFwfFrnjml9e6XnO9WW7W/TYwcD+CqAbQD0AfiJLdvT0mOLATwJ4KMAygCWAtgfwC0AtjYVs8iW7VOT/2spyuRJJ/RfB/De9KvfAjjZlm2d+c23AXwGydvj4wE0kNzIZwM425btmWld+wI4F8BOAKoALgFwki3bhqmYv6f1352+Ff6ILduLTMV8DMDJAGYCuBHAcbZsnx2jnQUAB2ICk39TMe8A8DUAC5G8OTvelu2D6bGlAL6HxP+3AfAbJG+qfg7gtUj88D22bHtS+4sBvA5AG4C707ruNxVzDJK3W9ZUzCcBXGvLdompmJ0AfB/AHgCWA/iCLdvLWPOuA3AwkoFEUdaJ+qj6qPLSxFTMdCQrCUfbsv09O3R5+t8IBVMxvwDwTiQvLT5ky/b2tI4FSJ4l9wcwCOCbtmy/nR47Dcnzbw3JxOMkUzFHAPgHgNcjefn/ZVMx77dluxdr10kADrBlO9bk6iAA1zPbNwF4M4Dtbdk+k37dh8QHOYtMxfwDwG4AbgZwmC3b1Wkd4z2bX+e0d1dTMY8COAHAJwHkTMVcCaBmy/bTrF2XIfHZb9qyrZmKuQPAWwGcP8Y1jctLZcUGAGAqZgsAbwNwl6mYhQCuQDJhmInkJn6JqZg5qflKAG8HMA3A0QC+aSrmlay6eenvFiGRfLnsCuDhcdoyG8ky2F2mYjqQTI4uADAXwPsBnGcqZufUfAjAkUhWeA4GcLypmEOdKg9AMri8FQBs2YYAHkMyU1aUjcWXkCxJ74Gk7+0LuUI6D0AJyQPIlwH8CIm8ZC8kDxOnmorZKrWNkLw4mA3g1Uje9vwnANiy3T+12T19Y3uRqZgDkTzcvBfAfCQ3xN80aed2AOJmLx5GMBWzPYALkdww5wC4EsDl6UPXCO9CeiMHsASJ3PWLqb0H4ERm++f03HMB3IlkZRW2bP83LX8jvZ4lpmLySAazv6b2Hwfwa1MxO7D6HoT6uDI51EfVR5WXJq9G4nt/WIfdO5D4TTeS1YfvAkC6enE5kgn5QiT++ElTMW9lvz0EySS7G2nfRjLpPwZAF5KXGlulE3aw479o0hb3WfdNAG5lk5pmHIbk2XouEmnpZ9JrWNezudvekRf3hyJZUNgZyWTlAyOrOenz9puQPGOPsN5++VJZsbnUVEyIZNZ4BZIlvhMBXGnL9srU5ipTMbcjmficb8v2Cvb7603F/BXJTf3O9LsYQNmWbR1j041EG+nybVMxZyOZrFyHREf5dgBLbdn+LLW5K13NeQ+AyshMNeUeUzEXIpnIXMq+P82W7ZBzroG0HYqysTgcwMdt2a4EAFMxFQA/BHBqejxAohWOTMX8BsD/AjjXlu0AgPtNxTyA5GbzpC3bO1i9S03F/BBJv28WpHg4gJ/asr0zPfcXAPSYilk8hq63mX+6vA/AFbZsr0rrPBvAJwD8GxL/BYDv2LJdkR6/AcBKW7Z3pZ//ALb8bsv2pyPl9O1Zj6mY6bZs+8Y4934AOgGcZcs2BvA3UzF/AvABAKelNurjymRRH1UfVV6azAKwOn0xPR43jjy7mor5JZJJPQDsA2COLdvT089PmIr5EZKX5X9Jv7vZlu3Is2PVVAyQxILfn34Xmoq5CMnLjC+ZJMh+MYA/NWmL66ezAEwkbvxntmwfSa/htyC55hEY59k8/Y63F+k1fM2W7dr0q1tNxfQh8eur0uu/buQekDKA5OXKpHmpTGwOtWV7Nf/CVMwiAO8xSXDiCHkA16bHD0Ii7doeyRuddgD3MttVtmxr45yzB8ls0uVEW7Y/HqMtrzIV08u+ziHVFpuKeRWAs5AsIRYAFAFc7NQ71uy4C0DvGN8ryovFAtAbFKTlBezzGlu2IwHM1fT//GZTRfKgMPIm9n+QxMW1I/EJ/iA11rlHXjzAlu2gqZg1SN5cLXVsm/nnWHVm12PLNjYV80xa5whu+5tdjw/gDCQvLOYgeTkCJG+7x3poWgDgmfSBaYSnnHOrjyuTRX1UfVR5abIGwGxTMbl1TG6eZ+VhACWTxJYsArDAeZb0AdzAPo/1rOh+dz6AC03FnIJkdeS347zEd/10DZLn5nXhXkNnWh732bxJe8f67nwkk6Sr0v+f6xxfb798qUxsxuIZAL+0Zfsx90CqQb4Eifzrj7ZsA1MxlyLJwjKCXUf99yBZop9oW663ZfvmJscvQLLUeFCqDRzROnNEe9JOvi2SJUlF2Vg8i+TGNPI2Zcv0u/Xh+wDuAvABW7YDqa793RM4NwAglXjOQqJ7d3kMgDEVs9AJehyrzl1ZnQbAFk3qXBeHIZEBvAnJQ9x0JIPCyH3Fvac8C2ALUzEee3DaEsAjzGYnqI8rk0N9tDnqo8qm5GYkmf4OxfrFZD2DZCV1u3Fsxnp2Fd/Zsv2nqZgGEpXSYel/zbgHciJzNYBPmIrZfF0y0iY0fTZv1t4m3/0KwH2mYnZH4oOXOsd3wujkXhPipTyx+RWA21Lt4dVIZoT7IbmZ9iFZFVmFZFnuIABvAXDfJOq/FUD3BG7KQLLEd5apmA+C9MZ7ABhMAyC7AKxNJzX7Iulkf11Hnfsikbdp4gBlY3IhgFNMxdyG5EbzZaznzQNJv+8HMGiS5BrHI/HJEVYA2BqJz46c+0JTMRcg0c+eCeCWMSQuSIObr0Yim7nAPc74LYDPm4p5I5L07J9AMvDctJ7XU0fyRqs9bR9n5HpGuAXJm6zPmYo5B8BrkMQH7MNsDgAgVoAVZR2oj45/PeqjyibBlm2fqZgvA/heGj7xVyTS0DcBeIMt28+to4pbAQyYijkZSaxMA8kDfJst29sm2ZxfIHmhHtjx00xfi
"text/plain": [
"<Figure size 1152x648 with 16 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "iNZWDHS1NQ8e",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "aef36399-f0b9-49f2-bf8c-c602f9260e37"
},
"source": [
"plt.figure(1) \n",
" \n",
" \n",
"plt.subplot(211) \n",
"plt.plot(history.history['accuracy']) \n",
"plt.plot(history.history['val_accuracy']) \n",
"plt.title('model accuracy') \n",
"plt.ylabel('accuracy') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
" \n",
" \n",
"plt.subplot(212) \n",
"plt.plot(history.history['loss']) \n",
"plt.plot(history.history['val_loss']) \n",
"plt.title('model loss') \n",
"plt.ylabel('loss') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
"plt.show()\n",
"\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9bn48c8zk0kmCdkIi5KAoKIFUUECYl2qohXcl6po9afe3qLV3trWetVel9Ze7/W2va21Wq1bq1VRxKXU4i5ovW4ERAVRQYsmYTUh+555fn+cEzJJJmGSzOQkM8/79ZrXnPme7clJ8n3mnO8536+oKsYYY0xXPq8DMMYYMzRZgjDGGBORJQhjjDERWYIwxhgTkSUIY4wxEVmCMMYYE5ElCGMAEfmziPxnlMtuEpHj4h2TMV6zBGGMMSYiSxDGJBARSfE6BpM4LEGYYcO9tHO1iHwgInUicr+IjBWR50SkRkReFpG8sOVPFZF1IlIpIitEZErYvBkistpd73Eg2GVfJ4vIGnfdN0XkoChjPElE3hORahEpEZGfdZl/hLu9Snf+xW55uoj8r4h8ISJVIvKGW3a0iJRGOA7HudM/E5ElIvKwiFQDF4vIbBF5y93HFhG5Q0RSw9Y/QEReEpEKEdkmIj8VkT1EpF5E8sOWO0REdohIIJqf3SQeSxBmuDkLOB7YDzgFeA74KTAa5+/5BwAish+wCPihO28Z8DcRSXUry2eAvwAjgSfc7eKuOwN4ALgUyAf+CCwVkbQo4qsD/h+QC5wEfE9ETne3u5cb7+/dmKYDa9z1fg3MBL7uxvTvQCjKY3IasMTd5yNAG/AjYBRwGDAXuNyNIQt4GXgeGAfsC7yiqluBFcA5Ydu9EHhMVVuijMMkGEsQZrj5vapuU9Uy4B/AO6r6nqo2Ak8DM9zlzgX+rqovuRXcr4F0nAp4DhAAblPVFlVdAqwM28dC4I+q+o6qtqnqg0CTu16vVHWFqn6oqiFV/QAnSX3DnX0+8LKqLnL3W66qa0TEB/wLcKWqlrn7fFNVm6I8Jm+p6jPuPhtUdZWqvq2qraq6CSfBtcdwMrBVVf9XVRtVtUZV33HnPQhcACAifuA8nCRqkpQlCDPcbAubbojweYQ7PQ74on2GqoaAEqDAnVemnXuq/CJsei/gKvcSTaWIVALj3fV6JSKHishy99JMFXAZzjd53G18FmG1UTiXuCLNi0ZJlxj2E5FnRWSre9npv6KIAeCvwFQRmYRzllalqu/2MyaTACxBmES1GaeiB0BEBKdyLAO2AAVuWbsJYdMlwC2qmhv2ylDVRVHs91FgKTBeVXOAu4H2/ZQA+0RY5yugsYd5dUBG2M/hx7k8Fa5rl8x3AR8Dk1U1G+cSXHgMe0cK3D0LW4xzFnEhdvaQ9CxBmES1GDhJROa6jaxX4VwmehN4C2gFfiAiARE5E5gdtu69wGXu2YCISKbb+JwVxX6zgApVbRSR2TiXldo9AhwnIueISIqI5IvIdPfs5gHgNyIyTkT8InKY2+bxKRB09x8Argd21xaSBVQDtSLyNeB7YfOeBfYUkR+KSJqIZInIoWHzHwIuBk7FEkTSswRhEpKqfoLzTfj3ON/QTwFOUdVmVW0GzsSpCCtw2iueClu3GPgucAewE9joLhuNy4GbRaQGuBEnUbVv90vgRJxkVYHTQH2wO/snwIc4bSEVwP8APlWtcrd5H87ZTx3Q6a6mCH6Ck5hqcJLd42Ex1OBcPjoF2ApsAI4Jm/9/OI3jq1U1/LKbSUJiAwYZY8KJyKvAo6p6n9exGG9ZgjDG7CIis4CXcNpQaryOx3jLLjEZYwAQkQdxnpH4oSUHA3YGYYwxpgd2BmGMMSaihOnYa9SoUTpx4kSvwzDGmGFl1apVX6lq12drgARKEBMnTqS4uNjrMIwxZlgRkR5vZ47bJSYReUBEtovI2h7mi4jcLiIbxemd85CweReJyAb3dVG8YjTGGNOzeLZB/BmY18v8+cBk97UQp3sARGQkcBNwKM7TrTdJWBfOxhhjBkfcLjGp6usiMrGXRU4DHnI7THtbRHJFZE/gaOAlVa0AEJGXcBJNNP3gGJeq0tDSRnVDK9WNLVQ3tLjv4Z9bqW5ooaaxc1ldU6vX4Rtj+mBaQQ4PXDwr5tv1sg2igM69UJa6ZT2VdyMiC3HOPpgwYUK3+S0tLZSWltLY2BijkAdXSBVV5z2kTqXfPh0+ry2kbK8P8cT6erbUdFT0baHeb2EOBnxkBwNkpwfIDqaQm5HKhPxMMlP9dOrGzhgzpBXmZex+oX4Y1o3UqnoPcA9AUVFRt9qwtLSUrKwsJk6ciAzxGq+5tY0vKxpoDYVoCymhkHbqorP9WqA/vEwEv0/wCeTWVXGufMWKzbiVfkpY5e98zgo6iSA7PUBWMIW0lPCtGWNMZ14miDKc7pfbFbplZTiXmcLLV/RnB42NjcMiOQCU1zXT0NxGTkYAv0/wu5W/3wd+EXw+6VTu8wm+sJ9LNYuU5lrOOGpKL3sxxpjoeZkglgLfF5HHcBqkq1R1i4i8APxXWMP0N4Hr+ruT4ZAcQqrsrGshK5jChJH9O1UcDj+nMWZ4iVuCEJFFOGcCo9xB12/CGeYRVb0bZ4zgE3G6Uq4HLnHnVYjIL+gYAvLm9gbrRFXb2EprKMTIzHSvQzHGmF3ieRfTebuZr8AVPcx7AGcAlWGvsrKSRx99lMsvv7zHZSrqmknx+8gKdvw6TjzxRB599FFyc3MHI0xjjOnG+mKKs8rKSv7whz90K29tdW4lbWkLUdPYSl5GoNNlomXLlllyMMZ4aljfxTQcXHvttXz22WdMnz6dQCBAMBgkLy+Pjz/+mE8//ZRTTzuNL74sgdYWfvjDK1m4cCHQ0XVIbW0t8+fP54gjjuDNN9+koKCAv/71r6Sn2+UoY0x8JU2C+Pnf1vHR5uqYbnPquGxuOuWAXpe59dZbWbt2LWvWrGHFihWcdNJJrF27lkmTJqGq/OxXdzAqP59xWX5mzZrFWWedRX5+fqdtbNiwgUWLFnHvvfdyzjnn8OSTT3LBBRfE9GcxxpiukiZBDBWzZ89m0qRJANQ3t/Hne+/iHy8tw+8TSkpK2LBhQ7cEMWnSJKZPnw7AzJkz2bRp02CHbYxJQkmTIHb3TX+wZGZm7pr++wsv884br/HWW2+RNSKTo48+OuJT32lpabum/X4/DQ0NgxKrMSa5WSN1nGVlZVFT0330xraQsvWrCvJH5pE1IpOPP/6Yt99+24MIjTEmsqQ5g/BKfn4+hx9+ONOmTSM9PZ2xY8cCUNXQzNe/MZdnH3+IKVOmsP/++zNnzhyPozXGmA4JMyZ1UVGRdh0waP369UyZMjS7nti4vZa2kLLf2BExewp6KP+8xpihSURWqWpRpHl2ickDjS1t1De3MjIzYF1kGGOGLLvE5IGd9c0IQm5GqtehGGO81tYKbU3Q2gRtzc57a5Nb1twxr7eyzDEw49sxD80SxCBr75gvOz2FgN9O4IzH2lqhsRIaKqFhp/NqDJtuL28va6wCVRAB8TkvxP3ctcwXm+UIm9epTMLecaeJUBZpub6U0fNyGoqyEu9a8YeVadvAf4/jDrEEkQhq3I758jLsSeghJRSC5lpoqoamGmh035uqnUrAHwB/mvOekgb+1LCyVEhJdcvCX4GwiiWOVKG5LroKfleZ+7m5+x12naRlQ3ouBHMhPQ9GjQHxO8cEdfatoY532j93LXOnQ6HdLBcCpWMeXbbfdbn2UVNUO/bT77J+bA9x/h5S0py/hZRU9z2t4+8kI7N7Wad1eisLhm3T/dxpP+1l8bkaYQlikO2saybQpWM+MwCq0NLQpWKvjlzRRyyr6Xh1GqIpRvypHYmlUyLpKdn0UOZPhZb6zpV+eMUf6mWYWF/AqdzT85zKPnscjD2go9JvL0/P61wWzAG//Z0mM/vtDyKnY74WRmelWeN0JC2NUFUKVSXOq3Z750q8a2Xf/jmaU/RAhvNtOC0Lgu571h5OWfvntKwuy7jTvpSOSwJtLc4lgrZm9zJCl9eus
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "IEEV1PwvbP2i",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "f2af12fd-f064-4a64-80e6-02ebbf67629d"
},
"source": [
"y_pred = model.predict(x_test)\n",
"zol = zero_one(y_pred, y_test)\n",
"\n",
"print(\"Zero-one Loss: \", zol)\n",
"\n",
"cnn_loss.append(zol)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Zero-one Loss: 0.016498625114573784\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0q1HqrTfODxB",
"colab_type": "text"
},
"source": [
"# 1.6 MobileNetV2"
]
},
{
"cell_type": "code",
"metadata": {
"id": "FfRYQkCGODjZ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "f6f4805e-9e5f-4ce3-b879-5999cc961a7c"
},
"source": [
"\n",
"model = MobileNetV2(input_shape=(32, 32, 3), alpha=1, weights=None,classes=10)\n",
"model.compile(loss='categorical_crossentropy', \n",
" optimizer='adam',\n",
" metrics=['accuracy'])\n",
"print(model.summary())\n",
"print('Compiled!')\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Model: \"mobilenetv2_1.00_32\"\n",
"__________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"==================================================================================================\n",
"input_12 (InputLayer) (None, 32, 32, 3) 0 \n",
"__________________________________________________________________________________________________\n",
"Conv1_pad (ZeroPadding2D) (None, 33, 33, 3) 0 input_12[0][0] \n",
"__________________________________________________________________________________________________\n",
"Conv1 (Conv2D) (None, 16, 16, 32) 864 Conv1_pad[0][0] \n",
"__________________________________________________________________________________________________\n",
"bn_Conv1 (BatchNormalization) (None, 16, 16, 32) 128 Conv1[0][0] \n",
"__________________________________________________________________________________________________\n",
"Conv1_relu (ReLU) (None, 16, 16, 32) 0 bn_Conv1[0][0] \n",
"__________________________________________________________________________________________________\n",
"expanded_conv_depthwise (Depthw (None, 16, 16, 32) 288 Conv1_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"expanded_conv_depthwise_BN (Bat (None, 16, 16, 32) 128 expanded_conv_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"expanded_conv_depthwise_relu (R (None, 16, 16, 32) 0 expanded_conv_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"expanded_conv_project (Conv2D) (None, 16, 16, 16) 512 expanded_conv_depthwise_relu[0][0\n",
"__________________________________________________________________________________________________\n",
"expanded_conv_project_BN (Batch (None, 16, 16, 16) 64 expanded_conv_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_expand (Conv2D) (None, 16, 16, 96) 1536 expanded_conv_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_expand_BN (BatchNormali (None, 16, 16, 96) 384 block_1_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_expand_relu (ReLU) (None, 16, 16, 96) 0 block_1_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_pad (ZeroPadding2D) (None, 17, 17, 96) 0 block_1_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_depthwise (DepthwiseCon (None, 8, 8, 96) 864 block_1_pad[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_depthwise_BN (BatchNorm (None, 8, 8, 96) 384 block_1_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_depthwise_relu (ReLU) (None, 8, 8, 96) 0 block_1_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_project (Conv2D) (None, 8, 8, 24) 2304 block_1_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_1_project_BN (BatchNormal (None, 8, 8, 24) 96 block_1_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_expand (Conv2D) (None, 8, 8, 144) 3456 block_1_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_expand_BN (BatchNormali (None, 8, 8, 144) 576 block_2_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_expand_relu (ReLU) (None, 8, 8, 144) 0 block_2_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_depthwise (DepthwiseCon (None, 8, 8, 144) 1296 block_2_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_depthwise_BN (BatchNorm (None, 8, 8, 144) 576 block_2_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_depthwise_relu (ReLU) (None, 8, 8, 144) 0 block_2_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_project (Conv2D) (None, 8, 8, 24) 3456 block_2_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_project_BN (BatchNormal (None, 8, 8, 24) 96 block_2_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_2_add (Add) (None, 8, 8, 24) 0 block_1_project_BN[0][0] \n",
" block_2_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_expand (Conv2D) (None, 8, 8, 144) 3456 block_2_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_expand_BN (BatchNormali (None, 8, 8, 144) 576 block_3_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_expand_relu (ReLU) (None, 8, 8, 144) 0 block_3_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_pad (ZeroPadding2D) (None, 9, 9, 144) 0 block_3_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_depthwise (DepthwiseCon (None, 4, 4, 144) 1296 block_3_pad[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_depthwise_BN (BatchNorm (None, 4, 4, 144) 576 block_3_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_depthwise_relu (ReLU) (None, 4, 4, 144) 0 block_3_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_project (Conv2D) (None, 4, 4, 32) 4608 block_3_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_3_project_BN (BatchNormal (None, 4, 4, 32) 128 block_3_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_expand (Conv2D) (None, 4, 4, 192) 6144 block_3_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_expand_BN (BatchNormali (None, 4, 4, 192) 768 block_4_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_expand_relu (ReLU) (None, 4, 4, 192) 0 block_4_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_depthwise (DepthwiseCon (None, 4, 4, 192) 1728 block_4_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_depthwise_BN (BatchNorm (None, 4, 4, 192) 768 block_4_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_depthwise_relu (ReLU) (None, 4, 4, 192) 0 block_4_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_project (Conv2D) (None, 4, 4, 32) 6144 block_4_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_project_BN (BatchNormal (None, 4, 4, 32) 128 block_4_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_4_add (Add) (None, 4, 4, 32) 0 block_3_project_BN[0][0] \n",
" block_4_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_expand (Conv2D) (None, 4, 4, 192) 6144 block_4_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_expand_BN (BatchNormali (None, 4, 4, 192) 768 block_5_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_expand_relu (ReLU) (None, 4, 4, 192) 0 block_5_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_depthwise (DepthwiseCon (None, 4, 4, 192) 1728 block_5_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_depthwise_BN (BatchNorm (None, 4, 4, 192) 768 block_5_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_depthwise_relu (ReLU) (None, 4, 4, 192) 0 block_5_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_project (Conv2D) (None, 4, 4, 32) 6144 block_5_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_project_BN (BatchNormal (None, 4, 4, 32) 128 block_5_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_5_add (Add) (None, 4, 4, 32) 0 block_4_add[0][0] \n",
" block_5_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_expand (Conv2D) (None, 4, 4, 192) 6144 block_5_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_expand_BN (BatchNormali (None, 4, 4, 192) 768 block_6_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_expand_relu (ReLU) (None, 4, 4, 192) 0 block_6_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_pad (ZeroPadding2D) (None, 5, 5, 192) 0 block_6_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_depthwise (DepthwiseCon (None, 2, 2, 192) 1728 block_6_pad[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_depthwise_BN (BatchNorm (None, 2, 2, 192) 768 block_6_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_depthwise_relu (ReLU) (None, 2, 2, 192) 0 block_6_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_project (Conv2D) (None, 2, 2, 64) 12288 block_6_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_6_project_BN (BatchNormal (None, 2, 2, 64) 256 block_6_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_expand (Conv2D) (None, 2, 2, 384) 24576 block_6_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_expand_BN (BatchNormali (None, 2, 2, 384) 1536 block_7_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_expand_relu (ReLU) (None, 2, 2, 384) 0 block_7_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_depthwise (DepthwiseCon (None, 2, 2, 384) 3456 block_7_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_depthwise_BN (BatchNorm (None, 2, 2, 384) 1536 block_7_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_depthwise_relu (ReLU) (None, 2, 2, 384) 0 block_7_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_project (Conv2D) (None, 2, 2, 64) 24576 block_7_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_project_BN (BatchNormal (None, 2, 2, 64) 256 block_7_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_7_add (Add) (None, 2, 2, 64) 0 block_6_project_BN[0][0] \n",
" block_7_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_expand (Conv2D) (None, 2, 2, 384) 24576 block_7_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_expand_BN (BatchNormali (None, 2, 2, 384) 1536 block_8_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_expand_relu (ReLU) (None, 2, 2, 384) 0 block_8_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_depthwise (DepthwiseCon (None, 2, 2, 384) 3456 block_8_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_depthwise_BN (BatchNorm (None, 2, 2, 384) 1536 block_8_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_depthwise_relu (ReLU) (None, 2, 2, 384) 0 block_8_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_project (Conv2D) (None, 2, 2, 64) 24576 block_8_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_project_BN (BatchNormal (None, 2, 2, 64) 256 block_8_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_8_add (Add) (None, 2, 2, 64) 0 block_7_add[0][0] \n",
" block_8_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_expand (Conv2D) (None, 2, 2, 384) 24576 block_8_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_expand_BN (BatchNormali (None, 2, 2, 384) 1536 block_9_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_expand_relu (ReLU) (None, 2, 2, 384) 0 block_9_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_depthwise (DepthwiseCon (None, 2, 2, 384) 3456 block_9_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_depthwise_BN (BatchNorm (None, 2, 2, 384) 1536 block_9_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_depthwise_relu (ReLU) (None, 2, 2, 384) 0 block_9_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_project (Conv2D) (None, 2, 2, 64) 24576 block_9_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_project_BN (BatchNormal (None, 2, 2, 64) 256 block_9_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_9_add (Add) (None, 2, 2, 64) 0 block_8_add[0][0] \n",
" block_9_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_10_expand (Conv2D) (None, 2, 2, 384) 24576 block_9_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_10_expand_BN (BatchNormal (None, 2, 2, 384) 1536 block_10_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_10_expand_relu (ReLU) (None, 2, 2, 384) 0 block_10_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_10_depthwise (DepthwiseCo (None, 2, 2, 384) 3456 block_10_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_10_depthwise_BN (BatchNor (None, 2, 2, 384) 1536 block_10_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_10_depthwise_relu (ReLU) (None, 2, 2, 384) 0 block_10_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_10_project (Conv2D) (None, 2, 2, 96) 36864 block_10_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_10_project_BN (BatchNorma (None, 2, 2, 96) 384 block_10_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_expand (Conv2D) (None, 2, 2, 576) 55296 block_10_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_expand_BN (BatchNormal (None, 2, 2, 576) 2304 block_11_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_expand_relu (ReLU) (None, 2, 2, 576) 0 block_11_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_depthwise (DepthwiseCo (None, 2, 2, 576) 5184 block_11_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_depthwise_BN (BatchNor (None, 2, 2, 576) 2304 block_11_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_depthwise_relu (ReLU) (None, 2, 2, 576) 0 block_11_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_project (Conv2D) (None, 2, 2, 96) 55296 block_11_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_project_BN (BatchNorma (None, 2, 2, 96) 384 block_11_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_11_add (Add) (None, 2, 2, 96) 0 block_10_project_BN[0][0] \n",
" block_11_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_expand (Conv2D) (None, 2, 2, 576) 55296 block_11_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_expand_BN (BatchNormal (None, 2, 2, 576) 2304 block_12_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_expand_relu (ReLU) (None, 2, 2, 576) 0 block_12_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_depthwise (DepthwiseCo (None, 2, 2, 576) 5184 block_12_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_depthwise_BN (BatchNor (None, 2, 2, 576) 2304 block_12_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_depthwise_relu (ReLU) (None, 2, 2, 576) 0 block_12_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_project (Conv2D) (None, 2, 2, 96) 55296 block_12_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_project_BN (BatchNorma (None, 2, 2, 96) 384 block_12_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_12_add (Add) (None, 2, 2, 96) 0 block_11_add[0][0] \n",
" block_12_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_expand (Conv2D) (None, 2, 2, 576) 55296 block_12_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_expand_BN (BatchNormal (None, 2, 2, 576) 2304 block_13_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_expand_relu (ReLU) (None, 2, 2, 576) 0 block_13_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_pad (ZeroPadding2D) (None, 3, 3, 576) 0 block_13_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_depthwise (DepthwiseCo (None, 1, 1, 576) 5184 block_13_pad[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_depthwise_BN (BatchNor (None, 1, 1, 576) 2304 block_13_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_depthwise_relu (ReLU) (None, 1, 1, 576) 0 block_13_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_project (Conv2D) (None, 1, 1, 160) 92160 block_13_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_13_project_BN (BatchNorma (None, 1, 1, 160) 640 block_13_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_expand (Conv2D) (None, 1, 1, 960) 153600 block_13_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_expand_BN (BatchNormal (None, 1, 1, 960) 3840 block_14_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_expand_relu (ReLU) (None, 1, 1, 960) 0 block_14_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_depthwise (DepthwiseCo (None, 1, 1, 960) 8640 block_14_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_depthwise_BN (BatchNor (None, 1, 1, 960) 3840 block_14_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_depthwise_relu (ReLU) (None, 1, 1, 960) 0 block_14_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_project (Conv2D) (None, 1, 1, 160) 153600 block_14_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_project_BN (BatchNorma (None, 1, 1, 160) 640 block_14_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_14_add (Add) (None, 1, 1, 160) 0 block_13_project_BN[0][0] \n",
" block_14_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_expand (Conv2D) (None, 1, 1, 960) 153600 block_14_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_expand_BN (BatchNormal (None, 1, 1, 960) 3840 block_15_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_expand_relu (ReLU) (None, 1, 1, 960) 0 block_15_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_depthwise (DepthwiseCo (None, 1, 1, 960) 8640 block_15_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_depthwise_BN (BatchNor (None, 1, 1, 960) 3840 block_15_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_depthwise_relu (ReLU) (None, 1, 1, 960) 0 block_15_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_project (Conv2D) (None, 1, 1, 160) 153600 block_15_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_project_BN (BatchNorma (None, 1, 1, 160) 640 block_15_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_15_add (Add) (None, 1, 1, 160) 0 block_14_add[0][0] \n",
" block_15_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_16_expand (Conv2D) (None, 1, 1, 960) 153600 block_15_add[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_16_expand_BN (BatchNormal (None, 1, 1, 960) 3840 block_16_expand[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_16_expand_relu (ReLU) (None, 1, 1, 960) 0 block_16_expand_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_16_depthwise (DepthwiseCo (None, 1, 1, 960) 8640 block_16_expand_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_16_depthwise_BN (BatchNor (None, 1, 1, 960) 3840 block_16_depthwise[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_16_depthwise_relu (ReLU) (None, 1, 1, 960) 0 block_16_depthwise_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_16_project (Conv2D) (None, 1, 1, 320) 307200 block_16_depthwise_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"block_16_project_BN (BatchNorma (None, 1, 1, 320) 1280 block_16_project[0][0] \n",
"__________________________________________________________________________________________________\n",
"Conv_1 (Conv2D) (None, 1, 1, 1280) 409600 block_16_project_BN[0][0] \n",
"__________________________________________________________________________________________________\n",
"Conv_1_bn (BatchNormalization) (None, 1, 1, 1280) 5120 Conv_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"out_relu (ReLU) (None, 1, 1, 1280) 0 Conv_1_bn[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_average_pooling2d_1 (Glo (None, 1280) 0 out_relu[0][0] \n",
"__________________________________________________________________________________________________\n",
"Logits (Dense) (None, 10) 12810 global_average_pooling2d_1[0][0] \n",
"==================================================================================================\n",
"Total params: 2,270,794\n",
"Trainable params: 2,236,682\n",
"Non-trainable params: 34,112\n",
"__________________________________________________________________________________________________\n",
"None\n",
"Compiled!\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "nh3LDJGzOJiG",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 392
},
"outputId": "8f23828f-5230-40f5-9df6-0dff6163bb94"
},
"source": [
"history = model.fit(x_train,y_train,\n",
" batch_size = 32,\n",
" epochs=10,\n",
" validation_data=(x_valid, y_valid),\n",
" verbose=2 )"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Train on 32607 samples, validate on 8724 samples\n",
"Epoch 1/10\n",
" - 152s - loss: 0.6746 - accuracy: 0.7860 - val_loss: 2.2316 - val_accuracy: 0.1977\n",
"Epoch 2/10\n",
" - 148s - loss: 0.1768 - accuracy: 0.9496 - val_loss: 2.2468 - val_accuracy: 0.1977\n",
"Epoch 3/10\n",
" - 148s - loss: 0.1177 - accuracy: 0.9681 - val_loss: 2.2137 - val_accuracy: 0.1977\n",
"Epoch 4/10\n",
" - 147s - loss: 0.1166 - accuracy: 0.9705 - val_loss: 2.1745 - val_accuracy: 0.1977\n",
"Epoch 5/10\n",
" - 147s - loss: 0.0993 - accuracy: 0.9756 - val_loss: 0.8243 - val_accuracy: 0.7942\n",
"Epoch 6/10\n",
" - 149s - loss: 0.0763 - accuracy: 0.9815 - val_loss: 0.9804 - val_accuracy: 0.7658\n",
"Epoch 7/10\n",
" - 146s - loss: 0.0685 - accuracy: 0.9836 - val_loss: 2.2230 - val_accuracy: 0.6530\n",
"Epoch 8/10\n",
" - 149s - loss: 0.0582 - accuracy: 0.9859 - val_loss: 9.0434 - val_accuracy: 0.4411\n",
"Epoch 9/10\n",
" - 148s - loss: 0.0601 - accuracy: 0.9856 - val_loss: 4.6619 - val_accuracy: 0.6551\n",
"Epoch 10/10\n",
" - 149s - loss: 0.0429 - accuracy: 0.9905 - val_loss: 2.4783 - val_accuracy: 0.7839\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "WMvIAWyKOPKS",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 540
},
"outputId": "81814288-ac21-44b5-8545-29a594fb3b0c"
},
"source": [
"y_pred = model.predict(x_test)\n",
"\n",
"# plot a random sample of test images, their predicted labels, and ground truth\n",
"fig = plt.figure(figsize=(16, 9))\n",
"for i, idx in enumerate(np.random.choice(x_test.shape[0], size=16, replace=False)):\n",
" ax = fig.add_subplot(4, 4, i + 1, xticks=[], yticks=[])\n",
" ax.imshow(np.squeeze(x_test[idx]))\n",
" pred_idx = np.argmax(y_pred[idx])\n",
" true_idx = np.argmax(y_test[idx])\n",
" ax.set_title(\"{} ({})\".format(TYPES[pred_idx], TYPES[true_idx]),\n",
" color=(\"green\" if pred_idx == true_idx else \"red\"))\n",
"\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAzYAAAILCAYAAAA63AmGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeZhkRZX237g3l1q7eoemabqVRQUVFwZQGXHQ0UFsdxgVBXVEwd1xG3UwSRTUGXADV0RFEEXFwQ0+wVFQGUHZEXBDGuh9rT23mze+P+7NG+dEVVZXF70l9f6eh4fIuifjRmbHuRGR8Z4TxloLQgghhBBCCOlkgj3dAEIIIYQQQgh5pHBhQwghhBBCCOl4uLAhhBBCCCGEdDxc2BBCCCGEEEI6Hi5sCCGEEEIIIR0PFzaEEEIIIYSQjmdWL2xM2dxoyuape7odM8GUzfmmbM7Y0+0gZCpM2bzelM1vH8H732LK5rM7s01T3Oubpmw+Pk3b35uyOWxXt4mQjsKYIoy5F8Ys2Q33WgFjLIzJTcN2JYy5Ype3iZC9kNk2193+A2HyG42Klz0AagCa6eu32JL99kzq3cE2WAAH25L92wzfvxLAiC3Z203ZfBnAa9NLBQAGyWcCgN/Ykj3+ETd4++35JoDVtmT/c5pvOQ/A703ZXGxLtr7rWkZmC6ZsrgdwOIB9bcnWtmO+yzFlUwDwnwCO9v7eB2A9dpNvtuE8AGcDeMUeuj95NGPMKgD7IBlXxwBcA+DtsHZ0qrftBbwZwK9h7Tr1V2POAlACcDSsvXm3t8ran8CYT8CYJ8Pau3b7/UlHwrnuzmd3zHVntLCxJdvXKpuyWQXgTbZkfzGTuvYgpwO4FABsyZ6evoYpm7MAHGRL9rXt37rnsSW7zpTNnwC8GMAP9nR7SGdjymYFgH8EMISkT31/T7Yn5SUA/mRLdo3391cgeRj/symbfW3Jrt/9TcOPAXx5D96fPPpZCWt/AWOWAvg5kkX+f+zhNiUYk4O10SRXTkeyuJG2BsApALam/9/9C5uE7yBp29v30P1Jh8G57p5nJnPdGS1s2mHKpgjgUwBOSv/0PQAftCVbM2XzHACXAfg8gPchWfWeAaAO4LMAFgI4z5bsuWldRwL4HIAnAKgAuBLAv9uSrZuy+XVa/53pavbfbMleYcrmNAAfBDAfwG8BnG5Ldu0k7SwAOA7AW6bxmV4M4BMAlgK4A8AZtmTvS6+tAvAFAK8DcCCA7wL4MIBvAjgGyQP8RFuy21L77yOZPHYDuDOt6x5TNm8GcDIAa8rm3QB+ZUt2pSmbJwD4EoCnAFgD4EO2ZH8smnc9gBPAhQ155JwC4CYkffZUiIVN+gtLFUkfPxrAbQBOsSX7YHrdAngXgHcDmAPgG0j8PvZvYsrm8QAuAPB0AJsAnGlL9ntt2nQ8gBsm+fupAL6cXn8tkl90WvWvAvAVJD65BMBVSPysKp5BXwTw7wBGAXyk3a9upmxeBODjAFYAuBfJ8+QuAEjruxXACwBc0qb9hDxyrF0DY64B8EQAgDFHA/g0gEMBPAjgXbD2+vTa9QB+B+C5AB4P4FcA3gBrt8KYFQAeQDLunYXk19rzYe156XsDAB8AcBqAuQD+F8Dp3nvfhGTnZRWAZ6t2GnMAgMdi4sLlH5H44psAfB7GvAc2/eXVmNen97sdic+uA/A2WPu/2/08PsYMpN/LCwHESJ5DJVjb+oX9eiT+z4UNeURwrrt3z3V3dozNR5BMfJ6CRNJyJJJfmVrsC6ALyRf3UQAXIZmYPB3Jl3CmKZvHpLZNAO9B0gmegeTB9lYAsCXbeqAebku2L/2HPg7JP8pJSB6iDyL58ifjYACxLdnVU30YUzaHIPmV590AFgG4GsBP0s7S4hUA/hnAIQBWIpEMfDi1DwC8U9hek957MZLJ4bfTz/PVtPxf6edZacomD+AnAK5N7d8B4NumbB4n6rsPyfdMyCPlFCR98NsAXmDKZh/v+skAPobEH+9I7SQvA3AEgKch2Wl5o38DUza9AK4DcDmSPv0qAF80ZXNomzY9CcCfvTqWA3iOaOspk7zvZCQLjgOR+KX/DFqI5Bl0KoCvej7Vus9TAXwdyYCwAMli6cfpgNaC/kd2PcYsQzJZvz3dvfkZkgX3fCQTpythzCLxjlOQ+N8SABGSCZbkn5CMQ88H8EEY87z07+8A8FIAxwLYD8A2JJMZybFIJmAvmKSlTwLw90l2ck5FMpa1fsBY6V0/CsD9SPyyBOCHMGb+DnyeFt9Mrx8E4Knp53uTuH4fgBUwZk6b9xMyXTjX3Yvnujt7YXMygLNtyW60JbsJQBnJCq9FA8A5tmQbSP4hFgL4nC3ZEVuy9yD5VfRwALAle6st2ZtsyUa2ZFchmVgcu517f92W7G1pfMCHADwjldj4zAUwMo3P868AfmZL9rq0zechWYE+U9hcYEt2QyqX+Q2Am23J3m5Ltgrgf5A8YJF+pq+nn7WG5Bezw03ZDLS599EA+gB80pZs3ZbsLwH8FMCrhc1I+lkImTGmbI4BsBzA92zJ3opkkvEaz+xntmR/nfbdjyDxrWXi+qdsyW61JfsQkl+lXo2JvAjAKluy30j9+nYkv06d2KZpk/np6wDcZUv2XiTPkMMmCYq80Jbsw7ZktwI4Z5K2nGlLtmZL9gYkk8STMJE3A/iKLdmbbck2bclegkT+JuN96H9kV3IVjBlE8ovsDQDORTI5uhrWXg1rY1h7HYBbkCx8WlwKa/8Ia8cAnAngJBgTiutlWDsGa+9GsqvR8o/TAXwE1q6GzcaoV3rB+Wel761M0t6J/mpMDxL/vhzWNpD84ur/GLERwGdhbQPWXoHkx4wTduDzAMbsk34H707btxHAZ5D8eNKi1Tb6LHmkcK67F891d6oUDcmvPA+K1w+mf2uxxZaybeHWg3GDuF5B8gFbK8hPI/kVuCdt663bufdtrRe2ZEdN2WxBsmJe5dluA9C//Y+jP48t2diUzcNpnS389rf7PCGSSdaJSFa4LZnOQiRxDZPd+2FPzvOgd+9+AIPT+ByETMWpAK61Jbs5fX15+rfPCJuHW4XUt7Yi7aP+dUz0+xbLARxlykb22RxS/e8kTOanpyD59Qu2ZNeYsrkhbevtk7V1krZssyU7Ns22nmrK5h3ibwXPlv5HdiUvhfX0/MYsB3AijJG7HnkkEq0Wfv/PIxln2l1/UlpeDuB/YIwcc5pIkhhM9l6fyfz1ZUh2Ua5OX38bwC9gzCJYuyn92xpYa702ST/b3udptT0PYB2Maf0t8N7baht9ljxSONfdi+e6O3vHZi2SB0yLA9K/zYQvAfgTkmwQc5BseZkp7NW9U9nLAiR6PZ+/ATCmbJZOcm2qOg2AZW3q3B6vQSLReR6AASS6fcB9JuvZrwWwzJSN/Dc6wLv3E5DoFwmZEaZsupHsWBxryma9KZv1SLbFDzdlI7d+l4n39CGRwayd7Dra+/3DAG6wJTtX/NdnS7ZdKse7kGx7t+77TCTb2x8SbT0KwGtMWf2qPFVb5qXPhum09RyvrT22ZL8jbOh/ZHfzMJIdjLniv15Y+0lh4/f/BoDNU1xv9f+HARzv1d0Fq5J3+OOU5C4Aj/F2eE5FMuF5CMasRxK7l4feEV6aJhiYrE3T+TytttcALBRtnwNrZUr2JwBYBWuHp/gMhEwHznXbs8fnujt7YfMdAP9pymaRKZuFSLSFl82wrn4AwwBG04Bjf/KzAUmgorz3G0zZPCXVwZ+LZKtslV+xTVLG/QJTb/cBiSb4BFM2z011gO9F8vD8vxl+nhqALUhW5ed61/3PczOAcQAfMGWTTwPSVkJrKY9FomUkZKa8FMmvsoci0Qs/BclD5DfQkpEXmrI5JtXcfgzATbZk5a+h7zdlMy+Vp70LwGRnRvwUwCGmbF6X9um8KZt/SAMHJ+NqaB89FUmMjmzrE5Fsmcs0lW8zZbO/KZv5SGRzflvKpmwKpmz+EYk8brIMcBcBON2UzVGmbIwpm15TNieYsukHAFM2XUj00te1aTshu4LLAKyEMS+AM
"text/plain": [
"<Figure size 1152x648 with 16 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "nUUJ3woiOQ_B",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "4a38780d-6bbd-47af-f1b2-fe561f69bf4c"
},
"source": [
"\n",
"plt.figure(1) \n",
" \n",
" \n",
"plt.subplot(211) \n",
"plt.plot(history.history['accuracy']) \n",
"plt.plot(history.history['val_accuracy']) \n",
"plt.title('model accuracy') \n",
"plt.ylabel('accuracy') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
" \n",
" \n",
"plt.subplot(212) \n",
"plt.plot(history.history['loss']) \n",
"plt.plot(history.history['val_loss']) \n",
"plt.title('model loss') \n",
"plt.ylabel('loss') \n",
"plt.xlabel('epoch') \n",
"plt.legend(['train', 'test'], loc='upper left') \n",
"plt.show()"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5fX48c+ZJJAEAoGAIHu0qCgqKIKKKNYNFMUVN9wtbnXp11q1X5dq2++P1ta6L7ivuIAoLiiiqFgVDZuCqICghH2HBBKSzPn98dxJJmEShmQmNzNz3q/XvDJztzkzSe655z73Po+oKsYYY0xNAb8DMMYY0zRZgjDGGBORJQhjjDERWYIwxhgTkSUIY4wxEVmCMMYYE5ElCGMAEXlWRP4W5bJLROTYeMdkjN8sQRhjjInIEoQxSURE0v2OwSQPSxAmYXindm4SkW9FpFhEnhKRDiIySUS2iMgUEWkTtvwpIjJPRDaKyCci0itsXl8Rmemt9yqQWeO9honIbG/dL0TkgChjPElEZonIZhFZKiJ/qTH/CG97G735F3vTs0Tk3yLyi4hsEpHPvWmDRaQwwvdwrPf8LyIyTkReFJHNwMUi0l9EvvTeY4WIPCQizcLW309EPhSR9SKySkT+LCIdRWSriOSFLXeQiKwRkYxoPrtJPpYgTKI5AzgO2As4GZgE/Bloj/t7vg5ARPYCxgI3ePPeA94WkWbezvJN4AWgLfC6t128dfsCTwNXAHnA48BEEWkeRXzFwIVALnAScJWInOptt7sX74NeTH2A2d56/wIOBg73YvoTEIzyOxkOjPPe8yWgAvgD0A44DDgGuNqLIQeYArwPdAJ+A3ykqiuBT4ARYdu9AHhFVcuijMMkGUsQJtE8qKqrVHUZMA2YrqqzVLUEmAD09ZY7G3hXVT/0dnD/ArJwO+BDgQzgPlUtU9VxwDdh7zEKeFxVp6tqhao+B5R669VJVT9R1e9UNaiq3+KS1FHe7POAKao61nvfdao6W0QCwKXA9aq6zHvPL1S1NMrv5EtVfdN7z22qOkNVv1LVclVdgktwoRiGAStV9d+qWqKqW1R1ujfvOWAkgIikAefikqhJUZYgTKJZFfZ8W4TXLb3nnYBfQjNUNQgsBTp785Zp9Z4qfwl73h240TtFs1FENgJdvfXqJCIDRGSqd2pmE3Al7kgebxuLIqzWDneKK9K8aCytEcNeIvKOiKz0Tjv9XxQxALwF7Csi+bgqbZOqfl3PmEwSsARhktVy3I4eABER3M5xGbAC6OxNC+kW9nwp8HdVzQ17ZKvq2Cje92VgItBVVVsDjwGh91kK7BlhnbVASS3zioHssM+Rhjs9Fa5ml8yPAj8APVW1Fe4UXHgMe0QK3KvCXsNVERdg1UPKswRhktVrwEkicozXyHoj7jTRF8CXQDlwnYhkiMjpQP+wdZ8ArvSqARGRFl7jc04U75sDrFfVEhHpjzutFPIScKyIjBCRdBHJE5E+XnXzNHCviHQSkTQROcxr8/gJyPTePwO4DdhZW0gOsBkoEpF9gKvC5r0D7C4iN4hIcxHJEZEBYfOfBy4GTsESRMqzBGGSkqr+iDsSfhB3hH4ycLKqblfV7cDpuB3helx7xRth6xYAvwMeAjYAC71lo3E1cLeIbAHuwCWq0HZ/BU7EJav1uAbqA73ZfwS+w7WFrAf+AQRUdZO3zSdx1U8xUO2qpgj+iEtMW3DJ7tWwGLbgTh+dDKwEFgBHh83/L65xfKaqhp92MylIbMAgY0w4EfkYeFlVn/Q7FuMvSxDGmEoicgjwIa4NZYvf8Rh/2SkmYwwAIvIc7h6JGyw5GLAKwhhjTC2sgjDGGBNR0nTs1a5dO+3Ro4ffYRhjTEKZMWPGWlWteW8NEMcEISJP427rX62qvSPMF+B+3GV/W4GLVXWmN+8i3PXeAH/zujqoU48ePSgoKIhV+MYYkxJEpNbLmeN5iulZYEgd84cCPb3HKNzdn4hIW+BOYADu5qU7w3voNMYY0zjiliBU9TPcDT+1GQ48r85XQK6I7A6cAHyoqutVdQPukru6Eo0xxpg48LMNojPVOxkr9KbVNn0HIjIKV33QrVu3SIsYYzyqiqrruCmoStB7Hf4zqG65SD+Dqm7dYPjyOy6zs+3WXKZy26oEg1XxVV8+tH7VMrWRSNMiTaxlukTcQuRlVaE8qFQEg95Ppawi7HWFVk0PBqu9rlyv2rRg5Tw3PRi2bNW08NcVQWXfTq14+uJD6vhW6iehG6lVdQwwBqBfv347/M2UlZVRWFhISUlJo8fW2DIzM+nSpQsZGTa2S11Uq//zhf9zllUEK/9Jq5ZRyiuClIUtX1bh5pdVBCu3U1bhliv3dhDlFUHKguHTdly2cn6Fsr3m/BrvVV6hNXbIYTtMbyfKDjtcUKq/Nv5ICwhpASHD+5meFnA/Q9NrvE4PWy4jLUBmRmhagPSAkJ5W/XW3vOydB1EPfiaIZbjeNUO6eNOWAYNrTP+kPm9QWFhITk4OPXr0QGo7hEgCqsq6desoLCwkPz8/6vUqgkpRSTmbS8rcY5t7XlxaXv2orsZRXEWwxhFd+NFiUGtd1x391T6/5vaCQaWs8igsuMOOvKxCw462gtWO0soqj7C8Haz3PNjIO8nQP3NGIOD+qdMCZHj/+JGnC5kZaWSkuX/8DG+50E4jIIIIiAgBgYC4aRB6DoGAtwzhy7h1pMbrQOU0vOfePELbqdqGQNj6QiBQfZ0dtgsEAlWvq94jbBs1PotEsUxofiSRkqDWUm9EXra27db+h7Pjzr367yzN+x4TkZ8JYiLwexF5BdcgvUlVV4jIB8D/hTVMHw/cWp83KCkpSerkENpZV6iSndOawuUreX/uisod/eZtZWwOJYCwaVtKyt3P0vK4xrfjzsi9TgvtAAJ17wACUrVzdUdZVf+AoaOq8H/OdG+nGtopV/6zhs2r2k4gbHtVR3SheeHbCf3D19xpp3vbqHWnn8A7BmMgvpe5jsVVAu28MXXvxI3ihao+hhsC8kRcT5lbgUu8eetF5K9UjfB1t6rW1di9szjqu2qjUlW2bq9wR8KqVASp3PkHvaPh0OuKoDetxlHNmqLtXDlxZuVrEchpnk6rrAxyMjNolZlO17bZtMrMoFVWuvfTTc8Jm9aiebq3c6v7KC50BFn9iK/qCDBRvntjTGRxSxCqeu5O5itwTS3znsb1j5/0VJWi0nJWbCqhpKyi2rxQmZ8WcEfdaQGheVrAHYUHwh4iBAJCcEMz3rtukNvRZ2XQslk6gYDtpI0x9ZPQjdSJYOPGjbz88stcffXVO8zbtt0lhqLScpqlB+jaNpvMjDTSRBh+8jBefvkl2rTJjfq9mqen0atTq1iGb4xJYdYXU5xt3LiRRx55pNq07eUVLF69mQWriygpq6BTbhZ7dcihTXYzsjLSaJYeYNKk92jTxu4PNMb4xyqIOLvllltYtGgRffr0IT0jg7SMZmS1bMWShQv4ctZ3XHnhOZWX4l5//fWMGjUKqOo6pKioiKFDh3LEEUfwxRdf0LlzZ9566y2ysrJ8/mTGmGSXMgnirrfn8f3yzTHd5r6dWnHnyfvVuczo0aOZO3cuH06bzjsffMjVF57NlP9+wyH770Oz9ADPPPMMbdu2Zdu2bRxyyCGcccYZ5OXlVdvGggULGDt2LE888QQjRoxg/PjxjBw5MqafxRhjakqZBOEHVWXztu2UlgdZsWkbmRlp9O9/CAP77lu5zAMPPMCECRMAWLp0KQsWLNghQeTn59OnTx8ADj74YJYsWdJon8EYk7pSJkHs7Eg/1opKylmxeRvLNpYgQH67FqxvnUVOy5aVy3zyySdMmTKFL7/8kuzsbAYPHhzxru/mzZtXPk9LS2Pbtm2N8RGMMSkuZRJEYykpq2DlphI2l5SRkRZgr87tKd1WTE7mjl1gbNq0iTZt2pCdnc0PP/zAV1995UPExhgTmSWIGCmrCLJqcwkbircTCAgdW2fSrkVzAoFWDBw4kN69e5OVlUWHDh0q1xkyZAiPPfYYvXr1Yu+99+bQQ
"text/plain": [
"<Figure size 432x288 with 2 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "mmG4dhPEbR5j",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "6490987f-0866-4d50-e9af-ea06296ff522"
},
"source": [
"y_pred = model.predict(x_test)\n",
"zol = zero_one(y_pred, y_test)\n",
"cnn_loss.append(zol)\n",
"print(\"Zero-one Loss: \", zol)\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"Zero-one Loss: 0.18973418881759854\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YLIwv_usqLED",
"colab_type": "text"
},
"source": [
"# 1.7 Summary results"
]
},
{
"cell_type": "code",
"metadata": {
"id": "cuFIMRMarrOd",
"colab_type": "code",
"colab": {}
},
"source": [
"df_cnn_loss = pd.DataFrame(\n",
"columns = ['type', 'zero_one']\n",
")\n",
"\n",
"types = [\"1VGG\",\"2VGG\",\"3VGG\",\"3VGG-drop\", \"3VGG-drop-norm\",\"LeNet\",\"MobileNetV2\"]\n",
"df_cnn_loss = pd.DataFrame( data = [types,cnn_loss], index = ['types', 'zero_one']).T\n",
"\n",
"\n",
"\n",
"df_new = df_loss.rename(columns={'components': 'types'})\n",
"df_new['types'] = df_new['types'].astype(str)\n",
"\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "lkmFEyZ6v4A6",
"colab_type": "code",
"colab": {}
},
"source": [
"frames = [df_new, df_cnn_loss]\n",
"result = pd.concat(frames)\n",
"result = result.sort_values(by=['zero_one'])\n",
"\n",
"result = result.reset_index(drop=True)\n",
"\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "17oa_fAY1P_l",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 295
},
"outputId": "8ca06857-dad7-4701-e1b6-2f537c622d81"
},
"source": [
"\n",
"plt.barh( result['types'].values,result['zero_one'].values , align='center', alpha=0.5)\n",
"plt.xlabel('Loss')\n",
"plt.ylabel(\"Loss\")\n",
"plt.title('Model Loss summary')\n",
"\n",
"plt.show()"
],
"execution_count": null,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAdAAAAEWCAYAAADW7MapAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5hddX32//dNQMIpnIKRk4ZDUAEhQIpQVBDlUFSQghhEBR7QHy3F8tDHKsWroECVaoFiVYqARIUAovggqJgqD2iLQIAQEg5NOAcop4RziAL374/1HVnZzCR775k9e2Zzv65rrqy91net9f3O5JrPrNO9ZJuIiIhozQrd7kBERMRolAIaERHRhhTQiIiINqSARkREtCEFNCIiog0poBEREW1IAY0YhSRNlGRJKzbR9jBJvxuOfkW8kaSARnSYpPsl/UHS+Ib5t5YiOLE7PWutEEfE0lJAI4bHfcDBfR8kvQtYtXvdicHKHx2RAhoxPH4AfLr2+VDg+/UGktaU9H1JT0h6QNKXJK1Qlo2R9A1JT0q6F/hQP+ueJ+lRSQ9LOkXSmMF0WNIGkq6QtFDSfEmfqS3bUdJMSc9KekzS6WX+WEk/lPSUpKcl3SRpwgDb/0Lp63OS7pb0gTL/Akmn1NrtJmlB7fP9kj4vabakF8q4J0j6RdnWf0hau7TtO8I+XNJDkhZJOkrSn5X1n5b0b7VtbybpN6X/T0q6UNJaDfv+gqTZwAulHz9uGNdZkv51MN/7GB1SQCOGx++BcZLeWQrbVOCHDW2+CawJbArsSlVwDy/LPgN8GNgOmAIc2LDuBcDLwOalzZ7AkYPs88XAAmCDsr9/krR7WfavwL/aHgdsBlxa5h9axrAxsC5wFLC4ccOS3g78DfBnttcA9gLub6FvBwB7AFsAHwF+AfwDsB7V77XPNbR/NzAJ+DhwJnAC8EFgK+AgSbv2dQ34ahnzO8s4TmrY1sFUf8CsRfUz3LuvyJaj0qk0/HEUvSkFNGL49B2F7gHcCTzct6BWVI+3/Zzt+4F/AT5VmhwEnGn7IdsLqX7J9607AdgHONb2C7YfB84o22uLpI2BXYAv2H7J9izgXF47iv4jsLmk8baft/372vx1gc1tv2L7ZtvP9rOLV4CVgS0lrWT7ftv3tNDFb9p+zPbDwG+BG2zfavsl4HKqPyLqTi7j+BXwAjDd9uO19bcDsD3f9gzbS2w/AZxO9cdM3Vnl57DY9qPAdcDHyrK9gSdt39zCWGKUSgGNGD4/AD4BHMbrj1DGAysBD9TmPQBsWKY3AB5qWNbnbWXdR8spyaeBfwfePIi+bgAstP3cAP05guro765ymvbDZf4PgKuBiyU9IumfJa3UuHHb84FjqY7uHpd0saQNWujfY7Xpxf18Xr2d9uVU8MXl1PKzVEeYS938xdI/B4BpwCfL9CepvgfxBpACGjFMbD9AdTPRPsBPGhY/SXX09rbavLfy2lHqo1SnE+vL+jwELAHG216rfI2zvdUguvsIsI6kNfrrj+15tg+mKtKnAZdJWs32H21/2faWwJ9TnXb+NP2wfZHt91CN2WU7UB0h1m+wessgxtGqfyp9eVc5Pf1JqtO6dY2vsPopsI2kranGe2HHexkjQgpoxPA6Atjd9gv1mbZfobqOeKqkNSS9DTiO166TXgp8TtJG5QaZL9bWfRT4FfAvksZJWqHcDNN46nFZVi43AI2VNJaqUP4X8NUyb5vS9x8CSPqkpPVsvwo8XbbxqqT3S3pXOSX9LNUfBa827kzS2yXtLmll4CWqo8C+drOAfSStI+ktVEeqw2UN4HngGUkbAp9f3grltPFlwEXAjbYf7GwXY6RIAY0YRrbvsT1zgMXHUB193Qv8juoX8vll2XepTo3eBtzC649gPw28CbgDWET1C339Frr2PFUR6/vanepmmYlUR6OXAyfa/o/Sfm9grqTnqW4ommp7MdXR4mVUxfNO4Fr6P6W5MvA1qiPv/6E6kj2+LPtBGef9VH8YXNLCOAbry8D2wDPAVbz++zyQacC7yOnbNxTlhdoREYMj6a3AXcBbBrhpKnpQjkAjIgahPKt7HHBxiucbS5I0IiLaJGk1qjt6H6A6rR1vIDmFGxER0Yacwo2IiGhDTuH2gPHjx3vixInd7kZExKhx8803P2l7vcFsIwW0B0ycOJGZMwd6MiIiIhpJemD5rZYtp3AjIiLakAIaERHRhhTQiIiINqSARkREtCEFNCIiog0poBEREW1IAY2IiGhDCmhEREQbEqTQAx579iXOmPHf3e5GRMSw+d97bNHtLuQINCIioh0poB0g6XxJj0uaU5u3jqQZkuaVf9ceYN1DS5t5kg4dvl5HREQrUkA74wJe/27ALwK/tj0J+HX5vBRJ6wAnAu8GdgROHKjQRkREd6WAdoDt64CFDbP3A6aV6WnAR/tZdS9ghu2FthcBM8hLeiMiRqQU0OEzwfajZfp/gAn9tNkQeKj2eUGZ9zqSPitppqSZLzyzaGh7GhERy5UC2gW2DXiQ2zjH9hTbU1ZbM2d5IyKGWwro8HlM0voA5d/H+2nzMLBx7fNGZV5ERIwwKaDD5wqg767aQ4H/20+bq4E9Ja1dbh7as8yLiIgRJkEKHSBpOrAbMF7SAqo7a78GXCrpCOAB4KDSdgpwlO0jbS+UdDJwU9nUV2w33oz0OhPGjR0RDxVHRLyRqLocF6PZlClTPHPmzG53IyJi1JB0s+0pg9lGjkB7QKL8IoZPzvZEn1wDjYiIaEMK6BCTNFbSjZJukzRX0pfL/E0k3SBpvqRLJL1pgPWPL23ulrTX8PY+IiKalQI69JYAu9veFpgM7C1pJ+A04AzbmwOLgCMaV5S0JTAV2IoqgejbksYMW88jIqJpKaBDzJXny8eVypeB3YHLyvyBovz2Ay62vcT2fcB8qkzciIgYYVJAO0DSGEmzqMISZgD3AE/bfrk0GSiiL1F+ERGjRApoB9h+xfZkqiShHYF3dGAfifKLiOiiFNAOsv00cA2wM7CWpL7HhgaK6EuUX0TEKJECOsQkrSdprTK9CrAHcCdVIT2wNBsoyu8KYKqklSVtAkwCbux8ryMiolUJUhh66wPTyt2zKwCX2r5S0h3AxZJOAW4FzgOQtC8wxfY/2p4r6VLgDuBl4Gjbryxvh4nyi4gYfony6wGJ8ouIaE2i/AJIlF/EUMhZnGhVroFGRES0YVQVUEmW9MPa5xUlPSHpyuWsd5Kk/9PP/A0kXVamd2tiO7uVPnykNu9KSbstZ73DJG1Qpk+U9NWG5ZMl3SlpVUlXSbqrxAB+bVnbjYiI7hlVBRR4Adi63N0K1R2ubT/mYfsR2wcuv+VSFgAntLjOYcAGZXo68PGG5VPLfIBv2H4HsB2wi6S/aHFfERExDEZbAQX4OfChMn0wrxUeJK0j6aeSZkv6vaRtauttK+l6SfMkfaa0nyhpTuMOJK0m6fwSCn+rpP1qi28DnpG0Rz/r7SDpWkk3S7pa0vqSDgSmABeWdKKHgEWS3l1b9SBguu0XbV8DYPsPwC1Uz4JGRMQIMxoL6MVUz0qOBbYBbqgt+zJwq+1tgH8Avl9btg1VHu3OwD/2nVIdwAnAb2zvCLwf+Lqk1WrLTwW+VF9B0krAN4EDbe8AnA+cavsyYCZwiO3JthdTFf2pZb2dgIW25zVsby3gI8Cv++tgovwiIrpr1N2Fa3u2pIlUR58/b1j8HuCA0u43ktaVNK4s+7+leC2WdA1VxN6sAXazJ7Bv7brpWOCttT5cJwlJ76mt83Zga2CGJIAxwKMDbP8S4L8k/R1Ln74Fqmu7Zd5Ztu/tbwO2zwHOAdh4i63zLFJExDAbdQW0uAL4BrAbsG6T6zQWmWUVHQEH2L57qZnShNrHvqPQl2vrzLW983I7Yj8k6T5gV6qC37jOOcA822cub1sREdEdo/EULlSnR79s+/aG+b8FDoHqjlngSdvPlmX7lZddr0tVeG9axvavBo5ROZSUtF1jA9u/AtamOjUMcDewnqSdyzorSdqqLHsOWKNhE9OBM4B7bS/om1mSi
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
}
]
}
Reference in New Issue Copy Permalink