cats-or-dogs-using-cnn-with-transfer-learning.ipynb

{"cells":[{"metadata":{"_uuid":"1fad66867637226998d21e199e23f70e4eea955d"},"cell_type":"markdown","source":"<h1><center><font size=\"6\">Cats or Dogs - using CNN with Transfer Learning</font></center></h1>\n\n\n<center><img src=\"https://www.theladders.com/wp-content/uploads/dog-cat-190709-1000x563.jpg\" width=\"900\"></img></center>\n\n\n# <a id='0'>Content</a>\n\n- <a href='#1'>Introduction</a>  \n- <a href='#2'>Load packages and set parameters</a>  \n- <a href='#3'>Read the data</a>  \n- <a href='#4'>Data exploration</a>\n    - <a href='#41'>Class distribution</a>\n    - <a href='#42'>Images samples</a>\n- <a href='#5'>Model</a>  \n    - <a href='#51'>Prepare the model</a>  \n    - <a href='#52'>Train the model</a>  \n    - <a href='#53'>Validation accuracy and loss</a>  \n    - <a href='#54'>Validation accuracy per class</a>  \n- <a href='#6'>Prepare submission</a>     \n- <a href='#7'>Conclusions</a>\n- <a href='#8'>References</a>\n\n"},{"metadata":{"_uuid":"9ac4b97e33b99764c02e38fb58e0b56a03c7fa5f"},"cell_type":"markdown","source":"# <a id=\"1\">Introduction</a>  \n\n\n## Dataset\n\nThe **train** folder contains **25,000** images of **dogs** and **cats**. Each image in this folder has the label as part of the filename. The **test** folder contains **12,500** images, named according to a numeric id.  \nFor each image in the test set, you should predict a probability that the image is a dog (**1** = **dog**, **0** = **cat**).\n\n\n## Method\n\nFor the solution of this problem we will use a pre-trained model, ResNet-50, replacing only the last layer."},{"metadata":{"_uuid":"f4c5a829054852b17a48ffda50c7e35c605911bb"},"cell_type":"markdown","source":"# <a id=\"2\">Load packages</a>"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os, cv2, random\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import classification_report\nfrom tqdm import tqdm\nfrom random import shuffle \nfrom IPython.display import SVG\nfrom keras.utils.vis_utils import model_to_dot\nfrom keras.utils import plot_model\nfrom tensorflow.python.keras.applications import ResNet50\nfrom tensorflow.python.keras.models import Sequential\nfrom tensorflow.python.keras.layers import Dense, Flatten, GlobalAveragePooling2D\n%matplotlib inline ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"90c407d19e9f09a44cef1714461a1b5d10f1b0d8"},"cell_type":"markdown","source":"## Parameters\n\nHere we set few parameters used in the model. The image size is **224**.    \nThe images are stored in two folders, **train** and **test**.  \nThere are two image classes: **Dog** and **Cat**.  \nWe will use a subset of the training data set (**20,000** images).  From the training set, **50%** will be used for training, **50%** for validation.  \nA pre-trained model from **ResNet-50** will be used.  \nA number of **10** epochs will be used for training.  \n\n"},{"metadata":{"trusted":true,"_uuid":"f265a0cfc1bc6379507962130087e7f212e12f83"},"cell_type":"code","source":"TEST_SIZE = 0.5\nRANDOM_STATE = 2018\nBATCH_SIZE = 64\nNO_EPOCHS = 20\nNUM_CLASSES = 2\nSAMPLE_SIZE = 20000\nPATH = '/kaggle/input/dogs-vs-cats-redux-kernels-edition/'\nTRAIN_FOLDER = './train/'\nTEST_FOLDER =  './test/'\nIMG_SIZE = 224\nRESNET_WEIGHTS_PATH = '/kaggle/input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5'","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a1c725fa996e10190a924b977354d82b767c68bf"},"cell_type":"markdown","source":"# <a id=\"3\">Read the data</a>\n\nWe set the train image list.   \nSetting the **SAMPLE_SIZE** value we can reduce/enlarge the size of the training set.    \nCurrently **SAMPLE_SIZE** is set to **20,000**.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_image_path = os.path.join(PATH, \"train.zip\")\ntest_image_path = os.path.join(PATH, \"test.zip\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import zipfile\nwith zipfile.ZipFile(train_image_path,\"r\") as z:\n    z.extractall(\".\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with zipfile.ZipFile(test_image_path,\"r\") as z:\n    z.extractall(\".\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"12d65488fa58182f965485608965ae36f345f183"},"cell_type":"code","source":"train_image_list = os.listdir(\"./train/\")[0:SAMPLE_SIZE]\ntest_image_list = os.listdir(\"./test/\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"729f56cad0895facac99467faec510d3d61156a9"},"cell_type":"markdown","source":"We set a function for parsing the image names to extract the first 3 letters from the image names, which gives the label of the image. It will be either a cat or a dog. We are using one hot encoder, storing [1,0] for **cat** and [0,1] for **dog**."},{"metadata":{"trusted":true,"_uuid":"1df3dd73913a3e63a059edec2496e2db1d570c34"},"cell_type":"code","source":"def label_pet_image_one_hot_encoder(img):\n    pet = img.split('.')[-3]\n    if pet == 'cat': return [1,0]\n    elif pet == 'dog': return [0,1]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e62d359a2c5c01658610c85c162da998afcbf14d"},"cell_type":"markdown","source":"We are defining as well a function to process the data (both train and test set). "},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"def process_data(data_image_list, DATA_FOLDER, isTrain=True):\n    data_df = []\n    for img in tqdm(data_image_list):\n        path = os.path.join(DATA_FOLDER,img)\n        if(isTrain):\n            label = label_pet_image_one_hot_encoder(img)\n        else:\n            label = img.split('.')[0]\n        img = cv2.imread(path,cv2.IMREAD_COLOR)\n        img = cv2.resize(img, (IMG_SIZE,IMG_SIZE))\n        data_df.append([np.array(img),np.array(label)])\n    shuffle(data_df)\n    return data_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cd19491249f9dd5bc2fd8bfbdfd124fdcdb0cbe1"},"cell_type":"markdown","source":"# <a id=\"4\">Data exploration</a>\n\n\n## <a id=\"41\">Class distribution</a>\n\nLet's inspect the train data to check the **cat**/**dog** distribution.   We show first the split in the reduced train data."},{"metadata":{"trusted":true,"_uuid":"2901e1126ebe160a5a5c5a03dd07d5b8792a5290"},"cell_type":"code","source":"def plot_image_list_count(data_image_list):\n    labels = []\n    for img in data_image_list:\n        labels.append(img.split('.')[-3])\n    sns.countplot(labels)\n    plt.title('Cats and Dogs')\n    \nplot_image_list_count(train_image_list)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1815a71df706af6e6d7592a7366caaba5c0cdad1"},"cell_type":"markdown","source":"Let's show also the class distribution in the full train data set."},{"metadata":{"trusted":true,"_uuid":"1f36b71d110658540e3cbf3e435e00810ebb2d19"},"cell_type":"code","source":"plot_image_list_count(os.listdir(TRAIN_FOLDER))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c76a6072f4d272ffc7906d56a3008449a569f45b"},"cell_type":"markdown","source":"## <a id=\"42\">Images samples</a>\n\nLet's represet some of the images. We start with a selection from the train set. We will show the first 25 images from the train set.\n\nFirst,  we process the train data, reading the images and creating a table with images and labels. If the data is trom train set, the label is the one calculated with one hot encoding; if the data is from test set, the label will be the image number."},{"metadata":{"trusted":true,"_uuid":"d1b8ee45b627b58a3a5d04bb0c64fb18d5e8f592"},"cell_type":"code","source":"train = process_data(train_image_list, TRAIN_FOLDER)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a3583fc9056b32953bae9b90a3aaab7ff451c025"},"cell_type":"markdown","source":"Then, we plot the image selection."},{"metadata":{"trusted":true,"_uuid":"8a2b305bce44360f693d678a4b7eb9f27b3a7d36"},"cell_type":"code","source":"def show_images(data, isTest=False):\n    f, ax = plt.subplots(5,5, figsize=(15,15))\n    for i,data in enumerate(data[:25]):\n        img_num = data[1]\n        img_data = data[0]\n        label = np.argmax(img_num)\n        if label  == 1: \n            str_label='Dog'\n        elif label == 0: \n            str_label='Cat'\n        if(isTest):\n            str_label=\"None\"\n        ax[i//5, i%5].imshow(img_data)\n        ax[i//5, i%5].axis('off')\n        ax[i//5, i%5].set_title(\"Label: {}\".format(str_label))\n    plt.show()\n\nshow_images(train)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7cc51ba41c85ca9a93289362ce93b852ef34be6d"},"cell_type":"markdown","source":"Let's also show a selection of the train set. We prepare the test set."},{"metadata":{"trusted":true,"_uuid":"cb0696a3111e5421b3809f91bcb040f32cbde799"},"cell_type":"code","source":"test = process_data(test_image_list, TEST_FOLDER, False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"417b99a48d96abddb8bc3895a42520690d31208a"},"cell_type":"markdown","source":"Then, we show a selection of the test set."},{"metadata":{"trusted":true,"_uuid":"485ff5d66200edf7159027b49d4e60619b8b404a"},"cell_type":"code","source":"show_images(test,True)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"968f2373e44ccea4585d7191b0afc39a92fdff75"},"cell_type":"markdown","source":"# <a id=\"5\">Model</a>\n\n## <a id=\"51\">Prepare the model</a>\n\nLet's start by preparing the model.\n\n### Prepare the train data"},{"metadata":{"trusted":true,"_uuid":"7651e9ef5ff8074165a5208257c5db81ab83b75f"},"cell_type":"code","source":"X = np.array([i[0] for i in train]).reshape(-1,IMG_SIZE,IMG_SIZE,3)\ny = np.array([i[1] for i in train])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"802f6921a5ab7e2b75b6cb239bedd71cb2c0e4cd"},"cell_type":"markdown","source":"### Prepare the model\n\nWe initialize the **ResNet-50** model, adding an additional last layer of type **Dense**, with **softmax** activation function.   \n\nWe also set the first layer of the model to be not trainable, becaise **ResNet-50** model was already trained."},{"metadata":{"trusted":true,"_uuid":"94892424cb4e898d6b7c3df1269cfab6371c4eb9"},"cell_type":"code","source":"model = Sequential()\nmodel.add(ResNet50(include_top=False, pooling='max', weights=RESNET_WEIGHTS_PATH))\nmodel.add(Dense(NUM_CLASSES, activation='softmax'))\n# ResNet-50 model is already trained, should not be trained\nmodel.layers[0].trainable = True","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cfb6886ef6d14fc9f1cdd1857fdd07f9639e6442"},"cell_type":"markdown","source":"### Compile the model\n\nWe compile the model, using a **sigmoid** optimized, the loss function as **categorical crossentropy** and the metric **accuracy**."},{"metadata":{"trusted":true,"_uuid":"c24a3f51ef3487e15efdc8c2e876aea7c429b876"},"cell_type":"code","source":"model.compile(optimizer='sgd', loss='categorical_crossentropy', metrics=['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"52e06decf4d5ea9e099b4669681b8ebb36ccd7bd"},"cell_type":"markdown","source":"### Model summary\n\nWe plot the model description. We can see that the **ResNet-50** model represent the 1st layer of our model, of type **Model**."},{"metadata":{"trusted":true,"_uuid":"3e2a3ff8c2dd896fd9ca725625587564165d0727"},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6bd39063f81d0b87f80542c6b8ce74035e2de28e"},"cell_type":"markdown","source":"Let's also show the model graphical representation using **plot_model**."},{"metadata":{"trusted":true,"_uuid":"64561287fa8faaa756ba4043a2a3c671810a9dbf"},"cell_type":"code","source":"plot_model(model, to_file='model.png')\nSVG(model_to_dot(model).create(prog='dot', format='svg'))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cba1cd58d8f3e3240fafe88ba5ac4c8cd3885008"},"cell_type":"markdown","source":"### Split the train data in train and validation\n\nWe split the train data in two parts. One will be reserved for train set, the second for validation set. Only the train subset of the data will be used for training the model; the validation set will be used for validation, during training."},{"metadata":{"trusted":true,"_uuid":"e80c5bd03d275807865c424187f752b521c99f15"},"cell_type":"code","source":"X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=TEST_SIZE, random_state=RANDOM_STATE)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cb28fa20f06729cb20a31e7b5c1f99970ebe62fa"},"cell_type":"markdown","source":"## <a id=\"52\">Train the model</a>\n\nWe are now ready to train our model."},{"metadata":{"trusted":true,"_uuid":"3899d6c135949c4546d5aa10291fb5b1ff4498da"},"cell_type":"code","source":"train_model = model.fit(X_train, y_train,\n                  batch_size=BATCH_SIZE,\n                  epochs=NO_EPOCHS,\n                  verbose=1,\n                  validation_data=(X_val, y_val))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e9c9159b0a2f2fc8bafe9bfc1983c30a846f099a"},"cell_type":"markdown","source":"## <a id=\"53\">Validation accuracy and loss</a>\n\nLet's show the train and validation accuracy on the same plot. As well, we will represent the train and validation loss on the same graph."},{"metadata":{"trusted":true,"_uuid":"5a7b3d7b9494a3a1a75db099d224a18b1bf1d8e3","collapsed":true},"cell_type":"code","source":"def plot_accuracy_and_loss(train_model):\n    hist = train_model.history\n    acc = hist['acc']\n    val_acc = hist['val_acc']\n    loss = hist['loss']\n    val_loss = hist['val_loss']\n    epochs = range(len(acc))\n    f, ax = plt.subplots(1,2, figsize=(14,6))\n    ax[0].plot(epochs, acc, 'g', label='Training accuracy')\n    ax[0].plot(epochs, val_acc, 'r', label='Validation accuracy')\n    ax[0].set_title('Training and validation accuracy')\n    ax[0].legend()\n    ax[1].plot(epochs, loss, 'g', label='Training loss')\n    ax[1].plot(epochs, val_loss, 'r', label='Validation loss')\n    ax[1].set_title('Training and validation loss')\n    ax[1].legend()\n    plt.show()\nplot_accuracy_and_loss(train_model)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"130f99193a6b24b40c5a28f84a97b84e6d39784c"},"cell_type":"markdown","source":"Let's also show the numeric validation accuracy and loss."},{"metadata":{"trusted":true,"_uuid":"0935a9e239a7af382208ecede99bd2afb5098482","collapsed":true},"cell_type":"code","source":"score = model.evaluate(X_val, y_val, verbose=0)\nprint('Validation loss:', score[0])\nprint('Validation accuracy:', score[1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ec8bbac69464197827260097132a0c297dd07b94"},"cell_type":"markdown","source":"## <a id=\"54\">Validation accuracy per class</a>\n\nLet's show the validation accuracy per each class.\n\nWe start by predicting the labels for the validation set."},{"metadata":{"trusted":true,"_uuid":"dd52f4c515120a0c9aa2ecd4f1fd7cfaa71a08d0","collapsed":true},"cell_type":"code","source":"#get the predictions for the test data\npredicted_classes = model.predict_classes(X_val)\n#get the indices to be plotted\ny_true = np.argmax(y_val,axis=1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"431664690b241560041cec6e5c93ddb0b045d63f"},"cell_type":"markdown","source":"We create two indices, **correct** and **incorrect**, for the images in the validation set with class predicted correctly and incorrectly, respectively."},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"9da776ea97666c99a141cf26d2e64705cf5d9069"},"cell_type":"code","source":"correct = np.nonzero(predicted_classes==y_true)[0]\nincorrect = np.nonzero(predicted_classes!=y_true)[0]","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6a1c361f6319484bc17f41ab71ecf1d07992dbe7"},"cell_type":"markdown","source":"\nWe saw what is the number of correctly vs. incorrectly predicted values in the validation set.    \n\nWe show here the classification report for the validation set, with the accuracy per class and overall."},{"metadata":{"trusted":true,"_uuid":"3a92b2e2751942fd9fff4792ee932ee0d5f9bef2","collapsed":true},"cell_type":"code","source":"target_names = [\"Class {}:\".format(i) for i in range(NUM_CLASSES)]\nprint(classification_report(y_true, predicted_classes, target_names=target_names))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"14d8d48765b4c4fe17c25be6605164eac34df534","collapsed":true},"cell_type":"markdown","source":"# <a id=\"6\">Prepare the submission</a>\n\n### Show test images with predicted class\n\nLet's show few of the test images with the predicted class. For this, we will have to predict the class.\n"},{"metadata":{"trusted":true,"_uuid":"07e788faee58139d33db79be0093cdb0900d8073","collapsed":true},"cell_type":"code","source":"f, ax = plt.subplots(5,5, figsize=(15,15))\nfor i,data in enumerate(test[:25]):\n    img_num = data[1]\n    img_data = data[0]\n    orig = img_data\n    data = img_data.reshape(-1,IMG_SIZE,IMG_SIZE,3)\n    model_out = model.predict([data])[0]\n    \n    if np.argmax(model_out) == 1: \n        str_predicted='Dog'\n    else: \n        str_predicted='Cat'\n    ax[i//5, i%5].imshow(orig)\n    ax[i//5, i%5].axis('off')\n    ax[i//5, i%5].set_title(\"Predicted:{}\".format(str_predicted))    \nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2e2199cdb3dbef1620298f0b9060908b7e2d4f90"},"cell_type":"markdown","source":"### Test data prediction"},{"metadata":{"trusted":true,"_uuid":"15afbef65cdfe94ffa3a2e63ed97f0fdd9a34d89","collapsed":true},"cell_type":"code","source":"pred_list = []\nimg_list = []\nfor img in tqdm(test):\n    img_data = img[0]\n    img_idx = img[1]\n    data = img_data.reshape(-1,IMG_SIZE,IMG_SIZE,3)\n    predicted = model.predict([data])[0]\n    img_list.append(img_idx)\n    pred_list.append(predicted[1])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a932e7efa19b93fc6a53d681ad7e195134acad17"},"cell_type":"markdown","source":"### Submission file\n\nLet's prepare now the submission file."},{"metadata":{"trusted":true,"collapsed":true,"_uuid":"9bd2fc5c0e3722cbc44c0211512574b2ab0cd249"},"cell_type":"code","source":"submission = pd.DataFrame({'id':img_list , 'label':pred_list})\nsubmission.head()\nsubmission.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"53f8f8c06fe12737d0c33fcf7e0609bbb7671326"},"cell_type":"markdown","source":"# <a id=\"7\">Conclusions</a>\n\nUsing a pretrained model for Keras, ResNet-50, with a Dense model with softmax activation added on top and training with a reduced set of  we were able to obtain quite good model in terms of validation accuracy.   \nThe model was used to predict the classes of the images from the independent test set and results were submitted to test the accuracy of the prediction with fresh data.  \n"},{"metadata":{"_uuid":"f668d5f2cd1b6a10fd5f78aca31463d66cf5addf"},"cell_type":"markdown","source":"# <a id=\"8\">References</a>\n\n[1] Dogs vs. Cats Redux: Kernels Edition, https://www.kaggle.com/c/dogs-vs-cats-redux-kernels-edition  \n[2] ResNet pretrained models for Keras, https://www.kaggle.com/keras/resnet50  \n\n\n\n"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}