From a9e9ff59fce71b159c2a65b7c3e4e11cfd675c9f Mon Sep 17 00:00:00 2001 From: afoix Date: Sat, 17 Aug 2024 17:34:09 +0000 Subject: [PATCH] Commit from GitHub Actions (Build Notebooks) --- exercise.ipynb | 304 ++++++++++++++++++++++----------------- solution.ipynb | 382 +++++++++++++++++++++++++++---------------------- 2 files changed, 383 insertions(+), 303 deletions(-) diff --git a/exercise.ipynb b/exercise.ipynb index 639a784..f58cf52 100644 --- a/exercise.ipynb +++ b/exercise.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "adda9842", + "id": "32b1e9f1", "metadata": {}, "source": [ "# Exercise 7: Failure Modes And Limits of Deep Learning" @@ -10,7 +10,7 @@ }, { "cell_type": "markdown", - "id": "e5d7e30e", + "id": "a3c84efc", "metadata": {}, "source": [ "In the following exercise, we explore the failure modes and limits of neural networks. \n", @@ -23,7 +23,7 @@ }, { "cell_type": "markdown", - "id": "f6dae011", + "id": "664ba6f6", "metadata": {}, "source": [ "\n", @@ -44,7 +44,7 @@ }, { "cell_type": "markdown", - "id": "76ccfd8d", + "id": "aa09118c", "metadata": {}, "source": [ "### Acknowledgements\n", @@ -53,7 +53,7 @@ }, { "cell_type": "markdown", - "id": "5b568cb9", + "id": "c2ff3769", "metadata": {}, "source": [ "### Data Loading\n", @@ -67,7 +67,7 @@ { "cell_type": "code", "execution_count": null, - "id": "839f5e5d", + "id": "9ad22ce4", "metadata": {}, "outputs": [], "source": [ @@ -90,7 +90,7 @@ }, { "cell_type": "markdown", - "id": "66fe78ff", + "id": "ae4bf898", "metadata": {}, "source": [ "### Part 1: Preparation of a Tainted Dataset\n", @@ -101,7 +101,7 @@ { "cell_type": "code", "execution_count": null, - "id": "a95679b1", + "id": "49be270b", "metadata": {}, "outputs": [], "source": [ @@ -115,7 +115,7 @@ { "cell_type": "code", "execution_count": null, - "id": "f81bdd3d", + "id": "134f9210", "metadata": {}, "outputs": [], "source": [ @@ -126,7 +126,7 @@ }, { "cell_type": "markdown", - "id": "f4cb3dc1", + "id": "468c1859", "metadata": {}, "source": [ "## Part 1.1: Local Corruption of Data\n", @@ -137,7 +137,7 @@ { "cell_type": "code", "execution_count": null, - "id": "2bedcac3", + "id": "e42789cb", "metadata": {}, "outputs": [], "source": [ @@ -149,7 +149,7 @@ { "cell_type": "code", "execution_count": null, - "id": "cbf5e737", + "id": "810977f7", "metadata": {}, "outputs": [], "source": [ @@ -172,7 +172,7 @@ }, { "cell_type": "markdown", - "id": "17c5d34f", + "id": "c35f41e3", "metadata": {}, "source": [ "

\n", @@ -183,7 +183,7 @@ }, { "cell_type": "markdown", - "id": "e8f04799", + "id": "af37ccef", "metadata": {}, "source": [ "

\n", @@ -194,7 +194,7 @@ }, { "cell_type": "markdown", - "id": "c3f572ee", + "id": "2d465f59", "metadata": {}, "source": [ "## Part 1.2: Global Corrution of data\n", @@ -204,7 +204,7 @@ }, { "cell_type": "markdown", - "id": "2fabdefb", + "id": "a8431945", "metadata": {}, "source": [ "You may have noticed that the images are stored as arrays of integers. First we cast them to float to be able to add textures easily without integer wrapping issues." @@ -213,7 +213,7 @@ { "cell_type": "code", "execution_count": null, - "id": "1f111005", + "id": "d4bb8cec", "metadata": {}, "outputs": [], "source": [ @@ -224,7 +224,7 @@ }, { "cell_type": "markdown", - "id": "f7826fb4", + "id": "3d71a89a", "metadata": {}, "source": [ "Then we create the grid texture and visualize it." @@ -233,7 +233,7 @@ { "cell_type": "code", "execution_count": null, - "id": "c8213ead", + "id": "b673baf2", "metadata": {}, "outputs": [], "source": [ @@ -249,7 +249,7 @@ }, { "cell_type": "markdown", - "id": "cd33949c", + "id": "1b153aa9", "metadata": {}, "source": [ "Next we add the texture to all 4s in the train and test set." @@ -258,7 +258,7 @@ { "cell_type": "code", "execution_count": null, - "id": "434e7f07", + "id": "bd299f0c", "metadata": {}, "outputs": [], "source": [ @@ -269,7 +269,7 @@ }, { "cell_type": "markdown", - "id": "e55e4b7b", + "id": "1b9cb99d", "metadata": {}, "source": [ "After adding the texture, we have to make sure the values are between 0 and 255 and then cast back to uint8. \n", @@ -279,7 +279,7 @@ { "cell_type": "code", "execution_count": null, - "id": "0840fe88", + "id": "d4e0464b", "metadata": {}, "outputs": [], "source": [ @@ -289,13 +289,13 @@ "\n", "# Cast back to byte:\n", "tainted_train_dataset.data = tainted_train_dataset.data.type(torch.uint8) \n", - "tainted_test_dataset.data = tainted_test_dataset.data.type(torch.uint8) \n" + "tainted_test_dataset.data = tainted_test_dataset.data.type(torch.uint8) " ] }, { "cell_type": "code", "execution_count": null, - "id": "5e2c971b", + "id": "0d4d3bf4", "metadata": {}, "outputs": [], "source": [ @@ -317,7 +317,7 @@ }, { "cell_type": "markdown", - "id": "0c80e649", + "id": "dc513b3b", "metadata": {}, "source": [ "

\n", @@ -329,7 +329,7 @@ { "cell_type": "code", "execution_count": null, - "id": "b7e6d2fa", + "id": "7be1949a", "metadata": {}, "outputs": [], "source": [ @@ -343,7 +343,7 @@ }, { "cell_type": "markdown", - "id": "5ce4fcf6", + "id": "e4bf92fc", "metadata": {}, "source": [ "Now we will train the neural network. A training function is provided below - this should be familiar, but make sure you look it over and understand what is happening in the training loop." @@ -352,8 +352,11 @@ { "cell_type": "code", "execution_count": null, - "id": "351d6d2b", - "metadata": {}, + "id": "f72635e7", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from tqdm import tqdm\n", @@ -377,7 +380,7 @@ }, { "cell_type": "markdown", - "id": "af1789c1", + "id": "7011f77b", "metadata": {}, "source": [ "We have to choose hyperparameters for our model. We have selected to train for two epochs, with a batch size of 64 for training and 1000 for testing. We are using the cross entropy loss, a standard multi-class classification loss." @@ -386,7 +389,7 @@ { "cell_type": "code", "execution_count": null, - "id": "d940e32c", + "id": "274f60f1", "metadata": {}, "outputs": [], "source": [ @@ -405,7 +408,7 @@ }, { "cell_type": "markdown", - "id": "a39797b5", + "id": "359def48", "metadata": {}, "source": [ "Next we initialize a clean model, and a tainted model. We want to have reproducible results, so we set the initial weights with a specific random seed. The seed number does not matter, just that it is the same!" @@ -414,7 +417,7 @@ { "cell_type": "code", "execution_count": null, - "id": "a516acac", + "id": "240cf939", "metadata": {}, "outputs": [], "source": [ @@ -442,7 +445,7 @@ }, { "cell_type": "markdown", - "id": "3731d055", + "id": "664c2725", "metadata": {}, "source": [ "Next we initialize the clean and tainted dataloaders, again with a specific random seed for reproducibility." @@ -451,7 +454,7 @@ { "cell_type": "code", "execution_count": null, - "id": "73f9588f", + "id": "4d8bdc2d", "metadata": {}, "outputs": [], "source": [ @@ -465,7 +468,7 @@ }, { "cell_type": "markdown", - "id": "478a64fc", + "id": "644c5b7e", "metadata": {}, "source": [ "Now it is time to train the neural networks! We are storing the training loss history for each model so we can visualize it later." @@ -474,7 +477,7 @@ { "cell_type": "code", "execution_count": null, - "id": "31498d2a", + "id": "b140b2a3", "metadata": {}, "outputs": [], "source": [ @@ -507,7 +510,7 @@ }, { "cell_type": "markdown", - "id": "a12b2049", + "id": "0b0f8da2", "metadata": {}, "source": [ "Now we visualize the loss history for the clean and tainted models." @@ -516,7 +519,7 @@ { "cell_type": "code", "execution_count": null, - "id": "36d68857", + "id": "25259f93", "metadata": {}, "outputs": [], "source": [ @@ -531,7 +534,7 @@ }, { "cell_type": "markdown", - "id": "0f0f81c9", + "id": "c19bdd9b", "metadata": {}, "source": [ "

\n", @@ -542,7 +545,7 @@ }, { "cell_type": "markdown", - "id": "1b6db647", + "id": "5fde5f20", "metadata": {}, "source": [ "

\n", @@ -553,7 +556,7 @@ }, { "cell_type": "markdown", - "id": "3ea8c3b3", + "id": "ddca6e25", "metadata": {}, "source": [ "

\n", @@ -564,7 +567,7 @@ }, { "cell_type": "markdown", - "id": "e3d2282a", + "id": "b2d75427", "metadata": {}, "source": [ "

\n", @@ -576,7 +579,7 @@ }, { "cell_type": "markdown", - "id": "7ad0a46a", + "id": "627bf942", "metadata": {}, "source": [ "

\n", @@ -591,7 +594,7 @@ }, { "cell_type": "markdown", - "id": "a7fde316", + "id": "c468a9dd", "metadata": {}, "source": [ "### Part 3: Examining the Results of the Clean and Tainted Networks\n", @@ -604,8 +607,11 @@ { "cell_type": "code", "execution_count": null, - "id": "5cd9755d", - "metadata": {}, + "id": "bd9c1d7d", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "import numpy as np\n", @@ -626,7 +632,7 @@ }, { "cell_type": "markdown", - "id": "ed703a9a", + "id": "288e65a0", "metadata": {}, "source": [ "Now we call the predict method with the clean and tainted models on the clean and tainted datasets." @@ -635,7 +641,7 @@ { "cell_type": "code", "execution_count": null, - "id": "1cff0217", + "id": "4ceb9f16", "metadata": {}, "outputs": [], "source": [ @@ -647,7 +653,7 @@ }, { "cell_type": "markdown", - "id": "42147aad", + "id": "7659d1e4", "metadata": {}, "source": [ "We can investivate the results using the confusion matrix, which you should recall from the Introduction to Machine Learning exercise. The function in the cell below will create a nicely annotated confusion matrix." @@ -656,8 +662,11 @@ { "cell_type": "code", "execution_count": null, - "id": "7f12688e", - "metadata": {}, + "id": "c1cb1393", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from sklearn.metrics import confusion_matrix\n", @@ -704,25 +713,44 @@ " cm.columns.name = 'Predicted'\n", " fig, ax = plt.subplots(figsize=figsize)\n", " ax=sns.heatmap(cm, annot=annot, fmt='', vmax=30)\n", - " ax.set_title(title)\n", - "#-\n", - "\n", - "# Now we will generate confusion matrices for each model/data combination. Take your time and try and interpret these, and then try and answer the questions below.\n", - "\n", + " ax.set_title(title)" + ] + }, + { + "cell_type": "markdown", + "id": "b8d1a35b", + "metadata": {}, + "source": [ + "Now we will generate confusion matrices for each model/data combination. Take your time and try and interpret these, and then try and answer the questions below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "607b7e66", + "metadata": {}, + "outputs": [], + "source": [ "cm_analysis(true_labels, pred_clean_clean, \"Clean Model on Clean Data\")\n", "cm_analysis(true_labels, pred_clean_tainted, \"Clean Model on Tainted Data\")\n", "cm_analysis(true_labels, pred_tainted_clean, \"Tainted Model on Clean Data\")\n", - "cm_analysis(true_labels, pred_tainted_tainted, \"Tainted Model on Tainted Data\")\n", - "\n", - "#

\n", - "# Task 3.1:

\n", - "# For the clean model and the clean dataset, which digit was least accurately predicted? What did the model predict instead? Why do you think these digits were confused by the model?\n", - "#
" + "cm_analysis(true_labels, pred_tainted_tainted, \"Tainted Model on Tainted Data\")" ] }, { "cell_type": "markdown", - "id": "4c670669", + "id": "ea3626f2", + "metadata": {}, + "source": [ + "

\n", + "Task 3.1:

\n", + "For the clean model and the clean dataset, which digit was least accurately predicted? What did the model predict instead? Why do you think these digits were confused by the model?\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "130e20e1", "metadata": {}, "source": [ "

\n", @@ -733,7 +761,7 @@ }, { "cell_type": "markdown", - "id": "fe2557a8", + "id": "bf8ed81e", "metadata": {}, "source": [ "

\n", @@ -744,7 +772,7 @@ }, { "cell_type": "markdown", - "id": "69e9728a", + "id": "40df4f2d", "metadata": {}, "source": [ "

\n", @@ -755,7 +783,7 @@ }, { "cell_type": "markdown", - "id": "64b8d401", + "id": "c8002432", "metadata": {}, "source": [ "

\n", @@ -767,7 +795,7 @@ }, { "cell_type": "markdown", - "id": "fb784988", + "id": "247ca0d8", "metadata": {}, "source": [ "

\n", @@ -782,7 +810,7 @@ }, { "cell_type": "markdown", - "id": "5bb27c64", + "id": "bf20ad38", "metadata": {}, "source": [ "### Part 4: Interpretation with Integrated Gradients\n", @@ -791,7 +819,7 @@ }, { "cell_type": "markdown", - "id": "0f945022", + "id": "f9fc68ea", "metadata": {}, "source": [ "\n", @@ -801,8 +829,11 @@ { "cell_type": "code", "execution_count": null, - "id": "dd31c2f2", - "metadata": {}, + "id": "e6f72533", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from captum.attr import IntegratedGradients\n", @@ -834,7 +865,7 @@ }, { "cell_type": "markdown", - "id": "0b840ebc", + "id": "21ae7cdc", "metadata": {}, "source": [ "Next we provide a function to visualize the output of integrated gradients, using the function above to actually run the algorithm." @@ -843,8 +874,11 @@ { "cell_type": "code", "execution_count": null, - "id": "12264ead", - "metadata": {}, + "id": "a3a6d67c", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from captum.attr import visualization as viz\n", @@ -882,7 +916,7 @@ }, { "cell_type": "markdown", - "id": "45f1b911", + "id": "846c5348", "metadata": {}, "source": [ "To start examining the results, we will call the `visualize_integrated_gradients` with the tainted and clean models on the tainted and clean sevens. \n", @@ -893,7 +927,7 @@ { "cell_type": "code", "execution_count": null, - "id": "d29a00d3", + "id": "caedad66", "metadata": {}, "outputs": [], "source": [ @@ -903,7 +937,7 @@ }, { "cell_type": "markdown", - "id": "8a1ae450", + "id": "c5fecf3e", "metadata": {}, "source": [ "

\n", @@ -914,7 +948,7 @@ }, { "cell_type": "markdown", - "id": "531d5887", + "id": "2951b347", "metadata": {}, "source": [ "Now let's look at the attention of the tainted model!" @@ -923,7 +957,7 @@ { "cell_type": "code", "execution_count": null, - "id": "b8b0a00c", + "id": "0a61a125", "metadata": {}, "outputs": [], "source": [ @@ -933,7 +967,7 @@ }, { "cell_type": "markdown", - "id": "50cef6fe", + "id": "2487b221", "metadata": {}, "source": [ "

\n", @@ -944,7 +978,7 @@ }, { "cell_type": "markdown", - "id": "316648a1", + "id": "b0796634", "metadata": {}, "source": [ "Now let's look at the regions of the image that Integrated Gradients highlights as important for classifying fours in the clean and tainted models." @@ -953,7 +987,7 @@ { "cell_type": "code", "execution_count": null, - "id": "dae40772", + "id": "3dbd3bfe", "metadata": {}, "outputs": [], "source": [ @@ -965,7 +999,7 @@ }, { "cell_type": "markdown", - "id": "788ce5d3", + "id": "c644f8c1", "metadata": {}, "source": [ "

\n", @@ -976,7 +1010,7 @@ }, { "cell_type": "markdown", - "id": "9fc7b89c", + "id": "1b20037d", "metadata": {}, "source": [ "

\n", @@ -987,7 +1021,7 @@ }, { "cell_type": "markdown", - "id": "cc55a255", + "id": "b792e773", "metadata": {}, "source": [ "

\n", @@ -1000,7 +1034,7 @@ }, { "cell_type": "markdown", - "id": "63756a82", + "id": "f15bb2c5", "metadata": {}, "source": [ "

\n", @@ -1014,7 +1048,7 @@ }, { "cell_type": "markdown", - "id": "2a72d908", + "id": "600744b8", "metadata": {}, "source": [ "## Part 5: Importance of using the right training data\n", @@ -1026,7 +1060,7 @@ }, { "cell_type": "markdown", - "id": "7ca0499e", + "id": "8acf0094", "metadata": {}, "source": [ "First, we will write a function to add noise to the MNIST dataset, so that we can train a model to denoise it." @@ -1035,31 +1069,32 @@ { "cell_type": "code", "execution_count": null, - "id": "c72a2158", - "metadata": {}, + "id": "53309b1e", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "import torch\n", "\n", "# A simple function to add noise to tensors:\n", "def add_noise(tensor, power=1.5):\n", - " return tensor * torch.rand(tensor.size()).to(tensor.device) ** power + 0.75*torch.randn(tensor.size()).to(tensor.device)\n", - "\n", - "\n" + " return tensor * torch.rand(tensor.size()).to(tensor.device) ** power + 0.75*torch.randn(tensor.size()).to(tensor.device)" ] }, { "cell_type": "markdown", - "id": "f7a6ab29", + "id": "69f3a8b8", "metadata": {}, "source": [ - "Next we will visualize a couple MNIST examples with and without noise.\n" + "Next we will visualize a couple MNIST examples with and without noise." ] }, { "cell_type": "code", "execution_count": null, - "id": "6093f082", + "id": "66b599cc", "metadata": {}, "outputs": [], "source": [ @@ -1088,7 +1123,7 @@ }, { "cell_type": "markdown", - "id": "ec029fc7", + "id": "1f1fb79a", "metadata": {}, "source": [ "### UNet model\n", @@ -1098,7 +1133,7 @@ }, { "cell_type": "markdown", - "id": "1ead30f9", + "id": "55ef489e", "metadata": {}, "source": [ "The training loop code is also provided here. It is similar to the code used to train the image classification model previously, but look it over to make sure there are no surprises." @@ -1107,8 +1142,11 @@ { "cell_type": "code", "execution_count": null, - "id": "631806b5", - "metadata": {}, + "id": "90639034", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from tqdm import tqdm\n", @@ -1155,7 +1193,7 @@ }, { "cell_type": "markdown", - "id": "b998875e", + "id": "56a26b76", "metadata": {}, "source": [ "Here we choose hyperparameters and initialize the model and data loaders." @@ -1164,7 +1202,7 @@ { "cell_type": "code", "execution_count": null, - "id": "e9d76a45", + "id": "fd5b1452", "metadata": {}, "outputs": [], "source": [ @@ -1202,7 +1240,7 @@ }, { "cell_type": "markdown", - "id": "5e96c67c", + "id": "af683527", "metadata": {}, "source": [ "Finally, we run the training loop!" @@ -1211,7 +1249,7 @@ { "cell_type": "code", "execution_count": null, - "id": "4587f15d", + "id": "bdb97058", "metadata": {}, "outputs": [], "source": [ @@ -1222,7 +1260,7 @@ }, { "cell_type": "markdown", - "id": "cc56b362", + "id": "45f1e2d5", "metadata": {}, "source": [ "As before, we will visualize the training loss. If all went correctly, it should decrease from around 1.0 to less than 0.2." @@ -1231,8 +1269,10 @@ { "cell_type": "code", "execution_count": null, - "id": "bcef2183", - "metadata": {}, + "id": "511dfa3a", + "metadata": { + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "# Loss Visualization\n", @@ -1245,7 +1285,7 @@ }, { "cell_type": "markdown", - "id": "ba61aa74", + "id": "900bbaed", "metadata": {}, "source": [ "### Check denoising performance\n", @@ -1256,7 +1296,7 @@ { "cell_type": "code", "execution_count": null, - "id": "370c076b", + "id": "caf3b358", "metadata": { "lines_to_next_cell": 1 }, @@ -1273,7 +1313,7 @@ { "cell_type": "code", "execution_count": null, - "id": "3ad20390", + "id": "b8636a86", "metadata": { "lines_to_next_cell": 1 }, @@ -1299,7 +1339,7 @@ }, { "cell_type": "markdown", - "id": "7f54e077", + "id": "40ab4204", "metadata": {}, "source": [ "We pick 8 images to show:" @@ -1308,7 +1348,7 @@ { "cell_type": "code", "execution_count": null, - "id": "b6754636", + "id": "a63651d3", "metadata": {}, "outputs": [], "source": [ @@ -1318,7 +1358,7 @@ }, { "cell_type": "markdown", - "id": "0abbf186", + "id": "9074c7cd", "metadata": {}, "source": [ "

\n", @@ -1329,7 +1369,7 @@ }, { "cell_type": "markdown", - "id": "347783c1", + "id": "fe8c80c8", "metadata": {}, "source": [ "### Apply trained model on 'wrong' data \n", @@ -1339,7 +1379,7 @@ }, { "cell_type": "markdown", - "id": "9074d884", + "id": "c1148bce", "metadata": {}, "source": [ "### Load the Fashion MNIST dataset\n", @@ -1350,7 +1390,7 @@ { "cell_type": "code", "execution_count": null, - "id": "9ec53c62", + "id": "45175e2a", "metadata": {}, "outputs": [], "source": [ @@ -1371,16 +1411,16 @@ }, { "cell_type": "markdown", - "id": "9f241941", + "id": "088f7608", "metadata": {}, "source": [ - "Next we apply the denoising model we trained on the MNIST data to FashionMNIST, and visualize the results.\n" + "Next we apply the denoising model we trained on the MNIST data to FashionMNIST, and visualize the results." ] }, { "cell_type": "code", "execution_count": null, - "id": "b7bd016c", + "id": "7ba8e696", "metadata": {}, "outputs": [], "source": [ @@ -1390,7 +1430,7 @@ }, { "cell_type": "markdown", - "id": "dc49133e", + "id": "c7028792", "metadata": {}, "source": [ "

\n", @@ -1401,7 +1441,7 @@ }, { "cell_type": "markdown", - "id": "6d2b670d", + "id": "3165c296", "metadata": {}, "source": [ "

\n", @@ -1412,7 +1452,7 @@ }, { "cell_type": "markdown", - "id": "8d9d850d", + "id": "e0b21688", "metadata": {}, "source": [ "### Train the denoiser on both MNIST and FashionMNIST\n", @@ -1423,7 +1463,7 @@ { "cell_type": "code", "execution_count": null, - "id": "00a1dc6e", + "id": "cf7a00f4", "metadata": {}, "outputs": [], "source": [ @@ -1460,7 +1500,7 @@ { "cell_type": "code", "execution_count": null, - "id": "5bdf014f", + "id": "62d0d815", "metadata": {}, "outputs": [], "source": [ @@ -1471,7 +1511,7 @@ { "cell_type": "code", "execution_count": null, - "id": "ded5d280", + "id": "3df9945d", "metadata": {}, "outputs": [], "source": [ @@ -1481,7 +1521,7 @@ }, { "cell_type": "markdown", - "id": "c9aaa117", + "id": "4ccdf0b4", "metadata": {}, "source": [ "

\n", @@ -1493,7 +1533,7 @@ { "cell_type": "code", "execution_count": null, - "id": "039cd974", + "id": "b5220bdd", "metadata": {}, "outputs": [], "source": [ @@ -1530,7 +1570,7 @@ { "cell_type": "code", "execution_count": null, - "id": "91cbce9e", + "id": "2c223391", "metadata": {}, "outputs": [], "source": [ @@ -1541,7 +1581,7 @@ { "cell_type": "code", "execution_count": null, - "id": "794e0d53", + "id": "ae1ce7a0", "metadata": {}, "outputs": [], "source": [ @@ -1551,7 +1591,7 @@ }, { "cell_type": "markdown", - "id": "6f5dddbc", + "id": "dd3d395e", "metadata": {}, "source": [ "

\n", diff --git a/solution.ipynb b/solution.ipynb index 630e90a..d598cb2 100644 --- a/solution.ipynb +++ b/solution.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "adda9842", + "id": "32b1e9f1", "metadata": {}, "source": [ "# Exercise 7: Failure Modes And Limits of Deep Learning" @@ -10,7 +10,7 @@ }, { "cell_type": "markdown", - "id": "e5d7e30e", + "id": "a3c84efc", "metadata": {}, "source": [ "In the following exercise, we explore the failure modes and limits of neural networks. \n", @@ -23,7 +23,7 @@ }, { "cell_type": "markdown", - "id": "f6dae011", + "id": "664ba6f6", "metadata": {}, "source": [ "\n", @@ -44,7 +44,7 @@ }, { "cell_type": "markdown", - "id": "76ccfd8d", + "id": "aa09118c", "metadata": {}, "source": [ "### Acknowledgements\n", @@ -53,7 +53,7 @@ }, { "cell_type": "markdown", - "id": "5b568cb9", + "id": "c2ff3769", "metadata": {}, "source": [ "### Data Loading\n", @@ -67,7 +67,7 @@ { "cell_type": "code", "execution_count": null, - "id": "839f5e5d", + "id": "9ad22ce4", "metadata": {}, "outputs": [], "source": [ @@ -90,7 +90,7 @@ }, { "cell_type": "markdown", - "id": "66fe78ff", + "id": "ae4bf898", "metadata": {}, "source": [ "### Part 1: Preparation of a Tainted Dataset\n", @@ -101,7 +101,7 @@ { "cell_type": "code", "execution_count": null, - "id": "a95679b1", + "id": "49be270b", "metadata": {}, "outputs": [], "source": [ @@ -115,7 +115,7 @@ { "cell_type": "code", "execution_count": null, - "id": "f81bdd3d", + "id": "134f9210", "metadata": {}, "outputs": [], "source": [ @@ -126,7 +126,7 @@ }, { "cell_type": "markdown", - "id": "f4cb3dc1", + "id": "468c1859", "metadata": {}, "source": [ "## Part 1.1: Local Corruption of Data\n", @@ -137,7 +137,7 @@ { "cell_type": "code", "execution_count": null, - "id": "2bedcac3", + "id": "e42789cb", "metadata": {}, "outputs": [], "source": [ @@ -149,7 +149,7 @@ { "cell_type": "code", "execution_count": null, - "id": "cbf5e737", + "id": "810977f7", "metadata": {}, "outputs": [], "source": [ @@ -172,7 +172,7 @@ }, { "cell_type": "markdown", - "id": "17c5d34f", + "id": "c35f41e3", "metadata": {}, "source": [ "

\n", @@ -183,7 +183,7 @@ }, { "cell_type": "markdown", - "id": "556f8bac", + "id": "e1c3dca2", "metadata": { "tags": [ "solution" @@ -199,7 +199,7 @@ }, { "cell_type": "markdown", - "id": "7dc94e89", + "id": "b39e1d1e", "metadata": { "tags": [ "solution" @@ -215,7 +215,7 @@ }, { "cell_type": "markdown", - "id": "e8f04799", + "id": "af37ccef", "metadata": {}, "source": [ "

\n", @@ -226,7 +226,7 @@ }, { "cell_type": "markdown", - "id": "74055b8a", + "id": "0d223612", "metadata": { "tags": [ "solution" @@ -240,7 +240,7 @@ }, { "cell_type": "markdown", - "id": "c0e60d36", + "id": "d8a309dc", "metadata": { "tags": [ "solution" @@ -261,7 +261,7 @@ }, { "cell_type": "markdown", - "id": "c3f572ee", + "id": "2d465f59", "metadata": {}, "source": [ "## Part 1.2: Global Corrution of data\n", @@ -271,7 +271,7 @@ }, { "cell_type": "markdown", - "id": "2fabdefb", + "id": "a8431945", "metadata": {}, "source": [ "You may have noticed that the images are stored as arrays of integers. First we cast them to float to be able to add textures easily without integer wrapping issues." @@ -280,7 +280,7 @@ { "cell_type": "code", "execution_count": null, - "id": "1f111005", + "id": "d4bb8cec", "metadata": {}, "outputs": [], "source": [ @@ -291,7 +291,7 @@ }, { "cell_type": "markdown", - "id": "f7826fb4", + "id": "3d71a89a", "metadata": {}, "source": [ "Then we create the grid texture and visualize it." @@ -300,7 +300,7 @@ { "cell_type": "code", "execution_count": null, - "id": "c8213ead", + "id": "b673baf2", "metadata": {}, "outputs": [], "source": [ @@ -316,7 +316,7 @@ }, { "cell_type": "markdown", - "id": "cd33949c", + "id": "1b153aa9", "metadata": {}, "source": [ "Next we add the texture to all 4s in the train and test set." @@ -325,7 +325,7 @@ { "cell_type": "code", "execution_count": null, - "id": "434e7f07", + "id": "bd299f0c", "metadata": {}, "outputs": [], "source": [ @@ -336,7 +336,7 @@ }, { "cell_type": "markdown", - "id": "e55e4b7b", + "id": "1b9cb99d", "metadata": {}, "source": [ "After adding the texture, we have to make sure the values are between 0 and 255 and then cast back to uint8. \n", @@ -346,7 +346,7 @@ { "cell_type": "code", "execution_count": null, - "id": "0840fe88", + "id": "d4e0464b", "metadata": {}, "outputs": [], "source": [ @@ -356,13 +356,13 @@ "\n", "# Cast back to byte:\n", "tainted_train_dataset.data = tainted_train_dataset.data.type(torch.uint8) \n", - "tainted_test_dataset.data = tainted_test_dataset.data.type(torch.uint8) \n" + "tainted_test_dataset.data = tainted_test_dataset.data.type(torch.uint8) " ] }, { "cell_type": "code", "execution_count": null, - "id": "5e2c971b", + "id": "0d4d3bf4", "metadata": {}, "outputs": [], "source": [ @@ -384,7 +384,7 @@ }, { "cell_type": "markdown", - "id": "0c80e649", + "id": "dc513b3b", "metadata": {}, "source": [ "

\n", @@ -395,7 +395,7 @@ }, { "cell_type": "markdown", - "id": "f5fe74f8", + "id": "1de5c94e", "metadata": { "tags": [ "solution" @@ -413,7 +413,7 @@ }, { "cell_type": "markdown", - "id": "e2feb73a", + "id": "f8347929", "metadata": { "tags": [ "solution" @@ -447,7 +447,7 @@ }, { "cell_type": "markdown", - "id": "6000419c", + "id": "ecf5740e", "metadata": { "tags": [ "solution" @@ -461,7 +461,7 @@ }, { "cell_type": "markdown", - "id": "a799bd0e", + "id": "6ea23472", "metadata": { "tags": [ "solution" @@ -497,7 +497,7 @@ { "cell_type": "code", "execution_count": null, - "id": "b7e6d2fa", + "id": "7be1949a", "metadata": {}, "outputs": [], "source": [ @@ -511,7 +511,7 @@ }, { "cell_type": "markdown", - "id": "5ce4fcf6", + "id": "e4bf92fc", "metadata": {}, "source": [ "Now we will train the neural network. A training function is provided below - this should be familiar, but make sure you look it over and understand what is happening in the training loop." @@ -520,8 +520,11 @@ { "cell_type": "code", "execution_count": null, - "id": "351d6d2b", - "metadata": {}, + "id": "f72635e7", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from tqdm import tqdm\n", @@ -545,7 +548,7 @@ }, { "cell_type": "markdown", - "id": "af1789c1", + "id": "7011f77b", "metadata": {}, "source": [ "We have to choose hyperparameters for our model. We have selected to train for two epochs, with a batch size of 64 for training and 1000 for testing. We are using the cross entropy loss, a standard multi-class classification loss." @@ -554,7 +557,7 @@ { "cell_type": "code", "execution_count": null, - "id": "d940e32c", + "id": "274f60f1", "metadata": {}, "outputs": [], "source": [ @@ -573,7 +576,7 @@ }, { "cell_type": "markdown", - "id": "a39797b5", + "id": "359def48", "metadata": {}, "source": [ "Next we initialize a clean model, and a tainted model. We want to have reproducible results, so we set the initial weights with a specific random seed. The seed number does not matter, just that it is the same!" @@ -582,7 +585,7 @@ { "cell_type": "code", "execution_count": null, - "id": "a516acac", + "id": "240cf939", "metadata": {}, "outputs": [], "source": [ @@ -610,7 +613,7 @@ }, { "cell_type": "markdown", - "id": "3731d055", + "id": "664c2725", "metadata": {}, "source": [ "Next we initialize the clean and tainted dataloaders, again with a specific random seed for reproducibility." @@ -619,7 +622,7 @@ { "cell_type": "code", "execution_count": null, - "id": "73f9588f", + "id": "4d8bdc2d", "metadata": {}, "outputs": [], "source": [ @@ -633,7 +636,7 @@ }, { "cell_type": "markdown", - "id": "478a64fc", + "id": "644c5b7e", "metadata": {}, "source": [ "Now it is time to train the neural networks! We are storing the training loss history for each model so we can visualize it later." @@ -642,7 +645,7 @@ { "cell_type": "code", "execution_count": null, - "id": "31498d2a", + "id": "b140b2a3", "metadata": {}, "outputs": [], "source": [ @@ -675,7 +678,7 @@ }, { "cell_type": "markdown", - "id": "a12b2049", + "id": "0b0f8da2", "metadata": {}, "source": [ "Now we visualize the loss history for the clean and tainted models." @@ -684,7 +687,7 @@ { "cell_type": "code", "execution_count": null, - "id": "36d68857", + "id": "25259f93", "metadata": {}, "outputs": [], "source": [ @@ -699,7 +702,7 @@ }, { "cell_type": "markdown", - "id": "0f0f81c9", + "id": "c19bdd9b", "metadata": {}, "source": [ "

\n", @@ -710,7 +713,7 @@ }, { "cell_type": "markdown", - "id": "73e8d38a", + "id": "2b205eaf", "metadata": { "tags": [ "solution" @@ -724,7 +727,7 @@ }, { "cell_type": "markdown", - "id": "b606a428", + "id": "a2c3f7f4", "metadata": { "tags": [ "solution" @@ -738,7 +741,7 @@ }, { "cell_type": "markdown", - "id": "1b6db647", + "id": "5fde5f20", "metadata": {}, "source": [ "

\n", @@ -749,7 +752,7 @@ }, { "cell_type": "markdown", - "id": "ecd13877", + "id": "1743a0ac", "metadata": { "tags": [ "solution" @@ -763,7 +766,7 @@ }, { "cell_type": "markdown", - "id": "53a4bae0", + "id": "7ff75eff", "metadata": { "tags": [ "solution" @@ -777,7 +780,7 @@ }, { "cell_type": "markdown", - "id": "3ea8c3b3", + "id": "ddca6e25", "metadata": {}, "source": [ "

\n", @@ -788,7 +791,7 @@ }, { "cell_type": "markdown", - "id": "e5e82742", + "id": "6a265b4c", "metadata": { "tags": [ "solution" @@ -802,7 +805,7 @@ }, { "cell_type": "markdown", - "id": "a35ef600", + "id": "576f7fc8", "metadata": { "tags": [ "solution" @@ -816,7 +819,7 @@ }, { "cell_type": "markdown", - "id": "e3d2282a", + "id": "b2d75427", "metadata": {}, "source": [ "

\n", @@ -828,7 +831,7 @@ }, { "cell_type": "markdown", - "id": "7ad0a46a", + "id": "627bf942", "metadata": {}, "source": [ "

\n", @@ -843,7 +846,7 @@ }, { "cell_type": "markdown", - "id": "a7fde316", + "id": "c468a9dd", "metadata": {}, "source": [ "### Part 3: Examining the Results of the Clean and Tainted Networks\n", @@ -856,8 +859,11 @@ { "cell_type": "code", "execution_count": null, - "id": "5cd9755d", - "metadata": {}, + "id": "bd9c1d7d", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "import numpy as np\n", @@ -878,7 +884,7 @@ }, { "cell_type": "markdown", - "id": "ed703a9a", + "id": "288e65a0", "metadata": {}, "source": [ "Now we call the predict method with the clean and tainted models on the clean and tainted datasets." @@ -887,7 +893,7 @@ { "cell_type": "code", "execution_count": null, - "id": "1cff0217", + "id": "4ceb9f16", "metadata": {}, "outputs": [], "source": [ @@ -899,7 +905,7 @@ }, { "cell_type": "markdown", - "id": "42147aad", + "id": "7659d1e4", "metadata": {}, "source": [ "We can investivate the results using the confusion matrix, which you should recall from the Introduction to Machine Learning exercise. The function in the cell below will create a nicely annotated confusion matrix." @@ -908,8 +914,11 @@ { "cell_type": "code", "execution_count": null, - "id": "7f12688e", - "metadata": {}, + "id": "c1cb1393", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from sklearn.metrics import confusion_matrix\n", @@ -956,25 +965,44 @@ " cm.columns.name = 'Predicted'\n", " fig, ax = plt.subplots(figsize=figsize)\n", " ax=sns.heatmap(cm, annot=annot, fmt='', vmax=30)\n", - " ax.set_title(title)\n", - "#-\n", - "\n", - "# Now we will generate confusion matrices for each model/data combination. Take your time and try and interpret these, and then try and answer the questions below.\n", - "\n", + " ax.set_title(title)" + ] + }, + { + "cell_type": "markdown", + "id": "b8d1a35b", + "metadata": {}, + "source": [ + "Now we will generate confusion matrices for each model/data combination. Take your time and try and interpret these, and then try and answer the questions below." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "607b7e66", + "metadata": {}, + "outputs": [], + "source": [ "cm_analysis(true_labels, pred_clean_clean, \"Clean Model on Clean Data\")\n", "cm_analysis(true_labels, pred_clean_tainted, \"Clean Model on Tainted Data\")\n", "cm_analysis(true_labels, pred_tainted_clean, \"Tainted Model on Clean Data\")\n", - "cm_analysis(true_labels, pred_tainted_tainted, \"Tainted Model on Tainted Data\")\n", - "\n", - "#

\n", - "# Task 3.1:

\n", - "# For the clean model and the clean dataset, which digit was least accurately predicted? What did the model predict instead? Why do you think these digits were confused by the model?\n", - "#
" + "cm_analysis(true_labels, pred_tainted_tainted, \"Tainted Model on Tainted Data\")" ] }, { "cell_type": "markdown", - "id": "649ab696", + "id": "ea3626f2", + "metadata": {}, + "source": [ + "

\n", + "Task 3.1:

\n", + "For the clean model and the clean dataset, which digit was least accurately predicted? What did the model predict instead? Why do you think these digits were confused by the model?\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "0453521d", "metadata": { "tags": [ "solution" @@ -988,7 +1016,7 @@ }, { "cell_type": "markdown", - "id": "5a6f0e64", + "id": "6e421289", "metadata": { "tags": [ "solution" @@ -1003,7 +1031,7 @@ }, { "cell_type": "markdown", - "id": "4c670669", + "id": "130e20e1", "metadata": {}, "source": [ "

\n", @@ -1014,7 +1042,7 @@ }, { "cell_type": "markdown", - "id": "cd4c2361", + "id": "8013831b", "metadata": { "tags": [ "solution" @@ -1028,7 +1056,7 @@ }, { "cell_type": "markdown", - "id": "364d3cfa", + "id": "07bafeac", "metadata": { "tags": [ "solution" @@ -1042,7 +1070,7 @@ }, { "cell_type": "markdown", - "id": "fe2557a8", + "id": "bf8ed81e", "metadata": {}, "source": [ "

\n", @@ -1053,7 +1081,7 @@ }, { "cell_type": "markdown", - "id": "8d3be153", + "id": "c0fe2ff0", "metadata": { "tags": [ "solution" @@ -1067,7 +1095,7 @@ }, { "cell_type": "markdown", - "id": "0a565fa6", + "id": "94e826a4", "metadata": { "tags": [ "solution" @@ -1084,7 +1112,7 @@ }, { "cell_type": "markdown", - "id": "69e9728a", + "id": "40df4f2d", "metadata": {}, "source": [ "

\n", @@ -1095,7 +1123,7 @@ }, { "cell_type": "markdown", - "id": "95d27f55", + "id": "cda393d7", "metadata": { "tags": [ "solution" @@ -1109,7 +1137,7 @@ }, { "cell_type": "markdown", - "id": "d74f6e25", + "id": "970fea68", "metadata": { "tags": [ "solution" @@ -1129,7 +1157,7 @@ }, { "cell_type": "markdown", - "id": "64b8d401", + "id": "c8002432", "metadata": {}, "source": [ "

\n", @@ -1141,7 +1169,7 @@ }, { "cell_type": "markdown", - "id": "fb784988", + "id": "247ca0d8", "metadata": {}, "source": [ "

\n", @@ -1156,7 +1184,7 @@ }, { "cell_type": "markdown", - "id": "5bb27c64", + "id": "bf20ad38", "metadata": {}, "source": [ "### Part 4: Interpretation with Integrated Gradients\n", @@ -1165,7 +1193,7 @@ }, { "cell_type": "markdown", - "id": "0f945022", + "id": "f9fc68ea", "metadata": {}, "source": [ "\n", @@ -1175,8 +1203,11 @@ { "cell_type": "code", "execution_count": null, - "id": "dd31c2f2", - "metadata": {}, + "id": "e6f72533", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from captum.attr import IntegratedGradients\n", @@ -1208,7 +1239,7 @@ }, { "cell_type": "markdown", - "id": "0b840ebc", + "id": "21ae7cdc", "metadata": {}, "source": [ "Next we provide a function to visualize the output of integrated gradients, using the function above to actually run the algorithm." @@ -1217,8 +1248,11 @@ { "cell_type": "code", "execution_count": null, - "id": "12264ead", - "metadata": {}, + "id": "a3a6d67c", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from captum.attr import visualization as viz\n", @@ -1256,7 +1290,7 @@ }, { "cell_type": "markdown", - "id": "45f1b911", + "id": "846c5348", "metadata": {}, "source": [ "To start examining the results, we will call the `visualize_integrated_gradients` with the tainted and clean models on the tainted and clean sevens. \n", @@ -1267,7 +1301,7 @@ { "cell_type": "code", "execution_count": null, - "id": "d29a00d3", + "id": "caedad66", "metadata": {}, "outputs": [], "source": [ @@ -1277,7 +1311,7 @@ }, { "cell_type": "markdown", - "id": "8a1ae450", + "id": "c5fecf3e", "metadata": {}, "source": [ "

\n", @@ -1288,7 +1322,7 @@ }, { "cell_type": "markdown", - "id": "153bfeaa", + "id": "bb08c804", "metadata": { "tags": [ "solution" @@ -1302,7 +1336,7 @@ }, { "cell_type": "markdown", - "id": "d397dcad", + "id": "0510e746", "metadata": { "tags": [ "solution" @@ -1312,12 +1346,12 @@ "**4.1 Answer from 2023 Students:**\n", "\n", "The network looks at the center of the 7s, same for clean and tainted 7s.\n", - "It looks like a 7, it is a 7. :)\n" + "It looks like a 7, it is a 7. :)" ] }, { "cell_type": "markdown", - "id": "531d5887", + "id": "2951b347", "metadata": {}, "source": [ "Now let's look at the attention of the tainted model!" @@ -1326,7 +1360,7 @@ { "cell_type": "code", "execution_count": null, - "id": "b8b0a00c", + "id": "0a61a125", "metadata": {}, "outputs": [], "source": [ @@ -1336,7 +1370,7 @@ }, { "cell_type": "markdown", - "id": "50cef6fe", + "id": "2487b221", "metadata": {}, "source": [ "

\n", @@ -1347,7 +1381,7 @@ }, { "cell_type": "markdown", - "id": "a13d1997", + "id": "c3db9a72", "metadata": { "tags": [ "solution" @@ -1361,7 +1395,7 @@ }, { "cell_type": "markdown", - "id": "6d780f65", + "id": "21b10c74", "metadata": { "tags": [ "solution" @@ -1379,7 +1413,7 @@ }, { "cell_type": "markdown", - "id": "316648a1", + "id": "b0796634", "metadata": {}, "source": [ "Now let's look at the regions of the image that Integrated Gradients highlights as important for classifying fours in the clean and tainted models." @@ -1388,7 +1422,7 @@ { "cell_type": "code", "execution_count": null, - "id": "dae40772", + "id": "3dbd3bfe", "metadata": {}, "outputs": [], "source": [ @@ -1400,7 +1434,7 @@ }, { "cell_type": "markdown", - "id": "788ce5d3", + "id": "c644f8c1", "metadata": {}, "source": [ "

\n", @@ -1411,7 +1445,7 @@ }, { "cell_type": "markdown", - "id": "e520d4b1", + "id": "14425198", "metadata": { "tags": [ "solution" @@ -1425,7 +1459,7 @@ }, { "cell_type": "markdown", - "id": "3264bcef", + "id": "27db03bf", "metadata": { "tags": [ "solution" @@ -1442,7 +1476,7 @@ }, { "cell_type": "markdown", - "id": "9fc7b89c", + "id": "1b20037d", "metadata": {}, "source": [ "

\n", @@ -1453,7 +1487,7 @@ }, { "cell_type": "markdown", - "id": "4db808a2", + "id": "c0677a4d", "metadata": { "tags": [ "solution" @@ -1467,7 +1501,7 @@ }, { "cell_type": "markdown", - "id": "4f3da547", + "id": "bd15604c", "metadata": { "tags": [ "solution" @@ -1483,7 +1517,7 @@ }, { "cell_type": "markdown", - "id": "cc55a255", + "id": "b792e773", "metadata": {}, "source": [ "

\n", @@ -1496,7 +1530,7 @@ }, { "cell_type": "markdown", - "id": "63756a82", + "id": "f15bb2c5", "metadata": {}, "source": [ "

\n", @@ -1510,7 +1544,7 @@ }, { "cell_type": "markdown", - "id": "2a72d908", + "id": "600744b8", "metadata": {}, "source": [ "## Part 5: Importance of using the right training data\n", @@ -1522,7 +1556,7 @@ }, { "cell_type": "markdown", - "id": "7ca0499e", + "id": "8acf0094", "metadata": {}, "source": [ "First, we will write a function to add noise to the MNIST dataset, so that we can train a model to denoise it." @@ -1531,31 +1565,32 @@ { "cell_type": "code", "execution_count": null, - "id": "c72a2158", - "metadata": {}, + "id": "53309b1e", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "import torch\n", "\n", "# A simple function to add noise to tensors:\n", "def add_noise(tensor, power=1.5):\n", - " return tensor * torch.rand(tensor.size()).to(tensor.device) ** power + 0.75*torch.randn(tensor.size()).to(tensor.device)\n", - "\n", - "\n" + " return tensor * torch.rand(tensor.size()).to(tensor.device) ** power + 0.75*torch.randn(tensor.size()).to(tensor.device)" ] }, { "cell_type": "markdown", - "id": "f7a6ab29", + "id": "69f3a8b8", "metadata": {}, "source": [ - "Next we will visualize a couple MNIST examples with and without noise.\n" + "Next we will visualize a couple MNIST examples with and without noise." ] }, { "cell_type": "code", "execution_count": null, - "id": "6093f082", + "id": "66b599cc", "metadata": {}, "outputs": [], "source": [ @@ -1584,7 +1619,7 @@ }, { "cell_type": "markdown", - "id": "ec029fc7", + "id": "1f1fb79a", "metadata": {}, "source": [ "### UNet model\n", @@ -1594,7 +1629,7 @@ }, { "cell_type": "markdown", - "id": "1ead30f9", + "id": "55ef489e", "metadata": {}, "source": [ "The training loop code is also provided here. It is similar to the code used to train the image classification model previously, but look it over to make sure there are no surprises." @@ -1603,8 +1638,11 @@ { "cell_type": "code", "execution_count": null, - "id": "631806b5", - "metadata": {}, + "id": "90639034", + "metadata": { + "lines_to_end_of_cell_marker": 0, + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "from tqdm import tqdm\n", @@ -1651,7 +1689,7 @@ }, { "cell_type": "markdown", - "id": "b998875e", + "id": "56a26b76", "metadata": {}, "source": [ "Here we choose hyperparameters and initialize the model and data loaders." @@ -1660,7 +1698,7 @@ { "cell_type": "code", "execution_count": null, - "id": "e9d76a45", + "id": "fd5b1452", "metadata": {}, "outputs": [], "source": [ @@ -1698,7 +1736,7 @@ }, { "cell_type": "markdown", - "id": "5e96c67c", + "id": "af683527", "metadata": {}, "source": [ "Finally, we run the training loop!" @@ -1707,7 +1745,7 @@ { "cell_type": "code", "execution_count": null, - "id": "4587f15d", + "id": "bdb97058", "metadata": {}, "outputs": [], "source": [ @@ -1718,7 +1756,7 @@ }, { "cell_type": "markdown", - "id": "cc56b362", + "id": "45f1e2d5", "metadata": {}, "source": [ "As before, we will visualize the training loss. If all went correctly, it should decrease from around 1.0 to less than 0.2." @@ -1727,8 +1765,10 @@ { "cell_type": "code", "execution_count": null, - "id": "bcef2183", - "metadata": {}, + "id": "511dfa3a", + "metadata": { + "lines_to_next_cell": 1 + }, "outputs": [], "source": [ "# Loss Visualization\n", @@ -1741,7 +1781,7 @@ }, { "cell_type": "markdown", - "id": "ba61aa74", + "id": "900bbaed", "metadata": {}, "source": [ "### Check denoising performance\n", @@ -1752,7 +1792,7 @@ { "cell_type": "code", "execution_count": null, - "id": "370c076b", + "id": "caf3b358", "metadata": { "lines_to_next_cell": 1 }, @@ -1769,7 +1809,7 @@ { "cell_type": "code", "execution_count": null, - "id": "3ad20390", + "id": "b8636a86", "metadata": { "lines_to_next_cell": 1 }, @@ -1795,7 +1835,7 @@ }, { "cell_type": "markdown", - "id": "7f54e077", + "id": "40ab4204", "metadata": {}, "source": [ "We pick 8 images to show:" @@ -1804,7 +1844,7 @@ { "cell_type": "code", "execution_count": null, - "id": "b6754636", + "id": "a63651d3", "metadata": {}, "outputs": [], "source": [ @@ -1814,7 +1854,7 @@ }, { "cell_type": "markdown", - "id": "0abbf186", + "id": "9074c7cd", "metadata": {}, "source": [ "

\n", @@ -1825,7 +1865,7 @@ }, { "cell_type": "markdown", - "id": "531bfe36", + "id": "b123e777", "metadata": { "tags": [ "solution" @@ -1839,7 +1879,7 @@ }, { "cell_type": "markdown", - "id": "a3409c51", + "id": "1544ae08", "metadata": { "tags": [ "solution" @@ -1853,7 +1893,7 @@ }, { "cell_type": "markdown", - "id": "347783c1", + "id": "fe8c80c8", "metadata": {}, "source": [ "### Apply trained model on 'wrong' data \n", @@ -1863,7 +1903,7 @@ }, { "cell_type": "markdown", - "id": "9074d884", + "id": "c1148bce", "metadata": {}, "source": [ "### Load the Fashion MNIST dataset\n", @@ -1874,7 +1914,7 @@ { "cell_type": "code", "execution_count": null, - "id": "9ec53c62", + "id": "45175e2a", "metadata": {}, "outputs": [], "source": [ @@ -1895,16 +1935,16 @@ }, { "cell_type": "markdown", - "id": "9f241941", + "id": "088f7608", "metadata": {}, "source": [ - "Next we apply the denoising model we trained on the MNIST data to FashionMNIST, and visualize the results.\n" + "Next we apply the denoising model we trained on the MNIST data to FashionMNIST, and visualize the results." ] }, { "cell_type": "code", "execution_count": null, - "id": "b7bd016c", + "id": "7ba8e696", "metadata": {}, "outputs": [], "source": [ @@ -1914,7 +1954,7 @@ }, { "cell_type": "markdown", - "id": "dc49133e", + "id": "c7028792", "metadata": {}, "source": [ "

\n", @@ -1925,7 +1965,7 @@ }, { "cell_type": "markdown", - "id": "b4197b25", + "id": "2d3010f1", "metadata": { "tags": [ "solution" @@ -1939,7 +1979,7 @@ }, { "cell_type": "markdown", - "id": "0bd27ffd", + "id": "8f33e6a1", "metadata": { "tags": [ "solution" @@ -1953,7 +1993,7 @@ }, { "cell_type": "markdown", - "id": "6d2b670d", + "id": "3165c296", "metadata": {}, "source": [ "

\n", @@ -1964,7 +2004,7 @@ }, { "cell_type": "markdown", - "id": "12882976", + "id": "f658d2a5", "metadata": { "tags": [ "solution" @@ -1978,7 +2018,7 @@ }, { "cell_type": "markdown", - "id": "b472427a", + "id": "d3b2dd9d", "metadata": { "tags": [ "solution" @@ -1993,7 +2033,7 @@ }, { "cell_type": "markdown", - "id": "8d9d850d", + "id": "e0b21688", "metadata": {}, "source": [ "### Train the denoiser on both MNIST and FashionMNIST\n", @@ -2004,7 +2044,7 @@ { "cell_type": "code", "execution_count": null, - "id": "00a1dc6e", + "id": "cf7a00f4", "metadata": {}, "outputs": [], "source": [ @@ -2041,7 +2081,7 @@ { "cell_type": "code", "execution_count": null, - "id": "5bdf014f", + "id": "62d0d815", "metadata": {}, "outputs": [], "source": [ @@ -2052,7 +2092,7 @@ { "cell_type": "code", "execution_count": null, - "id": "ded5d280", + "id": "3df9945d", "metadata": {}, "outputs": [], "source": [ @@ -2062,7 +2102,7 @@ }, { "cell_type": "markdown", - "id": "c9aaa117", + "id": "4ccdf0b4", "metadata": {}, "source": [ "

\n", @@ -2073,7 +2113,7 @@ }, { "cell_type": "markdown", - "id": "8fccf686", + "id": "d7ebaa83", "metadata": { "tags": [ "solution" @@ -2092,7 +2132,7 @@ { "cell_type": "code", "execution_count": null, - "id": "039cd974", + "id": "b5220bdd", "metadata": {}, "outputs": [], "source": [ @@ -2129,7 +2169,7 @@ { "cell_type": "code", "execution_count": null, - "id": "91cbce9e", + "id": "2c223391", "metadata": {}, "outputs": [], "source": [ @@ -2140,7 +2180,7 @@ { "cell_type": "code", "execution_count": null, - "id": "794e0d53", + "id": "ae1ce7a0", "metadata": {}, "outputs": [], "source": [ @@ -2150,7 +2190,7 @@ }, { "cell_type": "markdown", - "id": "6f5dddbc", + "id": "dd3d395e", "metadata": {}, "source": [ "

\n", @@ -2161,7 +2201,7 @@ }, { "cell_type": "markdown", - "id": "2fc4e5c7", + "id": "2795661c", "metadata": { "tags": [ "solution"