1_notmnist.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5hIbr52I7Z7U"
   },
   "source": [
    "Deep Learning\n",
    "=============\n",
    "\n",
    "Assignment 1\n",
    "------------\n",
    "\n",
    "The objective of this assignment is to learn about simple data curation practices, and familiarize you with some of the data we'll be reusing later.\n",
    "\n",
    "This notebook uses the [notMNIST](http://yaroslavvb.blogspot.com/2011/09/notmnist-dataset.html) dataset to be used with python experiments. This dataset is designed to look like the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset, while looking a little more like real data: it's a harder task, and the data is a lot less 'clean' than MNIST."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "collapsed": false,
    "id": "apJbCsBHl-2A"
   },
   "outputs": [],
   "source": [
    "# These are all the modules we'll be using later. Make sure you can import them\n",
    "# before proceeding further.\n",
    "from __future__ import print_function\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import os\n",
    "import sys\n",
    "import tarfile\n",
    "from IPython.display import display, Image\n",
    "from scipy import ndimage\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from six.moves.urllib.request import urlretrieve\n",
    "from six.moves import cPickle as pickle\n",
    "\n",
    "# Config the matlotlib backend as plotting inline in IPython\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jNWGtZaXn-5j"
   },
   "source": [
    "First, we'll download the dataset to our local machine. The data consists of characters rendered in a variety of fonts on a 28x28 image. The labels are limited to 'A' through 'J' (10 classes). The training set has about 500k and the testset 19000 labelled examples. Given these sizes, it should be possible to train models quickly on any machine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 186058,
     "status": "ok",
     "timestamp": 1444485672507,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "EYRJ4ICW6-da",
    "outputId": "0d0f85df-155f-4a89-8e7e-ee32df36ec8d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found and verified notMNIST_large.tar.gz\n",
      "Found and verified notMNIST_small.tar.gz\n"
     ]
    }
   ],
   "source": [
    "url = 'http://commondatastorage.googleapis.com/books1000/'\n",
    "last_percent_reported = None\n",
    "\n",
    "def download_progress_hook(count, blockSize, totalSize):\n",
    "  \"\"\"A hook to report the progress of a download. This is mostly intended for users with\n",
    "  slow internet connections. Reports every 1% change in download progress.\n",
    "  \"\"\"\n",
    "  global last_percent_reported\n",
    "  percent = int(count * blockSize * 100 / totalSize)\n",
    "\n",
    "  if last_percent_reported != percent:\n",
    "    if percent % 5 == 0:\n",
    "      sys.stdout.write(\"%s%%\" % percent)\n",
    "      sys.stdout.flush()\n",
    "    else:\n",
    "      sys.stdout.write(\".\")\n",
    "      sys.stdout.flush()\n",
    "      \n",
    "    last_percent_reported = percent\n",
    "        \n",
    "def maybe_download(filename, expected_bytes, force=False):\n",
    "  \"\"\"Download a file if not present, and make sure it's the right size.\"\"\"\n",
    "  if force or not os.path.exists(filename):\n",
    "    print('Attempting to download:', filename) \n",
    "    filename, _ = urlretrieve(url + filename, filename, reporthook=download_progress_hook)\n",
    "    print('\\nDownload Complete!')\n",
    "  statinfo = os.stat(filename)\n",
    "  if statinfo.st_size == expected_bytes:\n",
    "    print('Found and verified', filename)\n",
    "  else:\n",
    "    raise Exception(\n",
    "      'Failed to verify ' + filename + '. Can you get to it with a browser?')\n",
    "  return filename\n",
    "\n",
    "train_filename = maybe_download('notMNIST_large.tar.gz', 247336696)\n",
    "test_filename = maybe_download('notMNIST_small.tar.gz', 8458043)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cC3p0oEyF8QT"
   },
   "source": [
    "Extract the dataset from the compressed .tar.gz file.\n",
    "This should give you a set of directories, labelled A through J."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 186055,
     "status": "ok",
     "timestamp": 1444485672525,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "H8CBE-WZ8nmj",
    "outputId": "ef6c790c-2513-4b09-962e-27c79390c762"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "notMNIST_large already present - Skipping extraction of notMNIST_large.tar.gz.\n",
      "['notMNIST_large/A', 'notMNIST_large/B', 'notMNIST_large/C', 'notMNIST_large/D', 'notMNIST_large/E', 'notMNIST_large/F', 'notMNIST_large/G', 'notMNIST_large/H', 'notMNIST_large/I', 'notMNIST_large/J']\n",
      "notMNIST_small already present - Skipping extraction of notMNIST_small.tar.gz.\n",
      "['notMNIST_small/A', 'notMNIST_small/B', 'notMNIST_small/C', 'notMNIST_small/D', 'notMNIST_small/E', 'notMNIST_small/F', 'notMNIST_small/G', 'notMNIST_small/H', 'notMNIST_small/I', 'notMNIST_small/J']\n"
     ]
    }
   ],
   "source": [
    "num_classes = 10\n",
    "np.random.seed(133)\n",
    "\n",
    "def maybe_extract(filename, force=False):\n",
    "  root = os.path.splitext(os.path.splitext(filename)[0])[0]  # remove .tar.gz\n",
    "  if os.path.isdir(root) and not force:\n",
    "    # You may override by setting force=True.\n",
    "    print('%s already present - Skipping extraction of %s.' % (root, filename))\n",
    "  else:\n",
    "    print('Extracting data for %s. This may take a while. Please wait.' % root)\n",
    "    tar = tarfile.open(filename)\n",
    "    sys.stdout.flush()\n",
    "    tar.extractall()\n",
    "    tar.close()\n",
    "  data_folders = [\n",
    "    os.path.join(root, d) for d in sorted(os.listdir(root))\n",
    "    if os.path.isdir(os.path.join(root, d))]\n",
    "  if len(data_folders) != num_classes:\n",
    "    raise Exception(\n",
    "      'Expected %d folders, one per class. Found %d instead.' % (\n",
    "        num_classes, len(data_folders)))\n",
    "  print(data_folders)\n",
    "  return data_folders\n",
    "  \n",
    "train_folders = maybe_extract(train_filename)\n",
    "test_folders = maybe_extract(test_filename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4riXK3IoHgx6"
   },
   "source": [
    "---\n",
    "Problem 1\n",
    "---------\n",
    "\n",
    "Let's take a peek at some of the data to make sure it looks sensible. Each exemplar should be an image of a character A through J rendered in a different font. Display a sample of the images that we just downloaded. Hint: you can use the package IPython.display.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAACLElEQVR4nF2RS0iUcRTFf//HqJEU\nqKWFZGh9YzRFi1yF+cCFRCVBCGLYtk0RakEPQtJoU9Qim1YVRKQLoUUvyBrMQBLpgaKWqKNkPjAl\ne/j65rstJnXsbi7cwz3n3HNhqcpehedGQy+mm5JQrCpDtURERCQioeWpjTaPktHanjE1tz2Y+QgT\niV1UrP91DAOGg3Ie8x/r/oiDBYNfOlcrYqkbAN/JxiJokAB6NW3HbagUCUG5VC7xagAt6TseQxxu\nzhbeUozEgIoDsyGFy8zanQy1Fm7w1AooHH+2aCgffIoDL82+GFGNP1KE4ctdR4IQkPqYYwy3RnyQ\nIpcdmcxi88+wbzlBTbp3nTjOyfyiJwWkDEtedNWC9o6oh3gks/BkIuE9kw1nSltWJFv7AK2yMwBt\ncKTLsuR3q1yMcaA0jbILDVgUxfSjI0crjP3xITnhFNwpLetU/6JriWRA6rjIdHvHtOSj1ceZjShA\ns00eYFnXI3VYcuUC8ZyWsxjAUi0FGOidTCOOTXIfH443mBi19GnQYAhIDT7Fmm9tKENQKjDAHqnB\nWKokB43m9ZTBkOW2oYArshutafqdiMJwVdJQlmtSiCG+v1tZi2/sORoMJZKLtWTLG5TOy2wXEYpS\nv0ef95nDiGt6b+SeEBokfOlm84gri3vRKFLG5/LJrAh2y8IhwuKJTDXXD9WiQZE0KgsTrsj88L2A\ncvz6z9eBeRJcN5ptVXVf+7suL75vlr8kvt2zctwKgAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nn = 99\n",
    "nfolder = 0\n",
    "display(Image(train_folders[nfolder] + \"/\" + os.listdir(train_folders[nfolder])[nn]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PBdkjESPK8tw"
   },
   "source": [
    "Now let's load the data in a more manageable format. Since, depending on your computer setup you might not be able to fit it all in memory, we'll load each class into a separate dataset, store them on disk and curate them independently. Later we'll merge them into a single dataset of manageable size.\n",
    "\n",
    "We'll convert the entire dataset into a 3D array (image index, x, y) of floating point values, normalized to have approximately zero mean and standard deviation ~0.5 to make training easier down the road. \n",
    "\n",
    "A few images might not be readable, we'll just skip them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 30
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 399874,
     "status": "ok",
     "timestamp": 1444485886378,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "h7q0XhG3MJdf",
    "outputId": "92c391bb-86ff-431d-9ada-315568a19e59"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "notMNIST_large/A.pickle already present - Skipping pickling.\n",
      "notMNIST_large/B.pickle already present - Skipping pickling.\n",
      "notMNIST_large/C.pickle already present - Skipping pickling.\n",
      "notMNIST_large/D.pickle already present - Skipping pickling.\n",
      "notMNIST_large/E.pickle already present - Skipping pickling.\n",
      "notMNIST_large/F.pickle already present - Skipping pickling.\n",
      "notMNIST_large/G.pickle already present - Skipping pickling.\n",
      "notMNIST_large/H.pickle already present - Skipping pickling.\n",
      "notMNIST_large/I.pickle already present - Skipping pickling.\n",
      "notMNIST_large/J.pickle already present - Skipping pickling.\n",
      "notMNIST_small/A.pickle already present - Skipping pickling.\n",
      "notMNIST_small/B.pickle already present - Skipping pickling.\n",
      "notMNIST_small/C.pickle already present - Skipping pickling.\n",
      "notMNIST_small/D.pickle already present - Skipping pickling.\n",
      "notMNIST_small/E.pickle already present - Skipping pickling.\n",
      "notMNIST_small/F.pickle already present - Skipping pickling.\n",
      "notMNIST_small/G.pickle already present - Skipping pickling.\n",
      "notMNIST_small/H.pickle already present - Skipping pickling.\n",
      "notMNIST_small/I.pickle already present - Skipping pickling.\n",
      "notMNIST_small/J.pickle already present - Skipping pickling.\n"
     ]
    }
   ],
   "source": [
    "image_size = 28  # Pixel width and height.\n",
    "pixel_depth = 255.0  # Number of levels per pixel.\n",
    "\n",
    "def load_letter(folder, min_num_images):\n",
    "  \"\"\"Load the data for a single letter label.\"\"\"\n",
    "  image_files = os.listdir(folder)\n",
    "  dataset = np.ndarray(shape=(len(image_files), image_size, image_size),\n",
    "                         dtype=np.float32)\n",
    "  print(folder)\n",
    "  num_images = 0\n",
    "  for image in image_files:\n",
    "    image_file = os.path.join(folder, image)\n",
    "    try:\n",
    "      image_data = (ndimage.imread(image_file).astype(float) - \n",
    "                    pixel_depth / 2) / pixel_depth\n",
    "      if image_data.shape != (image_size, image_size):\n",
    "        raise Exception('Unexpected image shape: %s' % str(image_data.shape))\n",
    "      dataset[num_images, :, :] = image_data\n",
    "      num_images = num_images + 1\n",
    "    except IOError as e:\n",
    "      print('Could not read:', image_file, ':', e, '- it\\'s ok, skipping.')\n",
    "    \n",
    "  dataset = dataset[0:num_images, :, :]\n",
    "  if num_images < min_num_images:\n",
    "    raise Exception('Many fewer images than expected: %d < %d' %\n",
    "                    (num_images, min_num_images))\n",
    "    \n",
    "  print('Full dataset tensor:', dataset.shape)\n",
    "  print('Mean:', np.mean(dataset))\n",
    "  print('Standard deviation:', np.std(dataset))\n",
    "  return dataset\n",
    "        \n",
    "def maybe_pickle(data_folders, min_num_images_per_class, force=False):\n",
    "  dataset_names = []\n",
    "  for folder in data_folders:\n",
    "    set_filename = folder + '.pickle'\n",
    "    dataset_names.append(set_filename)\n",
    "    if os.path.exists(set_filename) and not force:\n",
    "      # You may override by setting force=True.\n",
    "      print('%s already present - Skipping pickling.' % set_filename)\n",
    "    else:\n",
    "      print('Pickling %s.' % set_filename)\n",
    "      dataset = load_letter(folder, min_num_images_per_class)\n",
    "      try:\n",
    "        with open(set_filename, 'wb') as f:\n",
    "          pickle.dump(dataset, f, pickle.HIGHEST_PROTOCOL)\n",
    "      except Exception as e:\n",
    "        print('Unable to save data to', set_filename, ':', e)\n",
    "  \n",
    "  return dataset_names\n",
    "\n",
    "train_datasets = maybe_pickle(train_folders, 45000)\n",
    "test_datasets = maybe_pickle(test_folders, 1800)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "vUdbskYE2d87"
   },
   "source": [
    "---\n",
    "Problem 2\n",
    "---------\n",
    "\n",
    "Let's verify that the data still looks good. Displaying a sample of the labels and images from the ndarray. Hint: you can use matplotlib.pyplot.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "l1 = load_letter(train_folders[0], 1)\n",
    "#or : ll = pickle.load(...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADJNJREFUeJzt3VusXOV5xvHniZ02B0cIRbUt4TY7qCpVI0VbREWK3AtH\nUIKqKiAkaEouIG0jLkJClJsSbrYc5SLJhRVuuOEkFwWVg0Tt3BBI3U1F2gSKcWPAIZWSISHBGysh\nEZZQhPHbi1mG8fbs+cazjub9/6Qlr72+dXj38npmHb6Z2Y4IAcjlXX0XAKB7BB9IiOADCRF8ICGC\nDyRE8IGEOgu+7Sts/9j2T2z/c1fbnZftke3/tf2M7ScHUM9dttds/2hi2vm2H7X9gu3v2j5vYPWt\n2H7J9sFquKLH+nbYPmD7OduHbX+xmj6IfTilvi9U0zvZh+6iH9/2uyT9RNKlkn4l6SlJn46IH7e+\n8TnZ/qmkj0XEq33XIkm2/0rScUn/EhEfraZ9Q9KvI+Kb1Yvn+RFxy4DqW5H0WkTs6aOmSba3S9oe\nEYdsb5H0tKQrJX1WA9iHM+r7O3WwD7s6418i6f8i4sWIeEPSv2r8Sw6JNaBbn4h4QtL6F6ErJe2t\nxvdKuqrToiZsUJ803o+9i4ijEXGoGj8u6YikHRrIPtygvguq5tb3YVcH+gWSfjHx80t6+5ccipD0\nmO2nbH+u72I2sDUi1qTxgSNpa8/1THOT7UO27+zzVmSS7SVJy5J+IGnb0PbhRH0/rCa1vg8Hc4Yb\ngJ0RcbGkv5H0+epSduiG9n7r2yVdGBHLko5KGsIl/xZJD0m6uTqzrt9nve7DKfV1sg+7Cv4vJf3J\nxM87qmmDEREvV/8ek/SwxrcnQ7Nme5v01j3iKz3Xc5qIOBZvPzS6Q9Jf9lmP7c0ah+reiNhXTR7M\nPpxWX1f7sKvgPyXpT21/yPYfSPq0pP0dbbvI9vuqV17Zfr+kyyU9229Vksb3epP3e/sl3VCNXy9p\n3/oFOnZafVWQTrla/e/DuyU9HxG3TUwb0j48o76u9mEnT/WlcXeepNs0frG5KyK+3smG52D7wxqf\n5UPSZknf7rs+2/dJ2iXpg5LWJK1I+jdJD0r6Y0kvSro2In47oPo+ofG96klJI0k3nrqf7qG+nZL+\nU9Jhjf9fQ9Ktkp6U9IB63ocz6rtOHezDzoIPYDh4uAckRPCBhAg+kBDBBxKqFfyhf/AGwHQLP9Wf\n94M3tuk2AHoSEVPf97+5xjrf+uCNJNk+9cGbMz5xtzIxvqpx5+9QrYr66ljVcOtb1XBrk5qvb/eM\ntjqX+ufCB28ATMHDPSChOpf6c3/wZnVi/D01NtiFpb4LKFjqu4CCpb4LmGGp7wIKlmouP6qGedR5\nuLdJ0gsaP9x7WeP3QP99RBxZN1+sTFkeQLt2q4WHexHxpu2bJD2qtz94c6SwGIABqHOpr4h4RNJF\nDdXSincX2j/QSRV4p3qt0P5GJ1WcPR7uAQkRfCAhgg8kRPCBhAg+kBDBBxIi+EBCtfrxzwWlftTf\ndFIFMCyc8YGECD6QEMEHEiL4QEIEH0iI4AMJEXwgocH345c+T1/qp7+20H40Pj6z/QOFT1y/qU2F\nLWDINunNme2vFb6xYbv/e2b7A4Xt1z2+F8UZH0iI4AMJEXwgIYIPJETwgYQIPpAQwQcSGnw/fqnA\nUj/nX1wzu/0jvrywhr56WtGNev+/cc3sfnw9OLu57vG9KM74QEIEH0iI4AMJEXwgIYIPJETwgYQI\nPpBQrX582yNJv5N0UtIbEXFJE0U16kTdFVxQaKcf/9xW6scfzW6ufXz1o+4beE5K2hURrzZRDIBu\n1L3UdwPrANCxuqENSY/Zfsr255ooCED76l7q74yIl23/kcYvAEci4on1M61OjC9VA4BmjVR8IvGW\nWsGPiJerf4/ZfljSJZLOCP6uOhsBMJclnX5SfXzGvAtf6tt+n+0t1fj7JV0u6dlF1wegO3XO+Nsk\nPWw7qvV8OyIebaYsAG1aOPgR8TNJyw3W0o7a3zjwy0I7/fjntlI/fsHgv9FiOrrigIQIPpAQwQcS\nIvhAQgQfSIjgAwkRfCChwfdCvl5z+fsL32se/7R79gzn6Oet0ZBCQu6/s97q6x7fi+KMDyRE8IGE\nCD6QEMEHEiL4QEIEH0iI4AMJOSLa3YAdK61uAcA0uyVFhKe1ccYHEiL4QEIEH0iI4AMJEXwgIYIP\nJETwgYQIPpAQwQcSIvhAQgQfSIjgAwkRfCAhgg8kRPCBhIrfq2/7Lkl/K2ktIj5aTTtf0v2SPiRp\nJOnaiPhdGwWW/np56a/TX1xo/9S3CjP8vtCOd7Y/nN28/0uz2w8WVl/3+F7UPGf8eyR9ct20WyR9\nLyIuknRA0leaLgxAe4rBj4gnJL26bvKVkvZW43slXdVwXQBatOg9/taIWJOkiDgqaWtzJQFoW1N/\nO2/mF/etTowvVQOAZo2qYR6LBn/N9raIWLO9XdIrs2beteBGAMxvSaefVB+fMe+8l/quhlP2S7qh\nGr9e0r451wNgAIrBt32fpP+S9Ge2f277s5K+Lumvbb8g6dLqZwDniOKlfkRct0HTZQ3XMlWpwFI/\n56eumd3uL5W+9b+vnlZ0o97/b1yze2b7wQdnr73u8b0o3rkHJETwgYQIPpAQwQcSIvhAQgQfSIjg\nAwk19V79d7D3FtrZhee2nO/T4IwPJETwgYQIPpAQwQcSIvhAQgQfSIjgAwnRCV30eqH9ndnPm8eJ\nvgvoBWd8ICGCDyRE8IGECD6QEMEHEiL4QEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQwQcSIvhAQgQf\nSKgYfNt32V6z/aOJaSu2X7J9sBquaLdMAE2a54x/j6RPTpm+JyIuroZHGq4LQIuKwY+IJyS9OqXJ\nzZcDoAt17vFvsn3I9p22z2usIgCtW/Q7926X9NWICNtfk7RH0j9uNPPqxPhSNQBo1qga5rFQ8CPi\n2MSPd0j6zqz5dy2yEQBnZUmnn1QfnzHvvJf61sQ9ve3tE21XS3p2zvUAGIDiGd/2fRqftD9o++eS\nViR9wvaypJMaX13c2GKNABpWDH5EXDdl8j0t1AKgI7xzD0iI4AMJEXwgIYIPJETwgYQIPpAQwQcS\nIvhAQgQfSIjgAwkRfCAhgg8kRPCBhAg+kBDBBxIi+EBCBB9IiOADCRF8ICGCDyRE8IGECD6QEMEH\nEiL4QEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQwQcSKgbf9g7bB2w/Z/uw7S9W08+3/ajtF2x/1/Z5\n7ZcLoAnznPFPSPpyRHxE0sclfd72n0u6RdL3IuIiSQckfaW9MgE0qRj8iDgaEYeq8eOSjkjaIelK\nSXur2fZKuqqtIgE066zu8W0vSVqW9ANJ2yJiTRq/OEja2nRxANqxed4ZbW+R9JCkmyPiuO1YN8v6\nn9+yOjG+VA0AmjWqhnnMFXzbmzUO/b0Rsa+avGZ7W0Ss2d4u6ZWNlt81ZzEAFrek00+qj8+Yd95L\n/bslPR8Rt01M2y/phmr8ekn71i8EYJiKZ3zbOyV9RtJh289ofEl/q6RvSHrA9j9IelHStW0WCqA5\nxeBHxPclbdqg+bJmywHQBd65ByRE8IGECD6QEMEHEiL4QEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQ\nwQcSIvhAQgQfSIjgAwkRfCAhgg8kRPCBhAg+kBDBBxIi+EBCBB9IiOADCRF8ICGCDyRE8IGECD6Q\nEMEHEiL4QEIEH0ioGHzbO2wfsP2c7cO2v1BNX7H9ku2D1XBF++UCaMLmOeY5IenLEXHI9hZJT9t+\nrGrbExF7Sit4d40CS8u+XlrBPL8hsKiax1fp+D5Rb/UbKpYdEUclHa3Gj9s+IumCqtkt1QWgRWd1\nj297SdKypB9Wk26yfcj2nbbPa7g2AC2ZO/jVZf5Dkm6OiOOSbpd0YUQsa3xFULzkBzAMc92h2N6s\ncejvjYh9khQRxyZmuUPSdzZa/t8nxj8s6cKzrxNAwU8l/WzOeed9NHG3pOcj4rZTE2xvr+7/Jelq\nSc9utPClc24EwOIu1Okn1f+YMW8x+LZ3SvqMpMO2n5EUkm6VdJ3tZUknJY0k3bhowQC6Nc9T/e9L\n2jSl6ZHmywHQhU56ud+osWztAtvqCAWk2sdXKRt1sjMLb9kFEiL4QEIEH0iI4AMJEXwgIYIPJETw\ngYQ66ce/rMay7y20P1lawXKh/cH5awHOUDi+thWOr0sKqy9+38QMu2e0ccYHEiL4QEIEH0io8+A/\n0/UGz9qo7wIKRn0XUDDqu4AZRn0XMNP/dLgtgn+GUd8FFIz6LqBg1HcBM4z6LmCmpzvcFpf6QEIE\nH0jIEdHuBux2NwBgQxEx9SvwWw8+gOHhUh9IiOADCRF8ICGCDyRE8IGE/h8wUhPVd/VX5AAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1169e5910>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "nn = 590\n",
    "plt.matshow(l1[nn])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "cYznx5jUwzoO"
   },
   "source": [
    "---\n",
    "Problem 3\n",
    "---------\n",
    "Another check: we expect the data to be balanced across classes. Verify that.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEACAYAAABVtcpZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADxJJREFUeJzt3H+s3XV9x/HnCysR9dI0Ji1Zi4hBsZj5g5iiErO7uKBo\n1vLHIKCJIIv/4ALRxdD6D/iXssQ4kskSorJqcKR0W2gWJrXpzhJNJv4AIdxamxhK6dZLnAzL9geU\nvPfH+VbOblq4Ped6vqf9PB/JSb7ne7/f8/ncEzjP+/mec5qqQpLUprP6noAkqT9GQJIaZgQkqWFG\nQJIaZgQkqWFGQJIa9qoRSPLNJItJHhvZtybJ7iT7kzyUZPXIz7YlOZBkX5IrRvZfmuSxJL9M8tcr\n/6tIkk7VclYC9wAfWbJvK7Cnqi4G9gLbAJJcAlwDbASuBO5Kku6cvwX+vKreDrw9ydLHlCRN2atG\noKp+ADy7ZPcWYHu3vR24qtveDNxXVceq6kngALApyXnAXFX9uDvu2yPnSJJ6Mu57AmurahGgqo4A\na7v964FDI8cd7vatB54e2f90t0+S1KOVemPYf3tCkk5Dq8Y8bzHJuqpa7C71PNPtPwycP3Lchm7f\nyfafUBKjIkljqKq8+lEvW+5KIN3tuF3ADd329cADI/uvTXJ2kguBi4CHu0tGzyXZ1L1R/KmRc06i\npnqbm/uzXsZ9eRF1sp/d1tO4ff2+r3Sb9Lk43X7flXguzqTfeTnPRX+/b1X1fhvHq64EknwXmAfe\nlOSp7hn/CnB/khuBgww/EURVLSTZASwALwI31csz+yzwd8DrgAer6ntjzViStGJeNQJV9YmT/OhP\nTnL8l4Evn2D/T4E/PKXZSZJ+r/zG8Myb73sCM2S+7wnMkPm+JzBD5vuewGnNCMy8+b4nMEPm+57A\nDJnvewIzZL7vCZzWjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLD\njIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAk\nNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDjIAkNcwISFLDJopAkm1J\nnkjyWJJ7k5ydZE2S3Un2J3koyeolxx9Isi/JFZNPX5I0ibEjkOQC4DPAe6vqXcAq4DpgK7Cnqi4G\n9gLbuuMvAa4BNgJXAnclyWTTlyRNYpKVwG+BF4A3JFkFnAMcBrYA27tjtgNXddubgfuq6lhVPQkc\nADZNML4kaUJjR6CqngW+CjzF8MX/uaraA6yrqsXumCPA2u6U9cChkYc43O2TJPVk1bgnJnkr8Dng\nAuA54P4knwRqyaFL7y/T7SPb891NknTcYDBgMBhM9BhjRwB4H/DDqvoNQJJ/Aj4ILCZZV1WLSc4D\nnumOPwycP3L+hm7fSdw+wdQk6cw3Pz/P/Pz87+5/6UtfOuXHmOQ9gf3A+5O8rnuD98PAArALuKE7\n5nrggW57F3Bt9wmiC4GLgIcnGF+SNKGxVwJV9fMk3wZ+CrwEPALcDcwBO5LcCBxk+IkgqmohyQ6G\noXgRuKmqxrxUJElaCZnF1+EkNfZbCWOam7uao0d3Mu1xh+K4jnuGjd3euLPwWpqEqjqlj977jWFJ\napgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgR\nkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSG\nGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJapgRkKSGGQFJathEEUiyOsn9SfYleSLJZUnW\nJNmdZH+Sh5KsHjl+W5ID3fFXTD59SdIkJl0J3Ak8WFUbgXcDvwC2Anuq6mJgL7ANIMklwDXARuBK\n4K4kmXB8SdIExo5AknOBD1XVPQBVdayqngO2ANu7w7YDV3Xbm4H7uuOeBA4Am8YdX5I0uUlWAhcC\nv05yT5KfJbk7yeuBdVW1CFBVR4C13fHrgUMj5x/u9kmSejJJBFYBlwJfr6pLgf9heCmolhy39L4k\naUasmuDcp4FDVfWT7v4/MIzAYpJ1VbWY5Dzgme7nh4HzR87f0O07idtHtue7myTpuMFgwGAwmOgx\nUjX+H+pJ/g34TFX9MsltwOu7H/2mqu5Iciuwpqq2dm8M3wtcxvAy0PeBt9UJJpCkpr2AmJu7mqNH\nd9LPwiWO67hn2NjtjTvJa+mKzSKhqk7pAzeTrAQAbgbuTfJa4FfAp4HXADuS3AgcZPiJIKpqIckO\nYAF4EbjpRAGQJE3PRCuB3xdXAo7ruKf72O2NOwuvpeOsBPzGsCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1\nzAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhI\nUsOMgCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsOM\ngCQ1zAhIUsOMgCQ1zAhIUsOMgCQ1zAhIUsMmjkCSs5L8LMmu7v6aJLuT7E/yUJLVI8duS3Igyb4k\nV0w6tiRpMiuxErgFWBi5vxXYU1UXA3uBbQBJLgGuATYCVwJ3JckKjC9JGtNEEUiyAfgY8I2R3VuA\n7d32duCqbnszcF9VHauqJ4EDwKZJxpckTWbSlcDXgC8ANbJvXVUtAlTVEWBtt389cGjkuMPdPklS\nT8aOQJKPA4tV9SjwSpd16hV+Jknq0aoJzr0c2JzkY8A5wFyS7wBHkqyrqsUk5wHPdMcfBs4fOX9D\nt+8kbh/Znu9ukqTjBoMBg8FgosdI1eR/qCf5I+Avq2pzkr8C/quq7khyK7CmqrZ2bwzfC1zG8DLQ\n94G31QkmkKSmvYCYm7uao0d30s/CJY7ruGfY2O2NuxKvpRPPIqGqTukDN5OsBE7mK8COJDcCBxl+\nIoiqWkiyg+EniV4EbjpRACRJ07MiK4GV5krAcR33dB+7vXFn4bV0nJWA3xiWpIYZAUlqmBGQpIYZ\nAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlq\nmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQ\npIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlqmBGQpIYZAUlq2NgRSLIhyd4kTyR5PMnN3f41SXYn2Z/k\noSSrR87ZluRAkn1JrliJX0CSNL5JVgLHgM9X1TuBDwCfTfIOYCuwp6ouBvYC2wCSXAJcA2wErgTu\nSpJJJi9JmszYEaiqI1X1aLf9PLAP2ABsAbZ3h20Hruq2NwP3VdWxqnoSOABsGnd8SdLkVuQ9gSRv\nAd4D/DuwrqoWYRgKYG132Hrg0Mhph7t9kqSerJr0AZK8EdgJ3FJVzyepJYcsvb9Mt49sz3c3SdJx\ng8GAwWAw0WOkaszXaCDJKuCfgX+pqju7ffuA+apaTHIe8K9VtTHJVqCq6o7uuO8Bt1XVj07wuDV2\nO8Y0N3c1R4/uZNrjDsVxHfcMG7u9cSd5LV2xWSRU1Sm91zrp5aBvAQvHA9DZBdzQbV8PPDCy/9ok\nZye5ELgIeHjC8SVJExj7clCSy4FPAo8neYRhfr8I3AHsSHIjcJDhJ4KoqoUkO4AF4EXgppqFdEpS\nwya6HPT74uUgx3Xc033s9sadhdfSPi4HSZJOY0ZAkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZA\nkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZAkhpm\nBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSpYUZAkhpmBCSp\nYUZAkhpmBCSpYVOPQJKPJvlFkl8muXXa40uSXjbVCCQ5C/gb4CPAO4HrkrxjmnM4/Qz6nsAMGfQ9\ngRky6HsCM2TQ9wROa9NeCWwCDlTVwap6EbgP2DLlOZxmBn1PYIYM+p7ADBn0PYEZMuh7Aqe1aUdg\nPXBo5P7T3T5JUg9W9T2Bkzn33D+d6ngvvPCTqY4nSbMgVTW9wZL3A7dX1Ue7+1uBqqo7lhw3vUlJ\n0hmkqnIqx087Aq8B9gMfBv4TeBi4rqr2TW0SkqTfmerloKp6KclfALsZvh/xTQMgSf2Z6kpAkjRb\nZuobw36RbCjJhiR7kzyR5PEkN/c9p74lOSvJz5Ls6nsufUqyOsn9SfZ1/31c1vec+pJkW/ccPJbk\n3iRn9z2naUnyzSSLSR4b2bcmye4k+5M8lGT1ch5rZiLgF8n+n2PA56vqncAHgM82/Fwcdwuw0Pck\nZsCdwINVtRF4N9Dk5dQkFwCfAd5bVe9ieGn72n5nNVX3MHytHLUV2FNVFwN7gW3LeaCZiQB+kex3\nqupIVT3abT/P8H/0Zr9PkWQD8DHgG33PpU9JzgU+VFX3AFTVsar6bc/T6stvgReANyRZBbwe+I9+\npzQ9VfUD4Nklu7cA27vt7cBVy3msWYqAXyQ7gSRvAd4D/KjfmfTqa8AXgNbfwLoQ+HWSe7pLY3cn\nOafvSfWhqp4Fvgo8BRwG/ruq9vQ7q96trapFGP4hCaxdzkmzFAEtkeSNwE7glm5F0JwkHwcWu5VR\nulurVgGXAl+vqkuB/2V4CaA5Sd4KfA64APgD4I1JPtHvrGbOsv5omqUIHAbePHJ/Q7evSd0Sdyfw\nnap6oO/59OhyYHOSXwF/D/xxkm/3PKe+PA0cqqrjX2/fyTAKLXof8MOq+k1VvQT8I/DBnufUt8Uk\n6wCSnAc8s5yTZikCPwYuSnJB9y7/tUDLnwT5FrBQVXf2PZE+VdUXq+rNVfVWhv9N7K2qT/U9rz50\nS/1DSd7e7fow7b5Zvh94f5LXJQnD56K1N8mXrox3ATd029cDy/rjcWb+7SC/SPayJJcDnwQeT/II\nw2XdF6vqe/3OTDPgZuDeJK8FfgV8uuf59KKqft6tCH8KvAQ8Atzd76ymJ8l3gXngTUmeAm4DvgLc\nn+RG4CBwzbIeyy+LSVK7ZulykCRpyoyAJDXMCEhSw4yAJDXMCEhSw4yAJDXMCEhSw4yAJDXs/wAx\nh3e2n57TEQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11adf9ad0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# checked files sizes of pickled files.\n",
    "#the following can be used after merge_dataset() is called:\n",
    "#plt.hist(train_labels)\n",
    "#plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LA7M7K22ynCt"
   },
   "source": [
    "Merge and prune the training data as needed. Depending on your computer setup, you might not be able to fit it all in memory, and you can tune `train_size` as needed. The labels will be stored into a separate array of integers 0 through 9.\n",
    "\n",
    "Also create a validation dataset for hyperparameter tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 411281,
     "status": "ok",
     "timestamp": 1444485897869,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "s3mWgZLpyuzq",
    "outputId": "8af66da6-902d-4719-bedc-7c9fb7ae7948"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training: (200000, 28, 28) (200000,)\n",
      "Validation: (10000, 28, 28) (10000,)\n",
      "Testing: (10000, 28, 28) (10000,)\n"
     ]
    }
   ],
   "source": [
    "def make_arrays(nb_rows, img_size):\n",
    "  if nb_rows:\n",
    "    dataset = np.ndarray((nb_rows, img_size, img_size), dtype=np.float32)\n",
    "    labels = np.ndarray(nb_rows, dtype=np.int32)\n",
    "  else:\n",
    "    dataset, labels = None, None\n",
    "  return dataset, labels\n",
    "\n",
    "def merge_datasets(pickle_files, train_size, valid_size=0):\n",
    "  num_classes = len(pickle_files)\n",
    "  valid_dataset, valid_labels = make_arrays(valid_size, image_size)\n",
    "  train_dataset, train_labels = make_arrays(train_size, image_size)\n",
    "  vsize_per_class = valid_size // num_classes\n",
    "  tsize_per_class = train_size // num_classes\n",
    "    \n",
    "  start_v, start_t = 0, 0\n",
    "  end_v, end_t = vsize_per_class, tsize_per_class\n",
    "  end_l = vsize_per_class+tsize_per_class\n",
    "  for label, pickle_file in enumerate(pickle_files):       \n",
    "    try:\n",
    "      with open(pickle_file, 'rb') as f:\n",
    "        letter_set = pickle.load(f)\n",
    "        # let's shuffle the letters to have random validation and training set\n",
    "        np.random.shuffle(letter_set)\n",
    "        if valid_dataset is not None:\n",
    "          valid_letter = letter_set[:vsize_per_class, :, :]\n",
    "          valid_dataset[start_v:end_v, :, :] = valid_letter\n",
    "          valid_labels[start_v:end_v] = label\n",
    "          start_v += vsize_per_class\n",
    "          end_v += vsize_per_class\n",
    "                    \n",
    "        train_letter = letter_set[vsize_per_class:end_l, :, :]\n",
    "        train_dataset[start_t:end_t, :, :] = train_letter\n",
    "        train_labels[start_t:end_t] = label\n",
    "        start_t += tsize_per_class\n",
    "        end_t += tsize_per_class\n",
    "    except Exception as e:\n",
    "      print('Unable to process data from', pickle_file, ':', e)\n",
    "      raise\n",
    "    \n",
    "  return valid_dataset, valid_labels, train_dataset, train_labels\n",
    "            \n",
    "            \n",
    "train_size = 200000\n",
    "valid_size = 10000\n",
    "test_size = 10000\n",
    "\n",
    "valid_dataset, valid_labels, train_dataset, train_labels = merge_datasets(\n",
    "  train_datasets, train_size, valid_size)\n",
    "_, _, test_dataset, test_labels = merge_datasets(test_datasets, test_size)\n",
    "\n",
    "print('Training:', train_dataset.shape, train_labels.shape)\n",
    "print('Validation:', valid_dataset.shape, valid_labels.shape)\n",
    "print('Testing:', test_dataset.shape, test_labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "GPTCnjIcyuKN"
   },
   "source": [
    "Next, we'll randomize the data. It's important to have the labels well shuffled for the training and test distributions to match."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "collapsed": true,
    "id": "6WZ2l2tN2zOL"
   },
   "outputs": [],
   "source": [
    "def randomize(dataset, labels):\n",
    "  permutation = np.random.permutation(labels.shape[0])\n",
    "  shuffled_dataset = dataset[permutation,:,:]\n",
    "  shuffled_labels = labels[permutation]\n",
    "  return shuffled_dataset, shuffled_labels\n",
    "train_dataset, train_labels = randomize(train_dataset, train_labels)\n",
    "test_dataset, test_labels = randomize(test_dataset, test_labels)\n",
    "valid_dataset, valid_labels = randomize(valid_dataset, valid_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "puDUTe6t6USl"
   },
   "source": [
    "---\n",
    "Problem 4\n",
    "---------\n",
    "Convince yourself that the data is still good after shuffling!\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "e\n",
      "b\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFlZJREFUeJzt3X+UlXWdB/D3J0lHY0NDgbPSOhkZKtkcBcWDbOOKVmtK\nW4aEdoRdCDXSLT1AxO48bGSKodi6mgsmBBoQpuBaRoJDYqJIYqCQmTu2pAz4A5VTeEQ++8e9NAPO\nfX8uPMy9t/N9v86Zw3Dfz/P9PveZ+czcO9/n+33M3SEiaXlPtQ9ARCpPhS+SIBW+SIJU+CIJUuGL\nJEiFL5KgihW+mX3KzDaa2bNmNqFS/ZbLzFrM7Ckze9LMHq+B47ndzFrN7DftHjvCzJaa2W/N7Odm\n1q3Gjq/JzDaZ2a+LH5+q4vH1NrPlZva0ma0zsyuKj9fEOezg+L5afLwi59AqMY5vZu8B8CyAswC8\nCGA1gOHuvrHTOy+TmT0P4BR3f63axwIAZnYGgO0AfujuJxUfuw7AK+4+rfjD8wh3n1hDx9cE4E13\nv6Eax9SemfUC0Mvd15pZVwBrAAwFMAo1cA7J8V2ICpzDSv3GPxXA79z9BXd/G8B8FJ5kLTHU0Fsf\nd18JYO8fQkMBzCl+PgfAZyt6UO2UOD6gcB6rzt03u/va4ufbAWwA0Bs1cg5LHN/RxbjTz2GlvtGP\nBvB/7f6/CW1PslY4gF+Y2WozG1Ptgymhh7u3AoVvHAA9qnw8HRlnZmvNbFY134q0Z2b1ABoArALQ\ns9bOYbvje6z4UKefw5r5DVcDBrn7yQD+EcBXii9la12tXW99C4Bj3b0BwGYAtfCSvyuARQCuLP5m\n3fucVfUcdnB8FTmHlSr8PwL4u3b/7118rGa4+0vFf7cCuAeFtye1ptXMegJ/eY+4pcrHswd33+pt\nfzSaCWBANY/HzLqgUFRz3X1x8eGaOYcdHV+lzmGlCn81gD5mdoyZHQxgOIAlFeo7ZGaHFX/ywsze\nB+AcAOure1QACu/12r/fWwJgZPHzSwAs3nuHCtvj+IqFtNvnUP1z+AMAz7j7Te0eq6Vz+K7jq9Q5\nrMhf9YHCcB6Am1D4YXO7u19bkY7LYGYfQuG3vAPoAuDOah+fmd0FoBFAdwCtAJoA3AvgxwA+COAF\nAMPcfVsNHd+ZKLxX3QWgBcDY3e+nq3B8gwD8EsA6FL6uDmASgMcBLESVzyE5vhGowDmsWOGLSO3Q\nH/dEEqTCF0mQCl8kQSp8kQTlKvxan3gjIh3b77/qlzvxxsw0bCBSJe7e4XX/XXK0+ZeJNwBgZrsn\n3nQw466p3efNKAz/7vbeoJu3g/wimvoXj6P5gh/t+f8fA/hCu/8PHxL83Pouj5sa+HyLbBXf3wbO\n3+uRPY/Qlw3n7Z/F258yI3h+wXdI07g9n18z9vzqZsv4/nbWIpr7sgtoHj6/ee2e390Z8Plszw0u\nXsP7/0h/mrf+jvffq29wfje2P55m7Hn2AOBQvj/+TLIpJZM8L/X/GibeiEgH9Mc9kQTleam/DxNv\nmtt9Xpejy853QrUPIFTbR1hf7QNgjm+s9hEE6nPu31L8iOUp/L9MvAHwEgoTb77Y8aaNObqprBOr\nfQCh2j7C+mofAHNCY7WPIFB/APZv38aKklvud+G7+ztmNg7AUrRNvNmwv+2JSOV0+iSdwnDeVLJF\ntABKMG3/fzIaN30m3ypGh/k4mo9/8WaaZ8GfOwcF/Z+/7RWa/+Tw7jR/jKbAud4v2IK73/is0dOC\n/Ydt59PhF3bl3x/R8zvT+bIKjct4C01D8n3/rPe5NL/bngtaqA9yVh+TSw7n6Y97IglS4YskSIUv\nkiAVvkiCVPgiCVLhiyRIhS+SoBoYx49Es/POo6k/yWdX4Roe25PB+Qnuuubf5ePAqzZ8nOan21ia\nt3gTzY+o20rzbl/K9/V/fS5/fq/tOIrm9fZtmq/162n+8QF8epxFsyuvbaaxzz2T7/88j+3GoP9t\nGc/D2auMxvFFpB0VvkiCVPgiCVLhiyRIhS+SIBW+SIJU+CIJqtA4PrkT8cjzeQOz+SqowH087pLx\n/Iyg+eZg/2gV1IuD2w3M2xS0PyvIo3HeaK0VtkprOaJVYHcGeXSdxmU8vqAnzxfx6wTC/o/MeN6L\nx1gf7I+TeRzWx0oSDtY4voi0UeGLJEiFL5IgFb5IglT4IglS4YskSIUvkqCKjOP76tJ5NoDvnzUH\n7V/Nj//hJ/h88Qd585iynrc/7sRpNO9ufBw/C5YLsCeCCd+YE+TROHtni64TGE1T7/dBmmd8WX8s\n86U0X3n82TSfvpF//7zBu8eUa/n3z5oJ/JZoS4zfoyYjt3WwcaVvk63f+CIJUuGLJEiFL5IgFb5I\nglT4IglS4YskSIUvkqBosjZlZi0AXgewC8Db7iVuRj4iRyd82XhgPh8nfbBPjr4BjD9xCs0n4Fqa\nfy9of+YTwQZ9PsTz6Pbq4Xz9aCQ68v4gD8bx+/am8cxgnD7yNdxI85UT+Tj+GyPz9X/6hOU0P3k2\nH6dfErT/u3H7eEBFuQofhYJvdPfXcrYjIhWU96W+HYA2RKTC8hatA/iFma02szEH4oBEpPPlfak/\nyN1fMrOjUPgBsMHd37UIWPZK2+eNhwKNh+XsVUTe5bHiRzlyFb67v1T8d6uZ3QPgVADvLvzueXoR\nkXKcVvzYjczf2f+X+mZ2mJl1LX7+PgDnAMj5N1gRqYQ8v/F7ArinsHw2ugC40z2YAykiNaEy6+oP\nLN3H0kcH0/3PGfMw72BWRuPjna9LPhJ30HyCncT7B2//PucXMZy3bBlvfkgW9B+N00fr1ne2fMfX\nbcelNF9xyN/TvMGeDvrn6+5/2vnX/8PBhRQ325949/0yGi9dl6M+Zpnm44tIGxW+SIJU+CIJUuGL\nJEiFL5IgFb5IglT4IgmqzDj+vNJ9eB++brnVBcfXkNF4RuGq4pJGv8XvP9+1biPvv99HaOxz+fNr\nbPgZzVfYKt5/zY/jd7a8z/94mi70eTQ/Ac/QvJ/x9Rww9WIa+5CgPrqS+uincXwRaUeFL5IgFb5I\nglT4IglS4YskSIUvkiAVvkiC8q65VxYfX3osMnsx2Pc0Po558Muv0/w160bz63n3+JbfRvPJW6bT\nPOvJ2z8fn6b5iobgOoa1Gc8RHABagzwSrasfjaMH6+6H7fcIcj5fvm4bX1f/aRvO86D3oX4Qze+d\nzr+/s4G8/YdRen82k1+/8UUSpMIXSZAKXyRBKnyRBKnwRRKkwhdJkApfJEEVmY9/r59Dcn4Pjg/4\nyTQfbPwG8z6S/2x7ZjaNceJkfn6O+tYfaH6rHUPzz9/P+7dzf8o3CO+WFs1XzyvvfP+88+kPDfLg\nOoHeGY19Fx9nXxNch9I/uA5j8pOTaH6QfYfmV3hdyay77dB8fBFpo8IXSZAKXyRBKnyRBKnwRRKk\nwhdJkApfJEHhOL6Z3Q7gMwBa3Qs3CzezIwAsAHAMgBYAw9y9w4nxZubAUyXbP8YPpv2/YB+meXR/\nc1yd8Xx00HzfYP9gvvtRfi7Nt37s73jz64P+jwzyIMa4G4INAjd/necTg/23Z8EG1b5vwHlBHq0X\ncGfO/S8N8jkkuzzXOP4dAD6512MTATzo7h8FsBzAN8poR0RqRFj47r4SwGt7PTwUbT9q5gD47AE+\nLhHpRPv7Hr+Hu7cCgLtvRrz+kYjUkAO15l5wwf+t7T7vD2DAAepWRNo8W/yI7W/ht5pZT3dvNbNe\nALbwzS/bz25EpHzHFT92Kz3Bq9yX+lb82G0JgJHFzy8BsLj8gxORagsL38zuAvArAMeZ2R/MbBSA\nawGcbWa/BXBW8f8i8leiIvPx/Y7SeTaK75+RfQHAfs6Pf9F8Pp96HW8eM3a8RPNJh/DrCP5kN9M8\n68f7t178+fmmYF32jbz9Nb6QbxA4xYbRvOkDfP/3vMrviwBE1xnknI+fe72CvOsF7OzE9idqPr6I\ntFHhiyRIhS+SIBW+SIJU+CIJUuGLJEiFL5KgA3WtPsdvIc/9OMj/ncfr5ufoG8CoQ/iFBBdiAc2D\nyxCwZn2wwQwePzYk2D8wIpgv3gXv0Dy6P/ycV4MN6oP56C3B/rlF4+TROH+UR9cRRPJep9Ax/cYX\nSZAKXyRBKnyRBKnwRRKkwhdJkApfJEEqfJEEVWQc3zaVnlM+w8fyfc++jTc+MKPxtzZvp/mXev6Q\n5nPsDZrPwIM0v4YtRgCg/3/dSHMMyWg8cHZw//VL+P3XR9hg3n9gsn+M5qNuv4Y3MDoLeujscfJI\nZ6/bH+mc56ff+CIJUuGLJEiFL5IgFb5IglT4IglS4YskSIUvkqCKrKuPTaXHIn0+n29cf9UGmr9g\npW8TBACPOh+nH7j0KZrbJ4PzczGPvT9f937ClRnNpxnvP3M+ztt02zSa283B8wuu9PBL+fObMnY8\nzTOL5pvL/puidfVFpI0KXyRBKnyRBKnwRRKkwhdJkApfJEEqfJEEheP4ZnY7gM8AaHX3k4qPNQEY\nA2BLcbNJ7v5Aif3de5RuP9tSOgOA7DSeH/rzV2g+4fDuvIHAYT6O5uO33EzzrCdvf1DQ//nb+PP7\nSfD8HgvaP9f7BVtw9xu/MUA0Sj9xYHAdwaosaKFz1p1vE60HEMk7nz/P88s3jn8HgE928PgN7n5y\n8aPDoheR2hQWvruvBPBaBxG/ZEtEalae9/jjzGytmc0ys24H7IhEpNPt75p7twD4D3d3M5sK4AYA\n/1Jq46zdsneNBxc+RORAa0G5Nxvcr8J3963t/jsTwH1s+6zr/vQiIvumvvix24qSW5b7Ut/Q7j29\nmfVql30OQHTPVxGpIeFvfDO7C0AjgO5m9gcATQDONLMGALtQeG3B18gWkZpSkfn40/2ykvnH7Fa6\n/8s+lOYj7Hqa+/jjaP7KdBrjyC8E52cqz5f24S+qzn6eN2/HPkxzX83XxZ83gLf/pe/nm48/fzQf\n3LlwFd/fBvL7DsTvWaNx9mgcvbPH6aNx+J2d2P5EzccXkTYqfJEEqfBFEqTCF0mQCl8kQSp8kQSp\n8EUSVJl19XFT6Q36XMEbeG5B0ANfdx8ofQ0BAKAumDC/IwvaD/QO9t/UHDQQ5UcHeX2QPxLkkWhF\ngeeCvDVn/53tvCB/f5DfmXP/S4N8Dsku1zi+iLRR4YskSIUvkiAVvkiCVPgiCVLhiyRIhS+SoIqM\n4zf7qSXzh+xxuv8X/Fia9zM+Tuyf4D/bFpRenQgAMDyYb3/MNzfSfKQdT/NsJu/fxvyUbxCunJ93\nvnmkluejA+G6+sF1Fr6Lrzew5kXefP/+/PsnWz2B92/TaH6F15XMutsOjeOLSBsVvkiCVPgiCVLh\niyRIhS+SIBW+SIJU+CIJqsg4vv9t6TwLxkGz03h+8P2v03zSkfnu59nFr6L55C18Yf4smO4fzca+\nqiH4+qzNghaiHt4I8kjede0j0fH3CHJ+nUfdNr4exITDuwftc2v9LprfO30EzbOreftDSDYY0Di+\niLRR4YskSIUvkiAVvkiCVPgiCVLhiyRIhS+SoHAc38x6A/ghgJ4AdgGY6e7fM7MjACwAcAwKNzEf\n5u7vGlQ3M8fs0n14Xz7f2eqCceyGjMYz/CWaj35rFs271q3j/fc9gcb+I/78Ght+RvMVFtxgvtPH\n0Wtd3ufP10tY6PNofgKeoXk/+zfe/dSRNPbGoD4OJ/XRz3KN4+8E8HV3PxHA6QC+YmZ9AUwE8KC7\nfxTAcgDfKKMtEakBYeG7+2Z3X1v8fDsKt67pDWAo2m7jMQfAZzvrIEXkwNqn9/hmVg+gAcAqAD3d\nvRUo/HBAfO2kiNSILuVuaGZdASwCcKW7by/cE28Ppd9s3JO1fd63ETi+cR8OUUTK8ngzsLq5rE3L\nKnwz64JC0c9198XFh1vNrKe7t5pZLwBbSjbwT1lZByMiOZzaWPjY7dYpJTct96X+DwA84+7tb3u7\nBMDI4ueXAFi8904iUpvC3/hmNgjARQDWmdmTKLyknwTgOgALzeyfAbwAYFhnHqiIHDhh4bv7IwAO\nKhGz6cBtvl86euDRT/B9x5TVQ0m3YSzN3zrk4KCF4P7xG/l87cUN59B8xbJPBf1H4/i1Lt84e7cd\n/P7wyw/5B5qfYk8E/fN162dhNM0/jN8H7bfweD6PH/hmUB+jgu5L0JV7IglS4YskSIUvkiAVvkiC\nVPgiCVLhiyRIhS+SoLKv1c/DXyk9pzjj043hnwjmIz+3i+bDjP9s+xPvHuM9o/kE1NP8e7aD5v8N\n/vy+3CdYj+C5jOf4QJC/GuSRnO33zWj8nTp+fpYEvd/tF9D883N+SvPTLfgGDZzuy2j+q9m8/ag+\nniXfP8eR/fQbXyRBKnyRBKnwRRKkwhdJkApfJEEqfJEEqfBFElSRcXywW4QPCPYtvXpQwcV8oDNa\nHWRhkE/7bRPN3/zo39C8B66i+ZhguvWXV2ziG4TeDPJovnze9gMbW2l82SC+exYsl3AjvsY3+C6P\n+/AYfwzyR6/j6wX8egJf1x+jNtD4IzNI+K+lI/3GF0mQCl8kQSp8kQSp8EUSpMIXSZAKXyRBKnyR\nBJl7MN87bwdmTm+yM/p83sCs3wQ9/ITHIzOe82XTgTOC/YP56O99eSTN3+7/ft58S9T/eTzuegrP\nt0ftB7oG+29fEzRwH4+D+fp4gK/Lj/r/DPp/g8dZ0H800H9xsD9O5nFYHytJOBju3uGFLvqNL5Ig\nFb5IglT4IglS4YskSIUvkiAVvkiCwsI3s95mttzMnjazdWb21eLjTWa2ycx+XfyI7vcsIjUiHMc3\ns14Aern7WjPrCmANgKEALgTwprvfEOzvwFSyRbQkwJ95fGRGY+/N5+svWMubHz6Pn58zLvoFzc+y\nc2ieBfPB7Wrev18VrMs+nbd/zcuv0/ygLjtpPuHw7rz/8bx/mxY8v2/z5/ftb/L2b9jJ1zN49fCj\naf7yn3n/f3yH9//xycF1MlMznuPQIGdfn8klx/HDhTjcfTOAzcXPt5vZBgC7z1a+uw2ISFXs03t8\nM6sH0ADgseJD48xsrZnNMrNuB/jYRKSTlF34xZf5iwBc6e7bAdwC4Fh3b0DhFQF9yS8itaOsNffM\nrAsKRT/X3RcDgLtvbbfJTNCLrtvfP+xDAI7d1+MUkdDzAP63rC3LXWzzBwCecfebdj9gZr2K7/8B\n4HMA1pfe/awyuxGR/Xcs9vyl+lDJLcPCN7NBAC4CsM7MngTgACYBGGFmDQB2AWgBMHb/D1hEKqmc\nv+o/AuCgDqIHDvzhiEglVGg+PhvH7xG0EKxcviijcdMF+UYc3/Fv0HziW9fR/Pq6XTQfEvQ/eCP/\n+jzUlz+/5qD9QX5GsAX3iLH54EBjsP+Zz/Hz81Af/vfn5qD9gc5vXPDpu3kLeb9/Hna+3sByeyJo\noT7IWX2UHsfXJbsiCVLhiyRIhS+SIBW+SIJU+CIJUuGLJEiFL5KgCo3js3vMR/dnD9ZNB1+X/i7/\nJc2/+ARZ8x+ADXiKd9/vJBr7OD4OfOHY2TRfaPza60u8J81nz76c5jYq39ff7+DPb+TIW2g+x1pp\nHj6/HwXP7/Lg+W3LaLzW76J5n7d+T/OudctoHl+JkKc+pmgcX0TaqPBFEqTCF0lQFQq/pfJd7oPm\n6FZvVddS7QOgmjdW+wiIt5urfQSBlor1pMLfiwo/n5ou/J3N1T6CQEvFetJLfZEEqfBFElShcXwR\nqYZS4/idXvgiUnv0Ul8kQSp8kQSp8EUSpMIXSZAKXyRB/w86pyNpX9cmVgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11a87e210>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD7tJREFUeJzt3X+s3XV9x/HXWwpqrTQIa+ta7W2RqllGbnAjGWXxGkSR\ndEIa02EhoWoqYZRhkLgKW+4pY0u7SAMLq5Hyw1phgETWQtRRLJeJTu2AslaKxdBTAemlAwY00dna\n9/4437anl3M+n9P7Pb/q+/lITjj3+/7+eN8vfZ3vOefz/X6vubsAxPKWXjcAoPsIPhAQwQcCIvhA\nQAQfCIjgAwF1Lfhmdo6ZPW1m283sb7q13VaZWdXMnjSzJ8zsp33Qz61mNmpm/1037QQze9DMfm5m\n/25mk/usv2Eze97MHi8e5/SwvxlmttHMfmZmW8zsr4vpfbEPG/R3eTG9K/vQujGOb2ZvkbRd0lmS\nfiVpk6QL3P3pjm+8RWb2rKQPufurve5FkszsTEl7JH3D3U8tpq2Q9LK7/1Px4nmCuy/to/6GJb3h\n7it70VM9M5smaZq7bzazSZIek3SepM+oD/Zhor+/VBf2YbeO+KdLesbdd7r7Xkl3qfZL9hNTH330\ncfdHJY19ETpP0pri+RpJ53e1qTpN+pNq+7Hn3H2Xu28unu+RtE3SDPXJPmzS3/Si3PF92K1/6NMl\nPVf38/M69Ev2C5e0wcw2mdniXjfTxBR3H5Vq/3AkTelxP40sMbPNZnZLLz+K1DOzAUmDkn4saWq/\n7cO6/n5STOr4PuybI1wfmOvup0k6V9JlxVvZftdv51uvkjTb3Qcl7ZLUD2/5J0m6V9IVxZF17D7r\n6T5s0F9X9mG3gv+CpPfW/TyjmNY33P3F4r+7Jd2n2seTfjNqZlOlg58RX+pxP4dx991+6Euj1ZL+\ntJf9mNkE1UK11t3XFZP7Zh826q9b+7Bbwd8k6X1mNtPMjpN0gaT1Xdp2lplNLF55ZWbvkPQxSVt7\n25Wk2me9+s976yUtKp5fLGnd2AW67LD+iiAdMF+934e3SXrK3W+sm9ZP+/BN/XVrH3blW32pNpwn\n6UbVXmxudfflXdlwC8xslmpHeZc0QdIdve7PzO6UNCTpREmjkoYl/Zukb0l6j6Sdkha4+//2UX8f\nUe2z6n5JVUmXHPg83YP+5kr6D0lbVPv/6pKulvRTSfeox/sw0d9CdWEfdi34APoHX+4BARF8ICCC\nDwRE8IGASgW/3y+8AdDYuL/Vb/XCGzNj2ADoEXdveN7/hBLrPHjhjSSZ2YELb950xd3auuffVu2s\nhAP2lWhAkvaWXH7s9h+QNK+H28+tf4Oks49g+Zyy/R875ufvq3YkOOADmeXnvytdt7Xp+nXnfjFZ\n/7sVXzn0w4aKdHbl8BmWrk5vIHuC6dsz9bF7aKzX656PqHZqRLssa1op81b/aLjwBkADfLkHBFTm\nrX7LF958u+75xBIb7IY5vW4gY3avG8iY1esGUmYP9bqDjIGSy1eLR16Z4B+88EbSi6pdePPpRjPO\nbzSxT/V78E/udQMZff3CdPJQrzvIGGjD8vXreKTpnOMOvrv/zsyWSHpQhy682Tbe9QHono5fpGNm\nPtzRLaCMazLXpX1/8oeT9UFtTtanPf1aegNb0mUtSZcrJa+mr/xhun75CyuS9ZvsjMwWHsrUc6MC\nv87UU5Y1Hc7jyz0gIIIPBETwgYAIPhAQwQcCIvhAQAQfCKjMmXvoA7lrv3JX3x13ZuY8jq2VzBo+\nkamXGYeWsr/h265JL/3868l6ZWn6D9WcmLnNhGcuP7RFmf27tJKu6/hMPf37NcMRHwiI4AMBEXwg\nIIIPBETwgYAIPhAQwQcCYhw/ukllV5C5Ta7eyNRzZxpkzgP4TSW99pPSi5syd4uops8T+OOB45L1\nm5c2vBz+oM9/JTPOf1UlXc+O8zfGER8IiOADARF8ICCCDwRE8IGACD4QEMEHAmIcP7o9ZVfwSqbe\n4evxs/+Ec/etzxj4h2T5U19Ij8OvvSE9jv/bv0/Xj9P2ZF26I1NvjCM+EBDBBwIi+EBABB8IiOAD\nARF8ICCCDwRUahzfzKqSXpO0X9Jedz+9HU0Bh+Su1y95PX/W1HT5hkqy/I/+WLL+C/tQsn6zX5us\nf95OTtabKXsCz35JQ+7+asn1AOiism/1rQ3rANBlZUPrkjaY2SYzW9yOhgB0Xtm3+nPd/UUz+wPV\nXgC2ufujY2caqXs+UDwAtFu1eOSVCr67v1j8d7eZ3SfpdElvCv5QmY0AaNGADj+sPtJ0znG/1Tez\niWY2qXj+Dkkfk7R1vOsD0D1ljvhTJd1nZl6s5w53f7A9bQHopHEH3913SBpsYy9AH8rdbyB9vf82\nW5+sVy5Lr/279s3M9l9L1JY1rTAUBwRE8IGACD4QEMEHAiL4QEAEHwiI4AMBcV99ICl3vf87M/X0\n/QAuuemGZP3d//KFZH2m/6ppbWfilv0c8YGACD4QEMEHAiL4QEAEHwiI4AMBEXwgIMbxgVL2lVr6\n5kuvSNaHlR7Hv02fbVo7K7EcR3wgIIIPBETwgYAIPhAQwQcCIvhAQAQfCIhxfKCU3PX6GfeWW/w5\nvSdR/c+mFY74QEAEHwiI4AMBEXwgIIIPBETwgYAIPhBQdhzfzG6VNE/SqLufWkw7QdLdkmZKqkpa\n4O6pP9QN/J46NlNP31df52cWvyVdflhDieo9TSutHPFvl/TxMdOWSnrI3d8vaaOkL7ewHgB9Iht8\nd39U0qtjJp8naU3xfI3yr1sA+sh4P+NPcfdRSXL3XZKmtK8lAJ3WrnP1PVUcqXs+UDwAtNeuke3a\nNbK9pXnHG/xRM5vq7qNmNk3SS6mZh8a5EQCtmzY0R9OG5hz8+cll32k6b6tv9a14HLBe0qLi+cWS\n1h1RhwB6Kht8M7tT0o8kzTGzX5rZZyQtl3S2mf1ctbv4Lu9smwDaKftW390XNil9tM29AEehcl+T\nzVj9THqGzDj+musvTVT/qmmFM/eAgAg+EBDBBwIi+EBABB8IiOADARF8ICDuq48Oy12v/vZMPXff\n+sz17qXl+ivnuevnJOsvH5NeftlV68e1XY74QEAEHwiI4AMBEXwgIIIPBETwgYAIPhAQ4/josHdl\n6q9k6rlx/Nw4e67+Rqaek+5/pl+QrFdsWbJe9VXpzdvj6XoTHPGBgAg+EBDBBwIi+EBABB8IiOAD\nARF8ICDG8VFS+np4X7c0WX/yk6ck64P2g8z2v5qp5+TOE7g8Wf26L07Wd9gHk/XKDemtm3Umohzx\ngYAIPhAQwQcCIvhAQAQfCIjgAwERfCCg7CChmd0qaZ6kUXc/tZg2LGmxpJeK2a529+91rEsctZ75\n5Ixk/T5L/334mzUtWV/8pUwD52bqX0uXv/6v6evld1h6+eGX03U7cW16Bv0iU8/93YLGWjni3y7p\n4w2mr3T304oHoQeOItngu/ujkl5tUMq81gHoV2U+4y8xs81mdouZTW5bRwA6brwnAq+SdK27u5ld\nJ2mlpM81m3mk7vlA8QDQbs9K2tHSnOMKvrvvrvtxtaT7U/MPjWcjAI7Q7OJxwMNN52z1rb6p7jO9\nmdV/1Tpf0tYj6A5Aj7UynHenagftE83sl5KGJX3EzAYl7ZdUlXRJB3sE0GbZ4Lv7wgaTb+9ALzgq\npe9bP8eafvVT8yeVZHnepm8l66frmmR91u+qyfqPPnxGsn78nROTddl3k+X/e2t6cWlnboaO4Mw9\nICCCDwRE8IGACD4QEMEHAiL4QEAEHwiI++qjwzLXi/9XJVl+IHMN6ANqdJrJkUifh5D7uwHH/s9d\n6cUnpa9f8xf+Nlm36cPp9Wf/LkBjHPGBgAg+EBDBBwIi+EBABB8IiOADARF8ICDG8dFhuXHm4zP1\nsv9Ex3ff+UPS/e09aWWyfuW+9PYrEzL755uVdP2iTL0JjvhAQAQfCIjgAwERfCAggg8ERPCBgAg+\nEBDj+Oix13vdQEbuPIO0yZXfJuvDmT86ff+FZyXrf3HRnx9xTxJHfCAkgg8ERPCBgAg+EBDBBwIi\n+EBABB8IKDuOb2YzJH1D0lRJ+yWtdvd/NrMTJN0taaakqqQF7v5aB3sFeiB9X/2s60aS5dxd/ec9\nsjEzR+fG8fdJutLd/0jSn0m6zMw+IGmppIfc/f2SNkr68rg6ANB12eC7+y5331w83yNpm6QZks6T\ntKaYbY2k8zvVJID2OqLP+GY2IGlQ0o8lTXX3Uan24iBpSrubA9AZLZ+rb2aTJN0r6Qp332NmPmaW\nsT8fNFL3fKB4AGi3avHIayn4ZjZBtdCvdfd1xeRRM5vq7qNmNk3SS82WH2qpFQDlDOjww+ojTeds\n9a3+bZKecvcb66atl7SoeH6xpHVjFwLQn1oZzpsr6UJJW8zsCdXe0l8taYWke8zss5J2SlrQyUYB\ntE82+O7+Q0nHNCl/tL3tAL9v3pesZs8SeG/bGjkMZ+4BARF8ICCCDwRE8IGACD4QEMEHAiL4QEDc\nVx9Iemem/kq6fNOMdH1Jurxo1qrM9kcz9cY44gMBEXwgIIIPBETwgYAIPhAQwQcCIvhAQIzjI7jc\nne33lVq7f9CS9Upm+TXvvjQzR24NjXHEBwIi+EBABB8IiOADARF8ICCCDwRE8IGAGMeP7m1lV5Ab\nB885tuTyuX/CuTvX58bp08t/0dPbr6SH8TXRMxfkWyVd1/GZemMc8YGACD4QEMEHAiL4QEAEHwiI\n4AMBZYNvZjPMbKOZ/czMtpjZ5cX0YTN73sweLx7ndL5dAO3Qyjj+PklXuvtmM5sk6TEz21DUVrr7\nys61h477TdkV5MbJy9ZzcucB7M3Uh5PVX+9JHxuXZ8bpKyPputkp6Rly9+0f5/7LBt/dd0naVTzf\nY2bbJE0vyplfG0A/OqLP+GY2IGlQ0k+KSUvMbLOZ3WJmk9vcG4AOaTn4xdv8eyVd4e57JK2SNNvd\nB1V7R8BbfuAo0dK5+mY2QbXQr3X3dZLk7rvrZlkt6f5my4/UPR8oHgDa7VlJO1qas9WLdG6T9JS7\n33hggplNKz7/S9J8SVubLTzU4kYAlDG7eBzwcNM5s8E3s7mSLpS0xcyekOSSrpa00MwGJe2XVJV0\nyfgbBtBNrXyr/0NJxzQofa/97QDoBq7HD+6ULU8m6898opJewUmZDeQudz8zUz/Hk+UvnXxtsr7i\n+mXJ+levSteXT0qWNd0vStbNzkivIPv37cuep9AYp+wCARF8ICCCDwRE8IGACD4QEMEHAiL4QECM\n4x/lxjeKe8hCG2xLH+N2V7nFT8vNkPn1JvqCZH3ZrLvTK8je9/71TL0z4/Q5HPGBgAg+EBDBBwLq\nevCr3d7gEar2uoGMaq8byKj2uoGEkTd63UHOs13bEsEfo9rrBjKqvW4go9rrBhJG9vS6g5zWbqLR\nDrzVBwIi+EBA5p6+3rn0Bsw6uwEATbl7w1vgdzz4APoPb/WBgAg+EBDBBwIi+EBABB8I6P8B7Z68\nlR2j0WMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b3db990>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "j\n",
      "j\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAD5NJREFUeJzt3X+MHPV5x/HPExtUEwcKUW23uOVCSmkUKbVIiqicKoeS\nAIma2kItUKCCtEFEDQlS2hSHRvKlSiRIK0vkD9rKNsQQaKC0jp1KYBOhM4XKQANuTDggaXsQE3y4\nARLckNbGT//Ycbx33H6fvdsfM+Z5v6SV9/aZ2XluvJ+b2f3OzJq7C0Aub6q7AQDDR/CBhAg+kBDB\nBxIi+EBCBB9IaGjBN7PzzOxJM3vazK4Z1nK7ZWaTZvbvZvaYmT3cgH42mtmUmX277bETzWy7mT1l\nZtvM7ISG9bfWzPaY2aPV7bwa+1tuZveZ2XfMbLeZfap6vBHrcJb+Plk9PpR1aMMYxzezN0l6WtL7\nJf1A0iOSLnL3Jwe+8C6Z2X9Kere7v1R3L5JkZu+VtF/SLe7+ruqx6yX90N2/VP3xPNHd1zSov7WS\nXnH3dXX01M7Mlkla5u67zGyxpG9JWiXpo2rAOiz0d6GGsA6HtcU/U9J33f0Zdz8g6Wtq/ZJNYmrQ\nWx93f0DSzD9CqyRtqu5vkrR6qE216dCf1FqPtXP3ve6+q7q/X9KEpOVqyDrs0N/JVXng63BYL/ST\nJX2/7ec9OvJLNoVLutfMHjGzK+pupoMl7j4ltV44kpbU3M9srjKzXWa2oc63Iu3MbETSCkk7JS1t\n2jps6++h6qGBr8PGbOEaYKW7nyHpw5I+Ue3KNl3Tjre+UdKp7r5C0l5JTdjlXyzpLklXV1vWmeus\n1nU4S39DWYfDCv5zkn6l7efl1WON4e7PV//uk7RZrbcnTTNlZkuln71HfKHmfqZx931+5EOj9ZJ+\ns85+zGyhWqG61d23VA83Zh3O1t+w1uGwgv+IpF81s1PM7FhJF0naOqRlh8zsuOovr8zszZLOkfR4\nvV1Jar3Xa3+/t1XS5dX9yyRtmTnDkE3rrwrSYeer/nV4k6Qn3P2GtseatA5f19+w1uFQPtWXWsN5\nkm5Q64/NRne/bigL7oKZvU2trbxLWijptrr7M7PbJY1KequkKUlrJX1d0j9I+mVJz0i6wN1fblB/\nZ6v1XvWQpElJVx5+P11Dfysl3S9pt1r/ry7pWkkPS7pTNa/DQn8XawjrcGjBB9AcfLgHJETwgYQI\nPpAQwQcS6in4TT/xBsDs5v2pfrcn3pgZwwZATdx91uP+F/bwnD878UaSzOzwiTeznHG3tu3+uFrD\nv001ruH2d0xQPzDj53FN7+/84ty+5DeK9bHguLWTymW9OuPneyV9sFCv07iG/8qL1t/BtvvbJJ07\no/7jYP41+zvXFi3uXOtlV/9oOPEGwCz4cA9IqJdd/TmceDPedv/neljkMIzU3UBgpO4Gik6tu4GC\nkbobCLy9x/nvv1+6/1+6m7aXD/cWSHpKrQ/3nlfrGOg/cPeJGdP59Pf4mG6u7/FnatZ7/LnW3+jm\n8h5/Nr2+x+/7h3vu/pqZXSVpu46ceDMRzAagAXrZ1Ze73yPp9D71klS0RY/8U7FqW8t7dL6hfJWn\nsQ3lpUf7K8cH9WiLVrfo94sC9GJQj/YIPvPT8sdwi856rVDt/H/Lh3tAQgQfSIjgAwkRfCAhgg8k\nRPCBhAg+kNDAL7bJkXtNF4y07/p0seyXBscBBBeH7vW4xbpF/f/Fs+W6/W6Qv11jPXTwuY5H7rHF\nBxIi+EBCBB9IiOADCRF8ICGCDyRE8IGEGMd/w4vOiI9GyqNr6IwWq/7c2cX6WHB51kXB0qPuej1O\nILp67Mf3Fi6BI+nQsr8KnqHXM/5L1jCOD+AIgg8kRPCBhAg+kBDBBxIi+EBCBB9IqKfr6uNo0OuV\n66Mrv48Xq/bO8nEitxau/S5J3wuWHo2CvyWoR9e9/20/pVg/ZNE4/dKgHh2JMJhvHmCLDyRE8IGE\nCD6QEMEHEiL4QEIEH0iI4AMJ9TSOb2aTkn4k6ZCkA+5+Zj+aQpNEI92Bl+8ulk8LZo/G8SPROH/k\nLv1ej88QGcw4faTXA3gOSRp195f60QyA4eh1V9/68BwAhqzX0Lqke83sETO7oh8NARi8Xnf1V7r7\n82b2C2r9AZhw9wdeP9l42/2R6gagvyarW6yn4Lv789W/+8xss6QzJc0S/NFeFgOgKyOavlHd0XHK\nee/qm9lxZra4uv9mSedICr4bFUAT9LLFXyppc+vy2Voo6TZ3396ftgAM0ryD7+7/JWlFH3tBI/V6\nZfqf9KuRWizQa3W3MBAMxQEJEXwgIYIPJETwgYQIPpAQwQcSIvhAQlxXHyh4TQvqbmEg2OIDCRF8\nICGCDyRE8IGECD6QEMEHEiL4QEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQwQcSIvhAQgQfSIjgAwkR\nfCAhgg8kRPCBhAg+kBDBBxIi+EBCYfDNbKOZTZnZt9seO9HMtpvZU2a2zcxOGGybAPqpmy3+zZLO\nnfHYGknfdPfTJd0n6bP9bgzA4ITBd/cHJL004+FVkjZV9zdJWt3nvgAM0Hzf4y9x9ylJcve9kpb0\nryUAg9av787zcnm87f5IdQPQX5PVLTbf4E+Z2VJ3nzKzZZJeKE8+Os/FAOjeiKZvVHd0nLLbXX2r\nbodtlXR5df8ySVu6bQ1A/boZzrtd0r9K+jUze9bMPirpOkkfNLOnJL2/+hnAUSLc1Xf3izuUPtDn\nXgAMCUfuAQkRfCAhgg8kRPCBhAg+kBDBBxIi+EBCBB9IiOADCRF8ICGCDyRE8IGECD6QEMEHEiL4\nQEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQwQcSIvhAQv367rwBOiaoLwrq0a94cMDz/zioA8PHFh9I\niOADCRF8ICGCDyRE8IGECD6QEMEHEgrH8c1so6TfkTTl7u+qHlsr6QpJL1STXevu98yvhWgcnnFy\noN+62eLfLOncWR5f5+5nVLd5hh5AHcLgu/sDkl6apWT9bwfAMPTyHv8qM9tlZhvM7IS+dQRg4OZ7\nrP6Nkv7S3d3MviBpnaQ/7jz5eNv9keoGoL8mq1tsXsF3931tP66X9I3yHKPzWQyAORnR9I3qjo5T\ndrurb2p7T29my9pq50t6vOveANSum+G829XaZL/VzJ6VtFbS2Wa2QtIhtfYtrhxgjwD6LAy+u188\ny8M3z20xpbH6V4N5R4vVkw6eVqzvXHBWsX7aD/YU61t+6ZxiffVt24p1XbquXOc4BNSAI/eAhAg+\nkBDBBxIi+EBCBB9IiOADCRF8IKEhXVe/MFb/8bHinD5ePglwLPgN7iyXdSCoL9L2Yn1tcJLi5/cE\nxyksvy7o4PigznEAmDu2+EBCBB9IiOADCRF8ICGCDyRE8IGECD6Q0HDG8c8a61j6/t8G4/TBU580\n52amezGoX/NsuW63eHmC5Vvn1M/rHe3j9McVq9FxFHX7Xx1bdwsDwRYfSIjgAwkRfCAhgg8kRPCB\nhAg+kBDBBxIayjj+/93Teaz+iz9fnndp8NzRVfmjUfCL/JRi3ezy4Bm+GNSbPlI9aEuK1YMDXnqv\na/9lBS9Q/aTHJdSDLT6QEMEHEiL4QEIEH0iI4AMJEXwgIYIPJBSO45vZckm3qDWkfkjSenf/spmd\nKOkOSadImpR0gbv/aLbneDQaCi2IzpePxmmj8/Xf8YeTwRRjQb08Ti09F9Tf6N5RrEbHYdTtFb0l\nmCIax2/mcRzdbPEPSvq0u79T0m9J+oSZ/bqkNZK+6e6nS7pP0mcH1yaAfgqD7+573X1XdX+/pAlJ\nyyWtkrSpmmyTpNWDahJAf83pPb6ZjUhaIWmnpKXuPiW1/jgo3ucF0BBdH6tvZosl3SXpanffb2Yz\nLzbX8eJz69vunyHp3XNqEUB3JqtbrKvgm9lCtUJ/q7tvqR6eMrOl7j5lZsskvdBp/iu6agVAb0aq\n22E7Ok7Z7a7+TZKecPcb2h7bKuny6v5lkrbMnAlAM3UznLdS0iWSdpvZY2rt0l8r6XpJd5rZH0l6\nRtIFg2wUQP+EwXf3ByUt6FD+QDcLmZxDQ/0Wnc+vf+t1CdGRBke76CUSjFOvLl9X/7tfn1s3c9Xr\n+f6TelswxVSPS6gHR+4BCRF8ICGCDyRE8IGECD6QEMEHEiL4QEJDua7+hVd2ro39XXneqMFFQX0i\nqK+a+PtifUvnrwSoRGf8vxLU6z4jPVqD0fno5esNXLp5fbH+YrB+jw+W3uv3KkTPv+Mz5wVT7Azq\ndf//zo4tPpAQwQcSIvhAQgQfSIjgAwkRfCAhgg8kZO4dL5XXnwWY+fv87o71UftQcf5eR8mjq5qv\nDOrnfDVYP5eOBc9wTFCv+7rrPfa3YaxY3vmx8kB951dGS9Rd9PqIzpYf++ty3f5sbfAMTfZ5ufus\n/wFs8YGECD6QEMEHEiL4QEIEH0iI4AMJEXwgoaGM40tf6Fj3ic8V5x8rf716eN38Xs/XPjmof8RP\nKNZ/cePL5Sf4WNTBeFCPRrrfVy5/tXzd+x9eUj5f/w77abEejaNH58NH12OIvtXgI0H9PaPB6398\nLHiG6HoGdZ6Pzzg+gDYEH0iI4AMJEXwgIYIPJETwgYTC4JvZcjO7z8y+Y2a7zeyT1eNrzWyPmT1a\n3aLrEANoiHAc38yWSVrm7rvMbLGkb0laJelCSa+4+7pgfpdK5zSXz4h/df85xfp1i4vlcJQ7GieO\n5o9G4SNnBPUPlQ8T0IHgC+Dv/p9yfXew/Eg0Dh9dbSD6/vpo/rHyy0N2bDBO/89jwRKafj2Fks7j\n+OEXarj7Xkl7q/v7zWxCR45rCb9uAkDzzOk9vpmNSFoh6aHqoavMbJeZbTCzYNsEoCm6Dn61m3+X\npKvdfb+kGyWd6u4r1NojKO7yA2iOrr47z8wWqhX6W919iyS5+762SdZL+kbnZxhvuz9S3QD012R1\ni3X7pZk3SXrC3W84/ICZLave/0vS+ZIe7zz7aJeLATB/I5q+Ud3Rccow+Ga2UtIlknab2WOSXNK1\nki42sxWSDqn1Z6bwnbgAmqSbT/UflLRgltI9/W8HwDDUfj5+LPjbNH5Nsew7yiOOXw4umx6d743e\nROPwH972j8X63RZtux4N6k0+n75XnI8PoA3BBxIi+EBCBB9IiOADCRF8ICGCDyQ0pHH80mB5dL5z\nNM4anREffIP66k8Vy2Oby8cJrH3oS+Xn/9Ny+YkHy/WHy+XwbPDgawn03vcEE/x5ufw3v39Zsf4n\n13+l/ARr7ggamAjq0etj0FdUaDLG8QG0IfhAQgQfSKiG4E8Of5FzsW+87g6Knqy7gcDT43vjiWrz\nvbobCEwObUkEf6b/Hq+7gyKC34v/qLuBwOTQlsSuPpAQwQcSGtI4PoA6dBrHH3jwATQPu/pAQgQf\nSIjgAwkRfCAhgg8k9P95MvG28T7YIQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b55b290>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEElJREFUeJzt3X+wXPVZx/HPU4IjkMAEJMmUtFyQihbKRFrQTnC8HQpF\nbQ2DmqYBJqBFrAQZ284AgZlsLGUo1rSxFn+EwKRJaUEcTOqMCMjc1LRCaUo0QPhRywVpySWFSInj\nOMQ8/rEnySbsfr979+zZs8nzfs3sZO95zo8n597PPbv3e85Zc3cBiOVtdTcAYPAIPhAQwQcCIvhA\nQAQfCIjgAwENLPhmdoGZPW1mz5rZtYPabrfMbNzM/s3MHjez7wxBP6vMbMLM/r1l2nQze8DMnjGz\nfzKzY4asv6Vm9pKZfa94XFBjf7PN7GEze9LMtpjZHxXTh2Iftunv6mL6QPahDWIc38zeJulZSedK\n+pGkxyQtcPenK994l8zsB5Le6+476u5FkszsHEk7JX3F3c8opn1O0qvufmvxy3O6u183RP0tlfSG\nuy+vo6dWZjZL0ix332xmUyVtkjRP0uUagn2Y6O+jGsA+HNQR/2xJz7n7C+7+pqSvq/mfHCamIXrr\n4+4bJR34S2iepNXF89WSLhxoUy069Cc192Pt3H2bu28unu+UtFXSbA3JPuzQ3wlFufJ9OKgf9BMk\n/WfL1y9p339yWLikB83sMTO7ou5mOpjh7hNS8wdH0oya+2lnsZltNrPb63wr0srMRiTNkfSIpJnD\ntg9b+nu0mFT5PhyaI9wQmOvuZ0r6dUlXFS9lh92wnW99m6ST3X2OpG2ShuEl/1RJ90q6pjiyHrjP\nat2HbfobyD4cVPB/KOmdLV/PLqYNDXd/ufh3u6T71Hx7MmwmzGymtPc94is197Mfd9/u+/5otFLS\nWXX2Y2ZT1AzVGndfV0wemn3Yrr9B7cNBBf8xSaeY2Ylm9lOSFkhaP6BtZ5nZkcVvXpnZUZLOl/RE\nvV1Jar7Xa32/t17SZcXzRZLWHbjAgO3XXxGkPS5S/fvwDklPufuKlmnDtA/f0t+g9uFA/qovNYfz\nJK1Q85fNKne/ZSAb7oKZnaTmUd4lTZH01br7M7O7JI1KOk7ShKSlkv5e0t9KeoekFyTNd/f/GqL+\nPqDme9XdksYlXbnn/XQN/c2V9E1JW9T8vrqkJZK+I+ke1bwPE/0t1AD24cCCD2B48Mc9ICCCDwRE\n8IGACD4QUKngD/uFNwDa6/mv+t1eeGNmDBsANXH3tuf9Tymxzr0X3kiSme258KbNFXdLW56PqTn8\nO6zGRH8tPt9Ilk/+1JP7ff1a48s6tnHV3q+n6Y3k8nP0eLJ+qp7N1J9J1n9W39/7/C8br+oTjeP2\nq7/79eeSyx++KVlujsKn5E6/aVm+8ZLUmL1/+blHlfSuKzvX7K8718q81D8YLrwB0AZ/3AMCKvNS\nfxIX3oy1PP/pEpschJG6G8gYqbuBpCNGa70uJ+l9o0fU3ULS6NHllh/7UfPRjTLB33vhjaSX1bzw\n5mPtZx0tsZlBG6m7gYyRuhtIOmJ0GC9qbDpr9Mi6W0gqG/zRtzcfeyxL/H2i5+C7+/+Z2WJJD2jf\nhTdbe10fgMGp/CKd5nDe0vyM6FHuMPGTdPm6RrK84pb0XaBey2z98Ez9zUz9UFfyu6eZvqhj7Q9t\ndcfhPP64BwRE8IGACD4QEMEHAiL4QEAEHwiI4AMBMY4f3sXp8invKrX2ec99LVn/bVuYrOc+fCF3\nEu6uTL2s3HkIufMYcuP0I5n65Y1EfhvGOD6AfQg+EBDBBwIi+EBABB8IiOADARF8ICDG8ZFR9or6\n+cmqn35ast7I3KU2N47/P5l61cr217gkXbe1NyWqNzKOD2Afgg8ERPCBgAg+EBDBBwIi+EBABB8I\nqMwn6eCQkBunn5apZ+6sP+fdyfLGzZnVZ1R9vX1Zub2bG8dfvuYT6RnW9vbJBBzxgYAIPhAQwQcC\nIvhAQAQfCIjgAwERfCCgUuP4ZjYu6XVJuyW96e5n96MpDFLFn1DfSJcfujBdL/v58XUru3c/delt\nmTkaPa237Ak8uyWNuvuOkusBMEBlX+pbH9YBYMDKhtYlPWhmj5nZFf1oCED1yr7Un+vuL5vZ8Wr+\nAtjq7hvfOttYy/MR5T8RDMDkjRePvFLBd/eXi3+3m9l9ks6W1Cb4o2U2A6ArI9r/oLqh45w9v9Q3\nsyPNbGrx/ChJ50vK3BMVwDAoc8SfKem+5u2zNUXSV939gf60BaBKPQff3Z+XNKePvWAolbszfWPe\ntcl6tZ/qUL3S983/WLq+bO0/TqadrjEUBwRE8IGACD4QEMEHAiL4QEAEHwiI4AMBcV99ZJQbx196\nw63JeiOzfMV3Cyit7Dj+xrvOTM/wtUczazg2U2+PIz4QEMEHAiL4QEAEHwiI4AMBEXwgIIIPBMQ4\nfni5T3DPjaQ3ktWNNy+bTDNvsavU0tV7reTyv3L3pswcjUy9tz3EER8IiOADARF8ICCCDwRE8IGA\nCD4QEMEHAmIcP7xpmXp6pPp4fzFZf8jSay97PXvVyp7lcPVh6fqyBWOT6Kad3vYQR3wgIIIPBETw\ngYAIPhAQwQcCIvhAQAQfCCg7jm9mqyR9WNKEu59RTJsu6W5JJ0oalzTf3V+vsE9UptwV7/frgmR9\nfWb5YR/Hz/WXG8ffuev49Aw2lllD2TMJ2uvmiH+npA8dMO06SQ+5+6mSHpZ0fU9bB1CLbPDdfaOk\nHQdMnidpdfF8taQL+9wXgAr1+h5/hrtPSJK7b5M0o38tAahav87V93R5rOX5SPEA0F/jxSOv1+BP\nmNlMd58ws1mSXknPPtrjZgB0b0T7H1Q3dJyz25f6Vjz2WC/psuL5Iknrum0NQP2ywTezuyR9W9LP\nmdmLZna5pFsknWdmz0g6t/gawEEi+1Lf3Rd2KH2wz72gErlx4J9k6ulv85m/mL5vfm4cv+5x+qqd\n/t9PZub4cqZe7n4JnXDmHhAQwQcCIvhAQAQfCIjgAwERfCAggg8ExH31D3m5b3Hmeu5LzkmWG2vT\ni+fOIqh7HD93vX3uLIdFmfqyqZnr8bOq2UMc8YGACD4QEMEHAiL4QEAEHwiI4AMBEXwgIMbxD3nl\nRtJvXvPHyfr/ZsbxS55FMPRO+ovMDIsbmRnq+WQBjvhAQAQfCIjgAwERfCAggg8ERPCBgAg+EBDj\n+Ie8cuPA1//ZF5P1Rmb5Yb8ev2x/H7nqnvQMi3P31a9nD3HEBwIi+EBABB8IiOADARF8ICCCDwRE\n8IGAsuP4ZrZK0oclTbj7GcW0pZKukPRKMdsSd7+/si6RUPJ67lMayfLGTy+bTDOT3Xrlqh4l/4fz\nficzR24cvx7dHPHvlPShNtOXu/uZxYPQAweRbPDdfaOkHW1K1v92AAxCmff4i81ss5ndbmbH9K0j\nAJXr9Vz92yT9ibu7md0kabmk3+s8+1jL85HiAaC/xotHXk/Bd/ftLV+ulPSN9BKjvWwGwKSMaP+D\n6oaOc3b7Ut/U8p7ezGa11C6S9ETXvQGoXTfDeXepecg+zsxelLRU0gfMbI6k3Wq+triywh4B9Fk2\n+O6+sM3kOyvoBT0pN1J9+CPpT4B/6GfSaz82s/XXMvWq5c5ySP/vpcY16fqyFZkPFsiq50wHztwD\nAiL4QEAEHwiI4AMBEXwgIIIPBETwgYC4r/5Br9wnzO+YOj1Z/9PM8rtKbX34PfjFc9IzrPh+Zg3D\neaYDR3wgIIIPBETwgYAIPhAQwQcCIvhAQAQfCIhx/KFX9s7w70lWj3rn7kl1c6ByZxFUL3e9fc75\nn/2XzByNTL3uTxZojyM+EBDBBwIi+EBABB8IiOADARF8ICCCDwTEOP7Qy90ZPjOSvva3kuXGJZPr\n5kB1X4+fO8shd55B45fS9WU3/t1k2mmj7j3UHkd8ICCCDwRE8IGACD4QEMEHAiL4QEAEHwgoO45v\nZrMlfUXSTEm7Ja109z83s+mS7pZ0oqRxSfPd/fUKew2q3KkWX7r448n6jzPj+MN5V/h9pmXquf5W\nPpLZAbYls4ajM/WydwSoRjdH/F2SPunup0l6v6SrzOznJV0n6SF3P1XSw5Kur65NAP2UDb67b3P3\nzcXznZK2SpotaZ6k1cVsqyVdWFWTAPprUu/xzWxE0hxJj0ia6e4TUvOXg6QZ/W4OQDW6fgNpZlMl\n3SvpGnffaWZ+wCwHft1irOX5SPEA0F/jxSOvq+Cb2RQ1Q7/G3dcVkyfMbKa7T5jZLEmvdF7DaFfN\nAChjRPsfVDd0nLPbl/p3SHrK3Ve0TFsv6bLi+SJJ6w5cCMBw6mY4b66kiyVtMbPH1XxJv0TS5yTd\nY2a/K+kFSfOrbBRA/2SD7+7fknRYh/IH+9tOL8pekV23XP+5kej0BeWLf2NZst7IrL3uu8KX3Tu/\nkKkvuHRNZo5Gpj7sP1/tceYeEBDBBwIi+EBABB8IiOADARF8ICCCDwR0ENxXP3df+eG8b3n3ct+C\nzDjxX/1astz4g/TiuXHyusfxc/cDmMjUf9mPT89gjcwacj9/de+h3nDEBwIi+EBABB8IiOADARF8\nICCCDwRE8IGABjSOnxotzl3PPDNTX5Spp69Hr/56/tz6y40T+zssWW+U3HrV//uy4/SNX03XzcYy\na7gnUz/YzxNpjyM+EBDBBwIi+EBABB8IiOADARF8ICCCDwQ0oHH83keDn/DPJOunPX95sm4nL81s\nITeSXHYkOzcOnL4z/Im+IFlvWPo8hbLX2+fG+cveLaH0OP1/JD6ysbmGTP3QvN4+hyM+EBDBBwIi\n+EBABB8IiOADARF8IKBs8M1stpk9bGZPmtkWM7u6mL7UzF4ys+8VjwuqbxdAP3Qzjr9L0ifdfbOZ\nTZW0ycweLGrL3X15fhXv6Vh5yX8/ueRKS49zv5jb9Ocb6fqnM/XSI9WZ8wCeTm//Jktfb/9CZuvT\nMvU3MvXcKHaunjtL4jc9/Qn2ZmU/vz7mOH1ONvjuvk3StuL5TjPbKumEopz+qQQwlCb1Ht/MRiTN\nkfRoMWmxmW02s9vN7Jg+9wagIl0Hv3iZf6+ka9x9p6TbJJ3s7nPUfEXQxUt+AMOgq3P1zWyKmqFf\n4+7rJMndt7fMslLSNzqv4e6W56dJOn2yfQLIGi8eed1epHOHpKfcfcWeCWY2q3j/L0kXSXqi8+If\n7XIzAHo3Ujz22NBxzmzwzWyupIslbTGzxyW5pCWSFprZHEm71fw1c2Wv7QIYrG7+qv8tSYe1Kd3f\n/3YADIK5565nLrkBM/9nf3/H+jftX5PLlx2FHcnUz/P0SPMCfT1Zf5++m6x/4eYlyXrjhmS59rv+\n3/D2dP3hH3b+3krSuWd9O72C7zYyHeR+AnJijtM3LZO7tx1y55RdICCCDwRE8IGACD4QEMEHAiL4\nQEAEHwhoIOP4So113/je5PJ/85lLk/UrHlibbuCadPmzT6frZcfJcxZl6id9JDPD4nT51vPTM1y7\n6kvpFXx8LNNArp5zdKaeG4ev+jt0MGMcH0ALgg8ERPCBgGoIfvrc9rr9oO4GMsZ+XHcHOeN1N5Aw\nXncDGeMD21INwd80+E1OwvN1N5Ax9mrdHeSM191AwnjdDWSMD2xLvNQHAiL4QEADGscHUIdO4/iV\nBx/A8OGlPhAQwQcCIvhAQAQfCIjgAwH9P0KQCLWhLY1HAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b536d50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "g\n",
      "d\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEstJREFUeJzt3XuMXPV1B/DvCSYQYl52sI3YlgVhcMFFW5KAKxtpKK5D\nKkemqHIoUNm0dZwIE2gitQ6l3dmISoBaF9JAaGweDgniYeLajlrqUDS0tOXNBoNtYihrCrEXgh3D\nFtICPv1jrtlhPXPO7Ny5D+t8P9LKs3Pu4+e7+915nPu7I6oKIorlY0UPgIjyx+ATBcTgEwXE4BMF\nxOATBcTgEwWUW/BF5DwR2SoiPxWRP8trv+0SkSER+YmIPCMij5dgPLeKyLCIPNtw39EislFEXhCR\nfxaRI0s2vn4ReVVEnk6+zitwfD0i8pCIPC8im0Tkq8n9pTiGTcZ3eXJ/LsdQ8ujji8jHAPwUwLkA\nfgbgCQAXqurWzHfeJhH5LwCfVtXdRY8FAERkDoARAN9T1dOT+64D8KaqXp/88TxaVZeXaHz9AN5W\n1RVFjKmRiEwDME1VB0VkIoCnACwAcClKcAyN8X0RORzDvB7xzwSwTVW3q+p7AO5G/T9ZJoISvfRR\n1UcAjP0jtADA6uT2agDn5zqoBi3GB9SPY+FUdaeqDia3RwBsAdCDkhzDFuM7Lilnfgzz+kU/DsB/\nN3z/Kkb/k2WhAH4sIk+IyJKiB9PCFFUdBuq/OACmFDyeZpaJyKCIrCrypUgjEekF0AfgUQBTy3YM\nG8b3WHJX5sewNI9wJTBbVc8A8DsALkueypZd2c63vhnAiaraB2AngDI85Z8IYA2AK5JH1rHHrNBj\n2GR8uRzDvIL/GoBfbfi+J7mvNFR1R/LvGwDWov7ypGyGRWQq8OFrxNcLHs9HqOobOvqm0UoAny1y\nPCIyAfVQ3amq65K7S3MMm40vr2OYV/CfAHCSiBwvIh8HcCGA9Tnt2yUihyV/eSEinwQwD8BzxY4K\nQP21XuPrvfUAFie3FwFYN3aFnH1kfEmQ9rkAxR/D2wBsVtUbG+4r0zHcb3x5HcNc3tUH6u08ADei\n/sfmVlW9Npcdt0FETkD9UV4BTADwg6LHJyJ3AagAmAxgGEA/gH8AcB+AXwGwHcBCVf1FicZ3Duqv\nVfcCGAKwdN/r6QLGNxvAvwLYhPrPVQFcBeBxAPei4GNojO8i5HAMcws+EZUH39wjCojBJwqIwScK\niMEnCihV8Ms+8YaImuv4Xf12J96ICNsGRAVR1abn/U9Isc0PJ94AgIjsm3jTZMZdf8PtGurt37wc\n7NTfG/N9DeMb38V2ecZ0uz7T2fz7Y77fUgV+rTr6/U5nfe8nPHb7413/qDHfv1AFTqmOft/jrD/k\n1B94x1ngeqfeqIb9f7bj/f3IUg3dzcZAy0qap/oHwsQbImqCb+4RBZTmqf44Jt7UGm4fmmKXeegt\negC2T1WKHoFtcqXoERh6ix6Aozfl+kPwXzvVpQn+hxNvAOxAfeLN7zdftJJiN3nrLXoAtmMqRY/A\nVuo/TL1FD8DR24X1G7fxcMslOw6+qn4gIssAbMToxJstnW6PiPKT+SSdejuv31+wY2nflb3ALldO\nt+uvOpt/cZuzwAan/pZTL7tPOHXn+M9yuiK9zubvfsxZ4J+cepne9R+vgZbtPL65RxQQg08UEINP\nFBCDTxQQg08UEINPFBCDTxTQAdDHT9lH7ak6dWfzj97vLLDJqWfN65NnzZvel3Wf+yy7fMnn7fr3\nvd//1jPc6rzj/65TzxL7+ETUgMEnCojBJwqIwScKiMEnCojBJwqIwScKqAR9/JR9+llVu+59yPCI\ns77LG7+nzPO581Dw8ZtRtetb017l9winnuX1FtjHJ6IGDD5RQAw+UUAMPlFADD5RQAw+UUAMPlFA\nOfXxrzGWcPqwJ1Xt+i+cAfzcWb/U86nJ550H4H1mjPPzvbZq14eczd/irJ9pn599fCJqwOATBcTg\nEwXE4BMFxOATBcTgEwXE4BMF5DU5TSIyBGAPgL0A3lPVM5svafXqF9o7+YwziLurzgIHep/e61Mf\n7tS9/59X9/rMniznmwP+fHzv5+/Ul1fN8nN6p1mfOecle/uX2NvP6vc3VfBRD3xFVXen3A4R5Sjt\nU33pwjaIKGdpQ6sAfiwiT4jIkm4MiIiyl/ap/mxV3SEix6D+B2CLqj6y/2K1htu9yRcRddcQ/MkD\ndamCr6o7kn/fEJG1AM4E0CT4lTS7IaK29OKjD6oPt1yy46f6InKYiExMbn8SwDz417QlohJI84g/\nFcDa+rRbTADwA1Xd2J1hEVGWOg6+qr4MoC/1CC451a5/f33KHXif3160lJ8rgF3dGkgLafvwaf9/\nXh/b237K8Tvz6U9bOGDWN97XdDr8h+bBux6GvX//+DTHVhxRQAw+UUAMPlFADD5RQAw+UUAMPlFA\nDD5RQGnP1W/Txa1Lr3rrPu3U0/aJs5Z2fFPN6hn662b9qSfPtjf/j3Z5118eatYnr3bmgy+u2vXU\n882d+oX2/vXrdp+9+lm7T29v3aeX2fuXm9Y4W3ixo/3yEZ8oIAafKCAGnyggBp8oIAafKCAGnygg\nBp8ooHz6+HOmt67V3EZ+yaX9fPYWH0WQ0DvnmfWq3QZO3WdG/y/tMuwBzNUzzPrZ8gV7/4dWzfLg\nuyeb9WfE6cPfbe/ePosCGHbqLvs0CfiXrdvU0W75iE8UEINPFBCDTxQQg08UEINPFBCDTxQQg08U\nUD59/JOM2iM/TLnxoufbe/PJ7eu6369/a9a9Pn2vs/chpz7JqX/1Fru+bWmPWZ8+aF9P4X61O+XP\nin0A1jrHJy2vT189y64f9fAOsz5wqHOA8XdO3TuPpDk+4hMFxOATBcTgEwXE4BMFxOATBcTgEwXE\n4BMF5PbxReRWAPMBDKvq6cl9RwO4B8DxqLeKF6rqnpYbGbH2kPXnu6eV9vPXF5nVSc588SOcrQ85\n9aozXfvYmS+Z9SvOOdHewNXOALz55j+yyzfoUrP+Dg4z6wfhfbN+CP7PrL+Nw826/MVfm3XMtcvo\nq9r1we84G+jsPJZ2HvFvB/C5MfctB/Cgqp4C4CEA3+ho70RUCDf4qvoIgN1j7l4AYHVyezWA87s8\nLiLKUKev8aeo6jAAqOpOAFO6NyQiylq3ztVXs/p8dfT2MRVgSqVLuyWiUdvQ7mfpdRr8YRGZqqrD\nIjINwOvm0qdVO9wNEbVvevK1zwMtl2z3qb4kX/usB7A4ub0IwLr2B0dERXODLyJ3AfgPACeLyCsi\ncimAawH8toi8AODc5HsiOkC4T/VV9aIWJa9DOaqzj/BOpP18+bS8+fbO/m85wSzXvmyv7vXxj3Pq\nMmC//YI1VWcL3v/f43x+fZ9dvhLHptx/1qpO/fN2+ShnQn/6K/c3xTP3iAJi8IkCYvCJAmLwiQJi\n8IkCYvCJAmLwiQLK57r65nz8skt3iHqWbjPrBzt9fG+2/9ecNu+Xpjp9fPdMAY/Tp3fPw/COb9rz\nCLLm/f+fNavzd79i1k+F/cEBqz7445a1XRNaX+uBj/hEATH4RAEx+EQBMfhEATH4RAEx+EQBMfhE\nAeXTxw/sIHyQ6faPOtL+/HU415X3+9BZ/4p4+/fqRfPOU3jNrG5Y9iWzXr3J3vqbv1ttWbPOAOAj\nPlFADD5RQAw+UUAMPlFADD5RQAw+UUAMPlFA+fTxvc9ILzWvD27b/vczzLr3qQCTnPqea6c5S3jz\n8b0RZP25BQe6lMfHyYb388cso7a2dYmP+EQBMfhEATH4RAEx+EQBMfhEATH4RAEx+EQBuX18EbkV\nwHwAw6p6enJfP4AlAF5PFrtKVR9ouZEeYwfPeSMouo+ccv9f3myW5zqrP+1t/0WnXrGvy47aV5wN\nvOPUvfMcvE8G2ODUi+bNt/d+P3rt8g/t8i5n6/ilt0Bz7Tzi3w7gc03uX6GqZyRfrUNPRKXjBl9V\nHwGwu0nJeSghorJK8xp/mYgMisgqETmyayMiosx1eq7+zQC+qaoqItcAWAHgj1ouva06entSBZhc\n6XC3RNRK7WWgNtTesh0FX1XfaPh2Jbx3aKZXO9kNEY1D5YT61z4DtdbLtvtUX9Dwml5EGqeEXYA2\n3psnovJop513F4AKgMki8gqAfgDniEgfgL0AhgAszXCMRNRlbvBV9aImd98+rr1MtIre57N7feCs\nedd198Z/r1l9SRea9V1ir+/2cZ906jOmmuW7tpxv1k+WdWb90yvt3cuSfnuB1H30tLyIOPufuNgs\n3/PypeMazX5O7Gw1nrlHFBCDTxQQg08UEINPFBCDTxQQg08UEINPFFA+19Ufsop2HxtY5dSL7vN6\nff5PmNXFYv//9TK7j3/0t+1zp66UY806ttrlD5xfEW82/bl/sMNeYMktzhbs45f9z9fbv3M9gkft\n8paZ4xrM/s7qbDU+4hMFxOATBcTgEwXE4BMFxOATBcTgEwXE4BMFlE8f/0nj2vJzT7XXfbC7Q+m+\ntH3kTWZVbtpor/4nzpX5f8+5GLIzn//fcKNZ/+7WNfYGDq3adfc8jKKvx+B9boD98997rH38B8Y5\nmrGun77MqH67ZYWP+EQBMfhEATH4RAEx+EQBMfhEATH4RAEx+EQBiapmuwMRrX8GRwuLq/YG7qg5\ne/DqRc/X92Q9vuOcun1dfeDplPsv+/FPOd8ef25W1zkfKm2fxeEfnYEHjfzOFahq0wHwEZ8oIAaf\nKCAGnyggBp8oIAafKCAGnyggBp8oIHc+voj0APge6g3fvQBWquq3RORoAPcAOB71K+cvVNU94x7B\nHU/Z9asrdv2amrODlJ9vnjlv/14f3Pv/vZay7u3f64MXPZ/ec4RTHzare97/uFlf4fx4vKN7+UF2\nfWCuk58W2nnEfx/A11T1NAC/CeAyEZkBYDmAB1X1FAAPAfhGRyMgoty5wVfVnao6mNweAbAFQA+A\nBQBWJ4utBnB+VoMkou4a12t8EekF0If6BwNNVdVhoP7HAcCUbg+OiLLR9jX3RGQigDUArlDVkfo5\n+B9hnDRca7jdm3wRUXc9CaC91/xtBV9EJqAe+jtVdV1y97CITFXVYRGZBuD11luotDUYIkrjM8nX\nPitbLtnuU/3bAGxW1cZLrq4HsDi5vQjAurErEVE5tdPOmw3gYgCbROQZ1J/SXwXgOgD3isgfAtgO\n//OuiagkcpqPf42xhNPH7q3a9VnOAO521nf70J53U65P6Xid8ElO3e7TT3p/iVm/fEKPWfeuhuCd\nRTFb55j1eXKuUR3gfHwiGsXgEwXE4BMFxOATBcTgEwXE4BMFxOATBVT8dfXTXnd9VtWuz3A2f4ez\nvsvrE3t9/nSfv37g886j8Oq70u3+uapZ/u5M+7r4aX+6Jzn1BdYUGADAXxm1q9nHJ6JRDD5RQAw+\nUUAMPlFADD5RQAw+UUAMPlFAbV9zLzten9rp4z5atetOn/Z4vdCsbxfvRABn/6ml7XNnzetUe53u\ntPXZZvVvdK1Zf0vsPr13loB3VX5v/eXqXCNPqilH0Bwf8YkCYvCJAmLwiQJi8IkCYvCJAmLwiQJi\n8IkCKkEf3+P1cZ0+5kjVLG8/267rdXaf90d/+ltm/Qu/8S9mHYPWfGogfZ+75CZWzfL8t+8z6xvm\nzDPrVfvHh6l22bnqvn/0D9ErzfoWWe9swevTv+XUm+MjPlFADD5RQAw+UUAMPlFADD5RQAw+UUBu\n8EWkR0QeEpHnRWSTiFye3N8vIq+KyNPJ13nZD5eIusG9rr6ITAMwTVUHRWQigKcALADwRQBvq+oK\nZ33nuvpZ8+are53YqlnVG+xGcdVu46LfuSy/PGzXV89caNYH0WfW/xeHmPVT8IJZvwh3mfVjlo2Y\n9ftvMsvYZJczV3HqV+p/mvWfyAPOFrLp09cNtLyuvnsCj6ruBLAzuT0iIlsAHJeUndMjiKiMxvUa\nX0R6AfQBeCy5a5mIDIrIKhE5sstjI6KMtB385Gn+GgBXqOoIgJsBnKiqfag/IzCf8hNRebR1rr6I\nTEA99Heq6joAUNU3GhZZCWBD6y3UGm73Jl9E1F1DyZev3Uk6twHYrKo37rtDRKYlr/8B4AIAz7Ve\nvdLmboioc7346INq63eG3eCLyGwAFwPYJCLPAFAAVwG4SET6AOxF/c/M0k6HS0T5audd/X8HcFCT\nktenIKKScvv4qXdQeB/fk7bP79haNcv6Hec8gBvNMjkWOfVX9EyzXjn8MbPuXe8h898vU+s+Pk/Z\nJQqIwScKiMEnCojBJwqIwScKiMEnCojBJwqIfXzXwSnXfy/d6vOrZvkrG+y5UTcPft3e/jed/Ttt\n7M0/s+vTnTmbBy+269tvOMasz/yf5836yMRP2TvAgFP3ZDmfPi328YmoAYNPFBCDTxRQAcEfyn+X\n4zJU9ABsP68VPQLT40UPwFQregCOodz2xODvZ6joAdjerBU9AtMTRQ/AVCt6AI6h3PbEp/pEATH4\nRAHl1McnoiK06uNnHnwiKh8+1ScKiMEnCojBJwqIwScKiMEnCuj/AfU0sFnebKjNAAAAAElFTkSu\nQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b06d190>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEIVJREFUeJzt3X2MXNV5x/HfQ03TBAdwEF43uGHruHlRU2qRQkBQxREQ\nSJrECLW8JRWmFUVtTEFN1ICLuuMWGlw1BCqLVgFDHCdOoTTUpC+8a6mggVDAxIBDSsiSQPHihNdN\nUWvXT/+Ya+94mTnP7NyZudec70caeXae+3J8d39zZ+bcc8bcXQDysk/VDQAwfAQfyBDBBzJE8IEM\nEXwgQwQfyNDQgm9mJ5nZd83se2b2uWHtt1tmNmFmj5jZw2b27Rq0Z62ZTZrZd1oem2dmt5nZE2Z2\nq5kdULP2jZnZM2b2UHE7qcL2LTSzu8zsMTPbbGZ/WDxei2PYpn3nFY8P5RjaMPrxzWwfSd+TdJyk\n/5L0gKTT3f27A995l8zsKUnvd/cXq26LJJnZsZKmJH3F3Q8rHlst6Sfu/pfFk+c8d7+wRu0bk/Sq\nu19eRZtamdkCSQvcfZOZzZX0oKRlks5WDY5hon2naQjHcFhn/CMl/ae7P+3u2yX9nZr/yTox1eit\nj7vfI2nmk9AySeuK++sknTzURrXo0D6peRwr5+5b3X1TcX9K0hZJC1WTY9ihfYcU5YEfw2H9oR8i\n6UctPz+j6f9kXbik283sATM7p+rGdDDf3Sel5h+OpPkVt6edFWa2ycyuqfKtSCszG5W0RNJ9kkbq\ndgxb2nd/8dDAj2FtznA1cIy7Hy7po5I+XbyUrbu6XW99laRF7r5E0lZJdXjJP1fSjZLOL86sM49Z\npcewTfuGcgyHFfxnJb2j5eeFxWO14e7PFf9uk3STmm9P6mbSzEak3e8Rn6+4PXtw920+/aHR1ZKO\nqLI9ZjZHzVCtd/eNxcO1OYbt2jesYzis4D8gabGZHWpmPyvpdEk3D2nfITN7S/HMKzPbT9KHJT1a\nbaskNd/rtb7fu1nS8uL+WZI2zlxhyPZoXxGkXU5R9cfwWkmPu/uVLY/V6Ri+rn3DOoZD+VRfanbn\nSbpSzSebte5+2VB23AUz+0U1z/IuaY6kr1XdPjPbIGmppIMkTUoak/SPkv5e0i9IelrSqe7+Uo3a\n9yE136vulDQh6dxd76craN8xkv5N0mY1f68uaaWkb0u6QRUfw0T7ztQQjuHQgg+gPvhwD8gQwQcy\nRPCBDBF8IEOlgl/3gTcA2uv5U/1uB96YGd0GQEXcve11/3NKbHP3wBtJMrNdA29eN+LOb52+31gv\nNX675ecT0zsZCRrR7w7OcTU7p+tqXLNr375B/W1BPbqQ/YQ37fnz53dIF7X8Ve3/iWADf5our3rf\nHyfrjfNWpzewprV+u6QTZizwWnr98AhGts9i2XH1969vVcdKmZf6e8PAGwBt8OEekKEyL/W7HnjT\nWD99/8D9SuxxCEarbkBgtOoGBI6t9alkUdUNCIyWXH+iuMXKBH/3wBtJz6k58OaMdgu2vqevu9Gq\nGxAYrboBgV+vdfDfWXUDAqN9WL91G3d3XLLn4Lv7/5nZCkm3aXrgzZZetwdgeAY+SMfMXGs678Pv\nS88y1PhqevvD/Mx1EPYP6lH7os+kc9f4TLpu5/1veoHRS0u24M1BfZC/wVUdu/Nq/cIMwGAQfCBD\nBB/IEMEHMkTwgQwRfCBDBB/I0HD68TWWWGJxcv3Vfn+y/t+2podWTYuuA6i6n7+sqBc5UvV1AtHo\nzBeCetnfX+OpdN0WPRJs4RtBfZB/gfTjA2hB8IEMEXwgQwQfyBDBBzJE8IEMEXwgQ2Vm4JmFVG/y\nk8k1P2cHJes/99JPkvW/OTC9ftu5wlqUHe8fHeBPeXqe24X3pv9/uiLYwX8E9aiBJ6XL+1z802T9\nvpGjkvUj3pH+FujGj5LlUDTfQaQRzNbly381WbcvR/+Ba4L6YMbzc8YHMkTwgQwRfCBDBB/IEMEH\nMkTwgQwRfCBDNRiPH/WUvzWop0dk+8bO3xgqSY1l6a2X/bbeRtvvFppmX08dmxwcn6z+nj+YrP+a\nXZCsl71OI/o24fD3H1xnYRdcFWwh6qdP1S9mPD6AaQQfyBDBBzJE8IEMEXwgQwQfyBDBBzJUajy+\nmU1IelnSTknb3f3I2W+l7Mzn6fHKz34i6olNXwdQdl56fTCofz3qSZ4f1KOZ5Qet7IwFdySrX2rb\nC91SX5O+DuVfVqQ38FB683olqEfXeTTSlxnoIn8uWf+8Refm3mYcKDsRx05JS939xZLbATBEZV/q\nWx+2AWDIyobWJd1uZg+Y2Tn9aBCAwSv7Uv8Yd3/OzA5W8wlgi7vf8/rFxlvujxY3AP31pKTvd7Vk\nqeC7Nz+ZcPdtZnaTpCMltQn+0jK7AdCVxdrzS2hv67hkzy/1zewtZja3uL+fpA9LSk+ZCqAWypzx\nRyTd1Bx2qzmSvubunZ9iANRGz8F39x9IWtLHtvSo6m9wLyu6juHVoF71/7/s/ktex7CikSx/9Jl0\n+8YWpq/UiAIS9fNHveyL7c/TC8wN5suYujTYQ3t0xQEZIvhAhgg+kCGCD2SI4AMZIvhAhgg+kKGy\n1+oDJUXXMTwf1IP5FhZeliw37kyv3jguXe9tNPy0aD6AG179eLJ+qr2/p/1yxgcyRPCBDBF8IEME\nH8gQwQcyRPCBDBF8IEP046Pmon7+HUE9Pd7fnkuPdx9Tel7+st8KEc1G8JsH/VOwxMWJ2qqOFc74\nQIYIPpAhgg9kiOADGSL4QIYIPpAhgg9kiH587OWief2DnvZPfTVZbgTfHNHYlK6nZ+2PrXohWOC+\nD3SuHdW5xBkfyBDBBzJE8IEMEXwgQwQfyBDBBzJE8IEMhf34ZrZW0sckTbr7YcVj8yRdL+lQSROS\nTnX3lwfYTqCDaER8NPP9k+nyPwerH5IuR+Pto3rkWx/ofKHB0Yn1ujnjXyfpxBmPXSjpDnd/t6S7\nJF3UxXYA1EQYfHe/R9KLMx5eJmldcX+dpJP73C4AA9Tre/z57j4pSe6+VdL8/jUJwKD161r99MRl\nGm+5P1rcAPTTQ+NTemh8qqtlew3+pJmNuPukmS1Q+M2GS3vcDYBuHb50rg5fOnf3z2tXTXZcttuX\n+lbcdrlZ0vLi/lmSNs6qhQAqFQbfzDZI+ndJ7zKzH5rZ2ZIuk3SCmT0h6bjiZwB7ifClvruf2aF0\nfJ/bAtTOuW+/Iln/eV1QavvRVQhR/aixR3raL1fuARki+ECGCD6QIYIPZIjgAxki+ECGCD6QIebV\nBxI2/LTTZSxNnynZj7+j1NrSK6t7W48zPpAhgg9kiOADGSL4QIYIPpAhgg9kiOADGaIfH0iYGj94\noNuPxttHvvE/va3HGR/IEMEHMkTwgQwRfCBDBB/IEMEHMkTwgQzRjw+kPFl1A9Je63E9zvhAhgg+\nkCGCD2SI4AMZIvhAhgg+kCGCD2Qo7Mc3s7WSPiZp0t0PKx4bk3SOpOeLxVa6+y0DayVQlcWD3fy+\nQT0ar//xRO0PErVuzvjXSTqxzeOXu/vhxY3QA3uRMPjufo+kF9uUrP/NATAMZd7jrzCzTWZ2jZkd\n0LcWARi4Xq/Vv0rSn7m7m9klki6X9LudFx9vuT9a3AD007eKWzd6Cr67b2v58WpJ30yvsbSX3QCY\nhaOL2y5fTCzb7Ut9U8t7ejNb0FI7RdKjXbcOQOW66c7boOYp+yAz+6GkMUkfMrMlknZKmpB07gDb\nCKDPwuC7e7svCL9uAG0BamfRbzxWdROSFn4wUby7c4kr94AMEXwgQwQfyBDBBzJE8IEMEXwgQwQf\nyBDz6uMNrtyf+CX6k2T9iVJbl94c1KPx+PrbRO29nUuc8YEMEXwgQwQfyBDBBzJE8IEMEXwgQwQf\nyBD9+HiDe7XU2md8YWOy3ii19S766QMnv2dDotpuKo0mzvhAhgg+kCGCD2SI4AMZIvhAhgg+kCGC\nD2SIfnzs5aIR7TvS5QMbyfL1n101q9bM9FpQj/rxR4P6qiPOSFTpxwfQguADGSL4QIYIPpAhgg9k\niOADGSL4QIbCfnwzWyjpK5JGJO2UdLW7/7WZzZN0vaRDJU1IOtXdXx5gW4E2on78F5LVq15cnqxv\nsfTW9w/2Xtbye9L1s49t9LTdbs74OyT9kbv/sqSjJX3azN4j6UJJd7j7uyXdJeminloAYOjC4Lv7\nVnffVNyfkrRF0kJJyyStKxZbJ+nkQTUSQH/N6j2+mY1KWiLpPkkj7j4pNZ8cJM3vd+MADEbX1+qb\n2VxJN0o6392nzMxnLDLz5xbjLfdHFV+BDGD2JopbrKvgm9kcNUO/3t13zT44aWYj7j5pZgskPd95\nC0u7agyAMka150n17o5LdvtS/1pJj7v7lS2P3SxpeXH/LEnp6UgB1EY33XnHSPqkpM1m9rCaL+lX\nSlot6QYz+x1JT0s6dZANBdA/YfDd/V5JP9OhfHx/mwPMNBLUJ9PlxY1k+RBLd9SnrwKIx9PvW3J9\nm0h8dCap15n9uXIPyBDBBzJE8IEMEXwgQwQfyBDBBzJE8IEMvQHm1U+Px97R8RKEuojGk0e/oqin\nuGpl/39BP31w3Zgfle6nbzyZ3nrU+nKzAUiNW9P1VSeuLtmC9jjjAxki+ECGCD6QIYIPZIjgAxki\n+ECGCD6QoRr040f90FE93VN66F3bZtWamaLvNw/dWXYPpVtQsWjEeeCkRrLsC4J++i+nNx/1gpf7\n65N+P6jbidF4+0uDem8R5owPZIjgAxki+ECGCD6QIYIPZIjgAxki+ECGzD3qRyy5AzOXLkssUbKf\n+o5Gsrz++HQ/77PB5nfMrjWvE/WyTnn628UvOfgv0hv48XeCPUTj2aP/4VuD+i+ly1ek58V/9Px3\nJut32lPJetSPHn1/ffT7ibb/3qB++iVBvi5uBFuIrjRI5WeV3L1tADjjAxki+ECGCD6QIYIPZIjg\nAxki+ECGwuCb2UIzu8vMHjOzzWZ2XvH4mJk9Y2YPFbeTBt9cAP0Q9uOb2QJJC9x9k5nNlfSgpGWS\nTpP0qrtfHqzv0lhiiUOS+7/Af5ysv8tWJutRL3bZ7y8ftLcF9dOC+sjbgwXelC77y+n6LUFH9/3B\n7iODntc+Eo2nX7Am6Kdf0Qi2EF1p8EpQT+ncjx+O4nf3rZK2FvenzGyLptOavjoGQC3N6j2+mY1K\nWqLpJ/IVZrbJzK4xswP63DYAA9J18IuX+TdKOt/dpyRdJWmRuy9R8xVB8iU/gProasIuM5ujZujX\nu/tGSXL31snsrpb0zc5bGG+5P1rcAPTXRHGLdTtT37WSHnf3K3c9YGYLivf/knSKpEc7r760y90A\n6N2o9jyp3t1xyTD4ZnaMpE9K2mxmD0tySSslnWlmSyTtVPNp5txemwtguLr5VP9eqe13Td/S/+YA\nGIbhjMf/bOd9dOhm3K3xhXL7r3s//SB7cSEdH9Tn+aJk/X32/WALjaBe5W+Y8fgAWhB8IEMEH8gQ\nwQcyRPCBDBF8IEMEH8jQUPrxfXPneuNX0uunZ2WPx1tX3U9fVnQdQqS3b0+fFs0HEP1+PhKM2dz3\nt9L17X+Vri874B+S9X+dd0p6Ay810vVQna/EoB8fQAuCD2SI4AMZGnrwxx8Y9h5nZ6LqBgTS3yRX\nvcTHOdXbPl51CwITQ9sTwZ9houoGBH5QdQMCiUkZqrdjvOoWBCaGtide6gMZIvhAhoYzHh9AJTr1\n4w88+ADqh5f6QIYIPpAhgg9kiOADGSL4QIb+H0lbCXn6sTkDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b168650>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c\n",
      "i\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEoJJREFUeJzt3XuwlPV5B/DvV2lGDfGCLdBK4hGvM1FLvY0dm/GYNF4y\nWm8T4jVqrTUXjWlso0EnrBZSzVQqTsdOR8EQL02ME4S09e5Aio0RL6QQUanxGIlwwEuItLVqefrH\nvsB63H2e5bxn3114vp+ZHZZ9dt/3d97d79nd87zv76WZQURy2a7bAxCR6in4Igkp+CIJKfgiCSn4\nIgkp+CIJVRZ8kseTfI7kCySvqGq97SI5QPJnJJ8h+UQPjGcWyUGS/9Fw224kHyT5PMkHSO7SY+Ob\nSnIlyaeLy/FdHN8Eko+S/DnJpSS/UtzeE9uwyfguLW6vZBuyij4+ye0AvADgUwBeBbAYwBlm9lzH\nV94mkr8AcKiZvdntsQAAyT8CsB7Ad83s4OK26wG8bmbfLn557mZmV/bQ+KYCeMvMZnRjTI1Ijgcw\n3syWkBwN4CkAJwO4AD2wDZ3xfQ4VbMOq3vGPALDCzF42s3cBfA/1H7KXED301cfMFgEY+kvoZABz\niutzAJxS6aAatBgfUN+OXWdmq81sSXF9PYDlACagR7Zhi/HtUZQ7vg2reqHvAeCVhv+vxOYfslcY\ngIdILiZ5UbcH08JYMxsE6i8cAGO7PJ5mLiG5hOSt3fwq0ohkH4BJAB4HMK7XtmHD+H5a3NTxbdgz\n73A94CgzOwTAZwB8ufgo2+t6bX/rmwFMNLNJAFYD6IWP/KMB3APgsuKddeg26+o2bDK+SrZhVcH/\nFYCPNfx/QnFbzzCzVcW/awHMRf3rSa8ZJDkO2PQdcU2Xx/M+ZrbWNv/R6BYAh3dzPCRHoR6q281s\nXnFzz2zDZuOrahtWFfzFAPYhuSfJDwE4A8D8itYdIrlT8ZsXJD8M4FgAy7o7KgD173qN3/fmAzi/\nuH4egHlDH1Cx942vCNJGp6H723A2gGfNbGbDbb20DT8wvqq2YSV/1Qfq7TwAM1H/ZTPLzK6rZMVt\nILkX6u/yBmAUgDu7PT6SdwHoB7A7gEEAUwHcC+AHAD4K4GUAk83s1z00vmNQ/666AcAAgIs3fp/u\nwviOAvBjAEtRf14NwBQATwC4G13ehs74zkIF27Cy4ItI79Af90QSUvBFElLwRRJS8EUSKhX8Xj/w\nRkSaG/Zf9ds98Iak2gYiXWJmTff7H1VimZsOvAEAkhsPvGlyxN3UhusLUG//9qoF2LLx7RzU/yeo\nvxvUf2vI/x9B/Xdt4cir3Ed/8if/7NYfeekkf/WP+OWhavOB2p803PBKy7vWbR/Uoz3V3wvqEzdf\nrX0PqJ0xpD7Jf/hZe81y6//Ec4MBTA/qja+fBwAcN6Re5vVzTctKmY/6W8OBNyLShP64J5JQmY/6\nW3DgzYKG6zuUWGUV+ro9gMBe3R6Aq3//bo+gtf4Duz2CyN4lHz9QXGJlgr/pwBsAq1A/8ObM5nft\nL7GaqvV1ewCBifFdukjBL2Ofko/vw/tfvwtb3nPYwTez/yN5CYAHsfnAm+XDXZ6IVKfMOz7M7H4A\nPfw7vh1D/2o+1I5B/TflHr/ycrds9waPf/Vqt7wwmFZgxzGvu/W3Z47x179shV/Hw0G97IFnUVfl\nL/zyNH+Wqxuvutit3/X1C936cdff69Yf5DNuPX59RvXm9Mc9kYQUfJGEFHyRhBR8kYQUfJGEFHyR\nhBR8kYQ6Ptlm/bDcqfEdOybqw0d90KBPf3/NLdvVwdmQPuuXefoG/w773OrXe+v0BU1Ez09Ujw7P\ni/aziJ7/6OjJL7pV+5fxbv3to/2l7zh6WrB+/+i8Vofl6h1fJCEFXyQhBV8kIQVfJCEFXyQhBV8k\nIQVfJKFtoI8f9XmjWUoPcqsD5vdp3+Fat77fsmD7Hljz68M83nqzaPtEoj522e0f1bst2v7RlBb+\nz/fOr1vPhAsA/7iLf7z/pZzgVNXHF5EGCr5IQgq+SEIKvkhCCr5IQgq+SEIKvkhCpebVr0bZPvFk\nt2qHfdx/+Hy/TPzMv0PYp4/mhY+eojeCetk+fHS8e7T+QF/NLe/7kr99p8M/r8Bn1/hnC8ZLfjmc\n9n9VUD/UL78dPL2v0T9bL6Y5+4lc3Zmz5YrIVkrBF0lIwRdJSMEXSUjBF0lIwRdJSMEXSahUH5/k\nAIB1ADYAeNfM/JOxNxUd7xz16WtudcMYf177RYsPceuf4EnB+u8L6lGfPvr5oj58pGyfPlj/kzW3\nbDsH5xV4yT8efSb+3K1P/tUP/OWfs4NfX+CXgdeDevD8HhC8vv3dEHCC/dC/wx/45VbK7sCzAUC/\nmb1ZcjkiUqGyH/U5AssQkYqVDa0BeIjkYpIXjcSARKTzyn7UP8rMVpH8HdR/ASw3s0UfvNuChut9\nxUVERtRbC4D1C9q6a6ngm9mq4t+1JOcCOAJAk+D3l1mNiLTjI/31y0aDHThIh+ROJEcX1z8M4FgA\ny4a7PBGpTpl3/HEA5tanz8YoAHea2YMjMywR6aSK5tX3zvEd9an73aodeIxbn730TLd+IfcL1j8m\nqL8V1Mv24SPRfgLB+eEn1Nyy3RH04T/mlzk7eH1Nm+7XO779tmWaV19EGij4Igkp+CIJKfgiCSn4\nIgkp+CIJKfgiCVU0r763Gr9P+4Kd6y/6X/1y3KePjlffyvv0y2pu2b7u9+k5OujDT/SXD0R9+ki0\nH0XJ+QTCx3da9PqLIhq9PpvTO75IQgq+SEIKvkhCCr5IQgq+SEIKvkhCCr5IQhX18Z2546fV3Efu\nu4c/7zpfjeYT8Jcf6+0+/Xj7vFtfdVzQp38i2H6H1fx6OP5o+0XnFXgjqG/tujPfgN7xRRJS8EUS\nUvBFElLwRRJS8EUSUvBFElLwRRKqqI/fml3g95nvuOp0fwGsBWsoeTx7acH50aP1n1Fzy6smBn36\ntUGffr2//Pp5UzyDQV16kd7xRRJS8EUSUvBFElLwRRJS8EUSUvBFElLwRRKimd/nJTkLwIkABs3s\n4OK23QB8H8CeAAYATDazdS0eb8A7LZdvp37IX/9Pgj706ppfD/vonT4eOpo33T8e3ab78xHsN2WJ\nW1/BucH6o3nrt/Xj4bdl18DMmu7o0c47/m0Ajhty25UAHjaz/QE8CuAb5QYoIlUKg29miwC8OeTm\nkwHMKa7PAXDKCI9LRDpouN/xx5rZIACY2WoAY0duSCLSaSO1r37wRfzahutHFxcRGVkDxSU23OAP\nkhxnZoMkxwNY49/9m8NcjYi0r6+4bLSw5T3b/ajP4rLRfADnF9fPAzCv3aGJSPeFwSd5F4B/B7Af\nyV+SvADAdQA+TfJ5AJ8q/i8iW4mwj196BaThydbrsCeD48m/MHWkhzTCSh7vf2PNLdsrwfa5YVqw\n/u7M2y69oFwfX0S2MQq+SEIKvkhCCr5IQgq+SEIKvkhCCr5IQpXMq3/Eoa13HcRHgwd/IVp6rx9v\n7/fx//qyv3TrC3GEv/gbop+v3HwAsm3SO75IQgq+SEIKvkhCCr5IQgq+SEIKvkhCCr5IQpX08b+E\nm1vWZo89s+TSoz51p/v45frgF+A2t/75TZMZt/JkUI/2c1AfPyO944skpOCLJKTgiySk4IskpOCL\nJKTgiySk4IskVEkffyf8d8vaa9i9iiGISAO944skpOCLJKTgiySk4IskpOCLJKTgiySk4IskRLPW\n564HAJKzAJwIYNDMDi5umwrgIgBrirtNMbP7Wzzejrb7Wi7//v86wV3/jqOnuvXuz6s/LqgPutVv\n2Tq3/gn82K/zpGD9mlc/r2tgZmxWaecd/zYAxzW5fYaZHVJcmoZeRHpTGHwzWwTgzSalpr9JRKT3\nlfmOfwnJJSRvJbnLiI1IRDpuuPvq3wzgWjMzktMAzABwYas7D9Tu2HR91/6DsWv/wcNcrYi0NlBc\nYsMKvpmtbfjvLQB+5N2/r3bOcFYjIlukr7hs1Ppkte1+1CcavtOTHN9QOw3AsrbHJiJdF77jk7wL\nQD+A3Un+EsBUAMeQnARgA+qfLS7u4BhFZISFwTezs5rc7E8GP8TC549vWdth8ZYsqZlO9+kj5frg\nU2b9nVu3pVHzJOrjq08vH6Q990QSUvBFElLwRRJS8EUSUvBFElLwRRJS8EUSqmRefRzo1PzD8QHU\nStY7fbz+b8qt/89qfv1yv7yvnerWV3CuvwCMCepvBHXZGukdXyQhBV8kIQVfJCEFXyQhBV8kIQVf\nJCEFXyShavr479Va1671H3r5/Glu/YZwrt9oXvlOH88fbWJ//fxf/7wHG3b3N8B2o/3HY33Nr5c8\nb4D0Jr3jiySk4IskpOCLJKTgiySk4IskpOCLJKTgiyRUTR/fwXV+n9kO9/vUN+DfgjU8HNQ7fbx+\nNK/9zn7572tu+TDzf3470t9+fDHo87/mrz8+nj/6+TXvfzfoHV8kIQVfJCEFXyQhBV8kIQVfJCEF\nXyQhBV8kobCPT3ICgO+ifmD2BgC3mNlNJHcD8H0AewIYADDZzNY1X4pzTHx/zR/AUr+86sAT3frv\n8qv+AvCRoP5WUO/0vPx+n/xp+vspcKXfJ7fPBX3+h4M+/6SaXw/3k4jq0fPzXlDv9f0Eys4XMbyf\nr513/PcAfM3MPg7gDwF8meQBAK4E8LCZ7Q/gUQDfGNYIRKRyYfDNbLWZLSmurwewHMAEACcDmFPc\nbQ6AUzo1SBEZWVv0HZ9kH4BJAB4HMM7MBoH6LwcAY0d6cCLSGW3vq09yNIB7AFxmZutJDv3y53wZ\nfKjh+kQAe2/BEEWkPQPFJdZW8EmOQj30t5vZvOLmQZLjzGyQ5HgAa1ov4dNtDUZEyugrLhstbHnP\ndj/qzwbwrJnNbLhtPoDzi+vnAZg39EEi0pvaaecdBeBsAEtJPoP6R/opAK4HcDfJPwXwMoDJnRyo\niIycMPhm9hiA7VuU/3hkh/NBPGi1W39t+/Fu/Wg70q0v5OPBCKLjzTvd54/OTx8czz/hOrfMfYL5\nEF4OTlyw3C9zVrAfwN9O9+vhz7+16/R5HZrTnnsiCSn4Igkp+CIJKfgiCSn4Igkp+CIJKfgiCdEs\n6LOWXQFpwFTnHtHxyMHxxofV3PILT/p96L+yu9z6PD7vrz/qo3foeOr2Rds3Op49GP9zNbds7wb7\nAbzql2cc+0W3fvnLM906zgmO91/kl4HB6A6+Xcf59SuDx+8Q1L96k1O8DGbW9AnQO75IQgq+SEIK\nvkhCCr5IQgq+SEIKvkhCCr5IQj3Qx4+U7PP/ds0tb9jg95kfeP1ot34CT/PXX/Z4+lA0L3+kw33+\nyAE1t/z7y/35Er6Ja936aa/e569/hV8On77o8af65Zv28+tfCea14sletq5RH19ENlPwRRJS8EUS\nUvBFElLwRRJS8EUSUvBFEtoK+viRqA8e9bnPc6u2eKL/8G/5ZW4fbN97an49PH98dGqETvfpS+5n\n0fPnry/5+vpOzS3bdf5+JHxuQ7D+a9ya+vgisomCL5KQgi+SkIIvkpCCL5KQgi+SUBh8khNIPkry\n5ySXkry0uH0qyZUkny4ux3d+uCIyEqImMFBv9H7NzJaQHA3gKZIPFbUZZjajc8NrR9Snj/rMd7tV\nHu7vgzDBznbrdngwr3wwbTv/JtgP4MaaX+/08fSl+/DR8xPV3wrq0c8X7Sfhv74+aYe59UcOCvr0\nJwbPb3Degnj7NBcG38xWA1hdXF9PcjmAPYpy8KoWkV60Rd/xSfYBmATgp8VNl5BcQvJWkruM8NhE\npEPaDn7xMf8eAJeZ2XoANwOYaGaTUP9E0OWP/CLSrna+44PkKNRDf7uZzQMAM1vbcJdbAPyo9RIW\nNFzvKy4iMrJeBPCLtu7ZVvABzAbwrJltOkMhyfHF938AOA3AstYP729zNSIyfHsXl40eaXnPMPgk\njwJwNoClJJ8BYACmADiL5CQAGwAMALh4+AMWkSq181f9xwBs36R0/8gPR0Sq0O5H/a1Y1GeO+rj+\n8dgreadbJ/yJ0Q8ZO8atW1/QMf2hX/6HU/35Br702Hf8BZzil/HaU8EdHgvq0cT10fO3T1A+xy2f\nvuIOt37PFef6y5/rl7ks6NMvq/n10vNNNKdddkUSUvBFElLwRRJS8EUSUvBFElLwRRJS8EUSStDH\nj0THa0f1qM+61K0+TX/5xHX+4s+/wi2feepst26/F+wncLNfxrqgviaoR6/AaDqByO5BHz4YHz8T\n9OH7rw8GMD2od6ZPH9E7vkhCCr5IQgq+SEJdCP5A9avcIgPdHkDgxW4PwLXg+W6PoLVeHlvdf1a2\nJgX/Awa6PYBAexMtdEsvh2vBC90eQaS6X+r6qC+SkIIvkhDNgj5l2RWQnV2BiLRkZk131Oh48EWk\n9+ijvkhCCr5IQgq+SEIKvkhCCr5IQv8PXza6JHrgIFEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11a903fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADUZJREFUeJzt3V+sHOV9xvHnCXYlwMI1qD6ucNstqkqkNK2VqqiRqeoo\naUJ7A+IiTYkUSKsIVSEhopEKqJJNW1VJlVrxDZXKPzkoqE2RwHBRMMi1G6ciQQEXGxyCFJYEig9W\n65RaSiMT/3qxY1jbZ+ddn9mdmZPf9yONPGfemZ3fGZ/nvDM78+5xRAhALu/qugAA7SP4QEIEH0iI\n4AMJEXwgIYIPJNRa8G1fZfs7tr9r+8/b2u+0bA9t/4ftZ21/qwf13GN70fZzY8vW2d5t+0Xbj9te\n27P6ttp+1fYz1XRVh/VttL3H9vO2D9r+bLW8F8dwifo+Uy1v5Ri6jfv4tt8l6buSPijpPyU9Lelj\nEfGdue98Sra/J+k3I+JY17VIku0rJR2X9JWI+PVq2Rcl/VdE/G31y3NdRNzao/q2SvrfiNjeRU3j\nbG+QtCEiDtheI+nbkq6W9En14BjW1PeHauEYttXjXyHppYh4JSJOSPpHjb7JPrF6dOkTEfslnflL\n6GpJO6v5nZKuabWoMRPqk0bHsXMRcSQiDlTzxyUdlrRRPTmGE+q7tGqe+zFs6wf9Ukk/GPv6Vb3z\nTfZFSHrC9tO2P9V1MROsj4hFafSDI2l9x/Us5SbbB2zf3eWlyDjbA0mbJD0laaFvx3Csvm9Wi+Z+\nDHvTw/XA5oh4n6Q/kPTp6lS27/r2vPWdki6LiE2Sjkjqwyn/GkkPSrq56lnPPGadHsMl6mvlGLYV\n/Nck/eLY1xurZb0REa9X/x6V9JBGlyd9s2h7QXr7GvGNjus5TUQcjXfeNLpL0m91WY/tVRqF6v6I\n2FUt7s0xXKq+to5hW8F/WtKv2P4l2z8j6WOSHmlp30W2L6h+88r2hZI+LOlQt1VJGl3rjV/vPSLp\nhmr+ekm7ztygZafVVwXplGvV/TG8V9ILEbFjbFmfjuFZ9bV1DFt5V18a3c6TtEOjXzb3RMQXWtnx\nFGz/ska9fEhaJemrXddn+wFJWyRdImlR0lZJD0v6Z0m/IOkVSR+NiB/2qL4PaHStelLSUNKNp66n\nO6hvs6R/k3RQo//XkHS7pG9J+po6PoY19V2nFo5ha8EH0B+8uQckRPCBhAg+kBDBBxJqFPy+D7wB\nsLRlv6s/7cAb29w2ADoSEUs+97+qwWu+PfBGkmyfGnizxIi7+8bmH9b4uIjfjQ1nrT1u78u/X9u+\neFl9kX9f36z3nvH1P2k0POqUhcL2Pyq0z9pXJH2i5X2ei7bru6jQ/i9j83s1evBg3LbCHXIvbC3s\nYVBoHxbax+3V2RU2ccfElian+ith4A2AJfDmHpBQk1P9cxh48/DY/AUNdjl/7+m6gILf6LqAgj7X\nN+i6gKJBw+2HmvbSoknw3x54I+l1jQbe/NHSq3b2eRHn7Ne6LqCgz8GS+l3foOsCigYz2H78NfZN\nXHPZwY+In9i+SdJuvTPw5vByXw9Ae5r0+IqIxyRdXl5zOLFlnye3SZJVGCT3s/WPD3z52I217QP/\nQ237o/V71+pC+4lCO5q5uOkLvDSLKlYe3twDEiL4QEIEH0iI4AMJEXwgIYIPJETwgYQa3cefjdL4\nqjfrm39Y//cGPuefr9/+8/Wjhk/eW//XjO747/qXP7++ufXRfYBEjw+kRPCBhAg+kBDBBxIi+EBC\nBB9IiOADCfXgPn7hPn1RacT7oL75S9tqm2+Jv6ltX+vba9tL9+kZz48u0OMDCRF8ICGCDyRE8IGE\nCD6QEMEHEiL4QEI9uI/fVOlOeeHPoRZ8+b231bbH+vr7+NveaLR7YC7o8YGECD6QEMEHEiL4QEIE\nH0iI4AMJEXwgoUb38W0PJf2PpJOSTkTEFbMoqlcOFT4v4N2F7Qv38Uv/AYzHxzw0fYDnpKQtEXFs\nFsUAaEfTU33P4DUAtKxpaEPSE7aftv2pWRQEYP6anupvjojXbf+cRr8ADkfE/rNX2zs2P1Dxc/AA\nLMOwmsoaBT8iXq/+PWr7IUlXSFoi+Fua7AbAVAY6vVPdN3HNZZ/q277A9ppq/kJJH5Z0aLmvB6A9\nTXr8BUkP2Y7qdb4aEbtnUxaAeVp28CPiZUmbZljLnLzVcPuX6pvXNHx5oAPcigMSIvhAQgQfSIjg\nAwkRfCAhgg8kRPCBhH4KPle/qfML7c/UN180s0KA1tDjAwkRfCAhgg8kRPCBhAg+kBDBBxIi+EBC\n3MfX6kL7a/XNa2dWCNAaenwgIYIPJETwgYQIPpAQwQcSIvhAQgQfSCjBffzSX5i/uND+Zn3zhedS\nC9AP9PhAQgQfSIjgAwkRfCAhgg8kRPCBhAg+kFAx+Lbvsb1o+7mxZets77b9ou3Hbecdlb6qMAE9\nNE2Pf5+kj5yx7FZJT0bE5ZL2SLpt1oUBmJ9i8CNiv6RjZyy+WtLOan6npGtmXBeAOVruNf76iFiU\npIg4Imn97EoCMG+zugqN+ua9Y/ODagIwW8NqKltu8BdtL0TEou0Nkt6oX33LMncDYHoDnd6p7pu4\n5rSn+q6mUx6RdEM1f72kXdOWBqB709zOe0DSv0v6Vdvft/1JSV+Q9Hu2X5T0weprACtE8VQ/Iq6b\n0PShGdcCoCU8uQckRPCBhAg+kBDBBxIi+EBCBB9IiOADCRF8ICGCDyRE8IGECD6QEMEHEiL4QEIE\nH0iI4AMJ8cnvTXEEsQLR4wMJEXwgIYIPJETwgYQIPpAQwQcSIvhAQtyF5hAgIXp8ICGCDyRE8IGE\nCD6QEMEHEiL4QEIEH0ioGHzb99hetP3c2LKttl+1/Uw1XTXfMnvsvMIE9NA0Pf59kj6yxPLtEfG+\nanpsxnUBmKNi8CNiv6RjSzR59uUAaEOTa/ybbB+wfbfttTOrCMDcLfdB9Tsl/WVEhO2/lrRd0p9M\nXn3v2PygmgDM1rCaypYV/Ig4OvblXZIerd9iy3J2A+CcDHR6p7pv4prTnupbY9f0tjeMtV0r6dDU\ntQHoXLHHt/2ARl32Jba/L2mrpA/Y3iTppEbnFjfOsUYAM1YMfkRct8Ti++ZQS0fearb5j2dTBdAm\nntwDEiL4QEIEH0iI4AMJEXwgIYIPJETwgYT4UPmmGj4GAHSBHh9IiOADCRF8ICGCDyRE8IGECD6Q\nEMEHEuI+vk4025zx+FiB6PGBhAg+kBDBBxIi+EBCBB9IiOADCRF8IKEE9/FXF9obDqj/v2abA12g\nxwcSIvhAQgQfSIjgAwkRfCAhgg8kRPCBhIrBt73R9h7bz9s+aPuz1fJ1tnfbftH247bXzr/ceThR\nmFbXTz9R/QT00DQ9/luSbomI90h6v6RP2363pFslPRkRl0vaI+m2+ZUJYJaKwY+IIxFxoJo/Lumw\npI2Srpa0s1ptp6Rr5lUkgNk6p2t82wNJmyQ9JWkhIhal0S8HSetnXRyA+Zj6WX3bayQ9KOnmiDhu\nO85Y5cyvx+wdmx9UE4DZGlZT2VTBt71Ko9DfHxG7qsWLthciYtH2BklvTH6FLVMVA6CJgU7vVPdN\nXHPaU/17Jb0QETvGlj0i6YZq/npJu87cCEA/FXt825slfVzSQdvPanRKf7ukL0r6mu0/lvSKpI/O\ns1AAs1MMfkR8Q9J5E5o/NNty+qjwniXj8bEC8eQekBDBBxIi+EBCBB9IiOADCRF8ICGCDySU4HP1\nS0qfq39xfTNj7rEC0eMDCRF8ICGCDyRE8IGECD6QEMEHEiL4QELcx9eJQvsl9c2Mx+9U6SmM+b/A\nyowQPT6QEMEHEiL4QEIEH0iI4AMJEXwgIYIPJLQyb0K2qvC5+ozH71TpKYyizxXaf3tbfftThXZd\nWmh/rdA+H/T4QEIEH0iI4AMJEXwgIYIPJETwgYSKwbe90fYe28/bPmj7M9XyrbZftf1MNV01/3IB\nzMI09/HfknRLRBywvUbSt20/UbVtj4jt8ytvFkrfYulO8EJ9M+PxO/WjQvtFhfZtB+rbY79r233l\n1wt7eLLQvrrQfn6hvXQEllYMfkQckXSkmj9u+7DeeSqh/qgA6KVzusa3PZC0SdI3q0U32T5g+27b\na2dcG4A5mTr41Wn+g5Jujojjku6UdFlEbNLojKDnp/wATpnqWX3bqzQK/f0RsUuSIuLo2Cp3SXp0\n8ivsHZsfVBOA2fqepJenWnPaQTr3SnohInacWmB7Q3X9L0nXSjo0efMtU+4GwPJdVk2n/OvENYvB\nt71Z0sclHbT9rKSQdLuk62xvknRS0lDSjcsvGECbpnlX/xuSzlui6bHZlwOgDYzHL7q4vvnH7VSB\n5Xmz0F68z39lfXvc/Du17V4XhR2U3hMvfQfLwyO7QEIEH0iI4AMJEXwgIYIPJETwgYQIPpAQ9/FL\nVhVGHjf8++qN/zw7GindJS+Nlt+2o779zsLI9T/9s/rtP/+lv6pt/7v3/8XkxqfumNhEjw8kRPCB\nhAg+kFAHwR+2v8tzMuy6gFrDrgsoGHZdQI1h1wUU7P1Be/si+GcZdl1ArWHXBRQMuy6gxrDrAgp+\nyoMPoGsEH0jIEYXxwk13YM93BwAmioglHySYe/AB9A+n+kBCBB9IiOADCRF8ICGCDyT0/xrcLFSj\nXbDVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b21f790>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "h\n",
      "j\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEcdJREFUeJzt3X2QVeV9B/DvN8FoFEqVEbBiXTFNYjV2a1tfBibdFDTE\nl4Ga+hJiB7SxTkeijH8khukMdzOx1fzBiGmc6YAaRG1FZgy+NAkwdknwBW0EK1GjlqyGBFYmMUaa\nZEbKr3/cA1zWe5/f3Xv2nnPg9/3M7HD3/p5zzrOH+91z7z7nOYdmBhGJ5QNld0BEiqfgiwSk4IsE\npOCLBKTgiwSk4IsEVFjwSc4i+QrJV0l+uajttovkIMkXSG4m+WwF+nMXySGS/93w3LEk15L8Mcnv\nkRxfsf4tJrmd5PPZ16wS+zeF5BMkf0TyRZI3ZM9XYh826d8Xs+cL2YcsYhyf5AcAvApgBoCfA3gO\nwJVm9krXN94mktsA/JmZvV12XwCA5HQAuwHca2ZnZs/dBuAXZvb17JfnsWZ2c4X6txjAu2a2pIw+\nNSI5GcBkM9tCciyAHwKYDeBqVGAfJvp3BQrYh0Ud8c8G8JqZvWFm7wH4d9R/yCohKvTRx8w2Ahj+\nS2g2gBXZ4xUA5hTaqQYt+gfU92PpzGynmW3JHu8G8DKAKajIPmzRvxOzctf3YVEv9BMB/LTh++04\n8ENWhQFYR/I5kteW3ZkWJprZEFB/4QCYWHJ/mllAcgvJ5WV+FGlEsgdAL4BnAEyq2j5s6N+m7Kmu\n78PKHOEqYJqZnQXgQgDXZ29lq65q51vfCWCqmfUC2AmgCm/5xwJYDeDG7Mg6fJ+Vug+b9K+QfVhU\n8H8G4A8bvp+SPVcZZrYj+3cXgIdR/3hSNUMkJwH7PyO+VXJ/DmJmu+zAH42WAfiLMvtDcgzqoVpp\nZmuypyuzD5v1r6h9WFTwnwPwEZInk/wQgCsBPFLQtl0kj85+84LkMQAuALC13F4BqH/Wa/y89wiA\n+dnjeQDWDF+gYAf1LwvSPpei/H14N4CXzGxpw3NV2ofv619R+7CQv+oD9eE8AEtR/2Vzl5ndWsiG\n20DyFNSP8gZgDID7y+4fyQcA9AGYAGAIwGIA3wbwEICTALwB4HIz+1WF+vcp1D+r7gUwCOC6fZ+n\nS+jfNADfB/Ai6v+vBmARgGcBrELJ+zDRv7koYB8WFnwRqQ79cU8kIAVfJCAFXyQgBV8koFzBr/rE\nGxFpruO/6rc78Yakhg1ESmJmTc/7H5Njnfsn3gAAyX0Tb5rMuFvc8HgA9eHffY7I0QUA+Otk9be7\nT0/W/2vswd/fDeCahu+9+bk39abrrDm/9+bUnC18eNj36wCc3/D9b53lizaAg/9/q2QA7+9b3tdf\n+gTPmq1P1he/9vUDbe8AajcMa/Baeuv/fOHClrVFvL1lLc9b/UNh4o2INKE/7okElOet/ggm3gw0\nPD4qxya770/L7oBratkdcPSU3YGEnrI7kNR3Tr7ltw38FNsGtrfVNk/w90+8AbAD9Yk3n2vetC/H\nZopV/eCfWnYHHD1ldyChp+wOJOUN/tS+kzC176T93z/Rv6ll246Db2b/R3IBgLU4MPHm5U7XJyLF\n6foknfpw3uJEi99z1vDrZHWrrUzWH+K2ZN37m+44p/5Lp/5JOy9Zn8ELnDUcl7MHkpbv9fdtezpZ\n38y1I+zPyNQS7xK4qfVwnv64JxKQgi8SkIIvEpCCLxKQgi8SkIIvEpCCLxJQnjP3Rkl6nNRz+vT0\nOP1judaef+7bjMefclrUnPqenD2IzjtTw3v9zUxWp7A/WfduDumdpeFd9H/GM4+2LvKSliUd8UUC\nUvBFAlLwRQJS8EUCUvBFAlLwRQJS8EUCKmgcPzWW+p6z7GnpsnMVUm8cPt9sbG+UF+i/+CdOC0/V\nrqJ7qPGuqPBuujwwPVl+tC+9uDdO3+Jadftd6tT/kenrPbSiI75IQAq+SEAKvkhACr5IQAq+SEAK\nvkhACr5IQAWN46c2443jp0fKN3oTlh15d8D0B5wGc1c4DYbfDXc4jePn413PIP36e2f6h5L1bzhr\nz3tXhN/Yn6Qb0OtBczriiwSk4IsEpOCLBKTgiwSk4IsEpOCLBKTgiwSUaxib5CCAdwDsBfCemZ09\nGp06yJQJyfKz2/Ot3rvqumf+5+5MN5g75KxB4/j55L1ufnrG+/Yx6evme2ehePPtvXH+807Z4rS4\nxak3l/f8lb0A+szs7ZzrEZEC5X2rz1FYh4gULG9oDcA6ks+RvHY0OiQi3Zf3rf40M9tB8njUfwG8\nbGYb399sXcPjqQBOzblZEXm/bQDau8ZjruCb2Y7s310kHwZwNoAmwT8/z2ZEpC1Ts699/rNly47f\n6pM8muTY7PExAC4AsLXT9YlIcfIc8ScBeJikZeu538zWjk63RKSbOg6+mf0EQG97rVNjrc44tXPh\n+l9/K133Rsm9+dCeFZf9g9Oi5tS9+eKSlvO6+ds/miyvmpJevMfZ+qBTn20npxuw5qzBuzNEcxqK\nEwlIwRcJSMEXCUjBFwlIwRcJSMEXCUjBFwmooOvq5/A3Tv1b6XLecfzaKel6/+r7nDV4NN8+H2//\npWfM2wvpV0jNWbs327/PqZ/GQaeFN9/euyJAczriiwSk4IsEpOCLBKTgiwSk4IsEpOCLBKTgiwRU\n0Dh+55s57aLnc23Zm63sjePv2jY23YCv5+yBNxIcnXcmhnM9g7G1ZHnZRenr5ud9/TxtC9MNcs+3\n7+z1oyO+SEAKvkhACr5IQAq+SEAKvkhACr5IQAq+SECVn49/BR5M1q3L25/6v9ucFt906pXfxRWX\n74oKt797XbL+M+bbeu0P0nXys84annTq3bleg474IgEp+CIBKfgiASn4IgEp+CIBKfgiASn4IgG5\ng8wk7wJwMYAhMzsze+5YAA8COBn1W4BfbmbvdKODf4t7k/V0FRhy6qc59d2/f7zT4jinfoRT9+Zb\n5z0PwJmv3nVe/7394103/pJk9UL+ebK+ylm7N4rOtc6ZJGfUnDV4///ljePfA+DTw567GcB6M/sY\ngCcAfGW0OyYi3eMG38w2Anh72NOzAazIHq8AMGeU+yUiXdTpZ/yJZjYEAGa2E8DE0euSiHTbaJ1I\n7nzQ+U7D448A+KNR2qyIHDCYffk6Df4QyUlmNkRyMoC30s0/0+FmRKR9PdnXPhtatmz3rT6zr30e\nATA/ezwPwJp2uyYi5XODT/IBAE8B+CjJN0leDeBWAOeT/DGAGdn3InKIcN/qm9ncFqWZ7W+m87HI\nqfft7HhZwB/F9kZRsec3TgPvyurSTV+yx5L1+5359p7a4+l6f+5x+nLuq6Az90QCUvBFAlLwRQJS\n8EUCUvBFAlLwRQJS8EUCKuii76lxfOfK5SvzbXmcU9/k1O0LxyTrfcu+k6xv2DArvYHdTgd6nLp3\nosKgU3/dqW936uemy+PnpM/DuOTIR5P1G3BHsv4mtybr3o93olPn9d6dG25x6t71BMqhI75IQAq+\nSEAKvkhACr5IQAq+SEAKvkhACr5IQDTr7h3mSRqwONHi8uTyP8Dpyfp6Z/ve/c3z3X1dyub9/3lX\ngjjaFiTrX+YEZw3VnG9f1w8za3pFAh3xRQJS8EUCUvBFAlLwRQJS8EUCUvBFAlLwRQIqaD5+wqw/\nTpbXfze9uHd3dW8ctzt3H5d2eaPg3vUUvPMsamek6+T5zhpeduqH5itIR3yRgBR8kYAUfJGAFHyR\ngBR8kYAUfJGAFHyRgNxxfJJ3AbgYwJCZnZk9txjAtQDeypotMjNnxL2FW526s1bvB5jp1M/xhmk9\nv3Dqbzn117pcd36+l5z+veusPq8Bp+5t3xtF578415voqzlr8M40OHzH8e8B8Okmzy8xs7Oyr85C\nLyKlcINvZhsBvN2k1PTKHiJSfXk+4y8guYXkcpLjR61HItJ1nZ6rfyeAr5qZkfwagCUA/q5184GG\nxz3wbwgnIiM3CP9miXUdBd/MdjV8uwxA+s6H6OtkMyIyIj04+KC6oWXLdt/qEw2f6UlObqhdCiB9\ny1IRqZR2hvMeQP2QPYHkm6hfK/tTJHsB7EX9vcV1XeyjiIyy0q+rv9COTC4/notybf9VuytZ/ze+\nmWv95Zvk1I/r8vLOOPbMc5LllevTg0Pe/e1r96TrvDp1Tweg2tfFz0vX1ReRBgq+SEAKvkhACr5I\nQAq+SEAKvkhACr5IQKVfV38R/ilZ/6azfI9T7z/hGqfFEqee9w7s7zn1PTmXH8pZz3tBgrTPrktf\nMOB1Z46ndxYBv+edh1Jz6jHpiC8SkIIvEpCCLxKQgi8SkIIvEpCCLxKQgi8SUOnj+Md/dXeu5eff\nmK5fvfSWXOsvfz72EV1ef96XQE+yuvrcK5P1mrP2q+yoZP1Gems4nOfbd05HfJGAFHyRgBR8kYAU\nfJGAFHyRgBR8kYAUfJGAChrH/0zr0vL+XGteevvfOw28+ezejO9fjqQ7XeD1P69xTj3984//3V8m\n67X0MDymOVufMM673oF3nka399+hSUd8kYAUfJGAFHyRgBR8kYAUfJGAFHyRgBR8kYDccXySUwDc\ni/qN1PcCWGZmd5A8FsCDAE4GMAjgcjN7p+lKelvfI/2+LSPu80EWrvlXp0XNqXvXtT/UefP5vXHy\nE5PVX112QrJec9b+us1LN9B8+65o54i/B8BNZnY6gPMAXE/y4wBuBrDezD4G4AkAX+leN0VkNLnB\nN7OdZrYle7wb9VuvTAEwG8CKrNkKAHO61UkRGV0j+oxPsgdAL4BnAEwysyGg/ssBwMTR7pyIdEfb\n5+qTHAtgNYAbzWw3yeE3LWt9E7MdtQOPx/YB4/pG0EURac9g9uVrK/gkx6Ae+pVmtiZ7eojkJDMb\nIjkZwFstV3BCra3OiEgePTj44qcbWrZs963+3QBeMrOlDc89AmB+9ngegDXDFxKRampnOG8agM8D\neJHkZtTf0i8CcBuAVSSvAfAGgMu72VERGT1u8M3sSQAfbFGe2dZWFrYuvT4/vehp3rpzjyUc7vO1\n8823n2wzkvWac3/7m45M18fzS+kGeNSpe+chSDM6c08kIAVfJCAFXyQgBV8kIAVfJCAFXyQgBV8k\noEKuq3/8vDdbF+enl73isnT9yoduG3F/DnaojwPnnW//iWR1xydOTdZrzto/+bun0w24yllD1e97\ncGjSEV8kIAVfJCAFXyQgBV8kIAVfJCAFXyQgBV8koELG8S/E4x0vu3PV+HQDeuPUh/t11z/s1NM/\n31WWHgf35tvXrkrX+7k53cDt/7tOXTqhI75IQAq+SEAKvkhACr5IQAq+SEAKvkhACr5IQIWM41+E\n/2hZe9VZdha+67Tw6oX8iF2Ub5weR9WS5QVMD9QPOFvnGa1vmViX3r5/nsWhfr2EatIRXyQgBV8k\nIAVfJCAFXyQgBV8kIAVfJCA3+CSnkHyC5I9Ivkjyi9nzi0luJ/l89jWr+90VkdHQziD3HgA3mdkW\nkmMB/JDkuqy2xMyWeCu47LXHWtZqzrIvnHCu08Ibx489Dmwvpcfpa1PTy8+0s9INWHN6cLhfD+HQ\n5AbfzHYC2Jk93k3yZQAnZmXnMg0iUkUj+oxPsgdAL4BN2VMLSG4huZykc6kcEamKtoOfvc1fDeBG\nM9sN4E4AU82sF/V3BO5bfhGphrZOZCc5BvXQrzSzNQBgZrsamiwD8Gir5Wt3HHjcd079S0RG22D2\n5Wt3BsvdAF4ys6X7niA5Ofv8DwCXAtjaauHaDW1uRURy6Mm+9tnQsqUbfJLTAHwewIskNwMwAIsA\nzCXZC2Av6r9mruu0uyJSrHb+qv8kgA82KXnjaCJSUTTz5lPn3ABp9leJBj93ln/la6Pan/d7r8vr\n93jz7Z3zED5eS5YXv+KM45+RXj23/iDdAE869T1Ovez9fzjrh5k1fQHolF2RgBR8kYAUfJGAFHyR\ngBR8kYAUfJGAFHyRgAq56DzvTYxFrz4qvfDCmrP2nOPgpTvCqTv9X54uf2F6us5znfM4ttbSdXf/\na5y+inTEFwlIwRcJSMEXCaj44D/1/cI3OTKDZXfAMVh2Bxz/U3YHEgbL7oBjsLAtFR/8pxX8fAbL\n7oBjW9kdSBgsuwOOwcK2pLf6IgEp+CIBFTIfv6sbEJGWWs3H73rwRaR69FZfJCAFXyQgBV8kIAVf\nJCAFXySg/weCB0XHAiyn0wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b041ed0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD7CAYAAABKWyniAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAEYRJREFUeJzt3X2MXNV5x/HfA05lgoECqdcVpiykIVStkEkKpTFVNg2h\nBNxASeUQSIXTBFEaiCuoBEFIHrepBFGwQiBECgbqkNAAlqh5Ca9xl9S0vNupDSY0hSWF4OXFBHBU\nJMBP/5hrPN7snGd2z8zcMef7kUYez3Pn3jN357d3Zs8955q7C0BZdqm7AQD6j+ADBSL4QIEIPlAg\ngg8UiOADBepb8M3sWDN7wsyeNLPz+rXdTpnZmJn9xMzWmtmDA9Ceq8xs3Mz+q+Wxvc3sLjP7qZnd\naWZ7DVj7lpjZs2b2aHU7tsb2zTWz1Wb2mJmtN7MvV48PxD6cpH1nV4/3ZR9aP/rxzWwXSU9K+rik\nX0h6SNLJ7v5EzzfeITN7StKH3f2VutsiSWZ2lKQtkr7r7odWj10s6WV3/1r1y3Nvdz9/gNq3RNLr\n7r6sjja1MrM5kua4+zozmyXpEUknSPq8BmAfJtr3GfVhH/briH+EpP9292fc/U1JP1DzRQ4S0wB9\n9XH3NZIm/hI6QdKK6v4KSSf2tVEt2rRPau7H2rn7JndfV93fImmjpLkakH3Ypn37VeWe78N+vdH3\nk/S/Lf9/Vttf5KBwSXeb2UNmdnrdjWljtruPS803jqTZNbdnMmeZ2TozW17nV5FWZjYsaZ6k+yUN\nDdo+bGnfA9VDPd+HA3OEGwDz3f1Dko6T9KXqo+ygG7Tzra+QdJC7z5O0SdIgfOSfJWmlpMXVkXXi\nPqt1H07Svr7sw34F/zlJv9Py/7nVYwPD3Z+v/n1R0k1qfj0ZNONmNiS98x3xhZrbswN3f9G3/9Ho\nSkmH19keM5uhZqiudfdV1cMDsw8na1+/9mG/gv+QpN81swPM7DcknSzp5j5tO2Rm761+88rMdpd0\njKQN9bZKUvO7Xuv3vZslLarunyZp1cQn9NkO7auCtM1Jqn8fXi3pcXe/tOWxQdqHv9a+fu3DvvxV\nX2p250m6VM1fNle5+0V92XAHzOxANY/yLmmGpO/X3T4zu07SiKR9JY1LWiLpXyXdKGl/Sc9IWuju\nvxyg9n1Mze+qWyWNSTpj2/fpGto3X9KPJa1X8+fqki6Q9KCkG1TzPky07xT1YR/2LfgABgd/3AMK\nRPCBAhF8oEAEHyhQVvAHfeANgMlN+6/6nQ68MTO6DYCauPuk5/3PyFjnOwNvJMnMtg28mWTE3ZKW\n+6Nqdv8OqlHRvhyj2rF9uwXLvxXUg7foyvQHzZc/vX37Fzfe1HmN9+xQ3+f0N9Lr35gu6y/S5WXn\nnpmsn2vfavlfo7q1Who0ILV/2w86zPmovzMMvAEwCf64BxQo56P+FAbejLbcn5mxyX4YrrsBgeG6\nGxAYrrsBbc0fGfTj3Ejm8/9H0lMdLZkT/HcG3kh6Xs2BN5+dfNGRjM3023DdDQgM192AwHDdDWjr\nqJFd625CYCTz+e+vbtv8qO2S0w6+u79tZmdJukvbB95EfwoBMAB6Pkin2Z23JF4QO6n3BPU3g/o5\nyaqvTU9A0zgsWH2P5b76s4MPIe/7pyCf5zcSxaVtu/MG/UsPgB4g+ECBCD5QIIIPFIjgAwUi+ECB\nCD5QoJwz91CE3J7qTyar/rm8fvo9g61HotZnjh3UPkH9srfTdb8pfTUt05OJavuRfRzxgQIRfKBA\nBB8oEMEHCkTwgQIRfKBABB8oEOPxEYhmyf2/ZPUW//dk/WFbnawPBVuv5VK8XRSdh/BaUF/6cCK/\nf2iMxwewHcEHCkTwgQIRfKBABB8oEMEHCkTwgQIxHr940Xj7dD+99MVkdcFQ+mqvDwdrj/qxd3bR\n3g1FEwa0wREfKBDBBwpE8IECEXygQAQfKBDBBwpE8IECZfXjm9mYpFclbZX0prsf0Y1GoZ+i8fbB\nzPO3zk2WGwum1pqJptlNPTByr0pwalBf+ndTaEyL3BN4tkoacfdXMtcDoI9yP+pbF9YBoM9yQ+uS\n7jazh8zs9G40CEDv5X7Un+/uz5vZb6n5C2Cju6/59cVGW+4PVzcAXfXqqPTaaEeLZgXf3Z+v/n3R\nzG6SdISkSYI/krMZAJ3Ya6R52+a5Hlw008zea2azqvu7SzpG0obprg9A/+Qc8Yck3dScPlszJH3f\n3e/qTrMA9NK0g+/uT0ua18W2oBZ5f+b5wvGXZz0/un785qy11y/zLAl9ID3dgbT84im0Zju64oAC\nEXygQAQfKBDBBwpE8IECEXygQAQfKBDz6hcvr6d8+QtnJ+uN4Pk7+3j7Xtt85cz0AsunNzM/R3yg\nQAQfKBDBBwpE8IECEXygQAQfKBDBBwpEP/67Xu7M7iPp8pHtp3fqRPb14QdctHcjH9WPgyVum9Z6\nOeIDBSL4QIEIPlAggg8UiOADBSL4QIEIPlAg+vGRNmskWV72dN7qozdgbj943XLnG9jwJ4cHS9CP\nD6BDBB8oEMEHCkTwgQIRfKBABB8oEMEHChT245vZVZIWSBp390Orx/aWdL2kAySNSVro7q/2sJ07\nsdzx8Lky17+lkSyfu9KTdf+yJeuNX6Q3H11fvu7x/Lk/3cb+6frSNY9PpTkd6+SIf42kP5vw2PmS\n7nH3D0paLekr3W4YgN4Jg+/uayS9MuHhEyStqO6vkHRil9sFoIem+x1/truPS5K7b5I0u3tNAtBr\n3TpXP/1FT6Mt94erG4DuGqtusekGf9zMhtx93MzmSHohvfjINDcDoHPD2vGgem/bJTv9qG/VbZub\nJS2q7p8maVWnTQNQvzD4ZnadpP+QdLCZ/dzMPi/pIkmfMLOfSvp49X8AO4nwo767n9KmdHSX29Ij\nUU9r1FMceS2oRz25Ufv2COqvZ24/ErTvLxvJss0I+vk/GvTzt/+0Kqn+fv5o++HeD16fDrohWCB6\n/0yOM/eAAhF8oEAEHygQwQcKRPCBAhF8oEAEHyjQu2Be/V73w+fKHbG9OXP9uduP6kPp8luNZNl2\nS/fzr1S6n399eus9nw0hN0DnHdgIlgiGwWifaW2XIz5QIIIPFIjgAwUi+ECBCD5QIIIPFIjgAwXa\nCfrxo3766ArkQU/tRY1k+UfnfSRZ/9Nv/Wd6/VFH83Hp8hmf+kay/h37dLCB5UE9t6d7PKjvmS7f\n0UiWn/Ez08+3byfLuePlo70TnWUR+dqqJcESjaA+vRkHOOIDBSL4QIEIPlAggg8UiOADBSL4QIEI\nPlAgc4/G+2ZuwMylVF9l7szojWTV7wzmbZ94AfAJej2eO/v66oek6zYz+PmuawRbyP35BP340XwI\nwbz9165M/3x/Fmx9erPSbxf9fE4N6gfrpWCJy4J66hVcKHefdAdxxAcKRPCBAhF8oEAEHygQwQcK\nRPCBAhF8oEDheHwzu0rSAknj7n5o9dgSSadLeqFa7AJ3v6P9WlJ9wUE/8G82kmUfzuunj3qZox2U\ne/31qB95dlBvPJGu+23p/WPHfzXYQvQKo1eQuYdWrkmWPxfMZ9D4YbqeOyFF1I//gXODBS7J6afv\npAWT6+SIf42kyeKzzN0/VN0SoQcwaMLgu/saSa9MUkofSgAMrJzv+GeZ2TozW25me3WtRQB6brpf\nca6Q9A/u7mb2VUnLJH2h/eJ3t9w/SNL7p7lZAO2NVbfYtILv7i+2/PdKSbekn/GJ6WwGwJQMV7dt\n7m27ZKcf9U0t3+nNbE5L7SRJGzpuG4DaddKdd52kEUn7mtnP1Rxj+zEzmydpq5qfLc7oYRsBdFkY\nfHc/ZZKHr5naZqbfl+v/FvTTH5Z+/n7B+l8I6rnj7SPRnglGq4euPz5aYrIfb6sVmS3I9WC6fGDe\n2nPH40c/vxu/viC9wCXRFqKI9q4fH8C7DMEHCkTwgQIRfKBABB8oEMEHCkTwgQLlDkfOd0gjWW4c\ntjRZj2Z9f25qrRk4uaOxj941WODt6EyGus1PlzfmrT3af9H7K7LwJ8HZ7MF1IXo13wFHfKBABB8o\nEMEHCkTwgQIRfKBABB8oEMEHClR7P/5BGx9LL1D4XL5RP3LUD71v+9mXmo66fQqtmY7MV3DUHyXL\nt6+eWmumanNQ/71oBV/sUkO6jCM+UCCCDxSI4AMFIvhAgQg+UCCCDxSI4AMFqr0ff+3b6YnxlwXP\n7/HV2WsX9YJH8+7//fx/DJZ4K6jvE9Sjnu49gnraLit/law/MCdZDlsfvfro/fOZoJ/+5OU3B2uI\n9OYdzBEfKBDBBwpE8IECEXygQAQfKBDBBwpE8IEChf34ZjZX0nclDUnaKulKd/+mme0t6XpJB0ga\nk7TQ3V+dfC1Dbde/56JeX4F+sEXnIUS95JFL7rwwWKKRuYVIXj/0/UNHJuu3Bc+P+ulzT2R54xvB\nAssfDRbIPU9iejo54r8l6Rx3/31JfyzpS2Z2iKTzJd3j7h+UtFrSV3rSQgBdFwbf3Te5+7rq/hY1\nr10yV9IJklZUi62QdGKvGgmgu6b0Hd/MhiXNk3S/pCF3H5eavxwkze524wD0RsdfccxslqSVkha7\n+xYz8wmLTPx/i1tb7h9c3QB011h1i3UUfDOboWbor3X3VdXD42Y25O7jZjZHUuLqiws6agyAHMPV\nbZv2M612+lH/akmPu/ulLY/dLGlRdf80SasmPgnAYOqkO2++pFMlrTeztWp+pL9A0sWSbjCzv5b0\njKSFvWwogO4Jg+/u90lqd5X1ozvayslnti2t+d7fdrSKdnb28fbRDyB6fdG07Uuzv2VFPeF7BvW8\nK9AfftSGZD3qx4+2nnsWyW5jiT9tSYrPk4j2b29w5h5QIIIPFIjgAwUi+ECBCD5QIIIPFIjgAwXq\nz7z6F7XvLb3nB+mnRvPK7+z9+Lm9uPdEC/xNUL88swFhT3j6J7TXG+kGNmaeP8X27Cg6y2A8qDeC\n8fZL/yC68kMkujJCb3DEBwpE8IECEXygQAQfKBDBBwpE8IECEXygQP3px382mj2+veiZO3s/ft5o\n9XiGNR+3ZN3+ORhPvijYwLx0+cK1FyTrH7HfTtYfCDbf/ooNTWE//f7pun09d7z9YJ6JwhEfKBDB\nBwpE8IECEXygQAQfKBDBBwpE8IECmXvUT5m5ATOX7mpb988ek3x+41/S698v2H7iul6S8udV39lF\n50lEF07YGNTHgnrurPxRL3hjcbpuK4L3/y8bwRYGs5++aancfdITOTjiAwUi+ECBCD5QIIIPFIjg\nAwUi+ECBwuCb2VwzW21mj5nZejM7u3p8iZk9a2aPVrdje99cAN3QyXj8tySd4+7rzGyWpEfM7O6q\ntszdO5hY/L62Fds93Y/60q7p8eSXvZ3ectRPHfXC9no+gNzzCKL27ZG5/tsznx+1L5pV/pNBfYVf\nkqybHRCsoRHUB7mffvrC4Lv7JkmbqvtbzGyjtp83k04lgIE0pe/4Zjas5pwr2yZGOcvM1pnZcjPb\nq8ttA9AjHQe/+pi/UtJid98i6QpJB7n7PDU/EeReSwhAn3Q0556ZzVAz9Ne6+ypJcvcXWxa5UtIt\n7dcw2nJ/uLoB6K4xxaMjmjqdbPNqSY+7+6XbHjCzOdX3f0k6SdKG9k8f6XAzAKZvWDseVO9tu2QY\nfDObL+lUSevNbK0kl3SBpFPMbJ6krWr+mjljus0F0F+d/FX/Pkm7TlK6o/vNAdAP/ZlXP9UXuryR\nfOb7lO7nf9b3TdYfsc3J+qPJ6uD30kbnAaRffb5oXvszF6Xrf3XNd5L1I+1T6RXYt4MW/Cyovzv7\n6SOcsgsUiOADBSL4QIEIPlAggg8UiOADBSL4QIH6NK/+ksQSPe5HndVIlhe8fmOyfuOvFibrM88K\ntv/DdPmBYOL/6ESLDx8YLHBcurz58pnJ+mJ9M1n/3p+fnt7Arenn559pEM3MH71/3s1XVmBefQAt\nCD5QIIIPFKiG4I/1f5NT8NLoY3U3IenhuhsQeWm07hYkPFV3AwJjfdsSwZ/g5QEP/iN1NyDy8mjd\nLUh4uu4GBMb6tiU+6gMFIvhAgfrUjw+gDu368XsefACDh4/6QIEIPlAggg8UiOADBSL4QIH+H4lI\nRemmA+UKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11b24df10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import string\n",
    "aa = string.ascii_lowercase\n",
    "\n",
    "for i in range(5):\n",
    "    plt.matshow(train_dataset[i,:,:])\n",
    "    print(aa[train_labels[i]])\n",
    "    plt.matshow(valid_dataset[i,:,:])\n",
    "    print(aa[valid_labels[i]])\n",
    "\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tIQJaJuwg5Hw"
   },
   "source": [
    "Finally, let's save the data for later reuse:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "collapsed": true,
    "id": "QiR_rETzem6C"
   },
   "outputs": [],
   "source": [
    "pickle_file = 'notMNIST.pickle'\n",
    "\n",
    "try:\n",
    "  f = open(pickle_file, 'wb')\n",
    "  save = {\n",
    "    'train_dataset': train_dataset,\n",
    "    'train_labels': train_labels,\n",
    "    'valid_dataset': valid_dataset,\n",
    "    'valid_labels': valid_labels,\n",
    "    'test_dataset': test_dataset,\n",
    "    'test_labels': test_labels,\n",
    "    }\n",
    "  pickle.dump(save, f, pickle.HIGHEST_PROTOCOL)\n",
    "  f.close()\n",
    "except Exception as e:\n",
    "  print('Unable to save data to', pickle_file, ':', e)\n",
    "  raise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "cellView": "both",
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "collapsed": false,
    "executionInfo": {
     "elapsed": 413065,
     "status": "ok",
     "timestamp": 1444485899688,
     "user": {
      "color": "#1FA15D",
      "displayName": "Vincent Vanhoucke",
      "isAnonymous": false,
      "isMe": true,
      "permissionId": "05076109866853157986",
      "photoUrl": "//lh6.googleusercontent.com/-cCJa7dTDcgQ/AAAAAAAAAAI/AAAAAAAACgw/r2EZ_8oYer4/s50-c-k-no/photo.jpg",
      "sessionId": "2a0a5e044bb03b66",
      "userId": "102167687554210253930"
     },
     "user_tz": 420
    },
    "id": "hQbLjrW_iT39",
    "outputId": "b440efc6-5ee1-4cbc-d02d-93db44ebd956"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Compressed pickle size: 718193801\n"
     ]
    }
   ],
   "source": [
    "statinfo = os.stat(pickle_file)\n",
    "print('Compressed pickle size:', statinfo.st_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gE_cRAQB33lk"
   },
   "source": [
    "---\n",
    "Problem 5\n",
    "---------\n",
    "\n",
    "By construction, this dataset might contain a lot of overlapping samples, including training data that's also contained in the validation and test set! Overlap between training and test can skew the results if you expect to use your model in an environment where there is never an overlap, but are actually ok if you expect to see training samples recur when you use it.\n",
    "Measure how much overlap there is between training, validation and test samples.\n",
    "\n",
    "Optional questions:\n",
    "- What about near duplicates between datasets? (images that are almost identical)\n",
    "- Create a sanitized validation and test set, and compare your accuracy on those in subsequent assignments.\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Answer:\n",
    "\n",
    "This is not correct since we should count how many overlaps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "size of train: 200000\n",
      "percent overlap for unique elements with valid: 0.509035\n",
      "percent overlap for unique elements with test: 0.615863\n",
      "size of valid: 10000\n",
      "percent overlap for unique elements with train: 9.653566\n",
      "percent overlap for unique elements with test: 0.557131\n",
      "size of test: 10000\n",
      "percent overlap for unique elements with train: 11.789366\n",
      "percent overlap for unique elements with valid: 0.562372\n"
     ]
    }
   ],
   "source": [
    "\n",
    "def Unique(dataset):\n",
    "    s = set()\n",
    "    for i in range(dataset.shape[0]):\n",
    "        s.add(dataset[i,:,:].tostring())\n",
    "    return dataset, s\n",
    "\n",
    "sets = [Unique(train_dataset), Unique(valid_dataset), Unique(test_dataset)]\n",
    "names = [\"train\", \"valid\", \"test\"]\n",
    "for i in range(3):\n",
    "    print(\"size of %s: %d\" % (names[i], sets[i][0].shape[0]))\n",
    "    for j in range(3):\n",
    "        if i == j:\n",
    "            continue\n",
    "        print(\"percent overlap for unique elements with %s: %f\" % (names[j], 100*len(sets[i][1].intersection(sets[j][1]))/float(len(sets[i][1]))))\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Todo\n",
    "Create sanitized version of datasets!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "L8oww1s4JMQx"
   },
   "source": [
    "---\n",
    "Problem 6\n",
    "---------\n",
    "\n",
    "Let's get an idea of what an off-the-shelf classifier can give you on this data. It's always good to check that there is something to learn, and that it's a problem that is not so trivial that a canned solution solves it.\n",
    "\n",
    "Train a simple model on this data using 50, 100, 1000 and 5000 training samples. Hint: you can use the LogisticRegression model from sklearn.linear_model.\n",
    "\n",
    "Optional question: train an off-the-shelf model on all the data!\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Samples: 50\n",
      "Accuracy for train: 1.000000\n",
      "Accuracy for test: 0.509000\n",
      "\n",
      "Samples: 100\n",
      "Accuracy for train: 1.000000\n",
      "Accuracy for test: 0.696600\n",
      "\n",
      "Samples: 1000\n",
      "Accuracy for train: 0.997000\n",
      "Accuracy for test: 0.833300\n",
      "\n",
      "Samples: 5000\n",
      "Accuracy for train: 0.949200\n",
      "Accuracy for test: 0.851100\n",
      "\n"
     ]
    }
   ],
   "source": [
    "lr = LogisticRegression()\n",
    "\n",
    "s = train_dataset.shape\n",
    "train_d = train_dataset.reshape((s[0], s[1]*s[2]))\n",
    "\n",
    "s = test_dataset.shape\n",
    "test_d = test_dataset.reshape((s[0], s[1]*s[2]))\n",
    "\n",
    "\n",
    "for n in (50,100,1000,5000):\n",
    "    lr.fit(train_d[:n], train_labels[:n])\n",
    "    train_score = lr.score(train_d[:n], train_labels[:n])\n",
    "    test_score = lr.score(test_d, test_labels)\n",
    "    \n",
    "    print(\"Samples: %d\\nAccuracy for train: %f\\nAccuracy for test: %f\\n\" % (n, train_score, test_score))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "default_view": {},
   "name": "1_notmnist.ipynb",
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}