zh-nbs/Lesson6_rossmann.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Practical Deep Learning for Coders, v3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson6_rossmann"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fastai.tabular import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rossmann\n",
    "# 连锁店销量预估"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data preparation 数据准备"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create the feature-engineered train_clean and test_clean from the Kaggle competition data, run `rossman_data_clean.ipynb`. One important step that deals with time series is this:<br><br>\n",
    "为了从Kaggle竞赛数据集中创建经过特征工程处理的train_clean和test_clean，运行`rossman_data_clean.ipynb`，其中一个处理时间序列数据的重要步骤如下：\n",
    "\n",
    "```python\n",
    "add_datepart(train, \"Date\", drop=False)\n",
    "add_datepart(test, \"Date\", drop=False)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path = Config().data_path()/'rossmann'\n",
    "train_df = pd.read_pickle(path/'train_clean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Store</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DayOfWeek</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Date</th>\n",
       "      <td>2015-07-31</td>\n",
       "      <td>2015-07-31</td>\n",
       "      <td>2015-07-31</td>\n",
       "      <td>2015-07-31</td>\n",
       "      <td>2015-07-31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Sales</th>\n",
       "      <td>5263</td>\n",
       "      <td>6064</td>\n",
       "      <td>8314</td>\n",
       "      <td>13995</td>\n",
       "      <td>4822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Customers</th>\n",
       "      <td>555</td>\n",
       "      <td>625</td>\n",
       "      <td>821</td>\n",
       "      <td>1498</td>\n",
       "      <td>559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Open</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Promo</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>StateHoliday</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SchoolHoliday</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Year</th>\n",
       "      <td>2015</td>\n",
       "      <td>2015</td>\n",
       "      <td>2015</td>\n",
       "      <td>2015</td>\n",
       "      <td>2015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Month</th>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Week</th>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Day</th>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "      <td>31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dayofweek</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Dayofyear</th>\n",
       "      <td>212</td>\n",
       "      <td>212</td>\n",
       "      <td>212</td>\n",
       "      <td>212</td>\n",
       "      <td>212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Is_month_end</th>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Is_month_start</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Is_quarter_end</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Is_quarter_start</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Is_year_end</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Is_year_start</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Elapsed</th>\n",
       "      <td>1438300800</td>\n",
       "      <td>1438300800</td>\n",
       "      <td>1438300800</td>\n",
       "      <td>1438300800</td>\n",
       "      <td>1438300800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>StoreType</th>\n",
       "      <td>c</td>\n",
       "      <td>a</td>\n",
       "      <td>a</td>\n",
       "      <td>c</td>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Assortment</th>\n",
       "      <td>a</td>\n",
       "      <td>a</td>\n",
       "      <td>a</td>\n",
       "      <td>c</td>\n",
       "      <td>a</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CompetitionDistance</th>\n",
       "      <td>1270</td>\n",
       "      <td>570</td>\n",
       "      <td>14130</td>\n",
       "      <td>620</td>\n",
       "      <td>29910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CompetitionOpenSinceMonth</th>\n",
       "      <td>9</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CompetitionOpenSinceYear</th>\n",
       "      <td>2008</td>\n",
       "      <td>2007</td>\n",
       "      <td>2006</td>\n",
       "      <td>2009</td>\n",
       "      <td>2015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Promo2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Promo2SinceWeek</th>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>14</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Min_Sea_Level_PressurehPa</th>\n",
       "      <td>1015</td>\n",
       "      <td>1017</td>\n",
       "      <td>1017</td>\n",
       "      <td>1014</td>\n",
       "      <td>1016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Max_VisibilityKm</th>\n",
       "      <td>31</td>\n",
       "      <td>10</td>\n",
       "      <td>31</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mean_VisibilityKm</th>\n",
       "      <td>15</td>\n",
       "      <td>10</td>\n",
       "      <td>14</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Min_VisibilitykM</th>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Max_Wind_SpeedKm_h</th>\n",
       "      <td>24</td>\n",
       "      <td>14</td>\n",
       "      <td>14</td>\n",
       "      <td>23</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mean_Wind_SpeedKm_h</th>\n",
       "      <td>11</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>16</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Max_Gust_SpeedKm_h</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Precipitationmm</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CloudCover</th>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Events</th>\n",
       "      <td>Fog</td>\n",
       "      <td>Fog</td>\n",
       "      <td>Fog</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>WindDirDegrees</th>\n",
       "      <td>13</td>\n",
       "      <td>309</td>\n",
       "      <td>354</td>\n",
       "      <td>282</td>\n",
       "      <td>290</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>StateName</th>\n",
       "      <td>Hessen</td>\n",
       "      <td>Thueringen</td>\n",
       "      <td>NordrheinWestfalen</td>\n",
       "      <td>Berlin</td>\n",
       "      <td>Sachsen</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CompetitionOpenSince</th>\n",
       "      <td>2008-09-15</td>\n",
       "      <td>2007-11-15</td>\n",
       "      <td>2006-12-15</td>\n",
       "      <td>2009-09-15</td>\n",
       "      <td>2015-04-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CompetitionDaysOpen</th>\n",
       "      <td>2510</td>\n",
       "      <td>2815</td>\n",
       "      <td>3150</td>\n",
       "      <td>2145</td>\n",
       "      <td>107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CompetitionMonthsOpen</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Promo2Since</th>\n",
       "      <td>1900-01-01</td>\n",
       "      <td>2010-03-29</td>\n",
       "      <td>2011-04-04</td>\n",
       "      <td>1900-01-01</td>\n",
       "      <td>1900-01-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Promo2Days</th>\n",
       "      <td>0</td>\n",
       "      <td>1950</td>\n",
       "      <td>1579</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Promo2Weeks</th>\n",
       "      <td>0</td>\n",
       "      <td>25</td>\n",
       "      <td>25</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AfterSchoolHoliday</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BeforeSchoolHoliday</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AfterStateHoliday</th>\n",
       "      <td>57</td>\n",
       "      <td>67</td>\n",
       "      <td>57</td>\n",
       "      <td>67</td>\n",
       "      <td>57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BeforeStateHoliday</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AfterPromo</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>BeforePromo</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SchoolHoliday_bw</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>StateHoliday_bw</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Promo_bw</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>SchoolHoliday_fw</th>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>StateHoliday_fw</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Promo_fw</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>93 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                    0           1                   2  \\\n",
       "index                               0           1                   2   \n",
       "Store                               1           2                   3   \n",
       "DayOfWeek                           5           5                   5   \n",
       "Date                       2015-07-31  2015-07-31          2015-07-31   \n",
       "Sales                            5263        6064                8314   \n",
       "Customers                         555         625                 821   \n",
       "Open                                1           1                   1   \n",
       "Promo                               1           1                   1   \n",
       "StateHoliday                    False       False               False   \n",
       "SchoolHoliday                       1           1                   1   \n",
       "Year                             2015        2015                2015   \n",
       "Month                               7           7                   7   \n",
       "Week                               31          31                  31   \n",
       "Day                                31          31                  31   \n",
       "Dayofweek                           4           4                   4   \n",
       "Dayofyear                         212         212                 212   \n",
       "Is_month_end                     True        True                True   \n",
       "Is_month_start                  False       False               False   \n",
       "Is_quarter_end                  False       False               False   \n",
       "Is_quarter_start                False       False               False   \n",
       "Is_year_end                     False       False               False   \n",
       "Is_year_start                   False       False               False   \n",
       "Elapsed                    1438300800  1438300800          1438300800   \n",
       "StoreType                           c           a                   a   \n",
       "Assortment                          a           a                   a   \n",
       "CompetitionDistance              1270         570               14130   \n",
       "CompetitionOpenSinceMonth           9          11                  12   \n",
       "CompetitionOpenSinceYear         2008        2007                2006   \n",
       "Promo2                              0           1                   1   \n",
       "Promo2SinceWeek                     1          13                  14   \n",
       "...                               ...         ...                 ...   \n",
       "Min_Sea_Level_PressurehPa        1015        1017                1017   \n",
       "Max_VisibilityKm                   31          10                  31   \n",
       "Mean_VisibilityKm                  15          10                  14   \n",
       "Min_VisibilitykM                   10          10                  10   \n",
       "Max_Wind_SpeedKm_h                 24          14                  14   \n",
       "Mean_Wind_SpeedKm_h                11          11                   5   \n",
       "Max_Gust_SpeedKm_h                NaN         NaN                 NaN   \n",
       "Precipitationmm                     0           0                   0   \n",
       "CloudCover                          1           4                   2   \n",
       "Events                            Fog         Fog                 Fog   \n",
       "WindDirDegrees                     13         309                 354   \n",
       "StateName                      Hessen  Thueringen  NordrheinWestfalen   \n",
       "CompetitionOpenSince       2008-09-15  2007-11-15          2006-12-15   \n",
       "CompetitionDaysOpen              2510        2815                3150   \n",
       "CompetitionMonthsOpen              24          24                  24   \n",
       "Promo2Since                1900-01-01  2010-03-29          2011-04-04   \n",
       "Promo2Days                          0        1950                1579   \n",
       "Promo2Weeks                         0          25                  25   \n",
       "AfterSchoolHoliday                  0           0                   0   \n",
       "BeforeSchoolHoliday                 0           0                   0   \n",
       "AfterStateHoliday                  57          67                  57   \n",
       "BeforeStateHoliday                  0           0                   0   \n",
       "AfterPromo                          0           0                   0   \n",
       "BeforePromo                         0           0                   0   \n",
       "SchoolHoliday_bw                    5           5                   5   \n",
       "StateHoliday_bw                     0           0                   0   \n",
       "Promo_bw                            5           5                   5   \n",
       "SchoolHoliday_fw                    7           1                   5   \n",
       "StateHoliday_fw                     0           0                   0   \n",
       "Promo_fw                            5           1                   5   \n",
       "\n",
       "                                    3           4  \n",
       "index                               3           4  \n",
       "Store                               4           5  \n",
       "DayOfWeek                           5           5  \n",
       "Date                       2015-07-31  2015-07-31  \n",
       "Sales                           13995        4822  \n",
       "Customers                        1498         559  \n",
       "Open                                1           1  \n",
       "Promo                               1           1  \n",
       "StateHoliday                    False       False  \n",
       "SchoolHoliday                       1           1  \n",
       "Year                             2015        2015  \n",
       "Month                               7           7  \n",
       "Week                               31          31  \n",
       "Day                                31          31  \n",
       "Dayofweek                           4           4  \n",
       "Dayofyear                         212         212  \n",
       "Is_month_end                     True        True  \n",
       "Is_month_start                  False       False  \n",
       "Is_quarter_end                  False       False  \n",
       "Is_quarter_start                False       False  \n",
       "Is_year_end                     False       False  \n",
       "Is_year_start                   False       False  \n",
       "Elapsed                    1438300800  1438300800  \n",
       "StoreType                           c           a  \n",
       "Assortment                          c           a  \n",
       "CompetitionDistance               620       29910  \n",
       "CompetitionOpenSinceMonth           9           4  \n",
       "CompetitionOpenSinceYear         2009        2015  \n",
       "Promo2                              0           0  \n",
       "Promo2SinceWeek                     1           1  \n",
       "...                               ...         ...  \n",
       "Min_Sea_Level_PressurehPa        1014        1016  \n",
       "Max_VisibilityKm                   10          10  \n",
       "Mean_VisibilityKm                  10          10  \n",
       "Min_VisibilitykM                   10          10  \n",
       "Max_Wind_SpeedKm_h                 23          14  \n",
       "Mean_Wind_SpeedKm_h                16          11  \n",
       "Max_Gust_SpeedKm_h                NaN         NaN  \n",
       "Precipitationmm                     0           0  \n",
       "CloudCover                          6           4  \n",
       "Events                            NaN         NaN  \n",
       "WindDirDegrees                    282         290  \n",
       "StateName                      Berlin     Sachsen  \n",
       "CompetitionOpenSince       2009-09-15  2015-04-15  \n",
       "CompetitionDaysOpen              2145         107  \n",
       "CompetitionMonthsOpen              24           3  \n",
       "Promo2Since                1900-01-01  1900-01-01  \n",
       "Promo2Days                          0           0  \n",
       "Promo2Weeks                         0           0  \n",
       "AfterSchoolHoliday                  0           0  \n",
       "BeforeSchoolHoliday                 0           0  \n",
       "AfterStateHoliday                  67          57  \n",
       "BeforeStateHoliday                  0           0  \n",
       "AfterPromo                          0           0  \n",
       "BeforePromo                         0           0  \n",
       "SchoolHoliday_bw                    5           5  \n",
       "StateHoliday_bw                     0           0  \n",
       "Promo_bw                            5           5  \n",
       "SchoolHoliday_fw                    1           1  \n",
       "StateHoliday_fw                     0           0  \n",
       "Promo_fw                            1           1  \n",
       "\n",
       "[93 rows x 5 columns]"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.head().T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "844338"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n = len(train_df); n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Experimenting with a sample 样例数据实验"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "idx = np.random.permutation(range(n))[:2000]\n",
    "idx.sort()\n",
    "small_train_df = train_df.iloc[idx[:1000]]\n",
    "small_test_df = train_df.iloc[idx[1000:]]\n",
    "small_cont_vars = ['CompetitionDistance', 'Mean_Humidity']\n",
    "small_cat_vars =  ['Store', 'DayOfWeek', 'PromoInterval']\n",
    "small_train_df = small_train_df[small_cat_vars + small_cont_vars + ['Sales']]\n",
    "small_test_df = small_test_df[small_cat_vars + small_cont_vars + ['Sales']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Store</th>\n",
       "      <th>DayOfWeek</th>\n",
       "      <th>PromoInterval</th>\n",
       "      <th>CompetitionDistance</th>\n",
       "      <th>Mean_Humidity</th>\n",
       "      <th>Sales</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>267</th>\n",
       "      <td>268</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4520.0</td>\n",
       "      <td>67</td>\n",
       "      <td>7492</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>604</th>\n",
       "      <td>606</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2260.0</td>\n",
       "      <td>61</td>\n",
       "      <td>7187</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>983</th>\n",
       "      <td>986</td>\n",
       "      <td>5</td>\n",
       "      <td>Feb,May,Aug,Nov</td>\n",
       "      <td>620.0</td>\n",
       "      <td>61</td>\n",
       "      <td>7051</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1636</th>\n",
       "      <td>525</td>\n",
       "      <td>4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1870.0</td>\n",
       "      <td>55</td>\n",
       "      <td>9673</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2348</th>\n",
       "      <td>123</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16760.0</td>\n",
       "      <td>50</td>\n",
       "      <td>10007</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Store  DayOfWeek    PromoInterval  CompetitionDistance  Mean_Humidity  \\\n",
       "267     268          5              NaN               4520.0             67   \n",
       "604     606          5              NaN               2260.0             61   \n",
       "983     986          5  Feb,May,Aug,Nov                620.0             61   \n",
       "1636    525          4              NaN               1870.0             55   \n",
       "2348    123          3              NaN              16760.0             50   \n",
       "\n",
       "      Sales  \n",
       "267    7492  \n",
       "604    7187  \n",
       "983    7051  \n",
       "1636   9673  \n",
       "2348  10007  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "small_train_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Store</th>\n",
       "      <th>DayOfWeek</th>\n",
       "      <th>PromoInterval</th>\n",
       "      <th>CompetitionDistance</th>\n",
       "      <th>Mean_Humidity</th>\n",
       "      <th>Sales</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>420510</th>\n",
       "      <td>829</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>110.0</td>\n",
       "      <td>55</td>\n",
       "      <td>6802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>420654</th>\n",
       "      <td>973</td>\n",
       "      <td>3</td>\n",
       "      <td>Jan,Apr,Jul,Oct</td>\n",
       "      <td>330.0</td>\n",
       "      <td>59</td>\n",
       "      <td>6644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>420990</th>\n",
       "      <td>194</td>\n",
       "      <td>2</td>\n",
       "      <td>Feb,May,Aug,Nov</td>\n",
       "      <td>16970.0</td>\n",
       "      <td>55</td>\n",
       "      <td>4720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>421308</th>\n",
       "      <td>512</td>\n",
       "      <td>2</td>\n",
       "      <td>Mar,Jun,Sept,Dec</td>\n",
       "      <td>590.0</td>\n",
       "      <td>72</td>\n",
       "      <td>6248</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>421824</th>\n",
       "      <td>1029</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1590.0</td>\n",
       "      <td>64</td>\n",
       "      <td>8004</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        Store  DayOfWeek     PromoInterval  CompetitionDistance  \\\n",
       "420510    829          3               NaN                110.0   \n",
       "420654    973          3   Jan,Apr,Jul,Oct                330.0   \n",
       "420990    194          2   Feb,May,Aug,Nov              16970.0   \n",
       "421308    512          2  Mar,Jun,Sept,Dec                590.0   \n",
       "421824   1029          2               NaN               1590.0   \n",
       "\n",
       "        Mean_Humidity  Sales  \n",
       "420510             55   6802  \n",
       "420654             59   6644  \n",
       "420990             55   4720  \n",
       "421308             72   6248  \n",
       "421824             64   8004  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "small_test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorify = Categorify(small_cat_vars, small_cont_vars)\n",
    "categorify(small_train_df)\n",
    "categorify(small_test_df, test=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Store</th>\n",
       "      <th>DayOfWeek</th>\n",
       "      <th>PromoInterval</th>\n",
       "      <th>CompetitionDistance</th>\n",
       "      <th>Mean_Humidity</th>\n",
       "      <th>Sales</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>420510</th>\n",
       "      <td>NaN</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>110.0</td>\n",
       "      <td>55</td>\n",
       "      <td>6802</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>420654</th>\n",
       "      <td>973.0</td>\n",
       "      <td>3</td>\n",
       "      <td>Jan,Apr,Jul,Oct</td>\n",
       "      <td>330.0</td>\n",
       "      <td>59</td>\n",
       "      <td>6644</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>420990</th>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>Feb,May,Aug,Nov</td>\n",
       "      <td>16970.0</td>\n",
       "      <td>55</td>\n",
       "      <td>4720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>421308</th>\n",
       "      <td>512.0</td>\n",
       "      <td>2</td>\n",
       "      <td>Mar,Jun,Sept,Dec</td>\n",
       "      <td>590.0</td>\n",
       "      <td>72</td>\n",
       "      <td>6248</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>421824</th>\n",
       "      <td>1029.0</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1590.0</td>\n",
       "      <td>64</td>\n",
       "      <td>8004</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Store DayOfWeek     PromoInterval  CompetitionDistance  \\\n",
       "420510     NaN         3               NaN                110.0   \n",
       "420654   973.0         3   Jan,Apr,Jul,Oct                330.0   \n",
       "420990     NaN         2   Feb,May,Aug,Nov              16970.0   \n",
       "421308   512.0         2  Mar,Jun,Sept,Dec                590.0   \n",
       "421824  1029.0         2               NaN               1590.0   \n",
       "\n",
       "        Mean_Humidity  Sales  \n",
       "420510             55   6802  \n",
       "420654             59   6644  \n",
       "420990             55   4720  \n",
       "421308             72   6248  \n",
       "421824             64   8004  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "small_test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Feb,May,Aug,Nov', 'Jan,Apr,Jul,Oct', 'Mar,Jun,Sept,Dec'], dtype='object')"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "small_train_df.PromoInterval.cat.categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "267    -1\n",
       "604    -1\n",
       "983     0\n",
       "1636   -1\n",
       "2348   -1\n",
       "dtype: int8"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "small_train_df['PromoInterval'].cat.codes[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fill_missing = FillMissing(small_cat_vars, small_cont_vars)\n",
    "fill_missing(small_train_df)\n",
    "fill_missing(small_test_df, test=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Store</th>\n",
       "      <th>DayOfWeek</th>\n",
       "      <th>PromoInterval</th>\n",
       "      <th>CompetitionDistance</th>\n",
       "      <th>Mean_Humidity</th>\n",
       "      <th>Sales</th>\n",
       "      <th>CompetitionDistance_na</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>185749</th>\n",
       "      <td>622</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2300.0</td>\n",
       "      <td>93</td>\n",
       "      <td>4508</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       Store DayOfWeek PromoInterval  CompetitionDistance  Mean_Humidity  \\\n",
       "185749   622         2           NaN               2300.0             93   \n",
       "\n",
       "        Sales  CompetitionDistance_na  \n",
       "185749   4508                    True  "
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "small_train_df[small_train_df['CompetitionDistance_na'] == True]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparing full data set 准备全部数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_df = pd.read_pickle(path/'train_clean')\n",
    "test_df = pd.read_pickle(path/'test_clean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(844338, 41088)"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train_df),len(test_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "procs=[FillMissing, Categorify, Normalize]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat_vars = ['Store', 'DayOfWeek', 'Year', 'Month', 'Day', 'StateHoliday', 'CompetitionMonthsOpen',\n",
    "    'Promo2Weeks', 'StoreType', 'Assortment', 'PromoInterval', 'CompetitionOpenSinceYear', 'Promo2SinceYear',\n",
    "    'State', 'Week', 'Events', 'Promo_fw', 'Promo_bw', 'StateHoliday_fw', 'StateHoliday_bw',\n",
    "    'SchoolHoliday_fw', 'SchoolHoliday_bw']\n",
    "\n",
    "cont_vars = ['CompetitionDistance', 'Max_TemperatureC', 'Mean_TemperatureC', 'Min_TemperatureC',\n",
    "   'Max_Humidity', 'Mean_Humidity', 'Min_Humidity', 'Max_Wind_SpeedKm_h', \n",
    "   'Mean_Wind_SpeedKm_h', 'CloudCover', 'trend', 'trend_DE',\n",
    "   'AfterStateHoliday', 'BeforeStateHoliday', 'Promo', 'SchoolHoliday']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dep_var = 'Sales'\n",
    "df = train_df[cat_vars + cont_vars + [dep_var,'Date']].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('2015-08-01', '2015-09-17')"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['Date'].min(), test_df['Date'].max()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "41395"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cut = train_df['Date'][(train_df['Date'] == train_df['Date'][len(test_df)])].index.max()\n",
    "cut"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "valid_idx = range(cut)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     5263\n",
       "1     6064\n",
       "2     8314\n",
       "3    13995\n",
       "4     4822\n",
       "Name: Sales, dtype: int64"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[dep_var].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = (TabularList.from_df(df, path=path, cat_names=cat_vars, cont_names=cont_vars, procs=procs,)\n",
    "                .split_by_idx(valid_idx)\n",
    "                .label_from_df(cols=dep_var, label_cls=FloatList, log=True)\n",
    "                .add_test(TabularList.from_df(test_df, path=path, cat_names=cat_vars, cont_names=cont_vars))\n",
    "                .databunch())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "doc(FloatList)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model 模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_log_y = np.log(np.max(train_df['Sales'])*1.2)\n",
    "y_range = torch.tensor([0, max_log_y], device=defaults.device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn = tabular_learner(data, layers=[1000,500], ps=[0.001,0.01], emb_drop=0.04, \n",
    "                        y_range=y_range, metrics=exp_rmspe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TabularModel(\n",
       "  (embeds): ModuleList(\n",
       "    (0): Embedding(1116, 81)\n",
       "    (1): Embedding(8, 5)\n",
       "    (2): Embedding(4, 3)\n",
       "    (3): Embedding(13, 7)\n",
       "    (4): Embedding(32, 11)\n",
       "    (5): Embedding(3, 3)\n",
       "    (6): Embedding(26, 10)\n",
       "    (7): Embedding(27, 10)\n",
       "    (8): Embedding(5, 4)\n",
       "    (9): Embedding(4, 3)\n",
       "    (10): Embedding(4, 3)\n",
       "    (11): Embedding(24, 9)\n",
       "    (12): Embedding(9, 5)\n",
       "    (13): Embedding(13, 7)\n",
       "    (14): Embedding(53, 15)\n",
       "    (15): Embedding(22, 9)\n",
       "    (16): Embedding(7, 5)\n",
       "    (17): Embedding(7, 5)\n",
       "    (18): Embedding(4, 3)\n",
       "    (19): Embedding(4, 3)\n",
       "    (20): Embedding(9, 5)\n",
       "    (21): Embedding(9, 5)\n",
       "    (22): Embedding(3, 3)\n",
       "    (23): Embedding(3, 3)\n",
       "  )\n",
       "  (emb_drop): Dropout(p=0.04)\n",
       "  (bn_cont): BatchNorm1d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (layers): Sequential(\n",
       "    (0): Linear(in_features=233, out_features=1000, bias=True)\n",
       "    (1): ReLU(inplace)\n",
       "    (2): BatchNorm1d(1000, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (3): Dropout(p=0.001)\n",
       "    (4): Linear(in_features=1000, out_features=500, bias=True)\n",
       "    (5): ReLU(inplace)\n",
       "    (6): BatchNorm1d(500, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (7): Dropout(p=0.01)\n",
       "    (8): Linear(in_features=500, out_features=1, bias=True)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learn.model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(data.train_ds.cont_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.\n"
     ]
    }
   ],
   "source": [
    "learn.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAELCAYAAADURYGZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xd8XNWd9/HPTxr1alnFsuXeC67CVJtiCBAglAAJgSwJbEiyG9gku5vdZ/Nsks1usklITzbFQIDNJmSpT2gxGAi2ARtbNrjj3uSiaku2ujTn+WPGIBxZzTNzp3zfr9e8PHPnju5X45F+uuece4455xARkcSV5HUAERHxlgqBiEiCUyEQEUlwKgQiIglOhUBEJMGpEIiIJLiwFQIz+42ZVZvZpm7bbjazzWbmN7PycB1bRET6L5xnBA8DV56ybRNwI7A8jMcVEZEB8IXrCzvnlpvZmFO2bQUws3AdVkREBkh9BCIiCS5sZwRnyszuBu4GyMrKmjdlyhSPE4mIxJa1a9fWOueK+tovaguBc24xsBigvLzcVVRUeJxIRCS2mNm+/uynpiERkQQXzuGjjwIrgclmVmlmd5nZDWZWCZwHPG9mL4br+CIi0j/hHDV062meejpcxxQRkYFT05CISIJTIRARSXAqBCIiCU6FQEQkwakQiIhEoeb2Tv7t2c3sq2sK+7FUCEREotDzGw7z0Bt7qWpsC/uxVAhERKLQYxUHGFeYxdljhoT9WCoEIiJRZlfNCdbsPcotZ4+MyGzNKgQiIlHmsTUHSE4ybpw7IiLHUyEQEYkiHV1+nlxXyaVTiinOSY/IMVUIRESiyKvvVlN7op2Pnz0yYsdUIRARiSL/u+YAxTlpXDSpz2UEQkaFQEQkShxpaOW1bdXcNK8MX3Lkfj2rEIiIRIkn1h7A7+CW8sg1C4EKgYhIVHDO8cTaSs4dV8CYwqyIHluFQEQkCmw62Mjeumaunx2ZIaPdqRCIiESB5zYcwpdkXDljWMSPrUIgIuIx5xzPbTjMgomF5GemRvz4KgQiIh5bt/8YB4+1cM3M4Z4cX4VARMRjz204RGpyEpdPL/Hk+CoEIiIe6vI7nt9wmIsmF5GbnuJJBhUCEREPrdlbT/XxNq6d5U2zEKgQiIh46rkNh0hPSWLRlGLPMqgQiIh4pLPLz582HmHRlBKy0nye5QhbITCz35hZtZlt6ratwMyWmtmO4L/hX3pHRCRKrdxdR11TO9fMLPU0RzjPCB4Grjxl2z8DrzjnJgKvBB+LiCSkl7dUkZ6SxCUeNgtBGAuBc245UH/K5uuAR4L3HwGuD9fxRUSi3YodtZw7bijpKcme5oh0H0GJc+4wQPBfb8ugiIhHDtQ3s7u2iQUTI7fuwOlEbWexmd1tZhVmVlFTU+N1HBGRkHp9Zy0ACycWepwk8oWgysxKAYL/Vp9uR+fcYudcuXOuvKjI+4opIhJKK3bUMCw3nQnF2V5HiXgheAa4I3j/DuCPET6+iIjnuvyO13fUsmBiIWbmdZywDh99FFgJTDazSjO7C/gOcLmZ7QAuDz4WEUkoGyqP0djayYIIrkvcm7BdweCcu/U0Ty0K1zFFRGLBih21mMGFE7zvH4Ao7iwWEYlXK3bUMGN4HgVZkV97oCcqBCIiEXS8tYN1+4+xIApGC52kQiAiEkErd9XR5XdRcf3ASSoEIiIRtGJHLZmpycwdne91lPeoEIiIRNCKHTWcO24oaT5vp5XoToVARCRCdlQdZ29dMxdFybDRk1QIREQi5LGKA/iSzPNpp0+lQiAiEgHtnX6eWneQy6aWMDQ7zes4H6BCICISAa++W0VdUzsfO3uk11H+ggqBiEgE/O+aAwzLTWdhlPUPgAqBiEjYHWloZdn2Gm6aV0ZykveTzJ1KhUBEJMyeWHsAv4NbyqOvWQhUCEREwsrvdzxWUcl544Yyamim13F6pEIgIhJGq/bUsb++OSo7iU9SIRARCaP/XXOAnHQfV84Y5nWU01IhEBEJk/qmdv608Qg3zBlBekr0TClxKhUCEZEwebziAO1dfm4/d7TXUXqlQiAiEgZ+v+P3q/czf0wBk0pyvI7TKxUCEZEweH1nLfvqmrnt3FFeR+mTCoGISBj87q19FGSlRnUn8UkqBCIiIXakoZWXt1Zzc3lZVK07cDoqBCIiIfaHNfvp8jtumx/dncQnqRCIiIRQZ5efP6w+wMJJRVF7JfGpVAhERELoz9tqONLYym3nRH8n8UmeFAIz+zsz22Rmm83si15kEBEJh2fWH2JIZgqXTin2Okq/RbwQmNkM4DPAfGAWcI2ZTYx0DhGRUGtp7+KVrVVcOaOUlOTYaXDxIulUYJVzrtk51wksA27wIIeISEi9+m41ze1dXBtlaxL3xYtCsAlYaGZDzSwT+DAQvdPyiYj003MbDlGYncY544Z6HWVAfJE+oHNuq5l9F1gKnADWA52n7mdmdwN3A4waFTudLiKSmE60dfLqu9V87OyRUbkKWW88acRyzj3onJvrnFsI1AM7ethnsXOu3DlXXlQUfWt8ioh098rWKto6/Vwzc7jXUQYs4mcEAGZW7JyrNrNRwI3AeV7kEBEJlWfXH2ZYbjrlo4d4HWXAPCkEwJNmNhToAP7WOXfUoxwiImesoaWD5dtr+OR5o0mKsWYh8KgQOOcWeHFcEZFwWLqlivYuP9fE2Gihk2JnoKuISJR6bsMhyoZkMHtkvtdRBkWFQETkDNSdaOP1HbVcPbMUs9hrFgIVAhGRM/LM+kN0+h03zinzOsqgqRCIiJyBp9YdZMaIXCYPi+7lKHujQiAiMkjbq46z8WBDTJ8NgAqBiMigPbmuEl+S8ZHZsXcRWXcqBCIig9Dld/y/tw9y8eQiCrPTvI5zRlQIREQG4c1dtVQ1tnHj3NhuFgIVAhGRQXlq3UFy030smho7C9CcjgqBiMgAnWjrZMmmI1w7azhpvmSv45wxFQIRkQH608bDtHR0xUWzEKgQiIgM2DPrDzF6aCZzR8XmlBKnUiEQERmAo03tvLmrjg+fFbtTSpxKhUBEZACWbqmiy++4+qzYnGm0JyoEIiID8MKmw4wsyGD68Fyvo4SMCoGISD81NHfwxs7auGoWAhUCEZF+W7q1io4ux4dnxE+zEKgQiIj02wsbDzMiP4OZZXleRwkpFQIRkX5obO1gxY4aPnzWsLhqFgIVAhGRfnl5S6BZ6Ko4Gi10kgqBiEg/vLDxCMPz0pkTo+sS90aFQESkD8dbO1i+o4YrZ8TXaKGTVAhERPqwclcd7Z1+PjS9xOsoYaFCICLSh1W760nzJTEnTuYWOpUnhcDMvmRmm81sk5k9ambpXuQQEemPt/bUMXfUkLiYcronES8EZjYCuBcod87NAJKBj0c6h4hIfzS0dLDlcCPnjCvwOkrYeNU05AMyzMwHZAKHPMohItKrNXvqcQ7OHTfU6yhhE/FC4Jw7CHwf2A8cBhqccy9FOoeISH+8taeOVF8Ss+Nw2OhJXjQNDQGuA8YCw4EsM7u9h/3uNrMKM6uoqamJdEwRESDQUTx7ZD7pKfHZPwD9LARmNt7M0oL3Lzaze81ssOXxMmCPc67GOdcBPAWcf+pOzrnFzrly51x5UVHRIA8lIjJ4ja0dbD7UENfNQtD/M4IngS4zmwA8SOCv+d8P8pj7gXPNLNMCV2YsArYO8muJiIRNxd56/A7OHRu/HcXQ/0Lgd851AjcAP3bOfQkY1IQbzrm3gCeAdcDGYIbFg/laIiLh9NbuelKTk5gzaojXUcLK18/9OszsVuAO4NrgtpTBHtQ593Xg64N9vYhIJKzaU8+skXlkpMZv/wD0/4zg08B5wLecc3vMbCzwP+GLJSLirRNtnWw62MA5Y+O7fwD6eUbgnNtC4CKwk6N+cpxz3wlnMBERL1XsrafL7+K+oxj6P2roNTPLNbMCYD3wkJn9MLzRRES8s2p3Pb4kY+7o+L1+4KT+Ng3lOecagRuBh5xz8wgMAxURiUurdtcxsyyPzNT+dqXGrv4WAp+ZlQK3AM+FMY+IiOcamjvYUHmMCyYUeh0lIvpbCL4JvAjscs6tMbNxwI7wxRIR8c7rO2vxO7hoUmJczNrfzuLHgce7Pd4NfDRcoUREvLRsezU56b64nl+ou/52FpeZ2dNmVm1mVWb2pJmVhTuciEikOedYvr2WCycU4ktOjLW7+vtdPgQ8Q2CSuBHAs8FtIiJxZXvVCY40tiZMsxD0vxAUOececs51Bm8PA4nzLolIwli+PTDb8UIVgr9Qa2a3m1ly8HY7UBfOYCIiXli2vYaJxdkMz8/wOkrE9LcQ3Elg6OgRAovJ3ERg2gkRkbjR3N7J6j31CdUsBP0sBM65/c65jzjnipxzxc656wlcXCYiEjfe2l1Pe5c/oZqF4MxWKPtyyFKIiESBZdtrSE9JYn6crz9wqjMpBBayFCIiUWD59hrOGTs0rpel7MmZFAIXshQiIh47UN/M7tqmhOsfgD6uLDaz4/T8C9+AxOlSF5G4tywBh42e1GshcM7lRCqIiIiXlm2vYUR+BuOLsryOEnGJcf20iEgv2jv9rNxVx0WTizBLvO5PFQIRSXjr9h/lRFtnQvYPgAqBiAjLttfgSzLOHx//y1L2RIVARBLesm01zBs9hJz0FK+jeEKFQEQSWnVjK1sON3LR5MRsFgIVAhFJcMt31AKJsxpZTyJeCMxsspm90+3WaGZfjHQOEREI9A8UZqcxdViu11E806+lKkPJObcNmA1gZsnAQeDpSOcQEenyO17fUcMlU4pJSkq8YaMned00tAjY5Zzb53EOEUlAGw82cLS5I6GbhcD7QvBx4FGPM4hIglq2rQYzWDBRhcATZpYKfAR4/DTP321mFWZWUVNTE9lwIpIQlm2vZmZZPgVZqV5H8ZSXZwRXAeucc1U9PemcW+ycK3fOlRcVJXa1FpHQO9bczjsHjnHRxEKvo3jOy0JwK2oWEhGPLNteg9/BJVOKvY7iOU8KgZllApcDT3lxfBGRl7dWU5idyqyyfK+jeC7iw0cBnHPNQGJO6iEinuvo8rNsWzVXTB+W0MNGT/J61JCISMSt3XeUxtZOFk1VsxCoEIhIAnplaxWpyUlcmODDRk9SIRCRhPPKu9WcM66A7DRPWsejjgqBiCSUPbVN7K5pYpFGC71HhUBEEsorWwOXLi2aWuJxkuihQiAiCeXVd6uZVJLNyIJMr6NEjYRqIOvyOxpaOjjW3M6xlg4aWjpobuuiqb2T5rZOhmansXBiEXmZiblKkUi8a2ztYPWeev56wTivo0SVhCgER5vaefjNvTyyci/Hmjt63Tc5ySgfPYSLJhfh9zsqj7ZQebQFh+P2c0aHZNyxc46jzR1UHm2m5ngbDS0dNLZ0cKKtk7yMFIpz0ynJTSc9JYn9dc3srw/cstN8zBiRx4zheYwsyMBM459FBmL59ho6/U7DRk8R14Xg0LEWHlixh0dX76elo4vLphZz4YRC8jNTyctMIS8jhaxUH5mpyWSmJrO3rolXtlbz6rvVfG/JNgAKs1MZMSSTo03tfP536xhXlMXnLhrPuWOH0tzRSVNbF01tnRxpbOVIQyuHG1qpbmylrqmdo83t1De1Y0Bmqo+M1GSSDA43tNLc3jWg7yUnzUdLRxedfhd4nO5jQnE24wqzGV+cxbjCLEYPzWL00EwyUz/43+qcU9EQAV7ZWk1+ZgpzRw3xOkpUietC8MOl23n67YNcN3s4n7toPJNKcnrdf2h2GvNGF/CVK6dQd6LtvV/eEGhWemHjYX7x2i6+8sSG036NwuxUinLSKcxOZfTQTIZkBmY1bG7vpLm9iy6/46JJxZQNyaBsSAYluenkZaSQm5FCdpqPhpYOqhpbqT7eSlNbFyMLMhldkEl+ZgptnX62Vx1n08FGNh9qYHdNEyt21PDkuspTMqThSzKa2ztp6ejCOZhUksNZI/KYUZbHzBF5TCnNIc2X/IHX+f2Olo4usjSkTuJQR5efV7ZWcdm0EpJ1NfEHmHPO6wx9Ki8vdxUVFQN+3cFjLTjnKBsSuk4h5xxv7KzjSGPre2cSWWk+SnLSKc5NIz0lue8vEmLHWzvYW9vMvvom9tU1c6C+GecgIzWZjNRkuvyOrYcb2RRchAMgJdmYWprLtNJcjjV3sKe2ib11TbR1+slJ81Gan05pXgZjC7OYWprD1NJcJpXkePL9iYTCih01fPLB1dz/V+VcPi0xRgyZ2VrnXHlf+8X1n34j8jNC/jXNjAujbNranPQUzirL46yyvF73c85x8FgLGysbWF/ZwPoDx1iy+QgFWamMK8xi4aRChmSlUt3YxqFjLRxqaGHN3vr3mrGSk4xJJTnMGZXP7JH5TC7JIT/YxJaTnqK/siSqLdl0hMzUZBZE2c9vNIjrQiAfZGaUDcmkbEgmV51V2q/X+P2O/fXNbD3cyOZDjayvPMaz6w/x+7f2/8W+WcGzo6w0H9lpvveKRH5mCiPyM5k+PJdpw3MpzE4L9bcm0qsuv+PFzVVcMrlYZ7U9UCGQXiUlGWMKsxhTmPVe8fD7HbtrT7CntpmG4DDchpYOTrR20tzeSVN7F8dbA9sOHm3haHP7e01SACW5aUwrzWVKaS5TS3MZX5RFQVYq+RmppKckqWNbQu7t/UepPdHGFTOGeR0lKqkQyIAlJRkTinOYUNx753t3Dc0dbD7cwJZDgTOLrYcbWbGj9r1RUCel+pIozUtnzNAsxhYGRkENy02nKCeN4pxAx7ov2fAlGylJSZpCWPplyaYjpCYncclkTTLXExUCiYi8zBTOH1/I+ePfb59t7/Szs/oE++qaONbSwbHmwMV+lcda2FvbRMXeepr6GGZbnJPGuKIsxhVlM64wi4klOUwqyWZYbrrOLAQI9I0t2XyECycWkpOui0V7okIgnkn1JTEt2G/QE+ccdU3tVDe2UXOijerGVhpbO+ns8tPpd7R1+jl4tIU9tSd4YePhD1wsmJ3mY1ppLvPGDKF89BDmjR5CfmZiL1CeqLYcbqTyaAv3XjrR6yhRS4VAopaZUZid1u/O5boTbeyoPhG4VR1nfWUD9y/fzS+DzU+jh2YyY3ge00fkBv4dnstQdVzHvRc3HSHJ0NXEvVAhkLgxNDuNodlpnDvu/VVQW9q7WF95jLX7jrLpYAMbDh7j+Y2H33u+NC+d6cMD10iMLQw0MY0vytLZQxxZsvkI88cWqOj3QoVA4lpGajLnjhv6geLQ0NzB5kMNbD4UuEJ706FGXttW84GO62mluVw8uYiLJxczd1Q+vmRN1BuLdtecYHvVCb5x7TSvo0Q1FQJJOHmZKZw/oZDzJ7zfcd3R5acy2N+w9fBxlm2v4dfLd/OL13aRkZLM9OG5nFWWx6yyfC6YUEhRjv66jAUvB9ceuHy6ho32Jq6nmBA5E42tHbyxo5bVe+vZWNnApkMNtHb4STKYP7aAq88q5YoZwyjOSfc6qpzGJ+5fRd2Jdl780kKvo3hCU0yInKHc9BSuOqv0vQvpOrv8vHvkOC9tqeKFjYf51z9u5mvPbGZWWT6XTyth0dRiJpfkaNhqlDjR1smavfXceeFYr6NEPRUCkX7yJScF1oMYkceXL5/E9qrjLNl0hFe2VnHfi9u478VtlA3J4LKpJXxoWglnjy0gRX0LnnlzZy0dXY6LJ2m0UF88KQRmlg88AMwAHHCnc26lF1lEBmtSSQ6TSnK4d9FEqhpbeWVrNS9vreL3q/fz8Jt7yctI4db5o7jrwrHqU/DAn7fVkJ3mo3yM1h7oi1dnBD8BljjnbjKzVECLh0pMK8lN5xPnjOIT54yiub2T5dtreXb9IRYv38VDb+zhY2eP5O6F40I6JbqcnnOOZduquWDCUJ2V9UPEC4GZ5QILgU8BOOfagfZI5xAJl8xUH1fOGMaVM4axp7aJX722i0dX7+fR1fu547wxfOHSCbpOIcx2VJ/gUEMr9y7S1cT94UWpHAfUAA+Z2dtm9oCZZXmQQyTsxhZm8d2bZrLsHy/hhjkjePCNPVx032s8sGI37Z1+r+PFrT+/Ww3ARZpkrl+8KAQ+YC7wS+fcHKAJ+OdTdzKzu82swswqampqIp1RJKSG52fwvZtm8cK9C5hZlsd/PL+VK3+8nGXb9dkOh9e21TBlWA6leaFfnCoeeVEIKoFK59xbwcdPECgMH+CcW+ycK3fOlRcVqapLfJhamstv7zqHhz59Ng644zerufu/KzhQ3+x1tLhxvLWDin31OhsYgIgXAufcEeCAmU0ObloEbIl0DhEvXTK5mCVfXMA/XTmF13fWctkPl/Fff96p5qIQeGNnHR1djksma9hof3nVnX4P8Dsz2wDMBr7tUQ4Rz6T5kvn8xeN55e8vYtHUYu57cRtX/3QFa/bWex0tpi3bXk1Omo95ozVstL88KQTOuXeCzT4znXPXO+eOepFDJBqU5mXwi9vm8eAd5TS3d3Hzr1byL09v5ERbp9fRYo7f7/jzuzVcMKFQw0YHQO+USJRYNLWEpV9eyGcWjOXR1fu56ifLWb1HZwcDsWpPHUcaW7nqLE0yNxAqBCJRJDPVx1evnsbjnz2PJDM+tngl335hK60dvS/ZKQFPrj1ITpqPD01TIRgIFQKRKFQ+poAX7l3AbeeMYvHy3Xzywbdo6LYUp/ylprZO/rTpMFfPLCUjNdnrODFFhUAkSmWl+fiP68/i55+Yw/oDDdz86zc5dKzF61hRa8mmIzS3d/HReWVeR4k5KgQiUe6amcN5+M6zOXyslRt/8Sbbq457HSkqPbG2ktFDMynXaKEBUyEQiQHnjy/ksc+dh985bvrlm1RoiOkHVB5tZuXuOj46t0zrQQyCCoFIjJhamstTf3M+Q7PTuP3Bt3htW7XXkaLG0+sOAnDDnBEeJ4lNKgQiMaRsSCaPf+48xhVm89ePVPDM+kNeR/Kcc44n11Vy3rihjCzQNN+DoUIgEmMKs9P4w2fPZe6oIfzdH97md2/t8zqSp9buO8reumZ1Ep8BFQKRGJSbnsIjd87nksnFfPXpTTywYrfXkTzzhzUHyExN5qoZunZgsFQIRGJURmoyv7p9HlfNGMZ/PL+Vn7+6w+tIEVfd2Mof3znIzfPKyErTEuyDpUIgEsNSfUn87NY53DBnBN9/aTv3vfguzjmvY0XMIyv30ul33HnhWK+jxDSVUJEY50tO4gc3zyI9JYn/+vMu0nzJCbFEY3N7J/+zaj9XTBvG6KFa5PBMqBCIxIGkJOPbN5xFW4efHy7dzqiCTK6P86GUT6ytpKGlg88s1NnAmVIhEIkTZsZ3PjqTQw0tfOWJDQzLS+fccUO9jhUWXX7HAyv2MGdUPvNGF3gdJ+apj0AkjqT6kvj17eWMLMjgs79dy66aE15HCoulW46wv76ZzywY53WUuKBCIBJn8jJTePjT80lJNj710GqqG1u9jhRy96/Yw8iCDK6YriGjoaBCIBKHRhZk8uAdZ1N3op07HlpDQ0v8TGH99v6jrN13lDsvGEtykuYVCgUVApE4NWtkPr/+5Dx2Vh/nM/9dETeL2zzw+h5y0n3cUj7S6yhxQ4VAJI4tmFjED2+ZzZq99dzz6Nt0dvm9jnRGDh5rYcmmI9w6f5QuIAshFQKROHftrOF849rpLN1Sxd8/vp6OGC4Gj7y5F4A7zh/jaY54o5IqkgDuOH8MTe2dfG/JNprbu/jZrXNIT4mt5Ryb2jp5dPV+rpwxjBH5GV7HiSs6IxBJEH9z8QS+eV3gzOCuR9bQ1NbpdaQBebziAMdbO7lL00mEnAqBSAL5q/PG8MNbZrFqdz23P/hWzAwt7fI7fvPGXuaMymfuKC1FGWqeFAIz22tmG83sHTOr8CKDSKK6cW4Zv7htLlsPN3LFj5fzp42HvY7Up5e3VrG/vpm/vlAXkIWDl2cElzjnZjvnyj3MIJKQrpg+jOfuWcDIgkw+/7t1fPmxd2hsjc5rDRqaO/jR0u2MyM/giuklXseJS2oaEklQE4qzefLz53Pvoon88Z1DXPaDZTy25gBd/uiZxrr6eCsfW7yS3TVNfPO66fiS9SsrHLx6Vx3wkpmtNbO7PcogkvBSkpP48uWTeOrz5zNiSAZfeXIDV/90Bcu213gdjQP1zdzyq5Xsq2vmwU+Vs2iqzgbCxbxYxMLMhjvnDplZMbAUuMc5t/yUfe4G7gYYNWrUvH37EntdVpFwc87xwsYjfHfJu+yvb+bqmaV8+/qzyMtMiXiWzYcauOvhCprbO3no0/OZN1odxINhZmv70/zuSSH4QACzbwAnnHPfP90+5eXlrqJCfcoikdDW2cX9y3fz45d3UJyTxo8+NptzIjSddWB66d18/6VtDMlM5ZE75zO1NDcix45H/S0EEW8aMrMsM8s5eR/4ELAp0jlEpGdpvmS+cOlEnvz8+aT6kvj4/av4wUvb8Ie57+DgsRZue2AV//mnd1k0pYQlX1yoIhAhXlxZXAI8bWYnj/9759wSD3KISC9mjczn+XsX8PVnNvOzV3dS19TOt66fQfBnN2TaOrt45M29/OyVnfid43s3zeTmeWUhP46cXsQLgXNuNzAr0scVkYHLSvNx300zKcxO41fLduFLMv7tI9ND8kvaOcfSLVV864Wt7Ktr5tIpxXzj2umMGpoZguQyEJprSER6ZWb805WT6fL7uX/FHpKTjK9dM+2MisHBYy38y1MbWba9hgnF2Txy53wumlQUwtQyECoEItInM+NfPjyVLj/85o09dPkdX7tm2oDH9TvneHT1Ab79wlb8zvH1a6fxyXNH6/oAj6kQiEi/mBn/es1UfMnG4uW72V3TxM8/MYf8zNQ+X9vU1skbO2t5ZOVe3thZx/njh/Ldj85kZIGagaKBCoGI9NvJM4MJxdn836c3cd1/vcH9f1XOpJKc9/ZxzlFzvI0thxvZcriRN3fWsXpPPe1dfnLSfXzrhhl8Yv4odQZHERUCERmwW8pHMr4om8/+di3X/fwNSvPTA084ONbSQX1T+3v7TizO5lMXjOHiyUWUjy4g1admoGijQiAigzJv9BCevecCfvLyDo63doKBAdlpPqYMy2FKaS5Th+V6cmWyDIwKgYgMWmleBt/56EyvY8gZ0jkR7OdCAAAIO0lEQVSaiEiCUyEQEUlwKgQiIglOhUBEJMGpEIiIJDgVAhGRBKdCICKS4FQIREQSnOdLVfaHmdUAx4CGU57K62NbX/dP/lsI1A4iWk/HH0i+vjL3lLX784PJPZjMveXq6XFPWc/kvY5k5u73o/3zES2Ze9quz0ffIvH5yHfO9T2/t3MuJm7A4oFu6+t+t38rQpUplJlPk7X7vgPOPZjMveXqz/t7pu91JDPH0ucjWjLr8xH9n4++brHUNPTsILb1db+n159ppr6eH0jm7o+9zNzT9t4e95T1THJHMnP3+9H++YiWzD1t1+ejb5H8fPQqJpqGws3MKpxz5V7nGKhYzK3MkRGLmSE2c8di5lPF0hlBOC32OsAgxWJuZY6MWMwMsZk7FjN/gM4IREQSnM4IREQSXNwVAjP7jZlVm9mmQbx2npltNLOdZvZT67aWnpndY2bbzGyzmX0v2jOb2TfM7KCZvRO8fTiUmcOVu9vz/2BmzswKQ5c4bO/1v5vZhuD7/JKZDY+BzPeZ2bvB3E+bWX4MZL45+PPnN7OQtcmfSdbTfL07zGxH8HZHt+29fuY9NZhhT9F8AxYCc4FNg3jtauA8Agst/Qm4Krj9EuBlIC34uDgGMn8D+IdYe6+Dz40EXgT2AYXRnhnI7bbPvcCvYiDzhwBf8P53ge/GQOapwGTgNaDc66zBHGNO2VYA7A7+OyR4f0hv31c03OLujMA5txyo777NzMab2RIzW2tmK8xsyqmvM7NSAj/QK13gf+2/geuDT38e+I5zri14jOoYyBx2Ycz9I+ArQMg7sMKR2TnX2G3XrFDnDlPml5xzncFdVwFlMZB5q3NuWyhznknW07gCWOqcq3fOHQWWAld6/bPal7grBKexGLjHOTcP+AfgFz3sMwKo7Pa4MrgNYBKwwMzeMrNlZnZ2WNMGnGlmgC8ET/1/Y2ZDwhf1A84ot5l9BDjonFsf7qDdnPF7bWbfMrMDwG3A18KY9aRQfD5OupPAX6jhFsrM4dafrD0ZARzo9vhk/mj5vnoU92sWm1k2cD7weLcmubSedu1h28m/7HwETvPOBc4GHjOzccHKHnIhyvxL4N+Dj/8d+AGBH/iwOdPcZpYJfJVAs0VEhOi9xjn3VeCrZvZ/gC8AXw9x1PeDhChz8Gt9FegEfhfKjH8RJISZw623rGb2aeDvgtsmAC+YWTuwxzl3A6fP7/n31Zu4LwQEznqOOedmd99oZsnA2uDDZwj84ux+elwGHArerwSeCv7iX21mfgLzi9REa2bnXFW3190PPBemrN2dae7xwFhgffAHsAxYZ2bznXNHojTzqX4PPE8YCwEhyhzsyLwGWBSuP2q6CfX7HE49ZgVwzj0EPARgZq8Bn3LO7e22SyVwcbfHZQT6Eirx/vs6Pa87KcJxA8bQreMHeBO4OXjfgFmned0aAn/1n+zM+XBw++eAbwbvTyJw6mdRnrm02z5fAv4QC+/1KfvsJcSdxWF6ryd22+ce4IkYyHwlsAUoCsfnIpyfDULcWTzYrJy+s3gPgRaEIcH7Bf39zHt18zxAGD58jwKHgQ4CVfguAn9lLgHWBz/8XzvNa8uBTcAu4Oe8f8FdKvA/wefWAZfGQObfAhuBDQT+0ioNZeZw5T5ln72EftRQON7rJ4PbNxCY32VEDGTeSeAPmneCt1CPdApH5huCX6sNqAJe9DIrPRSC4PY7g+/vTuDTA/nMe3XTlcUiIgkuUUYNiYjIaagQiIgkOBUCEZEEp0IgIpLgVAhERBKcCoHEJDM7EeHjPWBm00L0tbosMFPpJjN7tq+ZP80s38z+JhTHFumJho9KTDKzE8657BB+PZ97fxK2sOqe3cweAbY7577Vy/5jgOecczMikU8Sj84IJG6YWZGZPWlma4K3C4Lb55vZm2b2dvDfycHtnzKzx83sWeAlM7vYzF4zsycsMFf/707OGR/cXh68fyI4ydx6M1tlZiXB7eODj9eY2Tf7edaykvcn3Ms2s1fMbJ0F5q2/LrjPd4DxwbOI+4L7/mPwOBvM7N9C+DZKAlIhkHjyE+BHzrmzgY8CDwS3vwssdM7NITAz6Le7veY84A7n3KXBx3OALwLTgHHABT0cJwtY5ZybBSwHPtPt+D8JHr/PeWSC8+wsInDlN0ArcINzbi6BNTB+ECxE/wzscs7Nds79o5l9CJgIzAdmA/PMbGFfxxM5nUSYdE4Sx2XAtG4zRuaaWQ6QBzxiZhMJzPiY0u01S51z3eeiX+2cqwQws3cIzEHz+inHaef9SfzWApcH75/H+3PM/x74/mlyZnT72msJzFkPgTlovh38pe4ncKZQ0sPrPxS8vR18nE2gMCw/zfFEeqVCIPEkCTjPOdfSfaOZ/Qz4s3PuhmB7+2vdnm465Wu0dbvfRc8/Ix3u/c610+3Tmxbn3GwzyyNQUP4W+CmBtQyKgHnOuQ4z2wuk9/B6A/7TOffrAR5XpEdqGpJ48hKBtQAAMLOT0wjnAQeD9z8VxuOvItAkBfDxvnZ2zjUQWNryH8wshUDO6mARuAQYHdz1OJDT7aUvAncG583HzEaYWXGIvgdJQCoEEqsyzayy2+3LBH6plgc7ULcQmD4c4HvAf5rZG0ByGDN9Efiyma0GSoGGvl7gnHubwAyXHyewOEy5mVUQODt4N7hPHfBGcLjpfc65lwg0Pa00s43AE3ywUIgMiIaPioRIcIW1FuecM7OPA7c6567r63UiXlMfgUjozAN+Hhzpc4wwLw0qEio6IxARSXDqIxARSXAqBCIiCU6FQEQkwakQiIgkOBUCEZEEp0IgIpLg/j+Al2B53j4b5wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.recorder.plot()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "Total time: 11:27 <p><table style='width:300px; margin-bottom:10px'>\n",
       "  <tr>\n",
       "    <th>epoch</th>\n",
       "    <th>train_loss</th>\n",
       "    <th>valid_loss</th>\n",
       "    <th>exp_rmspe</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>1</th>\n",
       "    <th>0.023587</th>\n",
       "    <th>0.020941</th>\n",
       "    <th>0.140551</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>2</th>\n",
       "    <th>0.017678</th>\n",
       "    <th>0.023431</th>\n",
       "    <th>0.132211</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>3</th>\n",
       "    <th>0.017453</th>\n",
       "    <th>0.016929</th>\n",
       "    <th>0.120169</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>4</th>\n",
       "    <th>0.012608</th>\n",
       "    <th>0.016296</th>\n",
       "    <th>0.109245</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>5</th>\n",
       "    <th>0.010222</th>\n",
       "    <th>0.011238</th>\n",
       "    <th>0.105433</th>\n",
       "  </tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(5, 1e-3, wd=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.save('1')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZgAAAEKCAYAAAAvlUMdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXd4VFX6+D9vOoHQAoiCGqr0JoKIBURpKlhQwYqiWJYfu+vqirooYsNdv7aVVUFBbIBiQ8GGoggKUqUjAQKETigJJWWS9/fH3JnMTKZnJhngfJ5nntx77rnnnjtJ7nvf97xFVBWDwWAwGCJNXGVPwGAwGAwnJ0bAGAwGgyEqGAFjMBgMhqhgBIzBYDAYooIRMAaDwWCICkbAGAwGgyEqGAFjMBgMhqgQVQEjIn1FZIOIZIrIKC/Hk0VkunV8kYhkWO1dRGSF9flDRK4JNKaINLLG2GiNmRTNezMYDAaDf6ImYEQkHhgP9ANaAUNEpJVHt2HAQVVtCrwEPG+1rwY6q2oHoC/wpogkBBjzeeAlVW0GHLTGNhgMBkMlkRDFsbsAmaq6GUBEpgEDgbUufQYCY6ztGcBrIiKqesylTwrgSDfgdUwRWQdcCtxk9Ztijfu6vwnWqVNHMzIywrk3g8FgOGVZunTpflWtG6hfNAVMA2C7y3420NVXH1W1ichhIB3YLyJdgUnA2cCt1nFfY6YDh1TV5tLeINAEMzIyWLJkScg3ZjAYDKcyIrI1mH7RXIMRL22eic989lHVRaraGjgPeEREUvz0D+Za9guKDBeRJSKyZN++fT4nbzAYDIbyEU0Bkw2c6bLfENjpq4+IJAA1gAOuHVR1HXAUaONnzP1ATWsMX9dyjDdBVTuraue6dQNqeAaDwWAIk2gKmMVAM8u7KwkYDMz06DMTuN3aHgT8qKpqnZMAICJnA+cAWb7GVHtK6LnWGFhjfhG9WzMYDAZDIKK2BmOtmYwAvgXigUmqukZExgJLVHUm8DbwnohkYtdcBlunXwiMEpEioAS4X1X3A3gb0zrnYWCaiDwNLLfGNhgMpwhFRUVkZ2eTn59f2VM5aUhJSaFhw4YkJiaGdb6cyvVgOnfurGaR32A4OdiyZQtpaWmkp6cj4m1Z1hAKqkpOTg55eXk0atTI7ZiILFXVzoHGMJH8BoPhpCA/P98IlwgiIqSnp5dLIzQCxmAwnDQY4RJZyvt9GgFjCIm8/CK+WLGjsqdhMBhOAIyAMYTEqE9X8ddpK1i943BlT8VgiBlycnLo0KEDHTp0oH79+jRo0MC5X1hYGNQYd9xxBxs2bIjyTCuWaEbyG05Cdh+222OPFxVX8kwMhtghPT2dFStWADBmzBiqVavGgw8+6NZHVVFV4uK8v9dPnjw56vOsaIwGYwgJh0X2FHY+NBiCJjMzkzZt2nDvvffSqVMndu3axfDhw+ncuTOtW7dm7Nixzr4XXnghK1aswGazUbNmTUaNGkX79u3p1q0be/furcS7CB+jwRhCwrHmdyq7txtinye/XMPanbkRHbPVGdV54qrWIZ+3du1aJk+ezBtvvAHAuHHjqF27NjabjZ49ezJo0CBatXJPNH/48GEuueQSxo0bxwMPPMCkSZMYNapMxZOYx2gwhpAQS4cx4sVgCI4mTZpw3nnnOfenTp1Kp06d6NSpE+vWrWPt2rVlzqlSpQr9+vUD4NxzzyUrK6uiphtRjAZjCA3jBWo4AQhH04gWVatWdW5v3LiRV155hd9//52aNWtyyy23eI0zSUoqrZcYHx+PzWYr0+dEwGgwhrAwFjKDIXRyc3NJS0ujevXq7Nq1i2+//baypxRVjAZjCAnnIr8xkhkMIdOpUydatWpFmzZtaNy4Md27d6/sKUUVk4vM5CILicETfmPh5gN8eFdXLmhap7KnYzA4WbduHS1btqzsaZx0ePteTS4yQ1Qwi/wGgyFYjIAxhESpm3LlzsNgMMQ+RsAYQsLkEjQYDMFiBIwhLMwiv8FgCERUBYyI9BWRDSKSKSJlwlBFJFlEplvHF4lIhtV+uYgsFZFV1s9LrfY0EVnh8tkvIi9bx4aKyD6XY3dF895OVZxrMEa+GAyGAETNTVlE4oHxwOVANrBYRGaqqmvY6jDgoKo2FZHBwPPAjcB+4CpV3SkibbCXSG6gqnlAB5drLAU+dRlvuqqOiNY9GVzWYCp3GgaD4QQgmhpMFyBTVTeraiEwDRjo0WcgMMXangH0EhFR1eWqutNqXwOkiEiy64ki0gyoB/wStTuIAXKOFJC5N6+yp1GGU9m93WDwRo8ePcoETr788svcf//9Ps+pVq0aADt37mTQoEE+xw0UTvHyyy9z7Ngx537//v05dOhQsFOPGtEUMA2A7S772Vab1z6qagMOA+kefa4DlqtqgUf7EOwai+uT7joRWSkiM0TkzPLeQCxw2Ys/c9mL8yp7Gk4cFe6MeDEY3BkyZAjTpk1za5s2bRpDhgwJeO4ZZ5zBjBkzwr62p4CZPXs2NWvWDHu8SBFNAePN38jzueS3j4i0xm42u8dLv8HAVJf9L4EMVW0HzKFUM3K/oMhwEVkiIkv27dvnZ/qxwcFjRZU9BTecvzAjYQwGNwYNGsRXX31FQYH9XTgrK4udO3fSoUMHevXqRadOnWjbti1ffPFFmXOzsrJo06YNAMePH2fw4MG0a9eOG2+8kePHjzv73Xfffc5U/0888QQAr776Kjt37qRnz5707NkTgIyMDPbv3w/Aiy++SJs2bWjTpg0vv/yy83otW7bk7rvvpnXr1vTu3dvtOpEimqlisgFXLaIhsNNHn2wRSQBqAAcARKQh8Blwm6pucj1JRNoDCaq61NGmqjkuXSZiF0xlUNUJwASwR/KHflunNsZN2XBC8PUo2L0qsmPWbwv9xvk8nJ6eTpcuXfjmm28YOHAg06ZN48Ybb6RKlSp89tlnVK9enf3793P++eczYMAAn/XuX3/9dVJTU1m5ciUrV66kU6dOzmPPPPMMtWvXpri4mF69erFy5UpGjhzJiy++yNy5c6lTxz27xtKlS5k8eTKLFi1CVenatSuXXHIJtWrVYuPGjUydOpWJEydyww038Mknn3DLLbdE5ruyiKYGsxhoJiKNRCQJu8Yx06PPTOB2a3sQ8KOqqojUBGYBj6jqAi9jD8Fde0FETnfZHQCsi8A9GHxg3JQNhrK4mskc5jFV5dFHH6Vdu3Zcdtll7Nixgz179vgcY968ec4Hfbt27WjXrp3z2EcffUSnTp3o2LEja9as8Zrq35X58+dzzTXXULVqVapVq8a1117LL7/Yl60bNWpEhw52n6lolQSImgajqjYRGYHdAywemKSqa0RkLLBEVWcCbwPviUgmds1lsHX6CKApMFpERlttvVXVUdbtBqC/xyVHisgAwGaNNTRKt3ZKYypaGk4I/Gga0eTqq6/mgQceYNmyZRw/fpxOnTrxzjvvsG/fPpYuXUpiYiIZGRleU/S74k272bJlCy+88AKLFy+mVq1aDB06NOA4/pxxkpNL/abi4+OjYiKLahyMqs5W1eaq2kRVn7HaHreEC6qar6rXq2pTVe2iqput9qdVtaqqdnD57HUZt7Gqrve41iOq2lpV26tqT8/jhsjgXOQ3AsZgKEO1atXo0aMHd955p3Nx//Dhw9SrV4/ExETmzp3L1q1b/Y5x8cUX88EHHwCwevVqVq5cCdhT/VetWpUaNWqwZ88evv76a+c5aWlp5OWV9Ta9+OKL+fzzzzl27BhHjx7ls88+46KLLorU7QbEpOs3hERpun6DweCNIUOGcO211zpNZTfffDNXXXUVnTt3pkOHDrRo0cLv+ffddx933HEH7dq1o0OHDnTp0gWA9u3b07FjR1q3bl0m1f/w4cPp168fp59+OnPnznW2d+rUiaFDhzrHuOuuu+jYsWOFVcg06fpjPF1/xqhZAGQ+04+E+MrP7HPXlMXMWbeXCbeeS+/W9St7OgaDE5OuPzqYdP2nALaSWHkRMG5kBoMhOIyAMYRFrIg7g8EQuxgBc4IQK5ZMUw/GEMucyib/aFDe79MImBOEWIk7KTWQxcZ8DAYHKSkp5OTkGCETIVSVnJwcUlJSwh7DeJGdIMTK/4zRYAyxSsOGDcnOzuZESAF1opCSkkLDhg3DPt8ImBOEWHmeO+vBVPI8DAZPEhMTadSoUWVPw+CCMZFFkSm/ZrEvzzMJdHjEitpvNBiDwRAsRsBEicy9R3hi5hr+8uGyiIwXK8/zY4XFABQWF1fyTAwGQ6xjBEyUKCouAeBwiOn2x8xcw+2Tfi/TriURmVa5OW4JmLx8WyXPxGAwxDpmDSbKhJre/p1fs7yPEyOvAvWq2xPk1aiSWMkzMRgMsY4RMDHAgaOFrN2ZywMfrfDZJ1bWPHzVsDAYDAZPjICJATo//T0BM8HEioCxfsaKwDMYDLFLjBheTm2CSTMWK4GWcQ4vshiZj8FgiF2MgIkwj362ioxRsyL+hh8rGoPDRFYSI04HBoMhdomqgBGRviKyQUQyRWSUl+PJIjLdOr5IRDKs9stFZKmIrLJ+Xupyzk/WmCusTz1/Y1U0Hy7aBsDqHYeByHlbxYh8MfVgDAZD0ERNwIhIPDAe6Ae0AoaISCuPbsOAg6raFHgJeN5q3w9cpaptgduB9zzOu9lLpUtfY1UKr/ywEYAdh45zpMDGH9sPMeC1+RwtCE/gxEqgJc5AyxiZj8FgiFmiucjfBch0lEEWkWnAQGCtS5+BwBhrewbwmoiIqi536bMGSBGRZFX1Fxbva6xKeRLmF5UGIrZ54lvn9u9bDtCzRb2Qx4uVx3mcmFQxBoMhOKJpImsAbHfZz7bavPZRVRtwGEj36HMdsNxDuEy2zGOjpdRvNpixEJHhIrJERJZEMyleztFCr+3hLo7HisJQ6kUWIxMyGAwxSzQFjLeACc+nkt8+ItIau6nrHpfjN1ums4usz60hXA9VnaCqnVW1c926df1MP3TmrN0TsE+4i+Ox4rVlcpEZDIZgiaaAyQbOdNlvCOz01UdEEoAawAFrvyHwGXCbqm5ynKCqO6yfecCH2E1xfseqKO56d0nAPuE+l48WxEbuL5NN2WAwBEs0BcxioJmINBKRJGAwMNOjz0zsi/gAg4AfVVVFpCYwC3hEVRc4OotIgojUsbYTgSuB1f7GisJ9lYtwp/TH9kMRnkl4xFl/MbH3zRoMhlgjagLGWgcZAXwLrAM+UtU1IjJWRAZY3d4G0kUkE3gAcLgyjwCaAqM93JGTgW9FZCWwAtgBTAwwVkzx3sKtYZ1XaCtf4MnhY0XsOHS8XGPYseJgYljCqCoZo2bx3Ox1lT0Vg+GUJqqpYlR1NjDbo+1xl+184Hov5z0NPO1j2HN9XMvrWLHGLxv3u+1/tdLTauidAlv5TGQX/2cuh48XkTXuinKN41yDKdco0WXX4XwA3py3mUf6t6zk2RgMpy4mkr8S6P/KLyzdehCAH9bt9drngIcXWkF5NZjjoZUN8IXTkyKGNZjIaGoGg6G8GAFTCazdlct1r/8K4DPwstNT37vtl1fARIoTIQ4mziR8NhhiAiNgKplFW4JzdIsVAeMwkZUEk6GzkjAlBQyG2MAImEqmX5v6QfUr7xqMK2t35rIyOzyvtBMhF1mcETAGQ0xg6sFECMeaSqhsP3gsqH75hZETMP1f/QUgrAV/h3YQw0swxkRmMMQIRoOJEI41lVBZkJkTVL8pv7m7N3+xYgdZ+48GPE9Veen7P8OamzecJrIYljBGgzEYYgOjwVQyXRvVDnodBiA3v4gb31zIul25AIzq14J7Lm7sc91hb16BM7NzJHBE8m87cIxCWwlJCbH3jmLki8EQG8Te0+EE4JFP7UXFlm8LzyzmYPaqXZx7di2fx79YsaNM24KN+53CBWDc1+vZfsC3W24wikZufhGb9h0J3BGYtGALAO/+tpWHP1kZ1DkVTUKc+bM2GGIB858YBlN/txcVe2lO+TSDd3/LYvk234vt475eH9Q4thAyaB4rLHWL/mb1LrYfOMbgNxfS6/9+BuwF0zJGzWJvbn7AsT5bXlYA+qQ4MoXXgqFqcnyFXctgMPjGmMjKwbw/99Fh7HdhvzEv3OzfNOa6ltC+YQ2g/Oafu6aUJuS89/1lpKUkuFXd/GRZNgBbDxyjXvWU8l3MQdZ8+OoBuPF9qNs8MmMGQa3UxAq7lsFgKIvRYMrJoWNF7D/irw5aZCi2bF3HQvQm84zg/3WTu1OBZ0nn0novoc3PL/HJcPwAvHUZbJobwYH9E7tuCAbDqYERMDGMq7biSHb5wEd/lOm3/eBxtuV4d3fu8/K8sK8ZMc48D+7+EWo0gPevg8VvR+EiZYlhRzeD4ZTACJgYJvtg6eJ9UbHvp+Xtk37n4v+U1QxmLM0O6XrFLtH5Ea90UPMsGPYdNL0MZj0AX4+CkujWuFFVdh46TsaoWcxd7z3nm8FgiB5GwJwgXNPRs9q0f95buJUHPy6r7fjjpw17SwMpQzozMEcKbBwuToEhU+H8v8Ci12HqYMjPDXxymOTm25wZCxyOGQaDoeIwAuYEoV5aMkd8JMb0xujPVwfu5MGRAlt01mCALs/Mof3Y7yAuHvo+C1e+BJk/wKQ+cDC8GjnB4HCUiOHUaQbDSUtUBYyI9BWRDSKSKSJlCoCJSLKITLeOLxKRDKv9chFZKiKrrJ+XWu2pIjJLRNaLyBoRGecy1lAR2edSoOyuaN1Xl0a1ozW0TxT4bwQDJr1RUFTq7qwR1mHKOCd0vhNu+QRyd8DES2H77xG7lqtwLBUwRsIYDBVN1ASMiMQD44F+QCtgiIi08ug2DDioqk2Bl4Dnrfb9wFWq2hZ7GeT3XM55QVVbAB2B7iLSz+XYdFXtYH3eivxd2bmy3enRGtonJaq8OW9zVK9RYCsuXeSviOdxk54wbA4kp8E7V8LKjyM6fEKcEG8lJis2KozBUOFEU4PpAmSq6mZVLQSmAQM9+gwEpljbM4BeIiKqulxVHaUe1wApIpKsqsdUdS6ANeYyoGEU78ErvVsFlwE5klTE87G4RJ2pYEoU/jnjD1bvOBz0+SUlyr+/Wc+uwyEU/Krb3O5h1rAzfHoXzH02YvY5W4meELnTDIaTlWgKmAbAdpf9bKvNax9VtQGHgXSPPtcBy1XVLdhERGoCVwE/uPYVkZUiMkNEziz/LXinMjKRRNyryws2Fym2Jzefj5ZkM2zK4qDPX5F9iP/9tIm/T18R2oVTa8Otn0PHW+Dn52HGnVAUXlXK7uN+ZPCEhc79uErI/rxm52E+Wrw9cEeD4SQnmpH83iIqPP/N/fYRkdbYzWa93U4SSQCmAq+qqsNu9CUwVVULRORe7JrRpWUmJTIcGA5w1llnBXcnHlRGtt6KKPBVoqVv/A4hWuyShSaQNmOzXKnDMkclJMGA16BOc/j+CTi0FQZPhbTTQhrGs1yyw0RWkRrMFa/OB+CG86L2jmMwnBBE8108G3D9D2sI7PTVxxIaNYAD1n5D4DPgNlXd5HHeBGCjqr7saFDVHBctZyJwrrdJqeoEVe2sqp3r1q0b1o1VhoB5bW5m1K9RaCtxCpijBfZFedcsBbkeWQE8ceREO3isiLd+2Ry61iUC3f9qTymzdx1MvJTcLcsY8Np8th8Irm6OtyEhttdgsg8eqxAN1WCoaKIpYBYDzUSkkYgkAYOBmR59ZmJfxAcYBPyoqmqZv2YBj6jqAtcTRORp7ILobx7trivvA4B1EbsTDyqjoNX+I4UB+xw6VsimfUd477essK7xwnd/Otdgdh4qa6JaEaAKpiPnZubeIzw9ax3rduWFNQ9aXgl3fgNaQpX3+1N354+8/rPnO0ZwVIaJLBSWbTvIhc/PZboxqRlOQqJmIlNVm4iMAL4F4oFJqrpGRMYCS1R1JvA28J6IZGLXXAZbp48AmgKjRWS01dYbSAIeA9YDy6ygwNcsj7GRIjIAsFljDY3WvcVqzfceL/zEoWP+tYxA/J5lT8DpSE3jyuZ9/guceab8LyoOPsuzA1VlZfZh2p/ZHu7+kdy3rmWi7UVm7ckHbRNyLhuHgAklhqgiydxr/84WZx1kcJfwTLYGQ6wS1eVqVZ2tqs1VtYmqPmO1PW4JF1Q1X1WvV9WmqtrFsZ6iqk+ralUXl+MOqrpXVbNVVVS1pac7sqo+oqqtVbW9qvZU1eBy3YdBuBrMX3s1IyUxel95eYULlAqWLC+5zWwBBIbNwww1cPwCloVYM+fD37cxcPwCfly/B6qfzkdtJvB1yXlctXs8fPlXKA7tHuOtr3vtruhlDCgPTg3LpOY0nISYSP4w8KfB3N+jic9jCjzSr2UUZhR55qzbU6bt5z/3ee1bXKIsyNzv1RHhg4X+U7Ss25VLxqhZzjWWjXvsb/RZ++37x0liRNFI/mu7GpZNgfevhWPBVwCN9fLJ0cqcYDDEAkbARJiURN/FribN33JCl/M96ENDGj83k5vfWsS8jWUF0J4AhcveX2hPE/P16l1u7Y7nbfP6aShx/NFsBFzzJmxbaE/7vz84p4f4ylgwCwHH34NZ5DecjBgBE2Gqp5QuayV71KuP1XWA8rLZWnvZdbisMJmfud/vuR8ssms471mCxvOB63A6SEqIg/aD4baZkH8I3uoFWwKXIvDUYAptJRyNod+D834rdxoGQ1QwAiYMEny8FT91dRtuOf9s5/49Fzcu06drI8840pOH8sTqbD9g91oTj9CoUoFjNZzdDe76Aa1WH33vGlg6BX94aoy3vL2I1k98G/Y8I02se7kZDOXBCJgw8GYGa9OgOreefzYJ8aVfqbc3+nPqp3Fx8/Dib2Kdzfv9e5mBXTP5auVOZ1zKadWT3Y7HeQoUyu6vOlabdtkP8mtxK/hyJIveuI+f1rmb2ErHc5cwv28Jfv2mvIRi9jKpbAwnI0bARIgRPZuWaatRJZGnrm5Tpv2doedVxJQqhIxRs/h8hWf8rG++XLmLER8uZ4KVuHNA+zPcjrvmDiu0lTgFi+sD+KrX5pNHKrcVPARdhtN194cUfXgzFLi7SbuOVxkEo9BFq/6OwRALGAETRbo3rcOtLiYzB3F+Fp5bn1E9mlMKm183+V9LCZaRU5cDpYGcni/ucS4P3Ob/+pq/fLgMgO/WlvVqKyae4r7/5vGi27k0bhlM6svp5Lj1KQk9FCdiBMp8AC65koyEMZyEGAETJpe1dM+R5c3C0fJ0u7Do18aeffnJAa0Djjv+pk7ln1wUuGniooiO94PlBn3gmEeGghCzHy/fdpB3i/twZ9E/4dBWvkgeTXsp9TD7xYtnW0WxLYj0NiYOxnAyYwRMmLTy0DRqpCY6t//aqxkA9WukAJCalGD99O3C7CDW4zYixc7D+SzJOsCny3Y421bvOMybP9tNZ8EuSThMTD+XtIdh35OviUxPeoor4uwZlT2DPyuSYK7tNAlWoqZlMEQLI2DC5JzT0pzbE2/rzAVN6jj3/355c7LGXeHcD8UV9WSTLzWqJPo8du/7y9z2P1teKmy25rg7DJyXUcvrGG7WxnotuLrwKVZpI8YnvcqI+M/QMJ/c2w8cY3FW+RwCgtHCSuu7Vb4Gc/h4kVtyU4OhvEQzXf9JzTn1qwHwzDVtuLyV/5TywdjZJw3tTGJ8HLWqJkVmgjFCp7Nq+jzm+TBzFRYfLckO63oHqM7NhY/xXOJEHkz8mA0bjvBfbqSA0L7Xi/49F8DtRSFUgsngXMYNuxLp+uwc8otKynXPBoMrRoMJk6b10ljx+OXcFESCwmDqwrdtUJOLmtWlWnICn9x3gbN92IWN3PrVTUv2PDWmCeW56c886PjqcvPdF869jV9IIv8ouo9/F93AOXu/5oOkZ0nnsDOxZEURXIkAK3t1KFVAo0R+kbHTGSKLETDloGZqUlCZlR1xL63PqOH1+EXN6rgJjnQXLcZTJtWtVvECxlcOskjj77ssKi7h9km/027Md8GOxv+Kr+bhuH/QWrL4POlx7n/p/chMNEiCMZFlH7Q7AqzeEZvJOA2G8mAETAVwRbvTWf1kH9o29C5gTrecARwkuaSYKfZYQ6iMgLzv1uwO+9xQpusvbdgf2YfDEnQ/xF3ADYWPkyRFfJI0hh5xoZVzzjlSQMex3wWs5umNhCBqa8eCacxgiBZGwFQQ1ZLdl7v+O6Sjc9vTNJTokg3AX8wM2IVXtHHkC4s24SSmDPSAbtugOqu0MQMLnmKb1uPtxP9we/y3QS/+/7JxPwePFTkDQ0PB88XBGyebU4fB4IoRMJXEVS4R7J4PySQXAeOt8JcrvQM4GFQ2SqkZKBDhPWv9SxiH8N5NOtcXPsEPJZ14MnEKzH4wqNoy5UlGGYy2eaq4pRtOTaIqYESkr4hsEJFMERnl5XiyiEy3ji8SkQyr/XIRWSoiq6yfl7qcc67Vnikir4pluBeR2iLyvYhstH5692uNQdp4mM4SE0ofOp7ag+czK1A6/FjgwufnBtcxjIdtoGe4ayzKMVK4p+jvvGG7ClnyNnxwPRz3Xwa69Dqhi5hgBEyslxNwsC3nGDnGhdkQIlETMCISD4wH+gGtgCEi0sqj2zDgoKo2BV4Cnrfa9wNXqWpb4HbgPZdzXgeGA82sT1+rfRTwg6o2A36w9k8IPLMzu2ownuQcdf8nj3WvsnkhrJuE8xC/74Nlfo+38xDeShzjbEN4qGg4ZM2Hty+HA4HNX95mVlyiflPofLumbHobTwKZQGOFi/8zl27jfqzsaRhOMKKpwXQBMlV1s6oWAtOAgR59BgKOfOszgF4iIqq6XFUdGRTXACmWtnM6UF1Vf1P70+hd4GovY01xaY95PJ8xCX4EzP4j7qlVujet46PniceBo4WBO3mwL8//W/VPG7wLuI+Le8Btn8PRfTCxF3tW/si/v1lfRsj9sd1a3PciYd6ct4mbJi7yKUTbNvDu1OHKaTH+guBKIHOtweBJUAJGRJqISLK13UNERoqI7wg6Ow2A7S772Vab1z6qagMOA54FU64DlqtqgdXfNQLPdczTVHWXNdYuoF4w9xYLBOPq7I2scVdQLy2FD+/qGuEZVQ7RcCZY5cf7K69+V7jrB0hNp/an17Nn3mQ27XOPlXF48XmLtM+yyhM4zJcIAAAgAElEQVQ4Enc6cGiktVIDB3fWsQRMex8ehgbDiUywGswnQLGINAXeBhoBHwY4x9tT0/O/1G8fEWmN3Wx2Twhj+p+UyHARWSIiS/btq7xEiFD6htusXrWgz3nVxfvMwRk1q3jtO//hnjSqUzW8yfnhZHkYjv1yLXsTG3CLPM1SWvJ/SW9Q+7fn3BKDiZ+CYA5Ns8hHQKWifL92D5MXbPE5B8cf9B/ZobtBGwyxTrACpsTSMK4BXlbVvwOB/GOzgTNd9hsCnoVDnH1EJAGoARyw9hsCnwG3qeoml/4NfYy5xzKhYf3c621SqjpBVTuraue6dSu38Ncn913A+8O60vGs4P0R6nkxqew45D0KvGGtVOY+2MPveC/e0D7oazuIhtCqDA4fL+Ld37YyP7uYW/If4gNbL2ovHw8f30YV3J0nvAmYREtTsRW7m45Ka9rA3e8u4ckv1/qcQyAvsu/X7uFwEGn/I0lefsVez3DyEqyAKRKRIdgX3L+y2nxnMbSzGGgmIo1EJAkYDMz06DPTGhNgEPCjqqplfpsFPKKqCxydLdNXnoicb3mP3QZ84WWs213aY5akhDgubOZ9DaVPa7v7cb20ZKbc2cXZ7u1Bd8RLjflA+dEcnCheTNGgREu9zGwk8JjtTvZeMAbWz+KjpLGcxgF+WG9fqM8rKPvQ3bTPbiKbtdK9mqaj7HMoXmQNa5XVQnccOs7d7y7hr9OWB39TESAvv+zfk8EQDsEKmDuAbsAzqrpFRBoBfvNuWBrPCOBbYB3wkaquEZGxIjLA6vY2kC4imcADlHp+jQCaAqNFZIX1cayp3Ae8BWQCm4CvrfZxwOUishG43No/YWlQMxWAuy9qzCXN69KtsX1pypunVUZ6WY1ibxDuyzNHdA8vuDHkM2KTDXtyPTIlCAfaDYMh02gku/kieTTVD9q1jwWZ9kJmOUcKyDlSgKoyP9PuQbZk60Gv47/6w0af1+789Pe89P2fzv3khDgyRs3i3veWOtvyi4oB2JoTXBxRpDChOYZIEZSAUdW1qjpSVada8SVpqhrwAa6qs1W1uao2UdVnrLbHVXWmtZ2vqteralNV7aKqm632p1W1qqp2cPnstY4tUdU21pgjLG8yVDVHVXupajPrZ8UVX48CAzrYAzF7trCb8f52WTPSUhJo3aCGW64y8G42c9Vq6lSz9/esrtmuYc2QA/1evzk2C6KFw/YDxykqdheXqkDzPgwqHIONeD5OGkufuMUATPk1i3OfnsO5T8/h69Wl6XOGeCY8tb5SXx5sYPcGfMVFADneG75xScvjTOVv8skYTlCCStcvIj8BA6z+K4B9IvKzqj4Qxbmd0nQ4s6Zb2vSujdNZNaYPAN/87WJ2Hy6roVRPSSDXMm+4JtZ0aClpKWV/3aEqMM1Oq8bBY+l8scJzOe3ExOaRMsbxLF+vZ3F1wVNMSPo/3kx6iXFFg3lipuJ47LsGuNZKdbcWh6MAFLsIEVVFRML2LiwvRp4ZIkWwJrIaqpoLXAtMVtVzgcuiNy2DP+qmJbslzkxNtlfKvP2CDP51RUsA0quVajnJCfbjrs5OyVZCzVAfYiLCkC5n0qJ+WuDOJwDvL/TIlOBiANxPDYYU/ouZxd0YlTiNFxLfJAn7WoxrOWRPJ7Jw5ILrek2jR2aXKUtgMJyIBCtgEizPrBsoXeQ3xAjJCfFsfrY/D1ze3Ktb7ZQ7u3BfjyacVTvV2eY47M9EdpdHLRpHfxHhX1d4JmWAJ64q21aRRMKU5DlEAUmMLBrBS0XXMSh+Hu8lPUctcpm8IMvZJxIZrj1zb85ZW5oFoDyjHzhayOxVuwJ3dMEoMIZIEayAGYt9sX6Tqi4WkcaA7xVMQ4UTF2d/8HsTF43qVOXhvi04x1XrCOIp0sCLZ5PDpOYtgeUd3RuxYNSlZdoritmrwi8r4MC7rBBeKb6O/1c4gg6yic+THqeJlJZ3LvFQYcJJYOkpHGcszQ4z+ac79763lPs/WMbevNjPWWc4+Qh2kf9jVW2nqvdZ+5tV9broTs0QDqUleMs+Kc89uxYLRl2KCPzrSrspbcqvWT7H8szjBaUPz81WFLsnDXwEfVYEkXiITvzFd16yL0suYHDhv0iVfD5LeoIL41YB9vUT1+/bl2D4auVOHvr4D6/HPH9bv27KKT1WDpXCESNVYKpVGiqBYFPFNBSRz0Rkr4jsEZFPrEBIQ4zh9DzycbxBzSpsee4KbuuWAbh7oDWpG3wApedb+3PXtg1hlrHLLxv3sd9P1uDl2oyrC55ih6bzTuLz3BL/PZMXZNHokdl+c3V9sWIHIz5czsdLs70eD668cug4ap6FYsYzXmuGSBGsiWwy9kDGM7Dn/vrSajPEGP5Sm3ijepVSD6iP773A7Zi3Z55j3JTEeLf2Mq66lcCRCAQIlih0fnqO3z47qMugwjH8XNKepxMn80TCFOIppvm/7CFZ3hwn/jrNeyXNT5bZBY43AePwLPOWB82TA0cLyRg1iy//cPfuixdH0GfAIZwY+WKIFMEKmLqqOllVbdbnHaBy86wYIoJrqYDaVZPcXKM93W+h9GE3uMuZZY5VNn/uPRK4UwCCTctylCrcXfQP3rL1446Eb3kr8QWqYV+XCrR2cuhYadZoh7NAjpdM0jlW5uzjhcVljnlmkc607v3d37Lc2h0mzVA0pBOlhIChFFtxiTMwN5YIVsDsF5FbRCTe+twC5AQ8y1DhOMwbwa4z33ieb0HRtF5ZV2TH221qUlAhVBWK59t7tCkhjqdtt/JI0TAujFvNJ0lj4ODWgBLmrilLghrfUabZs0TDkqwDnPfMHGYGcb+OtbKtOd7XzLxhTGQnHoMnLKTF6G8qexplCFbA3IndRXk3sAt73rA7ojUpQ/gE437sSlMrk3PL06uHNP6pnMPMk6nFvbi96GHqywGYeCnNCnwntwTfqWU88ZV0cu2uXAB+31L2Hc+XbHh/4dagrulvDEPsEuzfVEUTrBfZNlUdoKp1VbWeql6NPejSEGOEulYsInx0Tzc+CLKmjMOm71mF81Tn15I2XFM4luPxVZma9AwD4hYEPikA+T6cBsTLukqg94mFm4PPnGQEjCFSlKeipUkTE4M40sHUrhq42JWDLo1qu/X/+N5uPNi7ude+Z6XbgzUT/VTdjARjKjloMxw26xl02/coy7UpryaN5+8JHyOE7x6c72XtBWCTtd7yoUuBtkDeg0XFwc8jEoGjoZCbXxQ1LzpD5VKep4R5hY1BruvUkKeubsO9lzQJe4zzMmoz4tJmAMx7qCeT7zgPgAHtz3D2SUqIK3clTUdJAm8M7V42i4Avzj07+Ho60eYQadxa+AjTbT34a8Jn/DfxNZIJvRQ0wLEi715x2w+UDXItrUHj/UHdKYSaQ6GY08pLoa2EdmO+Y/QXqyvsmoaKozwCxrxyxCDxccKt559NUkJkNIyz0lPpeU491o7tw0s3dnA7dkHTOsx54GLmPHBxyOP2b1ufl28sW50zHIIpTVyRFJHAw7a7eaboJvrHLWJ60lPU5VDI4xwr8K7BeHOD3rjHrtX4Uj4OHy8KevH+rfm+K3BGmkJLs/p8+Y4APQ0nIn6fQiKSJyK5Xj552GNiDKcIqUkJXhf2m9ZLK+NtdlX7M3jg8uacl+H7rfnhvi2okhRPl0a1yz233zbtd9uvmhTvo2dFIkwsvpJ7iv5Oc8nm8+TRtJTQNIOjhd41mNZnlHXIGPWpPauALyGyYU+eW9aGP/fkkTFqFiu2hy74IonjT8qs+5yc+BUwqpqmqtW9fNJUNfb8VA0xwX+HdGRkr2Zl2v/v+tLyzI5o+feGdSnTL1SaneYu4Cbc1pmH+7Yo97iR4PuSzlxf+ASCMiNpDL3ilnJGjZSgzs33kd6lih8B6u857Zp+ZqxVxvmrCnbt9iSU6p+GE4+ortSKSF8R2SAimSIyysvxZBGZbh1fJCIZVnu6iMwVkSMi8ppL/zSXCpcrRGS/iLxsHRsqIvtcjt0VzXszBGZxlrvrZNXk0neSo5b5x1FKoDx4erSlJMZxX4/w16AizRrNYGDBU2RqAyYmvsjVxz+lPBZmX+7L4K4J2DwW9l2v6KjGWVzJD3Zn7rxKnYUhWkRNwIhIPDAe6Ae0AoaIiKdr0DDgoKo2BV4Cnrfa84HRwIOunVU1z7XKJbAV+NSly3SX429F/q4M5SE5sfTPzbPQV3lIiPc03cWe/8k+anFj4Wi+LjmPf8a9z3MJb5FIeKlt/AllV02g3ZPfuR3zZj7znn8bJlXgOgxgJMxJSjQ1mC5AppV5uRCYBgz06DMQmGJtzwB6iYio6lFVnY9d0HhFRJoB9YBfIj91QzTo1jjduZ2WUpqG5tu/BXYSqJWayKP9vZu9cjwi3b3FhGSNu4LXburIjZ0rL8VNPsmMKBrJq7arGZIwlymJ46hB6Oltkv04cLh6+x7zcHP2pqx8vmIHGaNmcfmLP7u1j/3Kf7BopAjk/WY4sYmmgGkAbHfZz7bavPZRVRtwGEgnOIZg11hc/zKvE5GVIjJDRLw+SURkuIgsEZEl+/b5rpluKD89z3FPVycCl7WsB0Arl8wB5wRRHXP5470ZfnGp2cs184Bn9uMf1+31OsaV7c5g7NWtA088iihxvGi7gb8V3s+5cX/yWdLjNJLQCoLtyS2931fmbGTD7rzS8f08qL09xA9YOdA2+sjjtnFPnlvutGhhxMvJSTQFjDfd2/PvKJg+vhgMTHXZ/xLIUNV2wBxKNSP3wVUnqGpnVe1ct67J1xlNLmzm/v2qwms3dWLOA5e4rcd4Mmlo54Bjf/X/LqR3q9N47aaOZa6zO9d3XZiEuOgGiAbL5yUXclPhY9SQo3yW9Djd4tYEdd7h40VMWlBqvnppzp/0eXmec9+fIhBOLOPlL83jqtfmh35iiJj8Zycn0fxvywZctYiGgKfLirOPiCQANYCAOS1EpD2QoKpLHW2qmqOqjle7icC54U/dEAnu7J7BTw/24IXr21OnWjJJ8XGkJMY785/54tIWvgMwHcTHCRNu68yV7c7g2o7uivH5jX0rwQ5X6/Mbl989urws1XMYWPgUe7Um7yaO48b4uQHPCZQx119q/3DNUNsPHA/rvFAwgfwnJ9EUMIuBZiLSSESSsGscMz36zARut7YHAT9qcK8yQ3DXXhCR0112BwDrwpq1IWKICBl1qjLo3IYs+ddlUUsD38ojLiTQn1DWuCsYf1OnqMwlVLK1HtcVPsmCkjY8nziRRxM+IM5LepkDRwsZM3MNCzL3exmllD/3lL9kgcEQKaImYKw1lRHAt9gf9h+p6hoRGSsiA6xubwPpIpKJPbeZ05VZRLKAF4GhIpLt4YF2Ax4CBhgpImtE5A9gJDA0CrdlqEAe628v6/zLP3v67ecZAOqQL9dYms3MEd3LnJNeLblMW2WRRyrDih5ksq0PwxNm8Wbii6R6+Les2XmYd37N4oGPvJdcDgazkG6oaKIaLKmqs4HZHm2Pu2znA9f7ODfDz7iNvbQ9AjwS7lwNlUvVpHiOeng93XVRI4Z0PYtqLus1K8f0LnNuose6iuNB+vx17Xiwzzk0qFklCjOOLMXE86Ttdjbr6TyR8C5fJI1mQUlrDmh1DpBG7a05dJW95JDGAa3OIapR4uX9sHEd32WvN+8LviaML/KLipn6+zZu75ZhCpMZAmKi8Q0xwR3dG/Ha3Ey3NhFxEy4A1VPKVtmM94iDOc9KP5OUEBeUcBnRs2mZa1cW7xX3ZquexqMJH3JN/HxqiJXYcv5kprsoXcUqHKKaUwDlaHUOaBolBemwaAOkpnNB3CYOaHVyNI2DpLHrcD6zVu4qV3LQV3/YyP9+2kTN1ESu6diwnHdriDSq6jVXXWVhBIwhJrj74sZhP+Q9I/kz0n2/xXsjUolBI8W8kvbMK7Sn1UnARi3yGH/1Wbw88zfSyaW25FFbcp3b6ZJLM9lB7bhcahUdga8/AuBDjxyghzWVnBnV2ZtQkwmJVe1CydKIctwElX3bG3n5Nrefhthi8/6jNKnr34mmIjECxhATlKdCpqeACXWsWLb02EhgH7W44fM8oE3A/k3SU/jh/vZwdD83vvSVXRBJLrXJK9225XGm7KVD3CZqkUei+PBMe3YE85KqcIDqTg3pjJ1nUiW+hEbZG+HPlpBaB6qm238mVQ2+VrchKsSau7cRMIaYIL4cD6bylm+OtkmhSd2qbIrA+kcwFBMHVetA1Tos0k1BRJUp1TlKuuRRG0sAWdv/PL8OSxf8QTq5nCYHaRm3lXr7FtI9sRDWYP+4UBKfQlxVF4FTtY67AHLu14HUdEipEaVv4dQl1ty9jYAxxATlecaLCK8O6UhSvJCaFPqfdCjX7pJRm9+z7KFaHc+qyfJtgdPdd22cHlDAnN+4dkhljSOHkEs1crUaWzjdTSA91Kc/f//JzUeHu7pnMHXBOh7rUZcmqfm8+c3v3NiyClVsB1m3aQtXn53MafFH4Oh+yNkIR3OgyMe9xyWSlJrO10lJ5GgazPjELoBS070LpSq1IC4WSjHELjGmwBgBY4gNyluC2bXaZrBc2LSOM6uwN564qhUXNavDZS+WRso/P6gdPV/4CYD0IMtS+/PsctD6jBpOAXNNxwZ8FmYBrkg+X7yVMY6Lj+MoVTic0pCsKon8WGIjPaUhHy/NBi5g6p5UfnrIw6286Lhd4Bzbbxc4x/Y790uO7CN76TpqSy7sXG4/XnDY+4Qkzi5kXLUghwBKqw9tB53yWlGhLXJJZCOBETCGmKC8Zq5waNuwBvMz91Piw65wdnqqWzG1s9NTaVSnKv+6oiWtzqjOOwuygrpOTavi5uk1Uth12HsaG9c3zx7n1A1bwGzNOUbGqFkBY4eCobDY98PKV0xNVs4xDh4tpJar8E2sAjXPtH88sNmKuXvhN/ZzR15hNRbCMVdBlOMioFwE1b71kLUfjh8EFOa9AFe9DM37hH3PJzpfrdpJ24axI2SNgDGcsjjWfXzZrWtX9R6MeddF9jCsd38trVDpajrzpHfr0+i0qCYv3tCBHpb244nrA9ubK3ao/PRn+RO5fr1qd5k2x2uAL6EM8O2a3QzuclbI15u/cT8XNqsDCUlQ/XT7JxiKbbBzGcwcCR/eAO1uhL7jILXy0wFVNMXFsWUjiy3/TMMpT4cza1bYtRxKk6sp6PWbS1PIBJqLq9ZVLcX9Xe3nh3ow+Y7z2PJcf6qnJPLp/d3JCMJUBpFxxDp0tPwZkP/xse+sASVaWkvG85FWEKaZZunWg4E7eSM+Ac7sAvf8DJc8DKs/gfFdYO0X4Y13AlPZBeQ8MQLGEDOsebIPH93TrcKu17+d/Q25b5v6zrZmpwUfQ+BaQG3MVe5lAM5Or0rPc+qV8VAb2MH7WtE+l5IDkTAXRssxzuGsUKLqs67bLxv38fw36wMm5vTkzXmbAFi69QDbco6FPrmEZOj5KAz/CdJOh49us3+OeC/fcDLiT7OsDIyAMcQMVZMTKjTosUX96mSNu8Kttoy3/88eVl2b8zLcTS6PX9mKO7s34ru/X8xZ6alBXdOXM8Jul7WZuAhIBz/LJ+Vizro9AOw8dJw1O+yL8Z4vzXPW7eX1nzZxz3tLPU/3i6NA2nWv/8bF/wmcWdon9dvC3T9Cr8dhw9d2bWblR7HnYhUFjAZjMMQwNsuG3cKlCNq9l9gLnXkKh5qpSTx+VSuanxa4YJoDzzo4DjPcpS3qcUETe5mBTmeFn8rFwfrdueUewx978gqY8ttWv31+jsA6UNjEJ8JF/4B750N6U/j0bpg6GHI9K4bEBr9m7ufTZdnlHidaLxbhYhb5DQYX6qTZvZ9czWbnN05nzZN9/BZJCxbXWjj/uLw5Iy5typx1e+l5Tl2GXpDBzkPHqZJU/liPr1eXXaCPJAeOlpr09ub5LvBW6dQ9B+78Fha9CT+MhfFdoffT0Om2mMo6cNNbiwC4tlP58rsVl8SWhDECxmAA7u/RhIw6VamXlsLy0ZdTo4q7J1cwwkUksBWmjkuZgHZn1kREuLyVvcBaQnwczULQhiqT1TtKNaRDx4oiMmbUPNXj4qHb/XBOX7un2ZcjYc2ncNUrUCsj4OklJcrCLTlc0KROlCYYOWzGi8xgiD3+2bcFN3S2x2nUqpoUVir61WP68H/Xt2eVl5IC3jhZ6rOs2uEjMDJEol7OunZjuG0mXPEiZC+B/10AiyZAgLf+SQu2cNPERcxZuye684sAXWOgUqsrUf2NikhfEdkgIpkiMsrL8WQRmW4dXyQiGVZ7uojMFZEjIvKaxzk/WWOusD71/I1lMFQUVZMTuO7chqQFG8fiR768MrhDUEPUiaHCaeUl2vLFeZHzhsH9C+Gs8+Hrh+Cd/rDfdybvrBy759yuw9EvHV1eqoSRKimaRO1XKiLxwHigH9AKGOJRlRJgGHBQVZsCLwHPW+35wGjgQR/D36yqHayPwwfR11gGQ0yifiTMwA4NghrjzgszIjSbimfD7jxaPf6tc788CU9DpuaZcMsncPXrsHctvNEdFrxiD9r0wBHvE2MewF45ldyUuwCZqrpZVQuBacBAjz4DgSnW9gygl4iIqh5V1flAKKuHXscKf/oGQ3SJhIUsmIfyN3+7qPwXigLvL9zqFuQaCfdsX3y2PJs/tnskJhWBDjfBX36HJr3g+8fh7cthz9oy3SD2UuF7w1v+uMokmgKmAbDdZT/bavPaR1VtwGEgPYixJ1vmsdEuQiTcsQyGSiHYyH5/BBOU2aJ+9YB9KoqMUbN4/IvVgBcnrii+Dv59+h8MHL/A+8G0+jD4Axg0CQ5thTcvhp//DcV254UdB+2msQ17jkRvghHiVIqD8fbn4nn3wfTx5GZVbQtcZH1uDWUsERkuIktEZMm+fZXop2845QlUeXDkpU15f1hXNj3b32efRVvs+c9qpfpf9xl6QUbI84sET321tsyb/7tW/Myybe6pYYpL1K3vXVOWMOqTldGfJNilXZvr7NpMqwEw9xmY0BN2ruCH9XYr/PTF2ypmLuXgVDKRZQOu6VMbAp5RTs4+IpIA1AD8FsVQ1R3WzzzgQ+ymuKDHUtUJqtpZVTvXrVs3xFsyGCqOB3qfw4XN6vjVUjbszgPgYABX4TEDWvs9Hi3enr+FHi/8xPJtB92Ex+9bDri5OoM9kv+tX7Y49+es28O0xdupUKrWsWsygz+Eo/tg4qU8lDCNZApPiDWY6lXKnyg1kkRTwCwGmolIIxFJAgYDMz36zARut7YHAT+qH0OniCSISB1rOxG4ElgdzlgGQ2VxWct65R7jk/u68d6wLiTGx/4y49acY1zzv1/d1pxuePM3r32fmb2ugmYVgBZXwF8WQvsh/CVhJrOSHqWT/FnZs/KJI4C3SmJsFWSLmoCx1kFGAN8C64CPVHWNiIwVkQFWt7eBdBHJBB4AnK7MIpIFvAgMFZFsywMtGfhWRFYCK4AdwMRAYxkMscRbt59H1rgryjXGuWfX5qJmdd3yqIXKB3d1LdccQmVXbgxH/HujSi24ejx3lzxCFSlgRtKT8M2jUBhGIs4o41ByYy22KqpO06o6G5jt0fa4y3Y+cL2PczN8DHuuj/4+xzIYTlaevroNX63cxSf3XcCtby9yJowMhvMbV6wPTPdxP1bo9SLForiO9Ml/nocTpnHrwvGwYTYM+C80ih3vvLgAtY0qCxPJbzCcYNR2qRZZMzWJrHFXcO7ZtfjpwR5l+r55q9f3Mf7Ss0n0UrOcZCQlxHGEVEbb7oTbv7I3TrkSvnoACvIqd3IexJoGYwSMwXACMH34+cweaX9jLvKRMjchvuy/s6PUgCcP9WlRplbNicrCzTk8M2tt4I5hkuj6vTa6CO77FbqNgCWT4H/dIHNO1K4dLA4NJsbkixEwBsOJQNfG6ZxTP434OOHR/i299vEWdJlYIflXostHi7dTaCvhm9W7yRg1i/0uxdkABk9YyEQX77NIU6ZGUVIq9HkGhn0Hianw/nXw+f1wPMyKnBHA8WuONb+mE/+vz2A4RYiPEzY9258hPurdi5f/5nCSdsYa//xkJc9/s55737cXMLtj8mIADh0rxFYBBVASvWiGgFWmeZ697swf0+ylANbPivp8vGHWYAwGwwnDRc1iKzX92/NLNZRVOw5TYCumw9jvGf3FGmd7tNKk+BQwAIkp9sqZd/8IVevCtJtgxp1wdH9U5uILcQqY2JIwRsAYDAY3gbJyTG+ubHd6Jc4mMAU2u+Yy9ffS6Hpfa1PlJagy3md0gLvnQs/HYO1Me5nm1Z9U2KJIrLopGwFjMJwkpCUncM/FjZk+/PyQz31vWGlMTPWUxIhU74wm+UVl3bGjpcEkBRvMmpAEl/zTbjarebZdk5l2M+RFt7oolObJijH5YgSMwXCyICI80r8lLXwEXzqyKp8fRFGqs2qnRnRukabLMz+Uabvo33Pd9r9ZvYuuz86hwBZ8bJA3/JrIvHFaKxj2PVz+FGz6wa7NLP8gqk9/pxdZwFSOFUtsv6YYDIaQqZpkTxeSmuSeNqRF/er89GAPGtaq4vW8cde25ex0e4bndg1rRneSUeDA0ULn9p7cfO59fxkAuw6VL4NAyAIGID4Buo+Ec/rDzBHwxf12k9lVr9hr0fhAVX26j3+yNJsqSfH0b1vWfOlc5I++z0NIGA3GYDjJSIiPI2vcFawd27fMsYw6Vb3GywAM7nIW3ZqURvd/OeJC3rilU5l+7RvW8HntM2qkhDHjyJO1/6hzu7zv9GEJGAd1msLQ2dDvP7BtIfzvfFj8tk9J4M/K94+P/+D+D5Z5PSYxugZjNBiD4RRm6AUZdDzLu7bStmENjhSUVnicevf5ND+tGtVSEjjnX9+49W1WrxpTh59P7dQkGj8623OoCsd1Paa8sSFJCeV09Y6Lg67DoXkf+HIkzHoAVn8KA16F9CZuXW0lJcTHhZ6wMs54kRkMhgHUMDgAABlOSURBVFhjzIDWfsszu5YK6NYknfRqySQnlH0Afv/AJdSplhwTcTfFJYrNRcCUd+0/qTwajCu1zibv+o8p6P8K7F4Jr3eH38ZDSekaUbiOCo5ASxMHYzAYThjOqBkbJq9gWb87lyaPzub1nzY521zNZeHg6qa8x09G6OISZVuO/0zLbZ/8njZf1IP7F7IrvQt8+yhM6kMT2QHgJhhdCaSFOTSYU6lkssFgOMFpUNO7Q8DoK1s5t2eNvLCiphOQ5dsOAfDb5hxn245Dx8s1pqvG1vXZst5rDl6e8ycX/2cuW3P8C7SiYqWo2ul02zqcvxbeDzmZzE56hPvjv6C4sNRRYczMNWSMsmcG2LjXf7nmhDhjIjMYDCcYvjyahl3YyLnd+gzfi/4VjbfZnlM/zbkdznpM89P8l7Z28Nsmu1Dbk1sQoKcjUFT4ouRC+MvvzCnpxD8Tp1Pt/b6wexUA7/ya5ex/KEDFUocp01ZsBIzBYDgFqJnqv3zvyEubRvya3uSha9XPQBakvXn5ZIyaxeKs0mrrwSbSLM1oHPgh77pUlfH0Yv5S9DfuLfwbcUd2woQe8OMzJFEqVJJdzHR5+UX8usk9Fc0pucgvIn1FZIOIZIpImQqTIpIsItOt44tEJMNqTxeRuSJyRERec+mfKiKzRGS9iKwRkXEux4aKyD4RWWF97ormvRkMBv+seLy33+MD/DgXRBLXt/8xM9dwrNDms++slbsAePPn0jWcbQfc11VUlWm/byubmsYSGnvyAmsw3vimpAu7b/4Z2gyCef/my6THaC+ZACQnlj6qR05dzk0TF7lllXZoMKfMGoyIxAPjgX5AK2CIVfbYlWHAQVVtCrwEPG+15wOjgQe9DP2CqrYAOgLdRaSfy7HpqtrB+rwVwdsxGAwR5LH+LZ115CPJC9/9WaZt2JQlzu33Fm6l1ePfkpdvFzpv/ryJe99b6jz+7Ox1AMxZtxewOw148tjnqxn16SqaPfY1AM/NXsfSrQecWsnIqcvJGDXLbwaBowXejxUl14Jr34SbPqK6HOPTpCfgu9Eka6kw+XOPfT3muEv1Uof3XrEXDcZWXELGqFm89H3Z7ybaRFOD6QJkqupmVS0EpgEDPfoMBKZY2zOAXiIiqnpUVedjFzROVPWYqs61tguBZUDDKN6DwWDwwRu3nBv0Av9X/8+9n7cHYSTYF6T20HbMd3y+fAfPfb2eb9aU5gqL87Cx9X35lzLnfrR4u9v+m/M2c93rv5U5d90u39Uutx3w7ghwtNDGkQIbc0s60rvg30wv7gG/vkrD6b3pLOvtc3S6JJd+h45F/mIvazCOWKZXftjocz7RIpoCpgHg+pvIttq89lFVG3AYCKpQuIjUBK4CXN06rhORlSIyQ0R852MwGAzlpm+b+kEv8Dc/Lc1t3/Eovq6T/f0wmPxokeZv01c4t8fMXMOuw8edWZoBfli3J+AY/tZbprlkei57nvf24e8u5aGP/+COdxaTRyqP2u5mc78P0OIiPkp6iicSplBF7V5xrtYwR7E5b4L7mVnrAt5HtIimgPHm0OF598H0KTuwSAIwFXhVVTdbzV8CGaraDphDqWbkee5wEVkiIkv27dsX6FIGgyECiEDWuCtY82Qfhl6QwW3dMgD4vxvakzXuCnq3qu/Wf/LQ8yp0fu/8mkW35350a3M1rfkiv6hUIP26KcftmGsWBE98LZXsOHScLI9Ymks/E/rkj2NKcW9uj/+Od47/je5xqyi20s0Ulyj5ljmuxMvArhpaRRNNAZMNuGoRDYGdvvpYQqMGcIDATAA2qurLjgZVzVF1GionAud6O1FVJ6hqZ1XtXLeu93rlBoMhsjjMR1WTExgzoDVVkvynQ+nZoh7f/f3iiphauSjyk13Sn4BJ9FMCwFsyhC15wpO227mhcDTHS+L5IOk54r/6K+Qf5uFPVjJ7lV2IeNNgqiaVZgS77/2lLNycU6ZPtIimgFkMNBORRiKSBAwGZnr0mQncbm0PAn7UAD5+IvI0dkH0N4921xSjA4DK0wsNBoMb8QFSyLRpYDe19TynLhNv6wyUNavFCsG6Aufl+xYwU1xiXDxZs7OsY4GDJdqC/oXP8YbtKjK2fwbjz+fA8tLHqreaa67f/derd3P3u4E1s0gRtWSXqmoTkRHAt0A8MElV14jIWGCJqs4E3gbeE5FM7JrLYMf5IpIFVAeSRORqoDeQCzwGrAeWWUFgr1keYyNFZABgs8YaGq17MxhOJf51RUuahfmwv/eSJrzx8ya/aegBujSqzZJ/XUadasnhTrPCcLVCeTNJOWjbwH19yvXd+fMVnsac4CkgiXG2IZx/5Z10WPook5Je4NPiCxlbdKvTbOaKZyaDiswWF9Vsyqo6G5jt0fa4y3Y+cL2PczN8DOv1+1HVR4BHwpqowWDwyV0XNQ75nPSqSQCM6teCUf1aBHVOIOFy7tm1eO7atvR+aR4Na1Uh+2D5UsBEAl+5wwBmLM1mzIDWzv2vV0d2LSS3dlu452deGXMv98d/wUXJK5mbMwp7VEhsYCL5DQZDRNn4TD8WPdorImPd16M0nf3SrQedtVmqVWJJ56ou60f+NJgBHc5w28897j/dS6goQEIyL9kGMaDwaXZrbW7Y/ChH3h0CR/b6PC/Xj+ku0hgBYzAYIkpifJzPomah8nDfFrxxi91fp3vTdDLSU/lrr2bOdZrK4EyXctJd/CS/7NG8LqrKU1+tZcPuPOZtjKzXqqry5JdrAFinZ3N14VM8XzSYxE3fU/TqefDH9KiWaQ4GI2AMBkNM4yj4lZaciIjw98ubuz3kK5r1u30HULpSXKLsyS3g7flbuH3S7zSsFdk5/++nTUxekFV6PeJ5vXgA/QufZWV+XfhsOHx4I/WpOK8xT4yAMRgMMc0lzesx/OLGPH1NG6/H1z9VtjR0LOBe9ExpdXr1iI7/+xbvER2btAHXFz4BfZ6DLfP4Lvmf3Bg/l/IXjw4dI2AMBkNMEx8nPNq/ZRkngK/+34U82Ls5KYmhlxiuCI4U2Ci0MgPsDTMBZriUEMeMpAHcnPQSa0oa8XziRN5PfJaG4nttJhpU3kqZwWAwlIM2DWo442dikUc+XeW2/+Uf4bsmh8ODH/8BVOdXHmVIyVweSfiQ75Ie5nnbYCjpV5rULIoYDcZgMJzwdD67ltf2Lc/1r+CZ+Gbfkf/f3plHWVVdefj7UVVUMVQxllgCzSSDE4PigCiRQVQ0gjFGTLrF1izigK1tIo2xpUU7CcTuaBtcicY4ZalxioK2Ntpook0ERAVBIorKalFbnAKKE8LuP+55VfdVvVdFad0a7tvfWm/d8/Y959y933tV+55z7tm7eUcxGYx23LFzIsd8voCVu4Yxr+RWuGUKvLcx8Wu7g3Ecp81zQ56nyiRx0dFDgGQSnDWG2hsvm5u36MkZO2bzwy/Ohi3r4Y3liV/THYzjOG2eirL8s/3/MHEwm+Yfz0WThzajRnUZM2i3AsUnjLhv1zg4/3kY+b3Er+YOxnGcNk9Dsc6ak5NG5c7UOeuO55tZk3ro1CN3fukmxh2M4zhtntpxzvp278CFkwbXqbf8komcOXYAUw7Ys865puLqU0cm1ndTMeqKR5vlOu5gHMdJHU/NnsCFk4bUke/ZpYy539yXq08dyaEDurNszoSs85P26ZWzv8ryUn40uW5/+ejdtUPjFG5mPvxkB6+++3Hi13EH4zhOwVFaXMRdPxhD764d6B4Ccz532dHcOGM06684pk59M/jmiL3qyPNRO4Jxa2Tdm1sTv4Y7GMdxUkWHRm68vO+cw7ly2v7VjiYevmvxrLEA7Ni5qzrQZi4WnHxA4xVtJE0d4HPeg+ubtL9c+EZLx3FSxcMXHNmo+gN6dmJAz07V78tKiigpEvNO3J9eFWUAbP10R71r4n2bOM5YLurLkPlV+GD7F03aXy7cwTiOkyrizuKrUNROvPKTaIPmO9s+A6CspB2f78ifHrmqla+5tBSJTpFJOlbSBkkbJc3Jcb5U0l3h/ApJ/YO8h6QnJH0saWGtNgdJWhvaXKvw+Iik7pIek/RKOObe2us4TirZo7yUsXs37V6TzHRZRVkJXTqUVMtH9O3apNdJK4k5GElFwHXAcUQp1k6TVDvV2lnAh2a2N3A1sCDIPwMuA36Uo+tfATOBweGVCaU6B1hqZoOBpeG94zgFwspLJ3H79w9r0j7LwwbOkw/qQ6fYGsjcE7L/lWU2ep46um+DfeYLa7M7jOiTOxrAn2s9DddaSHIEcwiw0cxeM7MvgN8DU2vVmQrcGsr3AhMlycy2m9n/EDmaaiRVARVm9rRFCa5vA6bl6OvWmNxxHOcr0am0mL9ccSwXTx5K++Kaf5elxdn/Ont0LmXpD7/BldNypxSIE38abfzQyjrnjx9elbftollHVJf/9rC/AeCKqfuxVwNTdGcdMaCObPAenRvU9euSpIPpDbwRe785yHLWMbMvga1AfWPc3qGfXH32MrO3Q19vA3vk6kDSTEmrJK16992mzTDnOE766NC+iHYhUsAzl07i4mOG5oziPKiyc5YTykd8jeimMw6uLh/QuwtPzR7PL74zYrf0EpFOu5O08v+2fcawPcuzZJ2aIe10kg4m1zMXtT+K3anzderXrWx2g5mNNrPRlZV17x4cx3HyUVleynnjGw6aWV/omh07ax4WiEcgWPvmVvp270hpcd3HrIf2KmfqyOx9OJmmthseZvmr71NclK1TrkgHTU2SDmYzEJ+Q7APUTohQXUdSMdAFyJ2mraZ+nzx9vhOm0DJTac2bWcdxHCcQH+HU3rj56Y6dDba/4e8Oyoq+/F8XHsl/TB+VVacxkcSG9CqnqFb+l8y+nyRJ0sE8AwyWNEBSe2A6sLhWncXAjFD+NvC41eOOw9TXR5IOC0+PnQ4sytHXjJjccRynyblrZv4HCjIbL6cf3JeO7Yt59ac1eWmG9CrP16yayfvtyYPn16y31I61BlRP22VSM/94yjAAFn53FPNO3C+r7oDKThTXGlUN75P8k3CJTcKZ2ZeSZgFLgCLgJjN7UdIVwCozWwz8FvidpI1EI5fpmfaSNgEVQHtJ04DJZrYeOAe4BegAPBJeAPOBuyWdBfwvcEpStjmO4xw6sAdnHN6fL3fV3R8zbM8K1sydTEWH6F9sfMqsMZGfH7ngSJ7YkD0Zc+/ZY9jy0edsDxsv+/WI1nRmjhvEzHGDAPjTy9nry2bZ160vvUFTkuhVzOxh4OFasrmx8mfkcQRm1j+PfBVQ51ENM3sfmPg11HUcx2kUl9caKcTp0rEk6/2YgT3oVVHKoMrsp7emjtyLRatzp1Pep6qCfaoqsmSj+3cHorWXfaoqcj5w8I0hlSyeNZYTFy4D4PQx/bjyoZrQMLlGREngO/kdx3GagTvzTKn9+ykjWLT6Le4/9/BG9Scpp3PJEJ8C69ejI+2CU+nWsYTf5MkA2tS4g3Ecx2lBiovasWn+8YleQ4iykmjJ/brvHlg9CkoadzCO4zgpp6ykHT/71nCGLHudwwY2X+pmdzCO4zjNzM1nHEzH9o1LK/BV+O+LxvH4S1uQRGV5KbOPHZb4NeO4g3Ecx2lmxg/LGWikydl7j3L23qPhx6KTwhOOOY7jOIngDsZxHMdJBHcwjuM4TiK4g3Ecx3ESwR2M4ziOkwjuYBzHcZxEcAfjOI7jJII7GMdxHCcRtDvZ0NKKpI+ADS2tRzPTE3ivpZVoRgrNXig8mwvNXmh5m/uZWYMpgQt9J/8GM2uesKKtBEmrCsnmQrMXCs/mQrMX2o7NPkXmOI7jJII7GMdxHCcRCt3B3NDSCrQAhWZzodkLhWdzodkLbcTmgl7kdxzHcZKj0EcwjuM4TkK0eQcj6SZJWySti8m6S3pM0ivh2C3IJelaSRslvSDpwFibGaH+K5JmxOQHSVob2lwrhcTWLYikvpKekPQXSS9KuiDIU2m3pDJJKyWtCfbOC/IBklYE3e+S1D7IS8P7jeF8/1hflwT5BknHxOTHBtlGSXOa28Z8SCqS9Lykh8L7VNssaVP43a2WtCrIUvm7Dvp0lXSvpJfC3/OYVNlrZm36BYwDDgTWxWQ/B+aE8hxgQShPAR4BBBwGrAjy7sBr4dgtlLuFcyuBMaHNI8BxrcDmKuDAUC4HXgb2TavdQYfOoVwCrAh23A1MD/JfA+eE8rnAr0N5OnBXKO8LrAFKgQHAq0BReL0KDATahzr7tvT3HHS+CLgDeCi8T7XNwCagZy1ZKn/XQZ9bge+Hcnuga5rsbfE/oCb6kvqT7WA2AFWhXEW03wXgeuC02vWA04DrY/Lrg6wKeCkmz6rXWl7AIuDoQrAb6Ag8BxxKtNGsOMjHAEtCeQkwJpSLQz0BlwCXxPpaEtpVtw3yrHotaGsfYCkwAXgo2JB2mzdR18Gk8ncNVACvE9bC02hvm58iy0MvM3sbIBwz+Ul7A2/E6m0Osvrkm3PIWw1hKmQU0V19au0OU0WrgS3AY0R33381sy9DlbiO1XaF81uBHjT+c2hprgFmA7vC+x6k32YDHpX0rKSZQZbW3/VA4F3g5jANeqOkTqTI3rQ6mHzkmn+0ryBvFUjqDNwHXGhm2+qrmkPWpuw2s51mNpLorv4QYJ9c1cKxzdsr6QRgi5k9GxfnqJoamwNjzexA4DjgPEnj6qnb1m0uJpre/5WZjQK2E02J5aPN2ZtWB/OOpCqAcNwS5JuBvrF6fYC3GpD3ySFvcSSVEDmX283sD0GcervN7K/AH4nmoLtKyoQ7iutYbVc43wX4gMZ/Di3JWOBESZuA3xNNk11Dum3GzN4Kxy3A/UQ3E2n9XW8GNpvZivD+XiKHkx57W2r+sYnnMvuTvQZzFdmLZD8P5ePJXiRbGeTdieZCu4XX60D3cO6ZUDezSDalFdgr4DbgmlryVNoNVAJdQ7kD8BRwAnAP2Qve54byeWQveN8dyvuRveD9GtFid3EoD6BmwXu/lv6eY/YfRc0if2ptBjoB5bHyn4Fj0/q7Dvo8BQwN5cuDramxt8X/eJrgC7oTeBvYQeSxzyKae14KvBKOmQ9bwHVE8/drgdGxfs4ENobX38fko4F1oc1Cai3ItZDNRxANdV8AVofXlLTaDQwHng/2rgPmBvlAoqdkNhL94y0N8rLwfmM4PzDW16XBpg3EnqgJn9/L4dylLf0d17L/KGocTGptDratCa8XMzql9Xcd9BkJrAq/7QeIHERq7PWd/I7jOE4ipHUNxnEcx2lh3ME4juM4ieAOxnEcx0kEdzCO4zhOIriDcRzHcRLBHYyTaiTtDJF510h6TtLhDdTvKunc3ej3j5JafU705kTSLZK+3dJ6OK0HdzBO2vnUzEaa2QiigI4/a6B+V6LIxK2S2C5+x2n1uINxCokK4EOI4rhJWhpGNWslTQ115gODwqjnqlB3dqizRtL8WH+nKMpT87KkI0PdIklXSXom5Oz4QZBXSXoy9LsuUz9OyIWyIPS5UtLeQX6LpF9IegJYEPKFPBD6Xy5peMymm4OuL0g6OcgnS3o62HpPiGGHpPmS1oe6/xZkpwT91kh6sgGbJGlh6OM/qQnK6DhAFC7CcdJMhxCFuYwofPmEIP8MOMnMtknqCSyXtJgoNMf+FgXWRNJxwDTgUDP7RFL3WN/FZnaIpCnAvwCTiCJJbDWzgyWVAsskPQp8iyg8/k8kFRGlHcjFttDn6USxx04I8iHAJDPbKemXwPNmNk3SBKKwQSOBy8K1Dwi6dwu2/XNou13SPwEXSVoInAQMMzOT1DVcZy5wjJm9GZPls2kUMBQ4AOgFrAdu2q1vxSkI3ME4aefTmLMYA9wmaX+isBs/DdF6dxGFMe+Vo/0k4GYz+wTAzD6IncsEGX2WKB4ewGRgeGwtogswmCgm1E0hSOkDZrY6j753xo5Xx+T3mNnOUD4CODno87ikHpK6BF2nZxqY2YchKvO+RE4BorhjTwPbiJzsjWH08VBotgy4RdLdMfvy2TQOuDPo9Zakx/PY5BQo7mCcgsHMng539JVEcbgqgYPMbEeIWlyWo5nIH+L883DcSc3fkoDzzWxJnY4iZ3Y88DtJV5nZbbnUzFPeXkunXO1y6SrgMTM7LYc+hwATiZzSLGCCmZ0t6dCg52pJI/PZFEZuHmvKyYuvwTgFg6RhRJGE3ye6C98SnMt4oF+o9hFRGuoMjwJnSuoY+ohPkeViCXBOGKkgaYikTpL6hev9BvgtUVj2XJwaOz6dp86TwPdC/0cB71mUD+hRIkeRsbcbsBwYG1vP6Rh06gx0MbOHgQuJptiQNMjMVpjZXKKsmH3z2RT0mB7WaKqA8Q18Nk6B4SMYJ+1k1mAguhOfEdYxbgcelLSKKBr1SwBm9r6kZZLWAY+Y2cXhLn6VpC+Ah4Ef13O9G4mmy55TNCf1LtEazlHAxZJ2AB8Dp+dpXyppBdHNX51RR+ByoiyILwCfADOC/F+B64LuO4F5ZvYHSWcAd4b1E4jWZD4CFkkqC5/LP4ZzV0kaHGRLiSIbv5DHpvuJ1rTWEkVl/lM9n4tTgHg0ZcdpJYRputFm9l5L6+I4TYFPkTmO4ziJ4CMYx3EcJxF8BOM4juMkgjsYx3EcJxHcwTiO4ziJ4A7GcRzHSQR3MI7jOE4iuINxHMdxEuH/AQfJwynjF4jBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.recorder.plot_losses(last=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "learn.load('1');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "Total time: 11:32 <p><table style='width:300px; margin-bottom:10px'>\n",
       "  <tr>\n",
       "    <th>epoch</th>\n",
       "    <th>train_loss</th>\n",
       "    <th>valid_loss</th>\n",
       "    <th>exp_rmspe</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>1</th>\n",
       "    <th>0.012223</th>\n",
       "    <th>0.014312</th>\n",
       "    <th>0.116988</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>2</th>\n",
       "    <th>0.012001</th>\n",
       "    <th>0.017789</th>\n",
       "    <th>0.117619</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>3</th>\n",
       "    <th>0.011402</th>\n",
       "    <th>0.035596</th>\n",
       "    <th>0.114396</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>4</th>\n",
       "    <th>0.010067</th>\n",
       "    <th>0.015125</th>\n",
       "    <th>0.113652</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>5</th>\n",
       "    <th>0.009148</th>\n",
       "    <th>0.031326</th>\n",
       "    <th>0.116344</th>\n",
       "  </tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(5, 3e-4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "Total time: 11:31 <p><table style='width:300px; margin-bottom:10px'>\n",
       "  <tr>\n",
       "    <th>epoch</th>\n",
       "    <th>train_loss</th>\n",
       "    <th>valid_loss</th>\n",
       "    <th>exp_rmspe</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>1</th>\n",
       "    <th>0.011840</th>\n",
       "    <th>0.013236</th>\n",
       "    <th>0.110483</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>2</th>\n",
       "    <th>0.010765</th>\n",
       "    <th>0.057664</th>\n",
       "    <th>0.129586</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>3</th>\n",
       "    <th>0.010101</th>\n",
       "    <th>0.042744</th>\n",
       "    <th>0.111584</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>4</th>\n",
       "    <th>0.008820</th>\n",
       "    <th>0.116893</th>\n",
       "    <th>0.135458</th>\n",
       "  </tr>\n",
       "  <tr>\n",
       "    <th>5</th>\n",
       "    <th>0.009144</th>\n",
       "    <th>0.017969</th>\n",
       "    <th>0.126323</th>\n",
       "  </tr>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learn.fit_one_cycle(5, 3e-4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(10th place in the competition was 0.108)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_preds=learn.get_preds(DatasetType.Test)\n",
    "test_df[\"Sales\"]=np.exp(test_preds[0].data).numpy().T[0]\n",
    "test_df[[\"Id\",\"Sales\"]]=test_df[[\"Id\",\"Sales\"]].astype(\"int\")\n",
    "test_df[[\"Id\",\"Sales\"]].to_csv(\"rossmann_submission.csv\",index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}