fcc_book_recommendation_knn.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Bambillo1/ML_Course_9-11-20/blob/main/fcc_book_recommendation_knn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uGd4NYQX1Rf_"
      },
      "source": [
        "*Note: You are currently reading this using Google Colaboratory which is a cloud-hosted version of Jupyter Notebook. This is a document containing both text cells for documentation and runnable code cells. If you are unfamiliar with Jupyter Notebook, watch this 3-minute introduction before starting this challenge: https://www.youtube.com/watch?v=inN8seMm7UI*\n",
        "\n",
        "---\n",
        "\n",
        "In this challenge, you will create a book recommendation algorithm using **K-Nearest Neighbors**.\n",
        "\n",
        "You will use the [Book-Crossings dataset](http://www2.informatik.uni-freiburg.de/~cziegler/BX/). This dataset contains 1.1 million ratings (scale of 1-10) of 270,000 books by 90,000 users. \n",
        "\n",
        "After importing and cleaning the data, use `NearestNeighbors` from `sklearn.neighbors` to develop a model that shows books that are similar to a given book. The Nearest Neighbors algorithm measures distance to determine the “closeness” of instances.\n",
        "\n",
        "Create a function named `get_recommends` that takes a book title (from the dataset) as an argument and returns a list of 5 similar books with their distances from the book argument.\n",
        "\n",
        "This code:\n",
        "\n",
        "`get_recommends(\"The Queen of the Damned (Vampire Chronicles (Paperback))\")`\n",
        "\n",
        "should return:\n",
        "\n",
        "```\n",
        "[\n",
        "  'The Queen of the Damned (Vampire Chronicles (Paperback))',\n",
        "  [\n",
        "    ['Catch 22', 0.793983519077301], \n",
        "    ['The Witching Hour (Lives of the Mayfair Witches)', 0.7448656558990479], \n",
        "    ['Interview with the Vampire', 0.7345068454742432],\n",
        "    ['The Tale of the Body Thief (Vampire Chronicles (Paperback))', 0.5376338362693787],\n",
        "    ['The Vampire Lestat (Vampire Chronicles, Book II)', 0.5178412199020386]\n",
        "  ]\n",
        "]\n",
        "```\n",
        "\n",
        "Notice that the data returned from `get_recommends()` is a list. The first element in the list is the book title passed in to the function. The second element in the list is a list of five more lists. Each of the five lists contains a recommended book and the distance from the recommended book to the book passed in to the function.\n",
        "\n",
        "If you graph the dataset (optional), you will notice that most books are not rated frequently. To ensure statistical significance, remove from the dataset users with less than 200 ratings and books with less than 100 ratings.\n",
        "\n",
        "The first three cells import libraries you may need and the data to use. The final cell is for testing. Write all your code in between those cells."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "Y1onB6kUvo4Z"
      },
      "outputs": [],
      "source": [
        "# import libraries (you may add additional imports but you may not have to)\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from scipy.sparse import csr_matrix\n",
        "from sklearn.neighbors import NearestNeighbors\n",
        "import matplotlib.pyplot as plt"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "iAQGqqO_vo4d",
        "outputId": "e40f6806-9d7d-49c5-c97a-f56d2a0b75df",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-02-22 14:50:09--  https://cdn.freecodecamp.org/project-data/books/book-crossings.zip\n",
            "Resolving cdn.freecodecamp.org (cdn.freecodecamp.org)... 104.26.3.33, 172.67.70.149, 104.26.2.33, ...\n",
            "Connecting to cdn.freecodecamp.org (cdn.freecodecamp.org)|104.26.3.33|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 26085508 (25M) [application/zip]\n",
            "Saving to: ‘book-crossings.zip’\n",
            "\n",
            "book-crossings.zip  100%[===================>]  24.88M  27.9MB/s    in 0.9s    \n",
            "\n",
            "2022-02-22 14:50:10 (27.9 MB/s) - ‘book-crossings.zip’ saved [26085508/26085508]\n",
            "\n",
            "Archive:  book-crossings.zip\n",
            "  inflating: BX-Book-Ratings.csv     \n",
            "  inflating: BX-Books.csv            \n",
            "  inflating: BX-Users.csv            \n"
          ]
        }
      ],
      "source": [
        "# get data files\n",
        "!wget https://cdn.freecodecamp.org/project-data/books/book-crossings.zip\n",
        "\n",
        "!unzip book-crossings.zip\n",
        "\n",
        "books_filename = 'BX-Books.csv'\n",
        "ratings_filename = 'BX-Book-Ratings.csv'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "NClILWOiEd6Q"
      },
      "outputs": [],
      "source": [
        "# import csv data into dataframes\n",
        "df_books = pd.read_csv(\n",
        "    books_filename,\n",
        "    encoding = \"ISO-8859-1\",\n",
        "    sep=\";\",\n",
        "    header=0,\n",
        "    names=['isbn', 'title', 'author'],\n",
        "    usecols=['isbn', 'title', 'author'],\n",
        "    dtype={'isbn': 'str', 'title': 'str', 'author': 'str'})\n",
        "\n",
        "df_ratings = pd.read_csv(\n",
        "    ratings_filename,\n",
        "    encoding = \"ISO-8859-1\",\n",
        "    sep=\";\",\n",
        "    header=0,\n",
        "    names=['user', 'isbn', 'rating'],\n",
        "    usecols=['user', 'isbn', 'rating'],\n",
        "    dtype={'user': 'int32', 'isbn': 'str', 'rating': 'float32'})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "xAcXjkCFCh0A",
        "outputId": "eabf4c1c-412b-4e3b-cf61-301379debf6f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "              isbn  ...                author\n",
            "0       0195153448  ...    Mark P. O. Morford\n",
            "1       0002005018  ...  Richard Bruce Wright\n",
            "2       0060973129  ...          Carlo D'Este\n",
            "3       0374157065  ...      Gina Bari Kolata\n",
            "4       0393045218  ...       E. J. W. Barber\n",
            "...            ...  ...                   ...\n",
            "271374  0440400988  ...        Paula Danziger\n",
            "271375  0525447644  ...            Teri Sloat\n",
            "271376  006008667X  ...      Christine Wicker\n",
            "271377  0192126040  ...                 Plato\n",
            "271378  0767409752  ...   Christopher  Biffle\n",
            "\n",
            "[271379 rows x 3 columns]\n"
          ]
        }
      ],
      "source": [
        "# add your code here - consider creating a new cell for each section of code\n",
        "print(df_books)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(df_ratings)"
      ],
      "metadata": {
        "id": "TZyR1G8a7lHp",
        "outputId": "a0be7ee9-cd8b-4875-8f3f-df897dc2c4be",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "           user         isbn  rating\n",
            "0        276725   034545104X     0.0\n",
            "1        276726   0155061224     5.0\n",
            "2        276727   0446520802     0.0\n",
            "3        276729   052165615X     3.0\n",
            "4        276729   0521795028     6.0\n",
            "...         ...          ...     ...\n",
            "1149775  276704   1563526298     9.0\n",
            "1149776  276706   0679447156     0.0\n",
            "1149777  276709   0515107662    10.0\n",
            "1149778  276721   0590442449    10.0\n",
            "1149779  276723  05162443314     8.0\n",
            "\n",
            "[1149780 rows x 3 columns]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Calculate user and book rating counts\n",
        "user_RatingCount = df_ratings.groupby('user')['rating'].count().reset_index().rename(columns = {'rating':'userTotalRatingCount'})\n",
        "book_RatingCount = df_ratings.groupby('isbn')['rating'].count().reset_index().rename(columns = {'rating':'bookTotalRatingCount'})\n",
        "\n",
        "# Add to df_ratings\n",
        "df_ratings = df_ratings.merge(user_RatingCount,how='left', left_on='user', right_on='user')\n",
        "df_ratings = df_ratings.merge(book_RatingCount, how='left', left_on='isbn', right_on='isbn')\n",
        "\n",
        "# Filter data for statistical significance\n",
        "df_ratings_2 =df_ratings.loc[(df_ratings['userTotalRatingCount']>=200) & (df_ratings['bookTotalRatingCount']>=100)]"
      ],
      "metadata": {
        "id": "oXXF1cO57oNa"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# merge data sets\n",
        "books_with_ratings = pd.merge(df_ratings_2, df_books, on='isbn')\n",
        "print(books_with_ratings)"
      ],
      "metadata": {
        "id": "rmMOusOQ7oUA",
        "outputId": "3336cd10-54cb-4626-cfc3-27a6a9b167c0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "         user  ...             author\n",
            "0      277427  ...  James Finn Garner\n",
            "1        3363  ...  James Finn Garner\n",
            "2       11676  ...  James Finn Garner\n",
            "3       12538  ...  James Finn Garner\n",
            "4       13552  ...  James Finn Garner\n",
            "...       ...  ...                ...\n",
            "49512  238864  ...  Patricia Cornwell\n",
            "49513  251843  ...  Patricia Cornwell\n",
            "49514  253821  ...  Patricia Cornwell\n",
            "49515  265115  ...  Patricia Cornwell\n",
            "49516  266226  ...  Patricia Cornwell\n",
            "\n",
            "[49517 rows x 7 columns]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Remove duplicates\n",
        "books_with_ratings_2 = books_with_ratings.drop_duplicates(['title', 'user'])"
      ],
      "metadata": {
        "id": "Y0bO_Kih77Gj"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Preparing data table for analysis\n",
        "books_with_ratings_pivot = pd.pivot_table(data=books_with_ratings_2, values='rating', index='title', columns='user').fillna(0)\n",
        "print(books_with_ratings_pivot)"
      ],
      "metadata": {
        "id": "BfMXV0t58Jp-",
        "outputId": "76ec28aa-42e0-484e-e100-746a8cb6b6c6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "user                                                254     ...  278418\n",
            "title                                                       ...        \n",
            "1984                                                   9.0  ...     0.0\n",
            "1st to Die: A Novel                                    0.0  ...     0.0\n",
            "2nd Chance                                             0.0  ...     0.0\n",
            "4 Blondes                                              0.0  ...     0.0\n",
            "A Beautiful Mind: The Life of Mathematical Geni...     0.0  ...     0.0\n",
            "...                                                    ...  ...     ...\n",
            "Without Remorse                                        0.0  ...     0.0\n",
            "Year of Wonders                                        0.0  ...     0.0\n",
            "You Belong To Me                                       0.0  ...     0.0\n",
            "Zen and the Art of Motorcycle Maintenance: An I...     0.0  ...     0.0\n",
            "\\O\\\" Is for Outlaw\"                                    0.0  ...     0.0\n",
            "\n",
            "[673 rows x 888 columns]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert to 2D matrıx\n",
        "books_with_ratings_matrix = csr_matrix(books_with_ratings_pivot.values)"
      ],
      "metadata": {
        "id": "nN-x64Kn8Jwu"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Train Model\n",
        "model_knn = NearestNeighbors(algorithm='auto', metric='cosine')\n",
        "model_knn.fit(books_with_ratings_matrix)"
      ],
      "metadata": {
        "id": "lvDpwghY8agz",
        "outputId": "4577b30d-e4b3-453e-f4f3-f72fc53b97d0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "NearestNeighbors(metric='cosine')"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "K7tAYxAN8a4N"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        ""
      ],
      "metadata": {
        "id": "_yl-ZemM8J3P"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "f5ZUd-L1SQz7"
      },
      "outputs": [],
      "source": [
        "# function to return recommended books - this will be tested\n",
        "def get_recommends(book = \"\"):\n",
        "    X = books_with_ratings_pivot[books_with_ratings_pivot.index == book]\n",
        "    X = X.to_numpy().reshape(1,-1)\n",
        "    distances, indices = model_knn.kneighbors(X,n_neighbors=8)\n",
        "    recommended_books = []\n",
        "    for x in reversed(range(1,6)):\n",
        "          bookrecommended = [books_with_ratings_pivot.index[indices.flatten()[x]], distances.flatten()[x]]\n",
        "          recommended_books.append(bookrecommended)\n",
        "    recommended_books = [book, recommended_books]\n",
        "\n",
        "    return recommended_books"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eat9A2TKawHU"
      },
      "source": [
        "Use the cell below to test your function. The `test_book_recommendation()` function will inform you if you passed the challenge or need to keep trying."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "jd2SLCh8oxMh",
        "outputId": "d5d4afb8-7b9b-4635-8fc3-0f488d8d1536",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[\"Where the Heart Is (Oprah's Book Club (Paperback))\", [[\"I'll Be Seeing You\", 0.8016211], ['The Weight of Water', 0.77085835], ['The Surgeon', 0.7699411], ['I Know This Much Is True', 0.7677075], ['The Lovely Bones: A Novel', 0.7234864]]]\n",
            "You passed the challenge! 🎉🎉🎉🎉🎉\n"
          ]
        }
      ],
      "source": [
        "books = get_recommends(\"Where the Heart Is (Oprah's Book Club (Paperback))\")\n",
        "print(books)\n",
        "\n",
        "def test_book_recommendation():\n",
        "  test_pass = True\n",
        "  recommends = get_recommends(\"Where the Heart Is (Oprah's Book Club (Paperback))\")\n",
        "  if recommends[0] != \"Where the Heart Is (Oprah's Book Club (Paperback))\":\n",
        "    test_pass = False\n",
        "  recommended_books = [\"I'll Be Seeing You\", 'The Weight of Water', 'The Surgeon', 'I Know This Much Is True']\n",
        "  recommended_books_dist = [0.8, 0.77, 0.77, 0.77]\n",
        "  for i in range(2): \n",
        "    if recommends[1][i][0] not in recommended_books:\n",
        "      test_pass = False\n",
        "    if abs(recommends[1][i][1] - recommended_books_dist[i]) >= 0.05:\n",
        "      test_pass = False\n",
        "  if test_pass:\n",
        "    print(\"You passed the challenge! 🎉🎉🎉🎉🎉\")\n",
        "  else:\n",
        "    print(\"You haven't passed yet. Keep trying!\")\n",
        "\n",
        "test_book_recommendation()"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "fcc_book_recommendation_knn.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}