Spark RDD - Item-based Collaborative Filtering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Finding movie recommendations to a given movie through Cosine Similarity using Spark RDD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This project processed 1M MovieLens records downloaded from <a href = 'https://grouplens.org/datasets/movielens/'>groulens.org</a> using Spark RDD. The code below can be executed either on the Jupyter Notebook powered by an [Amazon EMR cluster](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-what-is-emr.html), or converted into a Python script and run from the master node of the cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyspark import SparkContext, SparkConf\n",
    "from math import sqrt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate SparkConf() object\n",
    "\n",
    "# Run the code below if executing locally\n",
    "conf = SparkConf().setMaster(\"local\").setAppName(\"SimilarMovie\")\n",
    "# conf = SparkConf().setAppName('SimilarMovie') \\\n",
    "#                   .set(\"spark.serializer\", \n",
    "#                        \"org.apache.spark.serializer.PickleSerializer\")\n",
    "\n",
    "# Run the code below if executing on AWS EMR cluster\n",
    "# conf = SparkConf()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate sc object\n",
    "sc = SparkContext(conf=conf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**About the dataset \"ratings.dat\"**\n",
    "\n",
    "All ratings are contained in the file \"./data/ml-1m/ratings.dat\" and are in the\n",
    "following format:\n",
    "\n",
    "`UserID::MovieID::Rating::Timestamp`\n",
    "\n",
    "- **UserIDs** range between 1 and 6040 \n",
    "- **MovieIDs** range between 1 and 3952\n",
    "- **Ratings** are made on a 5-star scale (whole-star ratings only)\n",
    "- **Timestamp** is represented in seconds since the epoch as returned by time(2)\n",
    "- Each user has at least 20 ratings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1::1193::5::978300760\n",
      "1::661::3::978302109\n",
      "1::914::3::978301968\n",
      "1::3408::4::978300275\n",
      "1::2355::5::978824291\n"
     ]
    }
   ],
   "source": [
    "# Preview the first 5 rows in ratings.dat\n",
    "!head -n 5 ./data/ml-1m/ratings.dat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read RDD from ratings.dat\n",
    "\n",
    "# Run the code below if executing locally\n",
    "RDD_movies = sc.textFile(\"./data/ml-1m/ratings.dat\")\n",
    "\n",
    "# Run the code below if executing on AWS EMR cluster\n",
    "# RDD_movies = sc.textFile('s3n://maxineproject/ml-1m/ratings.dat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['1::1193::5::978300760',\n",
       " '1::661::3::978302109',\n",
       " '1::914::3::978301968',\n",
       " '1::3408::4::978300275',\n",
       " '1::2355::5::978824291']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Show the first 5 records in RDD_movies\n",
    "# UserID::MovieID::Rating::Timestamp\n",
    "RDD_movies.collect()[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert each record to the following format:\n",
    "\n",
    "`(UserID, (MovieID, Rating))`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "RDD_ratings = RDD_movies.map(\n",
    "    lambda x: (int(x.split(\"::\")[0]), \n",
    "               (int(x.split(\"::\")[1]), float(x.split(\"::\")[2])))\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(1, (1193, 5.0)),\n",
       " (1, (661, 3.0)),\n",
       " (1, (914, 3.0)),\n",
       " (1, (3408, 4.0)),\n",
       " (1, (2355, 5.0))]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RDD_ratings.collect()[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Self-join RDD_ratings"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_**Tip**_: Call `partitionBy()` method to break RDD into smaller chunks before running large operations on RDD such as `join()`, `reduceByKey()`, `groupByKey()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "RDD_ratings = RDD_ratings.partitionBy(100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Join the same RDD by keys\n",
    "RDD_ratings_joined = RDD_ratings.join(RDD_ratings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(100, ((648, 2.0), (648, 2.0))),\n",
       " (100, ((648, 2.0), (800, 5.0))),\n",
       " (100, ((648, 2.0), (3948, 3.0))),\n",
       " (100, ((648, 2.0), (1408, 3.0))),\n",
       " (100, ((648, 2.0), (1196, 4.0)))]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RDD_ratings_joined.collect()[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove duplicate pairs\n",
    "RDD_ratings_filtered = RDD_ratings_joined.filter(\n",
    "    lambda x: x[1][0][0] < x[1][1][0]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(100, ((648, 2.0), (800, 5.0))),\n",
       " (100, ((648, 2.0), (3948, 3.0))),\n",
       " (100, ((648, 2.0), (1408, 3.0))),\n",
       " (100, ((648, 2.0), (1196, 4.0))),\n",
       " (100, ((648, 2.0), (1197, 4.0)))]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RDD_ratings_filtered.collect()[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert each record to the following format:\n",
    "\n",
    "`((movieId1, movieId2), (rating1, rating2))`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "RDD_ratings_paired = RDD_ratings_filtered.map(\n",
    "    lambda x: ((x[1][0][0], x[1][1][0]), (x[1][0][1], x[1][1][1]))\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[((648, 800), (2.0, 5.0)),\n",
       " ((648, 3948), (2.0, 3.0)),\n",
       " ((648, 1408), (2.0, 3.0)),\n",
       " ((648, 1196), (2.0, 4.0)),\n",
       " ((648, 1197), (2.0, 4.0))]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RDD_ratings_paired.collect()[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Group by keys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Group the records with same keys into the following format:\n",
    "\n",
    "( (movieId1, movieId2), [(rating1, rating2)$_1 $, (rating1, rating2)$_2 $, ... (rating1, rating2)$_i $] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "RDD_ratings_paired = RDD_ratings_paired.partitionBy(100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# group by key (movieId1, movieId2)\n",
    "RDD_ratings_grouped = RDD_ratings_paired.groupByKey().mapValues(list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((648, 800),\n",
       " [(3.0, 5.0),\n",
       "  (2.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (2.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (2.0, 2.0),\n",
       "  (2.0, 5.0),\n",
       "  (2.0, 5.0),\n",
       "  (5.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (2.0, 1.0),\n",
       "  (4.0, 5.0),\n",
       "  (1.0, 4.0),\n",
       "  (2.0, 3.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 2.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 3.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 5.0),\n",
       "  (2.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (2.0, 3.0),\n",
       "  (4.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (2.0, 5.0),\n",
       "  (4.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (2.0, 3.0),\n",
       "  (2.0, 3.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (2.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 3.0),\n",
       "  (2.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (5.0, 3.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 3.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (2.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (5.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 3.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 1.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 3.0),\n",
       "  (2.0, 3.0),\n",
       "  (4.0, 4.0),\n",
       "  (4.0, 3.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (3.0, 2.0),\n",
       "  (2.0, 5.0),\n",
       "  (4.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 3.0),\n",
       "  (2.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (2.0, 3.0),\n",
       "  (2.0, 4.0),\n",
       "  (2.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 2.0),\n",
       "  (1.0, 3.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 2.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 4.0),\n",
       "  (5.0, 5.0),\n",
       "  (2.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 4.0),\n",
       "  (4.0, 3.0),\n",
       "  (4.0, 4.0),\n",
       "  (1.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (5.0, 2.0),\n",
       "  (4.0, 5.0),\n",
       "  (2.0, 5.0),\n",
       "  (5.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (5.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (5.0, 3.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 3.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 3.0),\n",
       "  (2.0, 5.0),\n",
       "  (3.0, 3.0),\n",
       "  (2.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (4.0, 4.0),\n",
       "  (2.0, 4.0),\n",
       "  (2.0, 5.0),\n",
       "  (2.0, 4.0),\n",
       "  (1.0, 4.0),\n",
       "  (5.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (4.0, 1.0),\n",
       "  (4.0, 3.0),\n",
       "  (4.0, 3.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 4.0),\n",
       "  (1.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 3.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (4.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 3.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (1.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (2.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (5.0, 4.0),\n",
       "  (5.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 4.0),\n",
       "  (2.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 2.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (5.0, 5.0),\n",
       "  (2.0, 3.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 3.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 3.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 3.0),\n",
       "  (2.0, 5.0),\n",
       "  (5.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (2.0, 3.0),\n",
       "  (4.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 4.0),\n",
       "  (2.0, 5.0),\n",
       "  (4.0, 3.0),\n",
       "  (4.0, 3.0),\n",
       "  (2.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (4.0, 4.0),\n",
       "  (2.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (2.0, 4.0),\n",
       "  (5.0, 5.0),\n",
       "  (3.0, 3.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 4.0),\n",
       "  (3.0, 3.0),\n",
       "  (3.0, 3.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 3.0),\n",
       "  (1.0, 5.0),\n",
       "  (1.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (1.0, 4.0),\n",
       "  (4.0, 2.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 4.0),\n",
       "  (5.0, 4.0),\n",
       "  (4.0, 3.0),\n",
       "  (2.0, 4.0),\n",
       "  (4.0, 5.0),\n",
       "  (4.0, 3.0),\n",
       "  (2.0, 4.0),\n",
       "  (2.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (2.0, 5.0),\n",
       "  (4.0, 3.0),\n",
       "  (4.0, 3.0),\n",
       "  (4.0, 3.0),\n",
       "  (3.0, 1.0),\n",
       "  (4.0, 5.0),\n",
       "  (1.0, 3.0),\n",
       "  (3.0, 5.0),\n",
       "  (2.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 4.0),\n",
       "  (2.0, 4.0),\n",
       "  (4.0, 4.0),\n",
       "  (2.0, 4.0),\n",
       "  (2.0, 4.0),\n",
       "  (3.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 5.0),\n",
       "  (1.0, 5.0),\n",
       "  (4.0, 5.0),\n",
       "  (3.0, 3.0),\n",
       "  (2.0, 5.0),\n",
       "  (2.0, 5.0),\n",
       "  (3.0, 5.0)])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RDD_ratings_grouped.collect()[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Compute Cosine Similarity score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Declare a function that computes Cosine Similarity score for each movie pair\n",
    "def computeCosineSimilarity(ratingPairs):\n",
    "    # Initialize variables to keep track of the sums and the number of pairs\n",
    "    sumXX = 0\n",
    "    sumYY = 0\n",
    "    sumXY = 0\n",
    "    numPairs = 0\n",
    "\n",
    "    # Loop through each rating pair in the input list\n",
    "    for ratingPair in ratingPairs:\n",
    "        # Extract the individual ratings from the pair\n",
    "        ratingX = ratingPair[0]\n",
    "        ratingY = ratingPair[1]\n",
    "\n",
    "        # Calculate the sum of squares for both ratings (X and Y)\n",
    "        sumXX += ratingX * ratingX\n",
    "        sumYY += ratingY * ratingY\n",
    "        # Calculate the sum of the product of ratings (X and Y)\n",
    "        sumXY += ratingX * ratingY\n",
    "        # Increment the count of rating pairs processed\n",
    "        numPairs += 1\n",
    "\n",
    "    # Calculate the numerator and denominator for the cosine similarity\n",
    "    numerator = sumXY\n",
    "    denominator = sqrt(sumXX) * sqrt(sumYY)\n",
    "    # Calculate the cosine similarity score\n",
    "    score = numerator / denominator\n",
    "\n",
    "    # Return the Cosine Similarity score along with the number of rating pairs\n",
    "    # numPairs also indicates how many people watched the same pair of movies\n",
    "    return (score, numPairs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compute Cosine Similarity score for each movie pair\n",
    "RDD_pairs_similarities = RDD_ratings_grouped.mapValues(\n",
    "    lambda x: computeCosineSimilarity(x)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[((648, 800), (0.939054432011561, 274)),\n",
       " ((648, 3948), (0.9464824388307304, 414)),\n",
       " ((648, 1408), (0.9440372259647507, 565)),\n",
       " ((648, 1196), (0.9553752925210446, 1102)),\n",
       " ((648, 1197), (0.952789624869145, 908))]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "RDD_pairs_similarities.collect()[:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Configure parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define the target movie ID for which we want to find similar movies\n",
    "movieID = 50\n",
    "# Set the threshold for similarity score\n",
    "scoreThreshold = 0.97\n",
    "# Set the threshold for numPairs\n",
    "coOccurenceThreshold = 50"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Filter** the RDD to get only the movie pairs for the target movie that **exceed the specified thresholds** for both Cosine Similarity score and number of rating pairs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "RDD_results = RDD_pairs_similarities.filter(\n",
    "    lambda x: ((x[0][0] == movieID) or (x[0][1] == movieID))\n",
    "    and (x[1][0] > scoreThreshold)\n",
    "    and (x[1][1] > coOccurenceThreshold)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Find the top 10 movie recommendations based on their Cosine Similarity score of ratings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sort the RDD to get the Top 10 movies with highest similarity scores\n",
    "top_10 = RDD_results.sortBy(lambda x: x[1][0], ascending=False) \\\n",
    "                    .collect()[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[((50, 172), (0.9895522078385338, 345)),\n",
       " ((50, 181), (0.9857230861253026, 480)),\n",
       " ((50, 174), (0.981760098872619, 380)),\n",
       " ((50, 141), (0.9789385605497993, 68)),\n",
       " ((50, 178), (0.9776576120448436, 109)),\n",
       " ((50, 408), (0.9775948291054827, 92)),\n",
       " ((50, 498), (0.9764692222674887, 138)),\n",
       " ((50, 194), (0.9751512937740359, 204)),\n",
       " ((50, 169), (0.9748681355460885, 103)),\n",
       " ((50, 114), (0.9741816128302572, 58))]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "top_10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Print the full information about the top 10 movie recommendations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**movies.dat** contained a mapping of movie ID(s) to their respective movie names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1::Toy Story (1995)::Animation|Children's|Comedy\n",
      "2::Jumanji (1995)::Adventure|Children's|Fantasy\n",
      "3::Grumpier Old Men (1995)::Comedy|Romance\n",
      "4::Waiting to Exhale (1995)::Comedy|Drama\n",
      "5::Father of the Bride Part II (1995)::Comedy\n"
     ]
    }
   ],
   "source": [
    "# Preview the first 5 rows in movies.dat\n",
    "!head -n 5 ./data/ml-1m/movies.dat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the code below if executing locally\n",
    "movie_lines = open(\n",
    "    \"./data/ml-1m/movies.dat\", encoding=\"ascii\", errors=\"ignore\"\n",
    ").readlines()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are running the code on Amazon EMR cluster with the MovieLens 1M dataset stored in S3 bucket, please start by executing `aws s3 cp [s3-file-path] ./` on the Master public DNS to move **movies.dat** from S3 to the current directory. Afterward, you can proceed to run the code below to read lines from **movies.dat**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the code below if executing on AWS EMR cluster\n",
    "# movie_lines = open('movies.dat', encoding='ascii', errors='ignore').readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"1::Toy Story (1995)::Animation|Children's|Comedy\\n\",\n",
       " \"2::Jumanji (1995)::Adventure|Children's|Fantasy\\n\",\n",
       " '3::Grumpier Old Men (1995)::Comedy|Romance\\n',\n",
       " '4::Waiting to Exhale (1995)::Comedy|Drama\\n',\n",
       " '5::Father of the Bride Part II (1995)::Comedy\\n']"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_lines[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a dictionary with movieID as key and [movieName, category] as value\n",
    "movies = {}\n",
    "for line in movie_lines:\n",
    "    movies[int(line.split(\"::\")[0])] = [\n",
    "        line.split(\"::\")[1],\n",
    "        line.split(\"::\")[2][:-1].replace(\"|\", \"/\"),\n",
    "    ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{1: ['Toy Story (1995)', \"Animation/Children's/Comedy\"],\n",
       " 2: ['Jumanji (1995)', \"Adventure/Children's/Fantasy\"],\n",
       " 3: ['Grumpier Old Men (1995)', 'Comedy/Romance'],\n",
       " 4: ['Waiting to Exhale (1995)', 'Comedy/Drama'],\n",
       " 5: ['Father of the Bride Part II (1995)', 'Comedy'],\n",
       " 6: ['Heat (1995)', 'Action/Crime/Thriller'],\n",
       " 7: ['Sabrina (1995)', 'Comedy/Romance'],\n",
       " 8: ['Tom and Huck (1995)', \"Adventure/Children's\"],\n",
       " 9: ['Sudden Death (1995)', 'Action'],\n",
       " 10: ['GoldenEye (1995)', 'Action/Adventure/Thriller'],\n",
       " 11: ['American President, The (1995)', 'Comedy/Drama/Romance'],\n",
       " 12: ['Dracula: Dead and Loving It (1995)', 'Comedy/Horror'],\n",
       " 13: ['Balto (1995)', \"Animation/Children's\"],\n",
       " 14: ['Nixon (1995)', 'Drama'],\n",
       " 15: ['Cutthroat Island (1995)', 'Action/Adventure/Romance'],\n",
       " 16: ['Casino (1995)', 'Drama/Thriller'],\n",
       " 17: ['Sense and Sensibility (1995)', 'Drama/Romance'],\n",
       " 18: ['Four Rooms (1995)', 'Thriller'],\n",
       " 19: ['Ace Ventura: When Nature Calls (1995)', 'Comedy'],\n",
       " 20: ['Money Train (1995)', 'Action'],\n",
       " 21: ['Get Shorty (1995)', 'Action/Comedy/Drama'],\n",
       " 22: ['Copycat (1995)', 'Crime/Drama/Thriller'],\n",
       " 23: ['Assassins (1995)', 'Thriller'],\n",
       " 24: ['Powder (1995)', 'Drama/Sci-Fi'],\n",
       " 25: ['Leaving Las Vegas (1995)', 'Drama/Romance'],\n",
       " 26: ['Othello (1995)', 'Drama'],\n",
       " 27: ['Now and Then (1995)', 'Drama'],\n",
       " 28: ['Persuasion (1995)', 'Romance'],\n",
       " 29: ['City of Lost Children, The (1995)', 'Adventure/Sci-Fi'],\n",
       " 30: ['Shanghai Triad (Yao a yao yao dao waipo qiao) (1995)', 'Drama'],\n",
       " 31: ['Dangerous Minds (1995)', 'Drama'],\n",
       " 32: ['Twelve Monkeys (1995)', 'Drama/Sci-Fi'],\n",
       " 33: ['Wings of Courage (1995)', 'Adventure/Romance'],\n",
       " 34: ['Babe (1995)', \"Children's/Comedy/Drama\"],\n",
       " 35: ['Carrington (1995)', 'Drama/Romance'],\n",
       " 36: ['Dead Man Walking (1995)', 'Drama'],\n",
       " 37: ['Across the Sea of Time (1995)', 'Documentary'],\n",
       " 38: ['It Takes Two (1995)', 'Comedy'],\n",
       " 39: ['Clueless (1995)', 'Comedy/Romance'],\n",
       " 40: ['Cry, the Beloved Country (1995)', 'Drama'],\n",
       " 41: ['Richard III (1995)', 'Drama/War'],\n",
       " 42: ['Dead Presidents (1995)', 'Action/Crime/Drama'],\n",
       " 43: ['Restoration (1995)', 'Drama'],\n",
       " 44: ['Mortal Kombat (1995)', 'Action/Adventure'],\n",
       " 45: ['To Die For (1995)', 'Comedy/Drama'],\n",
       " 46: ['How to Make an American Quilt (1995)', 'Drama/Romance'],\n",
       " 47: ['Seven (Se7en) (1995)', 'Crime/Thriller'],\n",
       " 48: ['Pocahontas (1995)', \"Animation/Children's/Musical/Romance\"],\n",
       " 49: ['When Night Is Falling (1995)', 'Drama/Romance'],\n",
       " 50: ['Usual Suspects, The (1995)', 'Crime/Thriller'],\n",
       " 51: ['Guardian Angel (1994)', 'Action/Drama/Thriller'],\n",
       " 52: ['Mighty Aphrodite (1995)', 'Comedy'],\n",
       " 53: ['Lamerica (1994)', 'Drama'],\n",
       " 54: ['Big Green, The (1995)', \"Children's/Comedy\"],\n",
       " 55: ['Georgia (1995)', 'Drama'],\n",
       " 56: ['Kids of the Round Table (1995)', \"Adventure/Children's/Fantasy\"],\n",
       " 57: ['Home for the Holidays (1995)', 'Drama'],\n",
       " 58: ['Postino, Il (The Postman) (1994)', 'Drama/Romance'],\n",
       " 59: ['Confessional, The (Le Confessionnal) (1995)', 'Drama/Mystery'],\n",
       " 60: ['Indian in the Cupboard, The (1995)', \"Adventure/Children's/Fantasy\"],\n",
       " 61: ['Eye for an Eye (1996)', 'Drama/Thriller'],\n",
       " 62: [\"Mr. Holland's Opus (1995)\", 'Drama'],\n",
       " 63: [\"Don't Be a Menace to South Central While Drinking Your Juice in the Hood (1996)\",\n",
       "  'Comedy'],\n",
       " 64: ['Two if by Sea (1996)', 'Comedy/Romance'],\n",
       " 65: ['Bio-Dome (1996)', 'Comedy'],\n",
       " 66: ['Lawnmower Man 2: Beyond Cyberspace (1996)', 'Sci-Fi/Thriller'],\n",
       " 67: ['Two Bits (1995)', 'Drama'],\n",
       " 68: ['French Twist (Gazon maudit) (1995)', 'Comedy/Romance'],\n",
       " 69: ['Friday (1995)', 'Comedy'],\n",
       " 70: ['From Dusk Till Dawn (1996)', 'Action/Comedy/Crime/Horror/Thriller'],\n",
       " 71: ['Fair Game (1995)', 'Action'],\n",
       " 72: ['Kicking and Screaming (1995)', 'Comedy/Drama'],\n",
       " 73: ['Misrables, Les (1995)', 'Drama/Musical'],\n",
       " 74: ['Bed of Roses (1996)', 'Drama/Romance'],\n",
       " 75: ['Big Bully (1996)', 'Comedy/Drama'],\n",
       " 76: ['Screamers (1995)', 'Sci-Fi/Thriller'],\n",
       " 77: ['Nico Icon (1995)', 'Documentary'],\n",
       " 78: ['Crossing Guard, The (1995)', 'Drama'],\n",
       " 79: ['Juror, The (1996)', 'Drama/Thriller'],\n",
       " 80: ['White Balloon, The (Badkonake Sefid ) (1995)', 'Drama'],\n",
       " 81: [\"Things to Do in Denver when You're Dead (1995)\", 'Crime/Drama/Romance'],\n",
       " 82: [\"Antonia's Line (Antonia) (1995)\", 'Drama'],\n",
       " 83: ['Once Upon a Time... When We Were Colored (1995)', 'Drama'],\n",
       " 84: ['Last Summer in the Hamptons (1995)', 'Comedy/Drama'],\n",
       " 85: ['Angels and Insects (1995)', 'Drama/Romance'],\n",
       " 86: ['White Squall (1996)', 'Adventure/Drama'],\n",
       " 87: ['Dunston Checks In (1996)', \"Children's/Comedy\"],\n",
       " 88: ['Black Sheep (1996)', 'Comedy'],\n",
       " 89: ['Nick of Time (1995)', 'Action/Thriller'],\n",
       " 90: ['Journey of August King, The (1995)', 'Drama'],\n",
       " 92: ['Mary Reilly (1996)', 'Drama/Thriller'],\n",
       " 93: ['Vampire in Brooklyn (1995)', 'Comedy/Romance'],\n",
       " 94: ['Beautiful Girls (1996)', 'Drama'],\n",
       " 95: ['Broken Arrow (1996)', 'Action/Thriller'],\n",
       " 96: ['In the Bleak Midwinter (1995)', 'Comedy'],\n",
       " 97: ['Hate (Haine, La) (1995)', 'Drama'],\n",
       " 98: ['Shopping (1994)', 'Action/Thriller'],\n",
       " 99: ['Heidi Fleiss: Hollywood Madam (1995)', 'Documentary'],\n",
       " 100: ['City Hall (1996)', 'Drama/Thriller'],\n",
       " 101: ['Bottle Rocket (1996)', 'Comedy'],\n",
       " 102: ['Mr. Wrong (1996)', 'Comedy'],\n",
       " 103: ['Unforgettable (1996)', 'Thriller'],\n",
       " 104: ['Happy Gilmore (1996)', 'Comedy'],\n",
       " 105: ['Bridges of Madison County, The (1995)', 'Drama/Romance'],\n",
       " 106: ['Nobody Loves Me (Keiner liebt mich) (1994)', 'Comedy/Drama'],\n",
       " 107: ['Muppet Treasure Island (1996)', \"Adventure/Children's/Comedy/Musical\"],\n",
       " 108: ['Catwalk (1995)', 'Documentary'],\n",
       " 109: ['Headless Body in Topless Bar (1995)', 'Comedy'],\n",
       " 110: ['Braveheart (1995)', 'Action/Drama/War'],\n",
       " 111: ['Taxi Driver (1976)', 'Drama/Thriller'],\n",
       " 112: ['Rumble in the Bronx (1995)', 'Action/Adventure/Crime'],\n",
       " 113: ['Before and After (1996)', 'Drama/Mystery'],\n",
       " 114: [\"Margaret's Museum (1995)\", 'Drama'],\n",
       " 115: ['Happiness Is in the Field (1995)', 'Comedy'],\n",
       " 116: ['Anne Frank Remembered (1995)', 'Documentary'],\n",
       " 117: [\"Young Poisoner's Handbook, The (1995)\", 'Crime'],\n",
       " 118: ['If Lucy Fell (1996)', 'Comedy/Romance'],\n",
       " 119: ['Steal Big, Steal Little (1995)', 'Comedy'],\n",
       " 120: ['Race the Sun (1996)', 'Drama'],\n",
       " 121: ['Boys of St. Vincent, The (1993)', 'Drama'],\n",
       " 122: ['Boomerang (1992)', 'Comedy/Romance'],\n",
       " 123: ['Chungking Express (1994)', 'Drama/Mystery/Romance'],\n",
       " 124: [\"Star Maker, The (Uomo delle stelle, L') (1995)\", 'Drama'],\n",
       " 125: ['Flirting With Disaster (1996)', 'Comedy'],\n",
       " 126: ['NeverEnding Story III, The (1994)', \"Adventure/Children's/Fantasy\"],\n",
       " 127: ['Silence of the Palace, The (Saimt el Qusur) (1994)', 'Drama'],\n",
       " 128: [\"Jupiter's Wife (1994)\", 'Documentary'],\n",
       " 129: ['Pie in the Sky (1995)', 'Comedy/Romance'],\n",
       " 130: ['Angela (1995)', 'Drama'],\n",
       " 131: ['Frankie Starlight (1995)', 'Drama/Romance'],\n",
       " 132: ['Jade (1995)', 'Thriller'],\n",
       " 133: ['Nueba Yol (1995)', 'Comedy/Drama'],\n",
       " 134: ['Sonic Outlaws (1995)', 'Documentary'],\n",
       " 135: ['Down Periscope (1996)', 'Comedy'],\n",
       " 136: ['From the Journals of Jean Seberg (1995)', 'Documentary'],\n",
       " 137: ['Man of the Year (1995)', 'Documentary'],\n",
       " 138: ['Neon Bible, The (1995)', 'Drama'],\n",
       " 139: ['Target (1995)', 'Action/Drama'],\n",
       " 140: ['Up Close and Personal (1996)', 'Drama/Romance'],\n",
       " 141: ['Birdcage, The (1996)', 'Comedy'],\n",
       " 142: ['Shadows (Cienie) (1988)', 'Drama'],\n",
       " 143: ['Gospa (1995)', 'Drama'],\n",
       " 144: ['Brothers McMullen, The (1995)', 'Comedy'],\n",
       " 145: ['Bad Boys (1995)', 'Action'],\n",
       " 146: ['Amazing Panda Adventure, The (1995)', \"Adventure/Children's\"],\n",
       " 147: ['Basketball Diaries, The (1995)', 'Drama'],\n",
       " 148: ['Awfully Big Adventure, An (1995)', 'Drama'],\n",
       " 149: ['Amateur (1994)', 'Crime/Drama/Thriller'],\n",
       " 150: ['Apollo 13 (1995)', 'Drama'],\n",
       " 151: ['Rob Roy (1995)', 'Drama/Romance/War'],\n",
       " 152: ['Addiction, The (1995)', 'Horror'],\n",
       " 153: ['Batman Forever (1995)', 'Action/Adventure/Comedy/Crime'],\n",
       " 154: ['Belle de jour (1967)', 'Drama'],\n",
       " 155: ['Beyond Rangoon (1995)', 'Drama/War'],\n",
       " 156: ['Blue in the Face (1995)', 'Comedy'],\n",
       " 157: ['Canadian Bacon (1994)', 'Comedy/War'],\n",
       " 158: ['Casper (1995)', \"Adventure/Children's\"],\n",
       " 159: ['Clockers (1995)', 'Drama'],\n",
       " 160: ['Congo (1995)', 'Action/Adventure/Mystery/Sci-Fi'],\n",
       " 161: ['Crimson Tide (1995)', 'Drama/Thriller/War'],\n",
       " 162: ['Crumb (1994)', 'Documentary'],\n",
       " 163: ['Desperado (1995)', 'Action/Romance/Thriller'],\n",
       " 164: ['Devil in a Blue Dress (1995)', 'Crime/Film-Noir/Mystery/Thriller'],\n",
       " 165: ['Die Hard: With a Vengeance (1995)', 'Action/Thriller'],\n",
       " 166: ['Doom Generation, The (1995)', 'Comedy/Drama'],\n",
       " 167: ['Feast of July (1995)', 'Drama'],\n",
       " 168: ['First Knight (1995)', 'Action/Adventure/Drama/Romance'],\n",
       " 169: ['Free Willy 2: The Adventure Home (1995)',\n",
       "  \"Adventure/Children's/Drama\"],\n",
       " 170: ['Hackers (1995)', 'Action/Crime/Thriller'],\n",
       " 171: ['Jeffrey (1995)', 'Comedy'],\n",
       " 172: ['Johnny Mnemonic (1995)', 'Action/Sci-Fi/Thriller'],\n",
       " 173: ['Judge Dredd (1995)', 'Action/Adventure/Sci-Fi'],\n",
       " 174: ['Jury Duty (1995)', 'Comedy'],\n",
       " 175: ['Kids (1995)', 'Drama'],\n",
       " 176: ['Living in Oblivion (1995)', 'Comedy'],\n",
       " 177: ['Lord of Illusions (1995)', 'Horror'],\n",
       " 178: ['Love & Human Remains (1993)', 'Comedy'],\n",
       " 179: ['Mad Love (1995)', 'Drama/Romance'],\n",
       " 180: ['Mallrats (1995)', 'Comedy'],\n",
       " 181: ['Mighty Morphin Power Rangers: The Movie (1995)', \"Action/Children's\"],\n",
       " 182: ['Moonlight and Valentino (1995)', 'Drama/Romance'],\n",
       " 183: ['Mute Witness (1994)', 'Thriller'],\n",
       " 184: ['Nadja (1994)', 'Drama'],\n",
       " 185: ['Net, The (1995)', 'Sci-Fi/Thriller'],\n",
       " 186: ['Nine Months (1995)', 'Comedy'],\n",
       " 187: ['Party Girl (1995)', 'Comedy'],\n",
       " 188: ['Prophecy, The (1995)', 'Horror'],\n",
       " 189: ['Reckless (1995)', 'Comedy'],\n",
       " 190: ['Safe (1995)', 'Thriller'],\n",
       " 191: ['Scarlet Letter, The (1995)', 'Drama'],\n",
       " 192: ['Show, The (1995)', 'Documentary'],\n",
       " 193: ['Showgirls (1995)', 'Drama'],\n",
       " 194: ['Smoke (1995)', 'Drama'],\n",
       " 195: ['Something to Talk About (1995)', 'Comedy/Drama/Romance'],\n",
       " 196: ['Species (1995)', 'Horror/Sci-Fi'],\n",
       " 197: ['Stars Fell on Henrietta, The (1995)', 'Drama'],\n",
       " 198: ['Strange Days (1995)', 'Action/Crime/Sci-Fi'],\n",
       " 199: ['Umbrellas of Cherbourg, The (Parapluies de Cherbourg, Les) (1964)',\n",
       "  'Drama/Musical'],\n",
       " 200: ['Tie That Binds, The (1995)', 'Thriller'],\n",
       " 201: ['Three Wishes (1995)', 'Drama'],\n",
       " 202: ['Total Eclipse (1995)', 'Drama/Romance'],\n",
       " 203: ['To Wong Foo, Thanks for Everything! Julie Newmar (1995)', 'Comedy'],\n",
       " 204: ['Under Siege 2: Dark Territory (1995)', 'Action'],\n",
       " 205: ['Unstrung Heroes (1995)', 'Comedy/Drama'],\n",
       " 206: ['Unzipped (1995)', 'Documentary'],\n",
       " 207: ['Walk in the Clouds, A (1995)', 'Drama/Romance'],\n",
       " 208: ['Waterworld (1995)', 'Action/Adventure'],\n",
       " 209: [\"White Man's Burden (1995)\", 'Drama'],\n",
       " 210: ['Wild Bill (1995)', 'Western'],\n",
       " 211: ['Browning Version, The (1994)', 'Drama'],\n",
       " 212: ['Bushwhacked (1995)', 'Comedy'],\n",
       " 213: ['Burnt By the Sun (Utomlyonnye solntsem) (1994)', 'Drama'],\n",
       " 214: ['Before the Rain (Pred dozhdot) (1994)', 'Drama'],\n",
       " 215: ['Before Sunrise (1995)', 'Drama/Romance'],\n",
       " 216: ['Billy Madison (1995)', 'Comedy'],\n",
       " 217: ['Babysitter, The (1995)', 'Drama/Thriller'],\n",
       " 218: ['Boys on the Side (1995)', 'Comedy/Drama'],\n",
       " 219: ['Cure, The (1995)', 'Drama'],\n",
       " 220: ['Castle Freak (1995)', 'Horror'],\n",
       " 222: ['Circle of Friends (1995)', 'Drama/Romance'],\n",
       " 223: ['Clerks (1994)', 'Comedy'],\n",
       " 224: ['Don Juan DeMarco (1995)', 'Comedy/Drama/Romance'],\n",
       " 225: ['Disclosure (1994)', 'Drama/Thriller'],\n",
       " 226: ['Dream Man (1995)', 'Thriller'],\n",
       " 227: ['Drop Zone (1994)', 'Action'],\n",
       " 228: ['Destiny Turns on the Radio (1995)', 'Comedy'],\n",
       " 229: ['Death and the Maiden (1994)', 'Drama/Thriller'],\n",
       " 230: ['Dolores Claiborne (1994)', 'Drama/Thriller'],\n",
       " 231: ['Dumb & Dumber (1994)', 'Comedy'],\n",
       " 232: ['Eat Drink Man Woman (1994)', 'Comedy/Drama'],\n",
       " 233: ['Exotica (1994)', 'Drama'],\n",
       " 234: ['Exit to Eden (1994)', 'Comedy'],\n",
       " 235: ['Ed Wood (1994)', 'Comedy/Drama'],\n",
       " 236: ['French Kiss (1995)', 'Comedy/Romance'],\n",
       " 237: ['Forget Paris (1995)', 'Comedy/Romance'],\n",
       " 238: ['Far From Home: The Adventures of Yellow Dog (1995)',\n",
       "  \"Adventure/Children's\"],\n",
       " 239: ['Goofy Movie, A (1995)', \"Animation/Children's/Comedy/Romance\"],\n",
       " 240: ['Hideaway (1995)', 'Thriller'],\n",
       " 241: ['Fluke (1995)', \"Children's/Drama\"],\n",
       " 242: ['Farinelli: il castrato (1994)', 'Drama/Musical'],\n",
       " 243: ['Gordy (1995)', 'Comedy'],\n",
       " 244: ['Gumby: The Movie (1995)', \"Animation/Children's\"],\n",
       " 245: ['Glass Shield, The (1994)', 'Crime/Drama'],\n",
       " 246: ['Hoop Dreams (1994)', 'Documentary'],\n",
       " 247: ['Heavenly Creatures (1994)', 'Drama/Fantasy/Romance/Thriller'],\n",
       " 248: ['Houseguest (1994)', 'Comedy'],\n",
       " 249: ['Immortal Beloved (1994)', 'Drama/Romance'],\n",
       " 250: ['Heavyweights (1994)', \"Children's/Comedy\"],\n",
       " 251: ['Hunted, The (1995)', 'Action'],\n",
       " 252: ['I.Q. (1994)', 'Comedy/Romance'],\n",
       " 253: ['Interview with the Vampire (1994)', 'Drama/Horror'],\n",
       " 254: ['Jefferson in Paris (1995)', 'Drama'],\n",
       " 255: ['Jerky Boys, The (1994)', 'Comedy'],\n",
       " 256: ['Junior (1994)', 'Comedy/Sci-Fi'],\n",
       " 257: ['Just Cause (1995)', 'Mystery/Thriller'],\n",
       " 258: [\"Kid in King Arthur's Court, A (1995)\",\n",
       "  \"Adventure/Children's/Comedy/Fantasy/Romance\"],\n",
       " 259: ['Kiss of Death (1995)', 'Crime/Drama/Thriller'],\n",
       " 260: ['Star Wars: Episode IV - A New Hope (1977)',\n",
       "  'Action/Adventure/Fantasy/Sci-Fi'],\n",
       " 261: ['Little Women (1994)', 'Drama'],\n",
       " 262: ['Little Princess, A (1995)', \"Children's/Drama\"],\n",
       " 263: ['Ladybird Ladybird (1994)', 'Drama'],\n",
       " 264: [\"Enfer, L' (1994)\", 'Drama'],\n",
       " 265: ['Like Water for Chocolate (Como agua para chocolate) (1992)',\n",
       "  'Drama/Romance'],\n",
       " 266: ['Legends of the Fall (1994)', 'Drama/Romance/War/Western'],\n",
       " 267: ['Major Payne (1994)', 'Comedy'],\n",
       " 268: ['Little Odessa (1994)', 'Drama'],\n",
       " 269: ['My Crazy Life (Mi vida loca) (1993)', 'Drama'],\n",
       " 270: ['Love Affair (1994)', 'Drama/Romance'],\n",
       " 271: ['Losing Isaiah (1995)', 'Drama'],\n",
       " 272: ['Madness of King George, The (1994)', 'Drama'],\n",
       " 273: [\"Mary Shelley's Frankenstein (1994)\", 'Drama/Horror'],\n",
       " 274: ['Man of the House (1995)', 'Comedy'],\n",
       " 275: ['Mixed Nuts (1994)', 'Comedy'],\n",
       " 276: ['Milk Money (1994)', 'Comedy/Romance'],\n",
       " 277: ['Miracle on 34th Street (1994)', 'Drama'],\n",
       " 278: ['Miami Rhapsody (1995)', 'Comedy'],\n",
       " 279: ['My Family (1995)', 'Drama'],\n",
       " 280: ['Murder in the First (1995)', 'Drama/Thriller'],\n",
       " 281: [\"Nobody's Fool (1994)\", 'Drama'],\n",
       " 282: ['Nell (1994)', 'Drama'],\n",
       " 283: ['New Jersey Drive (1995)', 'Crime/Drama'],\n",
       " 284: ['New York Cop (1996)', 'Action/Crime'],\n",
       " 285: ['Beyond Bedlam (1993)', 'Drama/Horror'],\n",
       " 286: ['Nemesis 2: Nebula (1995)', 'Action/Sci-Fi/Thriller'],\n",
       " 287: ['Nina Takes a Lover (1994)', 'Comedy/Romance'],\n",
       " 288: ['Natural Born Killers (1994)', 'Action/Thriller'],\n",
       " 289: ['Only You (1994)', 'Comedy/Romance'],\n",
       " 290: ['Once Were Warriors (1994)', 'Crime/Drama'],\n",
       " 291: ['Poison Ivy II (1995)', 'Thriller'],\n",
       " 292: ['Outbreak (1995)', 'Action/Drama/Thriller'],\n",
       " 293: ['Professional, The (a.k.a. Leon: The Professional) (1994)',\n",
       "  'Crime/Drama/Romance/Thriller'],\n",
       " 294: ['Perez Family, The (1995)', 'Comedy/Romance'],\n",
       " 295: [\"Pyromaniac's Love Story, A (1995)\", 'Comedy/Romance'],\n",
       " 296: ['Pulp Fiction (1994)', 'Crime/Drama'],\n",
       " 297: ['Panther (1995)', 'Drama'],\n",
       " 298: ['Pushing Hands (1992)', 'Comedy'],\n",
       " 299: ['Priest (1994)', 'Drama'],\n",
       " 300: ['Quiz Show (1994)', 'Drama'],\n",
       " 301: ['Picture Bride (1995)', 'Drama/Romance'],\n",
       " 302: ['Queen Margot (La Reine Margot) (1994)', 'Drama/Romance'],\n",
       " 303: ['Quick and the Dead, The (1995)', 'Action/Adventure/Western'],\n",
       " 304: ['Roommates (1995)', 'Comedy/Drama'],\n",
       " 305: ['Ready to Wear (Pret-A-Porter) (1994)', 'Comedy'],\n",
       " 306: ['Three Colors: Red (1994)', 'Drama'],\n",
       " 307: ['Three Colors: Blue (1993)', 'Drama'],\n",
       " 308: ['Three Colors: White (1994)', 'Drama'],\n",
       " 309: ['Red Firecracker, Green Firecracker (1994)', 'Drama'],\n",
       " 310: ['Rent-a-Kid (1995)', 'Comedy'],\n",
       " 311: ['Relative Fear (1994)', 'Horror/Thriller'],\n",
       " 312: ['Stuart Saves His Family (1995)', 'Comedy'],\n",
       " 313: ['Swan Princess, The (1994)', \"Animation/Children's\"],\n",
       " 314: ['Secret of Roan Inish, The (1994)', 'Drama'],\n",
       " 315: ['Specialist, The (1994)', 'Action'],\n",
       " 316: ['Stargate (1994)', 'Action/Adventure/Sci-Fi'],\n",
       " 317: ['Santa Clause, The (1994)', \"Children's/Comedy/Fantasy\"],\n",
       " 318: ['Shawshank Redemption, The (1994)', 'Drama'],\n",
       " 319: ['Shallow Grave (1994)', 'Thriller'],\n",
       " 320: ['Suture (1993)', 'Film-Noir/Thriller'],\n",
       " 321: ['Strawberry and Chocolate (Fresa y chocolate) (1993)', 'Drama'],\n",
       " 322: ['Swimming with Sharks (1995)', 'Comedy/Drama'],\n",
       " 324: ['Sum of Us, The (1994)', 'Comedy'],\n",
       " 325: [\"National Lampoon's Senior Trip (1995)\", 'Comedy'],\n",
       " 326: ['To Live (Huozhe) (1994)', 'Drama'],\n",
       " 327: ['Tank Girl (1995)', 'Action/Comedy/Musical/Sci-Fi'],\n",
       " 328: ['Tales From the Crypt Presents: Demon Knight (1995)', 'Horror'],\n",
       " 329: ['Star Trek: Generations (1994)', 'Action/Adventure/Sci-Fi'],\n",
       " 330: ['Tales from the Hood (1995)', 'Comedy/Horror'],\n",
       " 331: ['Tom & Viv (1994)', 'Drama'],\n",
       " 332: ['Village of the Damned (1995)', 'Horror/Sci-Fi'],\n",
       " 333: ['Tommy Boy (1995)', 'Comedy'],\n",
       " 334: ['Vanya on 42nd Street (1994)', 'Drama'],\n",
       " 335: ['Underneath, The (1995)', 'Mystery/Thriller'],\n",
       " 336: ['Walking Dead, The (1995)', 'Drama/War'],\n",
       " 337: [\"What's Eating Gilbert Grape (1993)\", 'Drama'],\n",
       " 338: ['Virtuosity (1995)', 'Sci-Fi/Thriller'],\n",
       " 339: ['While You Were Sleeping (1995)', 'Comedy/Romance'],\n",
       " 340: ['War, The (1994)', 'Adventure/Drama'],\n",
       " 341: ['Double Happiness (1994)', 'Drama'],\n",
       " 342: [\"Muriel's Wedding (1994)\", 'Comedy/Romance'],\n",
       " 343: ['Baby-Sitters Club, The (1995)', \"Children's\"],\n",
       " 344: ['Ace Ventura: Pet Detective (1994)', 'Comedy'],\n",
       " 345: ['Adventures of Priscilla, Queen of the Desert, The (1994)',\n",
       "  'Comedy/Drama'],\n",
       " 346: ['Backbeat (1993)', 'Drama/Musical'],\n",
       " 347: ['Bitter Moon (1992)', 'Drama'],\n",
       " 348: ['Bullets Over Broadway (1994)', 'Comedy'],\n",
       " 349: ['Clear and Present Danger (1994)', 'Action/Adventure/Thriller'],\n",
       " 350: ['Client, The (1994)', 'Drama/Mystery/Thriller'],\n",
       " 351: ['Corrina, Corrina (1994)', 'Comedy/Drama/Romance'],\n",
       " 352: ['Crooklyn (1994)', 'Comedy'],\n",
       " 353: ['Crow, The (1994)', 'Action/Romance/Thriller'],\n",
       " 354: ['Cobb (1994)', 'Drama'],\n",
       " 355: ['Flintstones, The (1994)', \"Children's/Comedy\"],\n",
       " 356: ['Forrest Gump (1994)', 'Comedy/Romance/War'],\n",
       " 357: ['Four Weddings and a Funeral (1994)', 'Comedy/Romance'],\n",
       " 358: ['Higher Learning (1995)', 'Drama'],\n",
       " 359: ['I Like It Like That (1994)', 'Comedy/Drama/Romance'],\n",
       " 360: ['I Love Trouble (1994)', 'Action/Comedy'],\n",
       " 361: ['It Could Happen to You (1994)', 'Drama/Romance'],\n",
       " 362: ['Jungle Book, The (1994)', \"Adventure/Children's/Romance\"],\n",
       " 363: ['Wonderful, Horrible Life of Leni Riefenstahl, The (Die Macht der Bilder) (1993)',\n",
       "  'Documentary'],\n",
       " 364: ['Lion King, The (1994)', \"Animation/Children's/Musical\"],\n",
       " 365: ['Little Buddha (1993)', 'Drama'],\n",
       " 366: [\"Wes Craven's New Nightmare (1994)\", 'Horror'],\n",
       " 367: ['Mask, The (1994)', 'Comedy/Crime/Fantasy'],\n",
       " 368: ['Maverick (1994)', 'Action/Comedy/Western'],\n",
       " 369: ['Mrs. Parker and the Vicious Circle (1994)', 'Drama'],\n",
       " 370: ['Naked Gun 33 1/3: The Final Insult (1994)', 'Comedy'],\n",
       " 371: ['Paper, The (1994)', 'Comedy/Drama'],\n",
       " 372: ['Reality Bites (1994)', 'Comedy/Drama'],\n",
       " 373: ['Red Rock West (1992)', 'Thriller'],\n",
       " 374: ['Richie Rich (1994)', \"Children's/Comedy\"],\n",
       " 375: ['Safe Passage (1994)', 'Drama'],\n",
       " 376: ['River Wild, The (1994)', 'Action/Thriller'],\n",
       " 377: ['Speed (1994)', 'Action/Romance/Thriller'],\n",
       " 378: ['Speechless (1994)', 'Comedy/Romance'],\n",
       " 379: ['Timecop (1994)', 'Action/Sci-Fi'],\n",
       " 380: ['True Lies (1994)', 'Action/Adventure/Comedy/Romance'],\n",
       " 381: ['When a Man Loves a Woman (1994)', 'Drama'],\n",
       " 382: ['Wolf (1994)', 'Drama/Horror'],\n",
       " 383: ['Wyatt Earp (1994)', 'Western'],\n",
       " 384: ['Bad Company (1995)', 'Action'],\n",
       " 385: ['Man of No Importance, A (1994)', 'Drama'],\n",
       " 386: ['S.F.W. (1994)', 'Drama'],\n",
       " 387: ['Low Down Dirty Shame, A (1994)', 'Action/Comedy'],\n",
       " 388: ['Boys Life (1995)', 'Drama'],\n",
       " 389: ['Colonel Chabert, Le (1994)', 'Drama/Romance/War'],\n",
       " 390: ['Faster Pussycat! Kill! Kill! (1965)', 'Action/Comedy/Drama'],\n",
       " 391: [\"Jason's Lyric (1994)\", 'Crime/Drama'],\n",
       " 392: ['Secret Adventures of Tom Thumb, The (1993)', \"Adventure/Children's\"],\n",
       " 393: ['Street Fighter (1994)', 'Action'],\n",
       " 394: ['Coldblooded (1995)', 'Action'],\n",
       " 395: ['Desert Winds (1995)', 'Drama'],\n",
       " 396: ['Fall Time (1995)', 'Drama'],\n",
       " 397: ['Fear, The (1995)', 'Horror'],\n",
       " 398: ['Frank and Ollie (1995)', 'Documentary'],\n",
       " 399: ['Girl in the Cadillac (1995)', 'Drama'],\n",
       " 400: ['Homage (1995)', 'Drama'],\n",
       " 401: ['Mirage (1995)', 'Action/Thriller'],\n",
       " 402: ['Open Season (1996)', 'Comedy'],\n",
       " 403: ['Two Crimes (1995)', 'Comedy/Crime/Drama'],\n",
       " 404: ['Brother Minister: The Assassination of Malcolm X (1994)',\n",
       "  'Documentary'],\n",
       " 405: ['Highlander III: The Sorcerer (1994)', 'Action/Sci-Fi'],\n",
       " 406: ['Federal Hill (1994)', 'Drama'],\n",
       " 407: ['In the Mouth of Madness (1995)', 'Horror/Thriller'],\n",
       " 408: ['8 Seconds (1994)', 'Drama'],\n",
       " 409: ['Above the Rim (1994)', 'Drama'],\n",
       " 410: ['Addams Family Values (1993)', 'Comedy'],\n",
       " 411: ['You So Crazy (1994)', 'Comedy'],\n",
       " 412: ['Age of Innocence, The (1993)', 'Drama'],\n",
       " 413: ['Airheads (1994)', 'Comedy'],\n",
       " 414: ['Air Up There, The (1994)', 'Comedy'],\n",
       " 415: ['Another Stakeout (1993)', 'Comedy/Thriller'],\n",
       " 416: ['Bad Girls (1994)', 'Western'],\n",
       " 417: ['Barcelona (1994)', 'Comedy/Romance'],\n",
       " 418: ['Being Human (1993)', 'Drama'],\n",
       " 419: ['Beverly Hillbillies, The (1993)', 'Comedy'],\n",
       " 420: ['Beverly Hills Cop III (1994)', 'Action/Comedy'],\n",
       " 421: ['Black Beauty (1994)', \"Adventure/Children's\"],\n",
       " 422: ['Blink (1994)', 'Thriller'],\n",
       " 423: ['Blown Away (1994)', 'Action/Thriller'],\n",
       " 424: ['Blue Chips (1994)', 'Drama'],\n",
       " 425: ['Blue Sky (1994)', 'Drama/Romance'],\n",
       " 426: ['Body Snatchers (1993)', 'Horror/Sci-Fi/Thriller'],\n",
       " 427: ['Boxing Helena (1993)', 'Mystery/Romance/Thriller'],\n",
       " 428: ['Bronx Tale, A (1993)', 'Drama'],\n",
       " 429: ['Cabin Boy (1994)', 'Comedy'],\n",
       " 430: ['Calendar Girl (1993)', 'Drama'],\n",
       " 431: [\"Carlito's Way (1993)\", 'Crime/Drama'],\n",
       " 432: [\"City Slickers II: The Legend of Curly's Gold (1994)\",\n",
       "  'Comedy/Western'],\n",
       " 433: ['Clean Slate (1994)', 'Comedy'],\n",
       " 434: ['Cliffhanger (1993)', 'Action/Adventure/Crime'],\n",
       " 435: ['Coneheads (1993)', 'Comedy/Sci-Fi'],\n",
       " 436: ['Color of Night (1994)', 'Drama/Thriller'],\n",
       " 437: ['Cops and Robbersons (1994)', 'Comedy'],\n",
       " 438: ['Cowboy Way, The (1994)', 'Action/Comedy'],\n",
       " 439: ['Dangerous Game (1993)', 'Drama'],\n",
       " 440: ['Dave (1993)', 'Comedy/Romance'],\n",
       " 441: ['Dazed and Confused (1993)', 'Comedy'],\n",
       " 442: ['Demolition Man (1993)', 'Action/Sci-Fi'],\n",
       " 443: ['Endless Summer 2, The (1994)', 'Documentary'],\n",
       " 444: ['Even Cowgirls Get the Blues (1993)', 'Comedy/Romance'],\n",
       " 445: ['Fatal Instinct (1993)', 'Comedy'],\n",
       " 446: ['Farewell My Concubine (1993)', 'Drama/Romance'],\n",
       " 447: ['Favor, The (1994)', 'Comedy/Romance'],\n",
       " 448: ['Fearless (1993)', 'Drama'],\n",
       " 449: ['Fear of a Black Hat (1993)', 'Comedy'],\n",
       " 450: ['With Honors (1994)', 'Comedy/Drama'],\n",
       " 451: ['Flesh and Bone (1993)', 'Drama/Mystery/Romance'],\n",
       " 452: [\"Widows' Peak (1994)\", 'Drama'],\n",
       " 453: ['For Love or Money (1993)', 'Comedy'],\n",
       " 454: ['Firm, The (1993)', 'Drama/Thriller'],\n",
       " 455: ['Free Willy (1993)', \"Adventure/Children's/Drama\"],\n",
       " 456: ['Fresh (1994)', 'Drama'],\n",
       " 457: ['Fugitive, The (1993)', 'Action/Thriller'],\n",
       " 458: ['Geronimo: An American Legend (1993)', 'Drama/Western'],\n",
       " 459: ['Getaway, The (1994)', 'Action'],\n",
       " 460: ['Getting Even with Dad (1994)', 'Comedy'],\n",
       " 461: ['Go Fish (1994)', 'Drama/Romance'],\n",
       " 462: ['Good Man in Africa, A (1994)', 'Action/Adventure'],\n",
       " 463: ['Guilty as Sin (1993)', 'Crime/Drama/Thriller'],\n",
       " 464: ['Hard Target (1993)', 'Action/Adventure/Crime/Thriller'],\n",
       " 465: ['Heaven & Earth (1993)', 'Action/Drama/War'],\n",
       " 466: ['Hot Shots! Part Deux (1993)', 'Action/Comedy/War'],\n",
       " 467: ['Live Nude Girls (1995)', 'Comedy'],\n",
       " 468: ['Englishman Who Went Up a Hill, But Came Down a Mountain, The (1995)',\n",
       "  'Comedy/Romance'],\n",
       " 469: ['House of the Spirits, The (1993)', 'Drama/Romance'],\n",
       " 470: ['House Party 3 (1994)', 'Comedy'],\n",
       " 471: ['Hudsucker Proxy, The (1994)', 'Comedy/Romance'],\n",
       " 472: [\"I'll Do Anything (1994)\", 'Comedy/Drama'],\n",
       " 473: ['In the Army Now (1994)', 'Comedy/War'],\n",
       " 474: ['In the Line of Fire (1993)', 'Action/Thriller'],\n",
       " 475: ['In the Name of the Father (1993)', 'Drama'],\n",
       " 476: ['Inkwell, The (1994)', 'Comedy/Drama'],\n",
       " 477: [\"What's Love Got to Do with It? (1993)\", 'Drama'],\n",
       " 478: ['Jimmy Hollywood (1994)', 'Comedy'],\n",
       " 479: ['Judgment Night (1993)', 'Action'],\n",
       " 480: ['Jurassic Park (1993)', 'Action/Adventure/Sci-Fi'],\n",
       " 481: ['Kalifornia (1993)', 'Drama/Thriller'],\n",
       " 482: ['Killing Zoe (1994)', 'Thriller'],\n",
       " 483: ['King of the Hill (1993)', 'Drama'],\n",
       " 484: ['Lassie (1994)', \"Adventure/Children's\"],\n",
       " 485: ['Last Action Hero (1993)', 'Action/Comedy'],\n",
       " 486: ['Life with Mikey (1993)', 'Comedy'],\n",
       " 487: ['Lightning Jack (1994)', 'Comedy/Western'],\n",
       " 488: ['M. Butterfly (1993)', 'Drama'],\n",
       " 489: ['Made in America (1993)', 'Comedy'],\n",
       " 490: ['Malice (1993)', 'Thriller'],\n",
       " 491: ['Man Without a Face, The (1993)', 'Drama'],\n",
       " 492: ['Manhattan Murder Mystery (1993)', 'Comedy/Mystery'],\n",
       " 493: ['Menace II Society (1993)', 'Action/Crime/Drama'],\n",
       " 494: ['Executive Decision (1996)', 'Action/Thriller'],\n",
       " 495: ['In the Realm of the Senses (Ai no corrida) (1976)', 'Drama'],\n",
       " 496: ['What Happened Was... (1994)', 'Comedy/Drama/Romance'],\n",
       " 497: ['Much Ado About Nothing (1993)', 'Comedy/Romance'],\n",
       " 498: ['Mr. Jones (1993)', 'Drama/Romance'],\n",
       " 499: ['Mr. Wonderful (1993)', 'Comedy/Romance'],\n",
       " 500: ['Mrs. Doubtfire (1993)', 'Comedy'],\n",
       " 501: ['Naked (1993)', 'Drama'],\n",
       " 502: ['Next Karate Kid, The (1994)', \"Action/Children's\"],\n",
       " 503: ['New Age, The (1994)', 'Drama'],\n",
       " 504: ['No Escape (1994)', 'Action/Sci-Fi'],\n",
       " 505: ['North (1994)', 'Comedy'],\n",
       " 506: ['Orlando (1993)', 'Drama'],\n",
       " 507: ['Perfect World, A (1993)', 'Action/Drama'],\n",
       " 508: ['Philadelphia (1993)', 'Drama'],\n",
       " 509: ['Piano, The (1993)', 'Drama/Romance'],\n",
       " 510: ['Poetic Justice (1993)', 'Drama'],\n",
       " 511: ['Program, The (1993)', 'Action/Drama'],\n",
       " 512: [\"Robert A. Heinlein's The Puppet Masters (1994)\", 'Horror/Sci-Fi'],\n",
       " 513: ['Radioland Murders (1994)', 'Comedy/Mystery/Romance'],\n",
       " 514: ['Ref, The (1994)', 'Comedy'],\n",
       " 515: ['Remains of the Day, The (1993)', 'Drama'],\n",
       " 516: ['Renaissance Man (1994)', 'Comedy/Drama/War'],\n",
       " 517: ['Rising Sun (1993)', 'Action/Drama/Mystery'],\n",
       " 518: ['Road to Wellville, The (1994)', 'Comedy'],\n",
       " 519: ['Robocop 3 (1993)', 'Sci-Fi/Thriller'],\n",
       " 520: ['Robin Hood: Men in Tights (1993)', 'Comedy'],\n",
       " 521: ['Romeo Is Bleeding (1993)', 'Crime/Thriller'],\n",
       " 522: ['Romper Stomper (1992)', 'Action/Drama'],\n",
       " 523: ['Ruby in Paradise (1993)', 'Drama'],\n",
       " 524: ['Rudy (1993)', 'Drama'],\n",
       " 525: ['Saint of Fort Washington, The (1993)', 'Drama'],\n",
       " 526: ['Savage Nights (Nuits fauves, Les) (1992)', 'Drama'],\n",
       " 527: [\"Schindler's List (1993)\", 'Drama/War'],\n",
       " 528: ['Scout, The (1994)', 'Drama'],\n",
       " 529: ['Searching for Bobby Fischer (1993)', 'Drama'],\n",
       " 530: ['Second Best (1994)', 'Drama'],\n",
       " 531: ['Secret Garden, The (1993)', \"Children's/Drama\"],\n",
       " 532: ['Serial Mom (1994)', 'Comedy/Crime/Horror'],\n",
       " 533: ['Shadow, The (1994)', 'Action'],\n",
       " 534: ['Shadowlands (1993)', 'Drama/Romance'],\n",
       " 535: ['Short Cuts (1993)', 'Drama'],\n",
       " 536: ['Simple Twist of Fate, A (1994)', 'Drama'],\n",
       " 537: ['Sirens (1994)', 'Comedy/Drama'],\n",
       " 538: ['Six Degrees of Separation (1993)', 'Drama'],\n",
       " 539: ['Sleepless in Seattle (1993)', 'Comedy/Romance'],\n",
       " 540: ['Sliver (1993)', 'Thriller'],\n",
       " 541: ['Blade Runner (1982)', 'Film-Noir/Sci-Fi'],\n",
       " 542: ['Son in Law (1993)', 'Comedy'],\n",
       " 543: ['So I Married an Axe Murderer (1993)', 'Comedy/Romance/Thriller'],\n",
       " 544: ['Striking Distance (1993)', 'Action'],\n",
       " 545: ['Harlem (1993)', 'Drama'],\n",
       " 546: ['Super Mario Bros. (1993)', \"Action/Adventure/Children's/Sci-Fi\"],\n",
       " 547: ['Surviving the Game (1994)', 'Action/Adventure/Thriller'],\n",
       " 548: ['Terminal Velocity (1994)', 'Action'],\n",
       " 549: ['Thirty-Two Short Films About Glenn Gould (1993)', 'Documentary'],\n",
       " 550: ['Threesome (1994)', 'Comedy/Romance'],\n",
       " 551: ['Nightmare Before Christmas, The (1993)', \"Children's/Comedy/Musical\"],\n",
       " 552: ['Three Musketeers, The (1993)', 'Action/Adventure/Comedy'],\n",
       " 553: ['Tombstone (1993)', 'Western'],\n",
       " 554: ['Trial by Jury (1994)', 'Thriller'],\n",
       " 555: ['True Romance (1993)', 'Action/Crime/Romance'],\n",
       " 556: ['War Room, The (1993)', 'Documentary'],\n",
       " 557: ['Mamma Roma (1962)', 'Drama'],\n",
       " 558: ['Pagemaster, The (1994)',\n",
       "  \"Action/Adventure/Animation/Children's/Fantasy\"],\n",
       " 559: ['Paris, France (1993)', 'Comedy'],\n",
       " 560: ['Beans of Egypt, Maine, The (1994)', 'Drama'],\n",
       " 561: ['Killer (Bulletproof Heart) (1994)', 'Thriller'],\n",
       " 562: ['Welcome to the Dollhouse (1995)', 'Comedy/Drama'],\n",
       " 563: ['Germinal (1993)', 'Drama'],\n",
       " 564: ['Chasers (1994)', 'Comedy'],\n",
       " 565: ['Cronos (1992)', 'Horror'],\n",
       " 566: ['Naked in New York (1994)', 'Comedy/Romance'],\n",
       " 567: ['Kika (1993)', 'Drama'],\n",
       " 568: ['Bhaji on the Beach (1993)', 'Comedy/Drama'],\n",
       " 569: ['Little Big League (1994)', \"Children's/Comedy\"],\n",
       " 570: ['Slingshot, The (Kdisbellan ) (1993)', 'Comedy/Drama'],\n",
       " 571: ['Wedding Gift, The (1994)', 'Drama'],\n",
       " 572: ['Foreign Student (1994)', 'Drama'],\n",
       " 573: ['Ciao, Professore! (Io speriamo che me la cavo ) (1993)', 'Drama'],\n",
       " 574: ['Spanking the Monkey (1994)', 'Comedy/Drama'],\n",
       " 575: ['Little Rascals, The (1994)', \"Children's/Comedy\"],\n",
       " 576: ['Fausto (1993)', 'Comedy'],\n",
       " 577: ['Andre (1994)', \"Adventure/Children's\"],\n",
       " 578: ['Hour of the Pig, The (1993)', 'Drama/Mystery'],\n",
       " 579: ['Scorta, La (1993)', 'Thriller'],\n",
       " 580: ['Princess Caraboo (1994)', 'Drama'],\n",
       " 581: ['Celluloid Closet, The (1995)', 'Documentary'],\n",
       " 582: ['Metisse (Caf au Lait) (1993)', 'Comedy'],\n",
       " 583: ['Dear Diary (Caro Diario) (1994)', 'Comedy/Drama'],\n",
       " 584: [\"I Don't Want to Talk About It (De eso no se habla) (1993)\", 'Drama'],\n",
       " 585: ['Brady Bunch Movie, The (1995)', 'Comedy'],\n",
       " 586: ['Home Alone (1990)', \"Children's/Comedy\"],\n",
       " 587: ['Ghost (1990)', 'Comedy/Romance/Thriller'],\n",
       " 588: ['Aladdin (1992)', \"Animation/Children's/Comedy/Musical\"],\n",
       " 589: ['Terminator 2: Judgment Day (1991)', 'Action/Sci-Fi/Thriller'],\n",
       " 590: ['Dances with Wolves (1990)', 'Adventure/Drama/Western'],\n",
       " 591: ['Tough and Deadly (1995)', 'Action/Drama/Thriller'],\n",
       " 592: ['Batman (1989)', 'Action/Adventure/Crime/Drama'],\n",
       " 593: ['Silence of the Lambs, The (1991)', 'Drama/Thriller'],\n",
       " 594: ['Snow White and the Seven Dwarfs (1937)',\n",
       "  \"Animation/Children's/Musical\"],\n",
       " 595: ['Beauty and the Beast (1991)', \"Animation/Children's/Musical\"],\n",
       " 596: ['Pinocchio (1940)', \"Animation/Children's\"],\n",
       " 597: ['Pretty Woman (1990)', 'Comedy/Romance'],\n",
       " 598: ['Window to Paris (1994)', 'Comedy'],\n",
       " 599: ['Wild Bunch, The (1969)', 'Western'],\n",
       " 600: ['Love and a .45 (1994)', 'Thriller'],\n",
       " 601: [\"Wooden Man's Bride, The (Wu Kui) (1994)\", 'Drama'],\n",
       " 602: ['Great Day in Harlem, A (1994)', 'Documentary'],\n",
       " 603: ['Bye Bye, Love (1995)', 'Comedy'],\n",
       " 604: ['Criminals (1996)', 'Documentary'],\n",
       " 605: ['One Fine Day (1996)', 'Drama/Romance'],\n",
       " 606: ['Candyman: Farewell to the Flesh (1995)', 'Horror'],\n",
       " 607: ['Century (1993)', 'Drama'],\n",
       " 608: ['Fargo (1996)', 'Crime/Drama/Thriller'],\n",
       " 609: ['Homeward Bound II: Lost in San Francisco (1996)',\n",
       "  \"Adventure/Children's\"],\n",
       " 610: ['Heavy Metal (1981)', 'Action/Adventure/Animation/Horror/Sci-Fi'],\n",
       " 611: ['Hellraiser: Bloodline (1996)', 'Action/Horror/Sci-Fi'],\n",
       " 612: ['Pallbearer, The (1996)', 'Comedy'],\n",
       " 613: ['Jane Eyre (1996)', 'Drama/Romance'],\n",
       " 614: ['Loaded (1994)', 'Drama/Thriller'],\n",
       " 615: ['Bread and Chocolate (Pane e cioccolata) (1973)', 'Drama'],\n",
       " 616: ['Aristocats, The (1970)', \"Animation/Children's\"],\n",
       " 617: ['Flower of My Secret, The (La Flor de Mi Secreto) (1995)', 'Drama'],\n",
       " 618: ['Two Much (1996)', 'Comedy/Romance'],\n",
       " 619: ['Ed (1996)', 'Comedy'],\n",
       " 620: ['Scream of Stone (Schrei aus Stein) (1991)', 'Drama'],\n",
       " 621: ['My Favorite Season (1993)', 'Drama'],\n",
       " 623: ['Modern Affair, A (1995)', 'Romance'],\n",
       " 624: ['Condition Red (1995)', 'Action/Drama/Thriller'],\n",
       " 625: ['Asfour Stah (1990)', 'Drama'],\n",
       " 626: ['Thin Line Between Love and Hate, A (1996)', 'Comedy'],\n",
       " 627: ['Last Supper, The (1995)', 'Drama/Thriller'],\n",
       " 628: ['Primal Fear (1996)', 'Drama/Thriller'],\n",
       " 629: ['Rude (1995)', 'Drama'],\n",
       " 630: ['Carried Away (1996)', 'Drama/Romance'],\n",
       " 631: ['All Dogs Go to Heaven 2 (1996)', \"Animation/Children's/Musical\"],\n",
       " 632: ['Land and Freedom (Tierra y libertad) (1995)', 'War'],\n",
       " 633: ['Denise Calls Up (1995)', 'Comedy'],\n",
       " 634: ['Theodore Rex (1995)', 'Comedy'],\n",
       " 635: ['Family Thing, A (1996)', 'Comedy/Drama'],\n",
       " 636: ['Frisk (1995)', 'Drama'],\n",
       " 637: ['Sgt. Bilko (1996)', 'Comedy'],\n",
       " 638: ['Jack and Sarah (1995)', 'Romance'],\n",
       " 639: ['Girl 6 (1996)', 'Comedy'],\n",
       " 640: ['Diabolique (1996)', 'Drama/Thriller'],\n",
       " 641: ['Little Indian, Big City (Un indien dans la ville) (1994)', 'Comedy'],\n",
       " 642: ['Roula (1995)', 'Drama'],\n",
       " 643: ['Peanuts - Die Bank zahlt alles (1996)', 'Comedy'],\n",
       " 644: ['Happy Weekend (1996)', 'Comedy'],\n",
       " 645: ['Nelly & Monsieur Arnaud (1995)', 'Drama'],\n",
       " 647: ['Courage Under Fire (1996)', 'Drama/War'],\n",
       " 648: ['Mission: Impossible (1996)', 'Action/Adventure/Mystery'],\n",
       " 649: ['Cold Fever ( kldum klaka) (1994)', 'Comedy/Drama'],\n",
       " 650: ['Moll Flanders (1996)', 'Drama'],\n",
       " 651: ['Superweib, Das (1996)', 'Comedy'],\n",
       " 652: ['301, 302 (1995)', 'Mystery'],\n",
       " 653: ['Dragonheart (1996)', 'Action/Adventure/Fantasy'],\n",
       " 654: ['Und keiner weint mir nach (1996)', 'Drama/Romance'],\n",
       " 655: ['Mutters Courage (1995)', 'Comedy'],\n",
       " 656: ['Eddie (1996)', 'Comedy'],\n",
       " 657: ['Yankee Zulu (1994)', 'Comedy/Drama'],\n",
       " 658: [\"Billy's Holiday (1995)\", 'Drama'],\n",
       " 659: ['Purple Noon (1960)', 'Crime/Thriller'],\n",
       " 660: ['August (1996)', 'Drama'],\n",
       " 661: ['James and the Giant Peach (1996)', \"Animation/Children's/Musical\"],\n",
       " 662: ['Fear (1996)', 'Thriller'],\n",
       " 663: ['Kids in the Hall: Brain Candy (1996)', 'Comedy'],\n",
       " 664: ['Faithful (1996)', 'Comedy'],\n",
       " 665: ['Underground (1995)', 'War'],\n",
       " 666: ['All Things Fair (1996)', 'Drama'],\n",
       " 667: ['Bloodsport 2 (1995)', 'Action'],\n",
       " 668: ['Pather Panchali (1955)', 'Drama'],\n",
       " 669: ['Aparajito (1956)', 'Drama'],\n",
       " 670: ['World of Apu, The (Apur Sansar) (1959)', 'Drama'],\n",
       " 671: ['Mystery Science Theater 3000: The Movie (1996)', 'Comedy/Sci-Fi'],\n",
       " 672: ['Tarantella (1995)', 'Drama'],\n",
       " 673: ['Space Jam (1996)', \"Adventure/Animation/Children's/Comedy/Fantasy\"],\n",
       " 674: ['Barbarella (1968)', 'Adventure/Sci-Fi'],\n",
       " 675: ['Hostile Intentions (1994)', 'Action/Drama/Thriller'],\n",
       " 676: ['They Bite (1996)', 'Drama'],\n",
       " 678: ['Some Folks Call It a Sling Blade (1993)', 'Drama/Thriller'],\n",
       " 679: ['Run of the Country, The (1995)', 'Drama'],\n",
       " 680: ['Alphaville (1965)', 'Sci-Fi'],\n",
       " 681: ['Clean Slate (Coup de Torchon) (1981)', 'Crime'],\n",
       " 682: ['Tigrero: A Film That Was Never Made (1994)', 'Documentary/Drama'],\n",
       " 683: [\"Eye of Vichy, The (Oeil de Vichy, L') (1993)\", 'Documentary'],\n",
       " 684: ['Windows (1980)', 'Drama'],\n",
       " 685: [\"It's My Party (1995)\", 'Drama'],\n",
       " 687: ['Country Life (1994)', 'Drama/Romance'],\n",
       " 688: ['Operation Dumbo Drop (1995)', 'Action/Adventure/Comedy/War'],\n",
       " 690: ['Promise, The (Versprechen, Das) (1994)', 'Romance'],\n",
       " 691: ['Mrs. Winterbourne (1996)', 'Comedy/Romance'],\n",
       " 692: ['Solo (1996)', 'Action/Sci-Fi/Thriller'],\n",
       " 693: ['Under the Domin Tree (Etz Hadomim Tafus) (1994)', 'Drama'],\n",
       " 694: ['Substitute, The (1996)', 'Action'],\n",
       " 695: ['True Crime (1995)', 'Mystery/Thriller'],\n",
       " 696: ['Butterfly Kiss (1995)', 'Thriller'],\n",
       " 697: ['Feeling Minnesota (1996)', 'Drama/Romance'],\n",
       " 698: ['Delta of Venus (1994)', 'Drama'],\n",
       " 699: ['To Cross the Rubicon (1991)', 'Drama'],\n",
       " 700: ['Angus (1995)', 'Comedy'],\n",
       " 701: ['Daens (1992)', 'Drama'],\n",
       " 702: ['Faces (1968)', 'Drama'],\n",
       " 703: ['Boys (1996)', 'Drama'],\n",
       " 704: ['Quest, The (1996)', 'Action/Adventure'],\n",
       " 705: ['Cosi (1996)', 'Comedy'],\n",
       " 706: ['Sunset Park (1996)', 'Drama'],\n",
       " 707: ['Mulholland Falls (1996)', 'Crime/Film-Noir/Thriller'],\n",
       " 708: ['Truth About Cats & Dogs, The (1996)', 'Comedy/Romance'],\n",
       " 709: ['Oliver & Company (1988)', \"Animation/Children's\"],\n",
       " 710: ['Celtic Pride (1996)', 'Comedy'],\n",
       " 711: ['Flipper (1996)', \"Adventure/Children's\"],\n",
       " 712: ['Captives (1994)', 'Drama'],\n",
       " 713: ['Of Love and Shadows (1994)', 'Drama'],\n",
       " 714: ['Dead Man (1995)', 'Western'],\n",
       " 715: ['Horseman on the Roof, The (Hussard sur le toit, Le) (1995)', 'Drama'],\n",
       " 716: ['Switchblade Sisters (1975)', 'Crime'],\n",
       " 717: ['Mouth to Mouth (Boca a boca) (1995)', 'Comedy'],\n",
       " 718: ['Visitors, The (Les Visiteurs) (1993)', 'Comedy/Sci-Fi'],\n",
       " 719: ['Multiplicity (1996)', 'Comedy'],\n",
       " 720: ['Wallace & Gromit: The Best of Aardman Animation (1996)', 'Animation'],\n",
       " 721: ['Halfmoon (Paul Bowles - Halbmond) (1995)', 'Drama'],\n",
       " 722: ['Haunted World of Edward D. Wood Jr., The (1995)', 'Documentary'],\n",
       " 723: ['Two Friends (1986)', 'Drama'],\n",
       " 724: ['Craft, The (1996)', 'Drama/Horror'],\n",
       " 725: ['Great White Hype, The (1996)', 'Comedy'],\n",
       " 726: ['Last Dance (1996)', 'Drama'],\n",
       " 727: ['War Stories (1995)', 'Documentary'],\n",
       " 728: ['Cold Comfort Farm (1995)', 'Comedy'],\n",
       " 729: ['Institute Benjamenta, or This Dream People Call Human Life (1995)',\n",
       "  'Drama'],\n",
       " 730: ['Low Life, The (1994)', 'Drama'],\n",
       " 731: [\"Heaven's Prisoners (1996)\", 'Mystery/Thriller'],\n",
       " 732: ['Original Gangstas (1996)', 'Crime'],\n",
       " 733: ['Rock, The (1996)', 'Action/Adventure/Thriller'],\n",
       " 734: ['Getting Away With Murder (1996)', 'Comedy'],\n",
       " 735: ['Cemetery Man (Dellamorte Dellamore) (1994)', 'Comedy/Horror'],\n",
       " 736: ['Twister (1996)', 'Action/Adventure/Romance/Thriller'],\n",
       " 737: ['Barb Wire (1996)', 'Action/Sci-Fi'],\n",
       " 738: ['Garcu, Le (1995)', 'Drama'],\n",
       " 739: ['Honigmond (1996)', 'Comedy'],\n",
       " 741: ['Ghost in the Shell (Kokaku kidotai) (1995)', 'Animation/Sci-Fi'],\n",
       " 742: ['Thinner (1996)', 'Horror/Thriller'],\n",
       " 743: ['Spy Hard (1996)', 'Comedy'],\n",
       " 744: ['Brothers in Trouble (1995)', 'Drama'],\n",
       " 745: ['Close Shave, A (1995)', 'Animation/Comedy/Thriller'],\n",
       " 746: ['Force of Evil (1948)', 'Film-Noir'],\n",
       " 747: ['Stupids, The (1996)', 'Comedy'],\n",
       " 748: ['Arrival, The (1996)', 'Action/Sci-Fi/Thriller'],\n",
       " 749: ['Man from Down Under, The (1943)', 'Drama'],\n",
       " 750: ['Dr. Strangelove or: How I Learned to Stop Worrying and Love the Bomb (1963)',\n",
       "  'Sci-Fi/War'],\n",
       " 751: ['Careful (1992)', 'Comedy'],\n",
       " 752: ['Vermont Is For Lovers (1992)', 'Comedy/Romance'],\n",
       " 753: ['Month by the Lake, A (1995)', 'Comedy/Drama'],\n",
       " 754: ['Gold Diggers: The Secret of Bear Mountain (1995)',\n",
       "  \"Adventure/Children's\"],\n",
       " 755: ['Kim (1950)', \"Children's/Drama\"],\n",
       " 756: ['Carmen Miranda: Bananas Is My Business (1994)', 'Documentary'],\n",
       " 757: ['Ashes of Time (1994)', 'Drama'],\n",
       " 758: ['Jar, The (Khomreh) (1992)', 'Drama'],\n",
       " 759: ['Maya Lin: A Strong Clear Vision (1994)', 'Documentary'],\n",
       " 760: ['Stalingrad (1993)', 'War'],\n",
       " 761: ['Phantom, The (1996)', 'Adventure'],\n",
       " 762: ['Striptease (1996)', 'Comedy/Crime'],\n",
       " 763: ['Last of the High Kings, The (a.k.a. Summer Fling) (1996)', 'Drama'],\n",
       " 764: ['Heavy (1995)', 'Drama/Romance'],\n",
       " 765: ['Jack (1996)', 'Comedy/Drama'],\n",
       " 766: ['I Shot Andy Warhol (1996)', 'Drama'],\n",
       " 767: ['Grass Harp, The (1995)', 'Drama'],\n",
       " 768: [\"Someone Else's America (1995)\", 'Drama'],\n",
       " 769: ['Marlene Dietrich: Shadow and Light (1996)', 'Documentary'],\n",
       " 770: ['Costa Brava (1946)', 'Drama'],\n",
       " 771: ['Vie est belle, La (Life is Rosey) (1987)', 'Comedy/Drama'],\n",
       " 772: ['Quartier Mozart (1992)', 'Comedy'],\n",
       " 773: ['Touki Bouki (Journey of the Hyena) (1973)', 'Drama'],\n",
       " 774: [\"Wend Kuuni (God's Gift) (1982)\", 'Drama'],\n",
       " 775: ['Spirits of the Dead (Tre Passi nel Delirio) (1968)', 'Horror'],\n",
       " 776: ['Babyfever (1994)', 'Comedy/Drama'],\n",
       " 777: [\"Pharaoh's Army (1995)\", 'War'],\n",
       " 778: ['Trainspotting (1996)', 'Drama'],\n",
       " 779: [\"'Til There Was You (1997)\", 'Drama/Romance'],\n",
       " 780: ['Independence Day (ID4) (1996)', 'Action/Sci-Fi/War'],\n",
       " 781: ['Stealing Beauty (1996)', 'Drama'],\n",
       " 782: ['Fan, The (1996)', 'Thriller'],\n",
       " 783: ['Hunchback of Notre Dame, The (1996)', \"Animation/Children's/Musical\"],\n",
       " 784: ['Cable Guy, The (1996)', 'Comedy'],\n",
       " 785: ['Kingpin (1996)', 'Comedy'],\n",
       " 786: ['Eraser (1996)', 'Action/Thriller'],\n",
       " 787: ['Gate of Heavenly Peace, The (1995)', 'Documentary'],\n",
       " 788: ['Nutty Professor, The (1996)', 'Comedy/Fantasy/Romance/Sci-Fi'],\n",
       " 789: ['I, Worst of All (Yo, la peor de todas) (1990)', 'Drama'],\n",
       " 790: ['An Unforgettable Summer (1994)', 'Drama'],\n",
       " 791: ['Last Klezmer: Leopold Kozlowski, His Life and Music, The (1995)',\n",
       "  'Documentary'],\n",
       " 792: ['Hungarian Fairy Tale, A (1987)', 'Fantasy'],\n",
       " 793: [\"My Life and Times With Antonin Artaud (En compagnie d'Antonin Artaud) (1993)\",\n",
       "  'Drama'],\n",
       " 794: ['Midnight Dancers (Sibak) (1994)', 'Comedy/Drama'],\n",
       " 795: ['Somebody to Love (1994)', 'Drama'],\n",
       " 796: ['Very Natural Thing, A (1974)', 'Drama'],\n",
       " 797: ['Old Lady Who Walked in the Sea, The (Vieille qui marchait dans la mer, La) (1991)',\n",
       "  'Comedy'],\n",
       " 798: ['Daylight (1996)', 'Action/Adventure/Thriller'],\n",
       " 799: ['Frighteners, The (1996)', 'Comedy/Horror'],\n",
       " 800: ['Lone Star (1996)', 'Drama/Mystery'],\n",
       " 801: ['Harriet the Spy (1996)', \"Children's/Comedy\"],\n",
       " 802: ['Phenomenon (1996)', 'Drama/Romance'],\n",
       " 803: ['Walking and Talking (1996)', 'Romance'],\n",
       " 804: [\"She's the One (1996)\", 'Comedy/Romance'],\n",
       " 805: ['Time to Kill, A (1996)', 'Drama'],\n",
       " 806: ['American Buffalo (1996)', 'Drama'],\n",
       " 807: ['Rendezvous in Paris (Rendez-vous de Paris, Les) (1995)',\n",
       "  'Comedy/Romance'],\n",
       " 808: ['Alaska (1996)', \"Adventure/Children's\"],\n",
       " 809: ['Fled (1996)', 'Action/Adventure'],\n",
       " 810: ['Kazaam (1996)', \"Children's/Comedy/Fantasy\"],\n",
       " 811: ['Bewegte Mann, Der (1994)', 'Comedy'],\n",
       " 812: ['Magic Hunter (1994)', 'Drama'],\n",
       " 813: ['Larger Than Life (1996)', 'Comedy'],\n",
       " 814: ['Boy Called Hate, A (1995)', 'Drama'],\n",
       " 815: ['Power 98 (1995)', 'Action/Mystery/Thriller'],\n",
       " 816: ['Two Deaths (1995)', 'Drama'],\n",
       " 818: ['Very Brady Sequel, A (1996)', 'Comedy'],\n",
       " 819: ['Stefano Quantestorie (1993)', 'Comedy/Drama'],\n",
       " 820: ['Death in the Garden (Mort en ce jardin, La) (1956)', 'Drama'],\n",
       " 821: ['Crude Oasis, The (1995)', 'Romance'],\n",
       " 822: ['Hedd Wyn (1992)', 'Drama'],\n",
       " 823: ['Collectionneuse, La (1967)', 'Drama'],\n",
       " 824: ['Kaspar Hauser (1993)', 'Drama'],\n",
       " 825: ['Echte Kerle (1996)', 'Comedy/Romance'],\n",
       " 826: ['Diebinnen (1995)', 'Drama'],\n",
       " 827: ['Convent, The (Convento, O) (1995)', 'Drama'],\n",
       " 828: ['Adventures of Pinocchio, The (1996)', \"Adventure/Children's\"],\n",
       " 829: [\"Joe's Apartment (1996)\", 'Comedy/Musical'],\n",
       " 830: ['First Wives Club, The (1996)', 'Comedy'],\n",
       " 831: ['Stonewall (1995)', 'Drama'],\n",
       " 832: ['Ransom (1996)', 'Drama/Thriller'],\n",
       " 833: ['High School High (1996)', 'Comedy'],\n",
       " 834: ['Phat Beach (1996)', 'Comedy'],\n",
       " 835: ['Foxfire (1996)', 'Drama'],\n",
       " 836: ['Chain Reaction (1996)', 'Action/Adventure/Thriller'],\n",
       " 837: ['Matilda (1996)', \"Children's/Comedy\"],\n",
       " 838: ['Emma (1996)', 'Comedy/Drama/Romance'],\n",
       " 839: ['Crow: City of Angels, The (1996)', 'Action/Thriller'],\n",
       " 840: ['House Arrest (1996)', 'Comedy'],\n",
       " 841: ['Eyes Without a Face (1959)', 'Horror'],\n",
       " 842: ['Tales from the Crypt Presents: Bordello of Blood (1996)', 'Horror'],\n",
       " 843: ['Lotto Land (1995)', 'Drama'],\n",
       " 844: ['Story of Xinghua, The (1993)', 'Drama'],\n",
       " 845: ['Day the Sun Turned Cold, The (Tianguo niezi) (1994)', 'Drama'],\n",
       " 846: ['Flirt (1995)', 'Drama'],\n",
       " 847: ['Big Squeeze, The (1996)', 'Comedy/Drama'],\n",
       " 848: ['Spitfire Grill, The (1996)', 'Drama'],\n",
       " 849: ['Escape from L.A. (1996)', 'Action/Adventure/Sci-Fi/Thriller'],\n",
       " 850: ['Cyclo (1995)', 'Crime/Drama'],\n",
       " 851: ['Basquiat (1996)', 'Drama'],\n",
       " 852: ['Tin Cup (1996)', 'Comedy/Romance'],\n",
       " 853: ['Dingo (1992)', 'Drama'],\n",
       " 854: ['Ballad of Narayama, The (Narayama Bushiko) (1958)', 'Drama'],\n",
       " 855: ['Every Other Weekend (1990)', 'Drama'],\n",
       " 856: ['Mille bolle blu (1993)', 'Comedy'],\n",
       " 857: ['Crows and Sparrows (1949)', 'Drama'],\n",
       " 858: ['Godfather, The (1972)', 'Action/Crime/Drama'],\n",
       " 859: ['Hippie Revolution, The (1996)', 'Documentary'],\n",
       " 860: ['Maybe, Maybe Not (Bewegte Mann, Der) (1994)', 'Comedy'],\n",
       " 861: ['Supercop (1992)', 'Action/Thriller'],\n",
       " 862: ['Manny & Lo (1996)', 'Drama'],\n",
       " 863: ['Celestial Clockwork (1994)', 'Comedy'],\n",
       " 864: ['Wife, The (1995)', 'Comedy/Drama'],\n",
       " 865: ['Small Faces (1995)', 'Drama'],\n",
       " 866: ['Bound (1996)', 'Crime/Drama/Romance/Thriller'],\n",
       " 867: ['Carpool (1996)', 'Comedy/Crime'],\n",
       " 868: ['Death in Brunswick (1991)', 'Comedy'],\n",
       " 869: ['Kansas City (1996)', 'Crime'],\n",
       " 870: [\"Gone Fishin' (1997)\", 'Comedy'],\n",
       " 871: [\"Lover's Knot (1996)\", 'Comedy'],\n",
       " 872: ['Aiqing wansui (1994)', 'Drama'],\n",
       " 873: ['Shadow of Angels (Schatten der Engel) (1976)', 'Drama'],\n",
       " 874: ['Killer: A Journal of Murder (1995)', 'Crime/Drama'],\n",
       " 875: ['Nothing to Lose (1994)', 'Drama'],\n",
       " 876: ['Police Story 4: Project S (Chao ji ji hua) (1993)', 'Action'],\n",
       " 877: ['Girls Town (1996)', 'Drama'],\n",
       " 878: ['Bye-Bye (1995)', 'Drama'],\n",
       " 879: ['Relic, The (1997)', 'Horror'],\n",
       " 880: ['Island of Dr. Moreau, The (1996)', 'Sci-Fi/Thriller'],\n",
       " 881: ['First Kid (1996)', \"Children's/Comedy\"],\n",
       " 882: ['Trigger Effect, The (1996)', 'Drama/Thriller'],\n",
       " 884: ['Sweet Nothing (1995)', 'Drama'],\n",
       " 885: ['Bogus (1996)', \"Children's/Drama/Fantasy\"],\n",
       " 886: ['Bulletproof (1996)', 'Action'],\n",
       " 887: ['Talk of Angels (1998)', 'Drama'],\n",
       " 888: ['Land Before Time III: The Time of the Great Giving (1995)',\n",
       "  \"Animation/Children's\"],\n",
       " 889: ['1-900 (1994)', 'Romance'],\n",
       " 890: ['Baton Rouge (1988)', 'Thriller'],\n",
       " 891: ['Halloween: The Curse of Michael Myers (1995)', 'Horror/Thriller'],\n",
       " 892: ['Twelfth Night (1996)', 'Comedy/Drama/Romance'],\n",
       " 893: ['Mother Night (1996)', 'Drama'],\n",
       " 894: ['Liebelei (1933)', 'Romance'],\n",
       " 895: ['Venice/Venice (1992)', 'Drama'],\n",
       " 896: ['Wild Reeds (1994)', 'Drama'],\n",
       " 897: ['For Whom the Bell Tolls (1943)', 'Adventure/War'],\n",
       " 898: ['Philadelphia Story, The (1940)', 'Comedy/Romance'],\n",
       " 899: [\"Singin' in the Rain (1952)\", 'Musical/Romance'],\n",
       " 900: ['American in Paris, An (1951)', 'Musical/Romance'],\n",
       " 901: ['Funny Face (1957)', 'Comedy/Musical'],\n",
       " 902: [\"Breakfast at Tiffany's (1961)\", 'Drama/Romance'],\n",
       " 903: ['Vertigo (1958)', 'Mystery/Thriller'],\n",
       " 904: ['Rear Window (1954)', 'Mystery/Thriller'],\n",
       " 905: ['It Happened One Night (1934)', 'Comedy'],\n",
       " 906: ['Gaslight (1944)', 'Mystery/Thriller'],\n",
       " 907: ['Gay Divorcee, The (1934)', 'Comedy/Musical/Romance'],\n",
       " 908: ['North by Northwest (1959)', 'Drama/Thriller'],\n",
       " 909: ['Apartment, The (1960)', 'Comedy/Drama'],\n",
       " 910: ['Some Like It Hot (1959)', 'Comedy/Crime'],\n",
       " 911: ['Charade (1963)', 'Comedy/Mystery/Romance/Thriller'],\n",
       " 912: ['Casablanca (1942)', 'Drama/Romance/War'],\n",
       " 913: ['Maltese Falcon, The (1941)', 'Film-Noir/Mystery'],\n",
       " 914: ['My Fair Lady (1964)', 'Musical/Romance'],\n",
       " 915: ['Sabrina (1954)', 'Comedy/Romance'],\n",
       " 916: ['Roman Holiday (1953)', 'Comedy/Romance'],\n",
       " 917: ['Little Princess, The (1939)', \"Children's/Drama\"],\n",
       " 918: ['Meet Me in St. Louis (1944)', 'Musical'],\n",
       " 919: ['Wizard of Oz, The (1939)', \"Adventure/Children's/Drama/Musical\"],\n",
       " 920: ['Gone with the Wind (1939)', 'Drama/Romance/War'],\n",
       " 921: ['My Favorite Year (1982)', 'Comedy'],\n",
       " 922: ['Sunset Blvd. (a.k.a. Sunset Boulevard) (1950)', 'Film-Noir'],\n",
       " 923: ['Citizen Kane (1941)', 'Drama'],\n",
       " 924: ['2001: A Space Odyssey (1968)', 'Drama/Mystery/Sci-Fi/Thriller'],\n",
       " 925: ['Golden Earrings (1947)', 'Adventure/Romance'],\n",
       " 926: ['All About Eve (1950)', 'Drama'],\n",
       " 927: ['Women, The (1939)', 'Comedy'],\n",
       " 928: ['Rebecca (1940)', 'Romance/Thriller'],\n",
       " 929: ['Foreign Correspondent (1940)', 'Thriller'],\n",
       " 930: ['Notorious (1946)', 'Film-Noir/Romance/Thriller'],\n",
       " 931: ['Spellbound (1945)', 'Mystery/Romance/Thriller'],\n",
       " 932: ['Affair to Remember, An (1957)', 'Romance'],\n",
       " 933: ['To Catch a Thief (1955)', 'Comedy/Romance/Thriller'],\n",
       " 934: ['Father of the Bride (1950)', 'Comedy'],\n",
       " 935: ['Band Wagon, The (1953)', 'Comedy/Musical'],\n",
       " 936: ['Ninotchka (1939)', 'Comedy/Romance'],\n",
       " 937: ['Love in the Afternoon (1957)', 'Comedy/Romance'],\n",
       " 938: ['Gigi (1958)', 'Musical'],\n",
       " 939: ['Reluctant Debutante, The (1958)', 'Comedy/Drama'],\n",
       " 940: ['Adventures of Robin Hood, The (1938)', 'Action/Adventure'],\n",
       " 941: ['Mark of Zorro, The (1940)', 'Adventure'],\n",
       " 942: ['Laura (1944)', 'Crime/Film-Noir/Mystery'],\n",
       " 943: ['Ghost and Mrs. Muir, The (1947)', 'Drama/Romance'],\n",
       " 944: ['Lost Horizon (1937)', 'Drama'],\n",
       " 945: ['Top Hat (1935)', 'Comedy/Musical/Romance'],\n",
       " 946: ['To Be or Not to Be (1942)', 'Comedy/Drama/War'],\n",
       " 947: ['My Man Godfrey (1936)', 'Comedy'],\n",
       " 948: ['Giant (1956)', 'Drama'],\n",
       " 949: ['East of Eden (1955)', 'Drama'],\n",
       " 950: ['Thin Man, The (1934)', 'Mystery'],\n",
       " 951: ['His Girl Friday (1940)', 'Comedy'],\n",
       " 952: ['Around the World in 80 Days (1956)', 'Adventure/Comedy'],\n",
       " 953: [\"It's a Wonderful Life (1946)\", 'Drama'],\n",
       " 954: ['Mr. Smith Goes to Washington (1939)', 'Drama'],\n",
       " 955: ['Bringing Up Baby (1938)', 'Comedy'],\n",
       " 956: ['Penny Serenade (1941)', 'Drama/Romance'],\n",
       " 957: ['Scarlet Letter, The (1926)', 'Drama'],\n",
       " 958: ['Lady of Burlesque (1943)', 'Comedy/Mystery'],\n",
       " 959: ['Of Human Bondage (1934)', 'Drama'],\n",
       " 960: ['Angel on My Shoulder (1946)', 'Crime/Drama'],\n",
       " 961: ['Little Lord Fauntleroy (1936)', 'Drama'],\n",
       " 962: ['They Made Me a Criminal (1939)', 'Crime/Drama'],\n",
       " 963: ['Inspector General, The (1949)', 'Musical'],\n",
       " 964: ['Angel and the Badman (1947)', 'Western'],\n",
       " 965: ['39 Steps, The (1935)', 'Thriller'],\n",
       " 966: ['Walk in the Sun, A (1945)', 'Drama'],\n",
       " 967: ['Outlaw, The (1943)', 'Western'],\n",
       " 968: ['Night of the Living Dead (1968)', 'Horror/Sci-Fi'],\n",
       " 969: ['African Queen, The (1951)', 'Action/Adventure/Romance/War'],\n",
       " 970: ['Beat the Devil (1954)', 'Comedy/Drama'],\n",
       " 971: ['Cat on a Hot Tin Roof (1958)', 'Drama'],\n",
       " 972: ['Last Time I Saw Paris, The (1954)', 'Drama'],\n",
       " 973: ['Meet John Doe (1941)', 'Drama'],\n",
       " 974: ['Algiers (1938)', 'Drama/Romance'],\n",
       " 975: ['Something to Sing About (1937)', 'Comedy/Musical'],\n",
       " 976: ['Farewell to Arms, A (1932)', 'Romance/War'],\n",
       " 977: ['Moonlight Murder (1936)', 'Mystery'],\n",
       " 978: ['Blue Angel, The (Blaue Engel, Der) (1930)', 'Drama'],\n",
       " 979: ['Nothing Personal (1995)', 'Drama/War'],\n",
       " 980: ['In the Line of Duty 2 (1987)', 'Action'],\n",
       " 981: ['Dangerous Ground (1997)', 'Drama'],\n",
       " 982: ['Picnic (1955)', 'Drama'],\n",
       " 983: ['Madagascar Skin (1995)', 'Romance'],\n",
       " 984: ['Pompatus of Love, The (1996)', 'Comedy/Drama'],\n",
       " 985: ['Small Wonders (1996)', 'Documentary'],\n",
       " 986: ['Fly Away Home (1996)', \"Adventure/Children's\"],\n",
       " 987: ['Bliss (1997)', 'Drama/Romance'],\n",
       " 988: ['Grace of My Heart (1996)', 'Comedy/Drama'],\n",
       " 989: ['Schlafes Bruder (Brother of Sleep) (1995)', 'Drama'],\n",
       " 990: ['Maximum Risk (1996)', 'Action/Adventure/Thriller'],\n",
       " 991: ['Michael Collins (1996)', 'Drama/War'],\n",
       " 992: [\"Rich Man's Wife, The (1996)\", 'Thriller'],\n",
       " 993: ['Infinity (1996)', 'Drama'],\n",
       " 994: ['Big Night (1996)', 'Drama'],\n",
       " 996: ['Last Man Standing (1996)', 'Action/Drama/Western'],\n",
       " 997: ['Caught (1996)', 'Drama/Thriller'],\n",
       " 998: ['Set It Off (1996)', 'Action/Crime'],\n",
       " 999: ['2 Days in the Valley (1996)', 'Crime'],\n",
       " 1000: ['Curdled (1996)', 'Crime'],\n",
       " 1001: [\"Associate, The (L'Associe)(1982)\", 'Comedy'],\n",
       " 1002: [\"Ed's Next Move (1996)\", 'Comedy'],\n",
       " 1003: ['Extreme Measures (1996)', 'Drama/Thriller'],\n",
       " 1004: ['Glimmer Man, The (1996)', 'Action/Thriller'],\n",
       " 1005: ['D3: The Mighty Ducks (1996)', \"Children's/Comedy\"],\n",
       " 1006: ['Chamber, The (1996)', 'Drama'],\n",
       " 1007: ['Apple Dumpling Gang, The (1975)', \"Children's/Comedy/Western\"],\n",
       " 1008: ['Davy Crockett, King of the Wild Frontier (1955)', 'Western'],\n",
       " 1009: ['Escape to Witch Mountain (1975)', \"Adventure/Children's/Fantasy\"],\n",
       " 1010: ['Love Bug, The (1969)', \"Children's/Comedy\"],\n",
       " 1011: ['Herbie Rides Again (1974)', \"Adventure/Children's/Comedy\"],\n",
       " 1012: ['Old Yeller (1957)', \"Children's/Drama\"],\n",
       " ...}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top 10 movie recommendations for Usual Suspects, The (1995) (Crime/Thriller) with similar ratings:\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "345 viewers also watched:\n",
      "Johnny Mnemonic (1995)\n",
      "Category: Action/Sci-Fi/Thriller\n",
      "Similarity Score: 0.9895522078385338\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "480 viewers also watched:\n",
      "Mighty Morphin Power Rangers: The Movie (1995)\n",
      "Category: Action/Children's\n",
      "Similarity Score: 0.9857230861253026\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "380 viewers also watched:\n",
      "Jury Duty (1995)\n",
      "Category: Comedy\n",
      "Similarity Score: 0.981760098872619\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "68 viewers also watched:\n",
      "Birdcage, The (1996)\n",
      "Category: Comedy\n",
      "Similarity Score: 0.9789385605497993\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "109 viewers also watched:\n",
      "Love & Human Remains (1993)\n",
      "Category: Comedy\n",
      "Similarity Score: 0.9776576120448436\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "92 viewers also watched:\n",
      "8 Seconds (1994)\n",
      "Category: Drama\n",
      "Similarity Score: 0.9775948291054827\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "138 viewers also watched:\n",
      "Mr. Jones (1993)\n",
      "Category: Drama/Romance\n",
      "Similarity Score: 0.9764692222674887\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "204 viewers also watched:\n",
      "Smoke (1995)\n",
      "Category: Drama\n",
      "Similarity Score: 0.9751512937740359\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "103 viewers also watched:\n",
      "Free Willy 2: The Adventure Home (1995)\n",
      "Category: Adventure/Children's/Drama\n",
      "Similarity Score: 0.9748681355460885\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n",
      "58 viewers also watched:\n",
      "Margaret's Museum (1995)\n",
      "Category: Drama\n",
      "Similarity Score: 0.9741816128302572\n",
      "\n",
      "----------------------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# Print movie recommendations in full details\n",
    "\n",
    "print(\n",
    "    \"Top 10 movie recommendations for {} ({}) with similar ratings:\\n\".format(\n",
    "        movies[movieID][0], movies[movieID][1]\n",
    "    )\n",
    ")\n",
    "print(\"-\" * 80)\n",
    "\n",
    "for result in top_10:\n",
    "    # Get the name and category of similar movies\n",
    "    if result[0][0] == movieID:\n",
    "        similar_movie_name = movies[result[0][1]][0]\n",
    "        category = movies[result[0][1]][1]\n",
    "    if result[0][1] == movieID:\n",
    "        similar_movie_name = movies[result[0][0]][0]\n",
    "        category = movies[result[0][0]][1]\n",
    "\n",
    "    # Extract the similarity score and strength\n",
    "    similarity_score = result[1][0]\n",
    "    strength = result[1][1]\n",
    "\n",
    "    # Print the information about each similar movie, including its name, similarity score, category, and strength\n",
    "    print(\n",
    "        \"{} viewers also watched:\\n{}\\nCategory: {}\\nSimilarity Score: {}\\n\".format(\n",
    "            strength, similar_movie_name, category, similarity_score\n",
    "        )\n",
    "    )\n",
    "    print(\"-\" * 80)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run the driver Python script version of the code above from the master node of the EMR cluster (i.e., Master public DNS), use the following command:\n",
    "\n",
    "`spark-submit --executor-memory 1g [Spark-driver-script-name].py`\n",
    "\n",
    "Please note that since the default executor memory (512MB) might not be sufficient for processing one million movie ratings, it is essential to specify the executor memory to be 1GB."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.17"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "262.6px"
   },
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}