{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"\n",
    "    border: 2px solid #4CAF50; \n",
    "    padding: 15px; \n",
    "    background-color: #f4f4f4; \n",
    "    border-radius: 10px; \n",
    "    align-items: center;\">\n",
    "\n",
    "<h1 style=\"margin: 0; color: #4CAF50;\">Tabellarische Daten: Ein Beispiel</h1>\n",
    "<h2 style=\"margin: 5px 0; color: #555;\">DSAI</h2>\n",
    "<h3 style=\"margin: 5px 0; color: #555;\">Jakob Eggl</h3>\n",
    "\n",
    "<div style=\"flex-shrink: 0;\">\n",
    "    <img src=\"https://www.htl-grieskirchen.at/wp/wp-content/uploads/2022/11/logo_bildschirm-1024x503.png\" alt=\"Logo\" style=\"width: 250px; height: auto;\"/>\n",
    "</div>\n",
    "<p1> © 2024/25 Jakob Eggl. Nutzung oder Verbreitung nur mit ausdrücklicher Genehmigung des Autors.</p1>\n",
    "</div>\n",
    "<div style=\"flex: 1;\">\n",
    "</div>   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ein Beispiel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wählen Sie eines das folgende Dataset. Verwenden Sie bitte die Versionen, die auf Eduvidual bereitgestellt sind! (Die Links sind als Informationsquelle und als Referenzangabe angedacht).\n",
    "\n",
    "* Insurance `insurance.csv` (verwendet von https://www.kaggle.com/datasets/owaiskhan9654/health-insurance-lead-prediction-raw-data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wiederholen Sie nun jene Schritte der letzten beiden Stunden, die Sie für notwendig halten. Begründen Sie die Entscheidungen! Untersuchen und vorverarbeiten Sie dabei die gegebenen Daten (Visualisierungen sind nicht notwendig, diese werden erst später behandelt). Dokumentieren Sie Auffälligkeiten und Erkenntnisse."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Beachten Sie dabei insbesondere folgende Punkte (wählen Sie aber auch weitere Schritte, welche Sie für sinnvoll halten, basierend auf den besprochenen Möglichkeiten der letzten Stunden):\n",
    "* Um was geht es bei dem Dataset?x\n",
    "* Welche Features gibt es?x\n",
    "* Gibt es ein offensichtliches Label? Würden andere Spalten als Label auch Sinn machen? (Können auch numerisch, i.e. *float* sein)!x\n",
    "* Gibt es nominale, quantitative oder ordinale Spalten? Müssen diese umgerechnet werden?x\n",
    "* Wie gut ist die Datenqualität (NaN, Sanity Check, etc.) -> Wie gehen Sie mit diesen Daten um?x\n",
    "* Normalisieren Sie geeignete Spalten und speichern Sie diese in einer neuen Spalte. Welche Normalisierungsmethode wählen Sie und warum?x\n",
    "* Ist das Aggregieren sinnvoll? Wenden Sie auch eine Aggregierungsmehtode an um zBsp die Initialen von einem Namen herauszufinden oder ähnliches.\n",
    "* Welche Gruppierungen würden ggf. Sinn machen?\n",
    "* Falls möglich \"erfinden\" Sie ein weiteres Dataframe, welches Sie mergen können\n",
    "* etc."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dokumentieren Sie sämtliche Ergebnisse, indem Sie auch diese Zellen im Notebook belassen, welche keine zufriedenstellenden Ergebnisse liefern."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import Required Libraries\n",
    "import pandas as pd\n",
    "import os\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "import sklearn.linear_model as lm\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# Load Dataset\n",
    "path = os.path.join(\"..\", \"..\", \"_data\", \"insurance.csv\")\n",
    "df = pd.read_csv(path)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Um was geht es bei dem Dataset?\n",
    "\n",
    "Das Dataset bezieht sich auf Versicherungsdaten. Es enthält Informationen über verschiedene Versicherungsnehmer, einschließlich ihrer Altersgruppen, Wohnsituation, Art der empfohlenen Versicherung, Gesundheitsindikatoren und mehr. Ziel ist es, die Prämien und die Wahrscheinlichkeit einer positiven Antwort auf ein Versicherungsangebot zu analysieren.<br>\n",
    "Response -> Die Person hat eine Versicherung abgeschlossen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First few rows of the dataset:\n",
      "   ID City_Code  Region_Code Accomodation_Type Reco_Insurance_Type  Upper_Age  \\\n",
      "0   1        C3         3213            Rented          Individual         36   \n",
      "1   2        C5         1117             Owned               Joint         75   \n",
      "2   3        C5         3732             Owned          Individual         32   \n",
      "3   4       C24         4378             Owned               Joint         52   \n",
      "4   5        C8         2190            Rented          Individual         44   \n",
      "\n",
      "   Lower_Age Is_Spouse Health Indicator Holding_Policy_Duration  \\\n",
      "0         36        No               X1                     14+   \n",
      "1         22        No               X2                     NaN   \n",
      "2         32        No              NaN                     1.0   \n",
      "3         48        No               X1                     14+   \n",
      "4         44        No               X2                     3.0   \n",
      "\n",
      "   Holding_Policy_Type  Reco_Policy_Cat  Reco_Policy_Premium  Response  \n",
      "0                  3.0               22              11628.0         0  \n",
      "1                  NaN               22              30510.0         0  \n",
      "2                  1.0               19               7450.0         1  \n",
      "3                  3.0               19              17780.0         0  \n",
      "4                  1.0               16              10404.0         0  \n"
     ]
    }
   ],
   "source": [
    "# Understand the Dataset\n",
    "print(\"First few rows of the dataset:\")\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Summary statistics of the dataset:\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 50882 entries, 0 to 50881\n",
      "Data columns (total 14 columns):\n",
      " #   Column                   Non-Null Count  Dtype  \n",
      "---  ------                   --------------  -----  \n",
      " 0   ID                       50882 non-null  int64  \n",
      " 1   City_Code                50882 non-null  object \n",
      " 2   Region_Code              50882 non-null  int64  \n",
      " 3   Accomodation_Type        50882 non-null  object \n",
      " 4   Reco_Insurance_Type      50882 non-null  object \n",
      " 5   Upper_Age                50882 non-null  int64  \n",
      " 6   Lower_Age                50882 non-null  int64  \n",
      " 7   Is_Spouse                50882 non-null  object \n",
      " 8   Health Indicator         39191 non-null  object \n",
      " 9   Holding_Policy_Duration  30631 non-null  object \n",
      " 10  Holding_Policy_Type      30631 non-null  float64\n",
      " 11  Reco_Policy_Cat          50882 non-null  int64  \n",
      " 12  Reco_Policy_Premium      50882 non-null  float64\n",
      " 13  Response                 50882 non-null  int64  \n",
      "dtypes: float64(2), int64(6), object(6)\n",
      "memory usage: 5.4+ MB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(\"\\nSummary statistics of the dataset:\")\n",
    "print(df.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Data types of each column:\n",
      "ID                           int64\n",
      "City_Code                   object\n",
      "Region_Code                  int64\n",
      "Accomodation_Type           object\n",
      "Reco_Insurance_Type         object\n",
      "Upper_Age                    int64\n",
      "Lower_Age                    int64\n",
      "Is_Spouse                   object\n",
      "Health Indicator            object\n",
      "Holding_Policy_Duration     object\n",
      "Holding_Policy_Type        float64\n",
      "Reco_Policy_Cat              int64\n",
      "Reco_Policy_Premium        float64\n",
      "Response                     int64\n",
      "dtype: object\n",
      "14\n"
     ]
    }
   ],
   "source": [
    "\n",
    "print(\"\\nData types of each column:\")\n",
    "print(df.dtypes)\n",
    "print(df.dtypes.count())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welche Features gibt es?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Features in the dataset:\n",
      "['ID', 'City_Code', 'Region_Code', 'Accomodation_Type', 'Reco_Insurance_Type', 'Upper_Age', 'Lower_Age', 'Is_Spouse', 'Health Indicator', 'Holding_Policy_Duration', 'Holding_Policy_Type', 'Reco_Policy_Cat', 'Reco_Policy_Premium', 'Response']\n"
     ]
    }
   ],
   "source": [
    "# Identify Features and Labels\n",
    "features = df.columns.tolist()\n",
    "print(\"\\nFeatures in the dataset:\")\n",
    "print(features)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Was ist ein offensichtliches Label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Label in the dataset:\n",
      "Response\n"
     ]
    }
   ],
   "source": [
    "# Ein offensichtliches Label ist die Spalte Response-> ob die Person eine Versicherung abgeschlossen hat oder nicht\n",
    "label = 'Response'\n",
    "print(\"\\nLabel in the dataset:\")\n",
    "print(label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gibt es nominale, quantitative oder ordinale Spalten? Müssen diese umgerechnet werden?\n",
    "\n",
    "\n",
    "\n",
    "* ### Quantitative Spalten: Diese Spalten enthalten numerische Werte, die eine natürliche Reihenfolge und Abstände haben. ->Können nominalisiert werden\n",
    "Man kann damit fertig rechen -> eventuell nominalisieren\n",
    "<br><br>\n",
    "Upper_Age <br>\n",
    "Lower_Age <br>\n",
    "Reco_Policy_Premium <br>\n",
    "\n",
    "* ### Ordinale Spalten: Diese Spalten enthalten Kategorien mit einer natürlichen Reihenfolge.\n",
    "Man kann einteilen und es gibt eine Reihenfolge-> eventuell Umrechnen\n",
    "<br><br>\n",
    "Holding_Policy_Duration<br>\n",
    "Reco_Policy_Cat\n",
    "\n",
    "* ### Nominale Spalten: Diese Spalten enthalten Kategorien ohne eine natürliche Reihenfolge.\n",
    "Man kann einteilen, aber keine Reihenfolge -> eventuell Umrechnen\n",
    "<br><br>\n",
    "City_Code <br>\n",
    "Region_Code <br>\n",
    "Accomodation_Type <br>\n",
    "Reco_Insurance_Type <br>\n",
    "Is_Spouse <br>\n",
    "Health Indicator <br>\n",
    "Holding_Policy_Type <br>\n",
    "Response (binär) <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Wie gut ist die Datenqualität (NaN, etc.) -> Wie gehen Sie mit diesen Daten um?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Checking for missing values:\n",
      "ID                             0\n",
      "City_Code                      0\n",
      "Region_Code                    0\n",
      "Accomodation_Type              0\n",
      "Reco_Insurance_Type            0\n",
      "Upper_Age                      0\n",
      "Lower_Age                      0\n",
      "Is_Spouse                      0\n",
      "Health Indicator           11691\n",
      "Holding_Policy_Duration    20251\n",
      "Holding_Policy_Type        20251\n",
      "Reco_Policy_Cat                0\n",
      "Reco_Policy_Premium            0\n",
      "Response                       0\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "# Data Quality Check\n",
    "print(\"\\nChecking for missing values:\")\n",
    "print(df.isnull().sum())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "50882\n",
      "23548\n"
     ]
    }
   ],
   "source": [
    "# Handle missing values (if any)\n",
    "# For simplicity, let's drop rows with missing values\n",
    "print(df.count().max())\n",
    "df = df.dropna()\n",
    "print(df.count().max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalisieren Sie geeignete Spalten und speichern Sie diese in einer neuen Spalte. Welche Normalisierungsmethode wählen Sie und warum?\n",
    "MinMaxScaler->Minimalwert und Maximalwert aus der Spalte wird zu 0 und 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>City_Code</th>\n",
       "      <th>Region_Code</th>\n",
       "      <th>Accomodation_Type</th>\n",
       "      <th>Reco_Insurance_Type</th>\n",
       "      <th>Upper_Age</th>\n",
       "      <th>Lower_Age</th>\n",
       "      <th>Is_Spouse</th>\n",
       "      <th>Health Indicator</th>\n",
       "      <th>Holding_Policy_Duration</th>\n",
       "      <th>Holding_Policy_Type</th>\n",
       "      <th>Reco_Policy_Cat</th>\n",
       "      <th>Reco_Policy_Premium</th>\n",
       "      <th>Response</th>\n",
       "      <th>Upper_Age_Normalized</th>\n",
       "      <th>Lower_Age_Normalized</th>\n",
       "      <th>Reco_Policy_Premium_Normalized</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>C3</td>\n",
       "      <td>3213</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>36</td>\n",
       "      <td>36</td>\n",
       "      <td>No</td>\n",
       "      <td>X1</td>\n",
       "      <td>14+</td>\n",
       "      <td>3.0</td>\n",
       "      <td>22</td>\n",
       "      <td>11628.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.277778</td>\n",
       "      <td>0.338983</td>\n",
       "      <td>0.209596</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>C24</td>\n",
       "      <td>4378</td>\n",
       "      <td>Owned</td>\n",
       "      <td>Joint</td>\n",
       "      <td>52</td>\n",
       "      <td>48</td>\n",
       "      <td>No</td>\n",
       "      <td>X1</td>\n",
       "      <td>14+</td>\n",
       "      <td>3.0</td>\n",
       "      <td>19</td>\n",
       "      <td>17780.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.574074</td>\n",
       "      <td>0.542373</td>\n",
       "      <td>0.362881</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>C8</td>\n",
       "      <td>2190</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>44</td>\n",
       "      <td>44</td>\n",
       "      <td>No</td>\n",
       "      <td>X2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>16</td>\n",
       "      <td>10404.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.425926</td>\n",
       "      <td>0.474576</td>\n",
       "      <td>0.179098</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>C9</td>\n",
       "      <td>1785</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>52</td>\n",
       "      <td>52</td>\n",
       "      <td>No</td>\n",
       "      <td>X2</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>22</td>\n",
       "      <td>15264.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.574074</td>\n",
       "      <td>0.610169</td>\n",
       "      <td>0.300191</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>C1</td>\n",
       "      <td>3175</td>\n",
       "      <td>Owned</td>\n",
       "      <td>Joint</td>\n",
       "      <td>75</td>\n",
       "      <td>73</td>\n",
       "      <td>Yes</td>\n",
       "      <td>X4</td>\n",
       "      <td>9.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>17</td>\n",
       "      <td>29344.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.966102</td>\n",
       "      <td>0.651013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50875</th>\n",
       "      <td>50876</td>\n",
       "      <td>C6</td>\n",
       "      <td>231</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>36</td>\n",
       "      <td>36</td>\n",
       "      <td>No</td>\n",
       "      <td>X3</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>20</td>\n",
       "      <td>13574.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.277778</td>\n",
       "      <td>0.338983</td>\n",
       "      <td>0.258083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50878</th>\n",
       "      <td>50879</td>\n",
       "      <td>C5</td>\n",
       "      <td>4188</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>27</td>\n",
       "      <td>27</td>\n",
       "      <td>No</td>\n",
       "      <td>X3</td>\n",
       "      <td>7.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4</td>\n",
       "      <td>5408.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.111111</td>\n",
       "      <td>0.186441</td>\n",
       "      <td>0.054616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50879</th>\n",
       "      <td>50880</td>\n",
       "      <td>C1</td>\n",
       "      <td>442</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>63</td>\n",
       "      <td>63</td>\n",
       "      <td>No</td>\n",
       "      <td>X2</td>\n",
       "      <td>14+</td>\n",
       "      <td>1.0</td>\n",
       "      <td>12</td>\n",
       "      <td>11374.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.777778</td>\n",
       "      <td>0.796610</td>\n",
       "      <td>0.203267</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50880</th>\n",
       "      <td>50881</td>\n",
       "      <td>C1</td>\n",
       "      <td>4</td>\n",
       "      <td>Owned</td>\n",
       "      <td>Joint</td>\n",
       "      <td>71</td>\n",
       "      <td>49</td>\n",
       "      <td>No</td>\n",
       "      <td>X2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>16</td>\n",
       "      <td>28179.2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.925926</td>\n",
       "      <td>0.559322</td>\n",
       "      <td>0.621990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50881</th>\n",
       "      <td>50882</td>\n",
       "      <td>C3</td>\n",
       "      <td>3866</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>No</td>\n",
       "      <td>X3</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>18</td>\n",
       "      <td>11424.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.055556</td>\n",
       "      <td>0.135593</td>\n",
       "      <td>0.204513</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>23548 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          ID City_Code  Region_Code Accomodation_Type Reco_Insurance_Type  \\\n",
       "0          1        C3         3213            Rented          Individual   \n",
       "3          4       C24         4378             Owned               Joint   \n",
       "4          5        C8         2190            Rented          Individual   \n",
       "5          6        C9         1785            Rented          Individual   \n",
       "7          8        C1         3175             Owned               Joint   \n",
       "...      ...       ...          ...               ...                 ...   \n",
       "50875  50876        C6          231            Rented          Individual   \n",
       "50878  50879        C5         4188            Rented          Individual   \n",
       "50879  50880        C1          442            Rented          Individual   \n",
       "50880  50881        C1            4             Owned               Joint   \n",
       "50881  50882        C3         3866            Rented          Individual   \n",
       "\n",
       "       Upper_Age  Lower_Age Is_Spouse Health Indicator  \\\n",
       "0             36         36        No               X1   \n",
       "3             52         48        No               X1   \n",
       "4             44         44        No               X2   \n",
       "5             52         52        No               X2   \n",
       "7             75         73       Yes               X4   \n",
       "...          ...        ...       ...              ...   \n",
       "50875         36         36        No               X3   \n",
       "50878         27         27        No               X3   \n",
       "50879         63         63        No               X2   \n",
       "50880         71         49        No               X2   \n",
       "50881         24         24        No               X3   \n",
       "\n",
       "      Holding_Policy_Duration  Holding_Policy_Type  Reco_Policy_Cat  \\\n",
       "0                         14+                  3.0               22   \n",
       "3                         14+                  3.0               19   \n",
       "4                         3.0                  1.0               16   \n",
       "5                         5.0                  1.0               22   \n",
       "7                         9.0                  4.0               17   \n",
       "...                       ...                  ...              ...   \n",
       "50875                     2.0                  1.0               20   \n",
       "50878                     7.0                  3.0                4   \n",
       "50879                     14+                  1.0               12   \n",
       "50880                     2.0                  2.0               16   \n",
       "50881                     2.0                  3.0               18   \n",
       "\n",
       "       Reco_Policy_Premium  Response  Upper_Age_Normalized  \\\n",
       "0                  11628.0         0              0.277778   \n",
       "3                  17780.0         0              0.574074   \n",
       "4                  10404.0         0              0.425926   \n",
       "5                  15264.0         1              0.574074   \n",
       "7                  29344.0         1              1.000000   \n",
       "...                    ...       ...                   ...   \n",
       "50875              13574.0         1              0.277778   \n",
       "50878               5408.0         0              0.111111   \n",
       "50879              11374.0         0              0.777778   \n",
       "50880              28179.2         1              0.925926   \n",
       "50881              11424.0         0              0.055556   \n",
       "\n",
       "       Lower_Age_Normalized  Reco_Policy_Premium_Normalized  \n",
       "0                  0.338983                        0.209596  \n",
       "3                  0.542373                        0.362881  \n",
       "4                  0.474576                        0.179098  \n",
       "5                  0.610169                        0.300191  \n",
       "7                  0.966102                        0.651013  \n",
       "...                     ...                             ...  \n",
       "50875              0.338983                        0.258083  \n",
       "50878              0.186441                        0.054616  \n",
       "50879              0.796610                        0.203267  \n",
       "50880              0.559322                        0.621990  \n",
       "50881              0.135593                        0.204513  \n",
       "\n",
       "[23548 rows x 17 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Normalize Data\n",
    "quantitative_columns = ['Upper_Age', 'Lower_Age', 'Reco_Policy_Premium']\n",
    "quantitative_columns_normalized = ['Upper_Age_Normalized', 'Lower_Age_Normalized', 'Reco_Policy_Premium_Normalized']\n",
    "scaler = MinMaxScaler()\n",
    "df[quantitative_columns_normalized] = scaler.fit_transform(df[quantitative_columns])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ist das Aggregieren sinnvoll? Wenden Sie auch eine Aggregierungsmethode an um zBsp die Initialen von einem Namen herauszufinden oder ähnliches.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>City_Code</th>\n",
       "      <th>Region_Code</th>\n",
       "      <th>Accomodation_Type</th>\n",
       "      <th>Reco_Insurance_Type</th>\n",
       "      <th>Upper_Age</th>\n",
       "      <th>Lower_Age</th>\n",
       "      <th>Is_Spouse</th>\n",
       "      <th>Health Indicator</th>\n",
       "      <th>Holding_Policy_Duration</th>\n",
       "      <th>Holding_Policy_Type</th>\n",
       "      <th>Reco_Policy_Cat</th>\n",
       "      <th>Reco_Policy_Premium</th>\n",
       "      <th>Response</th>\n",
       "      <th>Upper_Age_Normalized</th>\n",
       "      <th>Lower_Age_Normalized</th>\n",
       "      <th>Reco_Policy_Premium_Normalized</th>\n",
       "      <th>Accomodation_Type_Short</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>C3</td>\n",
       "      <td>3213</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>36</td>\n",
       "      <td>36</td>\n",
       "      <td>No</td>\n",
       "      <td>X1</td>\n",
       "      <td>14+</td>\n",
       "      <td>3.0</td>\n",
       "      <td>22</td>\n",
       "      <td>11628.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.277778</td>\n",
       "      <td>0.338983</td>\n",
       "      <td>0.209596</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>C24</td>\n",
       "      <td>4378</td>\n",
       "      <td>Owned</td>\n",
       "      <td>Joint</td>\n",
       "      <td>52</td>\n",
       "      <td>48</td>\n",
       "      <td>No</td>\n",
       "      <td>X1</td>\n",
       "      <td>14+</td>\n",
       "      <td>3.0</td>\n",
       "      <td>19</td>\n",
       "      <td>17780.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.574074</td>\n",
       "      <td>0.542373</td>\n",
       "      <td>0.362881</td>\n",
       "      <td>O</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>C8</td>\n",
       "      <td>2190</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>44</td>\n",
       "      <td>44</td>\n",
       "      <td>No</td>\n",
       "      <td>X2</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>16</td>\n",
       "      <td>10404.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.425926</td>\n",
       "      <td>0.474576</td>\n",
       "      <td>0.179098</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>C9</td>\n",
       "      <td>1785</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>52</td>\n",
       "      <td>52</td>\n",
       "      <td>No</td>\n",
       "      <td>X2</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>22</td>\n",
       "      <td>15264.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.574074</td>\n",
       "      <td>0.610169</td>\n",
       "      <td>0.300191</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>8</td>\n",
       "      <td>C1</td>\n",
       "      <td>3175</td>\n",
       "      <td>Owned</td>\n",
       "      <td>Joint</td>\n",
       "      <td>75</td>\n",
       "      <td>73</td>\n",
       "      <td>Yes</td>\n",
       "      <td>X4</td>\n",
       "      <td>9.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>17</td>\n",
       "      <td>29344.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.966102</td>\n",
       "      <td>0.651013</td>\n",
       "      <td>O</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50875</th>\n",
       "      <td>50876</td>\n",
       "      <td>C6</td>\n",
       "      <td>231</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>36</td>\n",
       "      <td>36</td>\n",
       "      <td>No</td>\n",
       "      <td>X3</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>20</td>\n",
       "      <td>13574.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0.277778</td>\n",
       "      <td>0.338983</td>\n",
       "      <td>0.258083</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50878</th>\n",
       "      <td>50879</td>\n",
       "      <td>C5</td>\n",
       "      <td>4188</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>27</td>\n",
       "      <td>27</td>\n",
       "      <td>No</td>\n",
       "      <td>X3</td>\n",
       "      <td>7.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4</td>\n",
       "      <td>5408.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.111111</td>\n",
       "      <td>0.186441</td>\n",
       "      <td>0.054616</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50879</th>\n",
       "      <td>50880</td>\n",
       "      <td>C1</td>\n",
       "      <td>442</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>63</td>\n",
       "      <td>63</td>\n",
       "      <td>No</td>\n",
       "      <td>X2</td>\n",
       "      <td>14+</td>\n",
       "      <td>1.0</td>\n",
       "      <td>12</td>\n",
       "      <td>11374.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.777778</td>\n",
       "      <td>0.796610</td>\n",
       "      <td>0.203267</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50880</th>\n",
       "      <td>50881</td>\n",
       "      <td>C1</td>\n",
       "      <td>4</td>\n",
       "      <td>Owned</td>\n",
       "      <td>Joint</td>\n",
       "      <td>71</td>\n",
       "      <td>49</td>\n",
       "      <td>No</td>\n",
       "      <td>X2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>16</td>\n",
       "      <td>28179.2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.925926</td>\n",
       "      <td>0.559322</td>\n",
       "      <td>0.621990</td>\n",
       "      <td>O</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50881</th>\n",
       "      <td>50882</td>\n",
       "      <td>C3</td>\n",
       "      <td>3866</td>\n",
       "      <td>Rented</td>\n",
       "      <td>Individual</td>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>No</td>\n",
       "      <td>X3</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>18</td>\n",
       "      <td>11424.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.055556</td>\n",
       "      <td>0.135593</td>\n",
       "      <td>0.204513</td>\n",
       "      <td>R</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>23548 rows × 18 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          ID City_Code  Region_Code Accomodation_Type Reco_Insurance_Type  \\\n",
       "0          1        C3         3213            Rented          Individual   \n",
       "3          4       C24         4378             Owned               Joint   \n",
       "4          5        C8         2190            Rented          Individual   \n",
       "5          6        C9         1785            Rented          Individual   \n",
       "7          8        C1         3175             Owned               Joint   \n",
       "...      ...       ...          ...               ...                 ...   \n",
       "50875  50876        C6          231            Rented          Individual   \n",
       "50878  50879        C5         4188            Rented          Individual   \n",
       "50879  50880        C1          442            Rented          Individual   \n",
       "50880  50881        C1            4             Owned               Joint   \n",
       "50881  50882        C3         3866            Rented          Individual   \n",
       "\n",
       "       Upper_Age  Lower_Age Is_Spouse Health Indicator  \\\n",
       "0             36         36        No               X1   \n",
       "3             52         48        No               X1   \n",
       "4             44         44        No               X2   \n",
       "5             52         52        No               X2   \n",
       "7             75         73       Yes               X4   \n",
       "...          ...        ...       ...              ...   \n",
       "50875         36         36        No               X3   \n",
       "50878         27         27        No               X3   \n",
       "50879         63         63        No               X2   \n",
       "50880         71         49        No               X2   \n",
       "50881         24         24        No               X3   \n",
       "\n",
       "      Holding_Policy_Duration  Holding_Policy_Type  Reco_Policy_Cat  \\\n",
       "0                         14+                  3.0               22   \n",
       "3                         14+                  3.0               19   \n",
       "4                         3.0                  1.0               16   \n",
       "5                         5.0                  1.0               22   \n",
       "7                         9.0                  4.0               17   \n",
       "...                       ...                  ...              ...   \n",
       "50875                     2.0                  1.0               20   \n",
       "50878                     7.0                  3.0                4   \n",
       "50879                     14+                  1.0               12   \n",
       "50880                     2.0                  2.0               16   \n",
       "50881                     2.0                  3.0               18   \n",
       "\n",
       "       Reco_Policy_Premium  Response  Upper_Age_Normalized  \\\n",
       "0                  11628.0         0              0.277778   \n",
       "3                  17780.0         0              0.574074   \n",
       "4                  10404.0         0              0.425926   \n",
       "5                  15264.0         1              0.574074   \n",
       "7                  29344.0         1              1.000000   \n",
       "...                    ...       ...                   ...   \n",
       "50875              13574.0         1              0.277778   \n",
       "50878               5408.0         0              0.111111   \n",
       "50879              11374.0         0              0.777778   \n",
       "50880              28179.2         1              0.925926   \n",
       "50881              11424.0         0              0.055556   \n",
       "\n",
       "       Lower_Age_Normalized  Reco_Policy_Premium_Normalized  \\\n",
       "0                  0.338983                        0.209596   \n",
       "3                  0.542373                        0.362881   \n",
       "4                  0.474576                        0.179098   \n",
       "5                  0.610169                        0.300191   \n",
       "7                  0.966102                        0.651013   \n",
       "...                     ...                             ...   \n",
       "50875              0.338983                        0.258083   \n",
       "50878              0.186441                        0.054616   \n",
       "50879              0.796610                        0.203267   \n",
       "50880              0.559322                        0.621990   \n",
       "50881              0.135593                        0.204513   \n",
       "\n",
       "      Accomodation_Type_Short  \n",
       "0                           R  \n",
       "3                           O  \n",
       "4                           R  \n",
       "5                           R  \n",
       "7                           O  \n",
       "...                       ...  \n",
       "50875                       R  \n",
       "50878                       R  \n",
       "50879                       R  \n",
       "50880                       O  \n",
       "50881                       R  \n",
       "\n",
       "[23548 rows x 18 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Aggregation\n",
    "df['Accomodation_Type_Short'] = df['Accomodation_Type'].apply(lambda x: x[0])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welche Gruppierungen würden ggf. Sinn machen?\n",
    "Mit City_Code gruppieren"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City_Code</th>\n",
       "      <th>Response</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>C1</td>\n",
       "      <td>1085</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>C2</td>\n",
       "      <td>958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>C3</td>\n",
       "      <td>544</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>C4</td>\n",
       "      <td>390</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>C7</td>\n",
       "      <td>227</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>C9</td>\n",
       "      <td>208</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>C6</td>\n",
       "      <td>205</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>C8</td>\n",
       "      <td>204</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>C10</td>\n",
       "      <td>175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>C5</td>\n",
       "      <td>147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>C11</td>\n",
       "      <td>135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>C17</td>\n",
       "      <td>126</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>C15</td>\n",
       "      <td>126</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>C16</td>\n",
       "      <td>115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>C13</td>\n",
       "      <td>108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>C20</td>\n",
       "      <td>99</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>C19</td>\n",
       "      <td>89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>C14</td>\n",
       "      <td>85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>C18</td>\n",
       "      <td>82</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>C21</td>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>C12</td>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>C24</td>\n",
       "      <td>62</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>C22</td>\n",
       "      <td>56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>C23</td>\n",
       "      <td>49</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>C26</td>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>C25</td>\n",
       "      <td>44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>C29</td>\n",
       "      <td>42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>C28</td>\n",
       "      <td>33</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>C27</td>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>C33</td>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>C32</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>C34</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>C35</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>C30</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>C36</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>C31</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   City_Code  Response\n",
       "0         C1      1085\n",
       "11        C2       958\n",
       "22        C3       544\n",
       "30        C4       390\n",
       "33        C7       227\n",
       "35        C9       208\n",
       "32        C6       205\n",
       "34        C8       204\n",
       "1        C10       175\n",
       "31        C5       147\n",
       "2        C11       135\n",
       "8        C17       126\n",
       "6        C15       126\n",
       "7        C16       115\n",
       "4        C13       108\n",
       "12       C20        99\n",
       "10       C19        89\n",
       "5        C14        85\n",
       "9        C18        82\n",
       "13       C21        79\n",
       "3        C12        77\n",
       "16       C24        62\n",
       "14       C22        56\n",
       "15       C23        49\n",
       "18       C26        48\n",
       "17       C25        44\n",
       "21       C29        42\n",
       "20       C28        33\n",
       "19       C27        28\n",
       "26       C33        26\n",
       "25       C32        15\n",
       "27       C34        14\n",
       "28       C35         8\n",
       "23       C30         7\n",
       "29       C36         3\n",
       "24       C31         1"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Group by City_Code and get The count of Positive Responses\n",
    "grouped_df = df.groupby('City_Code').agg({'Response': 'sum'}).reset_index().sort_values('Response', ascending=False)\n",
    "grouped_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Holding_Policy_Duration</th>\n",
       "      <th>Response</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>14+</td>\n",
       "      <td>849</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2.0</td>\n",
       "      <td>788</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>771</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>3.0</td>\n",
       "      <td>647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>4.0</td>\n",
       "      <td>522</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>5.0</td>\n",
       "      <td>422</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>6.0</td>\n",
       "      <td>350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>7.0</td>\n",
       "      <td>322</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>8.0</td>\n",
       "      <td>266</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>9.0</td>\n",
       "      <td>207</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>10.0</td>\n",
       "      <td>162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>11.0</td>\n",
       "      <td>107</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13.0</td>\n",
       "      <td>105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>14.0</td>\n",
       "      <td>91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>12.0</td>\n",
       "      <td>91</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Holding_Policy_Duration  Response\n",
       "5                      14+       849\n",
       "7                      2.0       788\n",
       "0                      1.0       771\n",
       "8                      3.0       647\n",
       "9                      4.0       522\n",
       "10                     5.0       422\n",
       "11                     6.0       350\n",
       "12                     7.0       322\n",
       "13                     8.0       266\n",
       "14                     9.0       207\n",
       "1                     10.0       162\n",
       "2                     11.0       107\n",
       "4                     13.0       105\n",
       "6                     14.0        91\n",
       "3                     12.0        91"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Group by Holding_Policy_Duration and get The count of Positive Responses\n",
    "grouped_df = df.groupby('Holding_Policy_Duration').agg({'Response': 'sum'}).reset_index().sort_values('Response', ascending=False)\n",
    "grouped_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Holding_Policy_Type</th>\n",
       "      <th>Response</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3.0</td>\n",
       "      <td>2484</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1528</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.0</td>\n",
       "      <td>873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4.0</td>\n",
       "      <td>815</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Holding_Policy_Type  Response\n",
       "2                  3.0      2484\n",
       "0                  1.0      1528\n",
       "1                  2.0       873\n",
       "3                  4.0       815"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Group by Holding_Policy_Duration and get The count of Positive Responses\n",
    "grouped_df = df.groupby('Holding_Policy_Type').agg({'Response': 'sum'}).reset_index().sort_values('Response', ascending=False)\n",
    "grouped_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Accomodation_Type</th>\n",
       "      <th>Response</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Owned</td>\n",
       "      <td>3509</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Rented</td>\n",
       "      <td>2191</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Accomodation_Type  Response\n",
       "0             Owned      3509\n",
       "1            Rented      2191"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Group by Holding_Policy_Duration and get The count of Positive Responses\n",
    "grouped_df = df.groupby('Accomodation_Type').agg({'Response': 'sum'}).reset_index().sort_values('Response', ascending=False)\n",
    "grouped_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Health Indicator</th>\n",
       "      <th>Response</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>X1</td>\n",
       "      <td>1891</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>X2</td>\n",
       "      <td>1528</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>X3</td>\n",
       "      <td>915</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>X4</td>\n",
       "      <td>885</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>X5</td>\n",
       "      <td>251</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>X6</td>\n",
       "      <td>166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>X7</td>\n",
       "      <td>43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>X8</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>X9</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Health Indicator  Response\n",
       "0               X1      1891\n",
       "1               X2      1528\n",
       "2               X3       915\n",
       "3               X4       885\n",
       "4               X5       251\n",
       "5               X6       166\n",
       "6               X7        43\n",
       "7               X8        13\n",
       "8               X9         8"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Group by Holding_Policy_Duration and get The count of Positive Responses\n",
    "grouped_df = df.groupby('Health Indicator').agg({'Response': 'sum'}).reset_index().sort_values('Response', ascending=False)\n",
    "grouped_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualisierung"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def linear_regression(data: pd.DataFrame, x: str, y: str) -> None:\n",
    "    \"\"\"\n",
    "    Perform linear regression on data. Makes a legend cell which contains the function of the regression.\n",
    "    \"\"\"\n",
    "    # Create linear regression object\n",
    "    regr = lm.LinearRegression()\n",
    "\n",
    "    # Train the model using the training sets\n",
    "    regr.fit(data[x].values.reshape(-1, 1), data[y])\n",
    "\n",
    "    # Make predictions using the testing set\n",
    "    y_pred = regr.predict(data[x].values.reshape(-1, 1))\n",
    "\n",
    "    # Plot outputs\n",
    "    plt.scatter(data[x], data[y])\n",
    "    plt.plot(data[x], y_pred, color='blue', linewidth=3)\n",
    "\n",
    "    # Add legend\n",
    "    plt.legend([f\"y = {regr.coef_[0]:.2f}x + {regr.intercept_:.2f}\"])\n",
    "\n",
    "    # add axis labels\n",
    "    plt.xlabel(x)\n",
    "    plt.ylabel(y)\n",
    "\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 53764 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "linear_regression(df, 'Upper_Age', 'Reco_Policy_Premium')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_pairplot(data, hue, palette, columns):\n",
    "    sns.pairplot(data, hue=hue, palette=palette, vars=columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 508716 characters]",
      "text/plain": [
       "<Figure size 827.486x750 with 12 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_pairplot(df, 'Response', 'viridis', ['Upper_Age', 'Lower_Age', 'Reco_Policy_Premium'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dsai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.20"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
