{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"\n",
    "    border: 2px solid #4CAF50; \n",
    "    padding: 15px; \n",
    "    background-color: #f4f4f4; \n",
    "    border-radius: 10px; \n",
    "    align-items: center;\">\n",
    "\n",
    "<h1 style=\"margin: 0; color: #4CAF50;\">Datenverarbeitung in Pandas</h1>\n",
    "<h2 style=\"margin: 5px 0; color: #555;\">DSAI</h2>\n",
    "<h3 style=\"margin: 5px 0; color: #555;\">Jakob Eggl</h3>\n",
    "\n",
    "<div style=\"flex-shrink: 0;\">\n",
    "    <img src=\"https://www.htl-grieskirchen.at/wp/wp-content/uploads/2022/11/logo_bildschirm-1024x503.png\" alt=\"Logo\" style=\"width: 250px; height: auto;\"/>\n",
    "</div>\n",
    "<p1> © 2024/25 Jakob Eggl. Nutzung oder Verbreitung nur mit ausdrücklicher Genehmigung des Autors.</p1>\n",
    "</div>\n",
    "<div style=\"flex: 1;\">\n",
    "</div>   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ersten Manipulationen von Dataframes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Beginnen wir mit dem Hinzufügen von Daten.\n",
    "Wir arbeiten dabei wieder hauptsächlich mit dem *Titanic* Dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wir fügen nun einen fiktionalen Datensatz hinzu\n",
    "path = os.path.join(\"..\", \"..\", \"_data\", \"titanic.csv\")\n",
    "titanic = pd.read_csv(path)\n",
    "\n",
    "new_row = {'pclass': 1, 'survived': 1, 'name': 'Alice Burger', 'sex': 'female', 'age': 35, 'sibsp': 0, 'parch': 0, 'ticket': '12345', 'fare': 50.0, 'cabin': 'C10', 'embarked': 'S', 'home.dest': 'New York'}\n",
    "new_df = pd.DataFrame(new_row, index=[0])\n",
    "titanic = pd.concat([titanic, new_df], ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Für mehrere Daten zum hinzufügen können wir auch eine Liste von Dictionaries verwenden\n",
    "new_row_1 = {'pclass': 1, 'survived': 1, 'name': 'Alice Burger', 'sex': 'female', 'age': 35, 'sibsp': 0, 'parch': 0, 'ticket': '12345', 'fare': 50.0, 'cabin': 'C10', 'embarked': 'S', 'home.dest': 'New York'} # sind zBsp Zwillinge :D\n",
    "new_row_2 = {'pclass': 1, 'survived': 1, 'name': 'Bob Burger', 'sex': 'male', 'age': 35, 'sibsp': 0, 'parch': 0, 'ticket': '12345', 'fare': 50.0, 'cabin': 'C10', 'embarked': 'S', 'home.dest': 'New York'} # sind zBsp Zwillinge :D\n",
    "\n",
    "new_data = [new_row_1, new_row_2]\n",
    "new_df = pd.DataFrame(new_data, index=[0, 1])\n",
    "\n",
    "titanic = pd.concat([titanic, new_df], ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Duplikate können auch entfernt werden"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic.drop_duplicates(inplace=True, ignore_index=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nun betrachten wir die eindeutigen Zeilenwerte in einem Dataframe. Dafür benutzen wir den Befehl `unique()` bzw. `nunique()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Anzahl der eindeutige Werte in allen Spalten anzeigen lassen\n",
    "titanic.nunique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Eindeutige Werte in einer Spalte anzeigen lassen\n",
    "\n",
    "column = 'sibsp' # Andere interessante Fälle: 'boat'\n",
    "titanic[column].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wir können auch die eindeutigen Werte und deren Anzahl anzeigen lassen\n",
    "\n",
    "column = 'sibsp'\n",
    "\n",
    "titanic[column].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In vielen Fällen sind nicht alle Spalten relevant, diese können mit dem Befehl `drop()` entfernt werden. Welche Spalten soll man entfernen, welche Spalten sollte man nicht entfernen?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Drop columns\n",
    "titanic.drop(columns=['boat', 'body'], inplace=True) # erneut inplace Option um Speicherplatz zu sparen"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wie bereits gesehen können aber auch Zeilen \"gedroppt\" werden, dies passiert mittels geschickter indizierung. Für den Fall, dass <em>leere</em> Werte gelöscht werden sollen, gibt es nach wie vor den Befehl `dropna()`. Dieses mal spezifizieren wir dabei eine spezielle Achse. Merksatz in diesem Fall (und auch bei Matrizen): \"Zeilen zuerst, Spalten später\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Spalten, welche leere Werte enthalten, können auch auch mit dem dropna() Befehl entfernt werden, dazu braucht es das Argument axis=1\n",
    "titanic.dropna(axis=1, inplace=True) # Vergleiche mit letzter Woche: dropna() entfernt Zeilen mit leeren Werten"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir laden nun das Dataset erneut um die bisherigen Änderungen zu verwerfen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wollen wir Daten aggregieren, so können wir den `agg()` Befehl verwenden. Wo kann dies nützlich sein? Für welche Datentypen funktionieren welche Aggregierungsmöglichkeiten?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'titanic' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[1], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m agg_df \u001b[38;5;241m=\u001b[39m \u001b[43mtitanic\u001b[49m\u001b[38;5;241m.\u001b[39magg([\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcount\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124msize\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mnunique\u001b[39m\u001b[38;5;124m'\u001b[39m])\n",
      "\u001b[1;31mNameError\u001b[0m: name 'titanic' is not defined"
     ]
    }
   ],
   "source": [
    "agg_df = titanic.agg(['age: count', 'size', 'nunique'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "agg_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Weitere Möglichkeiten sind `sum`, `mean`, `median`, `std`, `var`, `min`, `max`, etc. --> Wir werden später genauer auf Statistiken eingehen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Eine weitere Möglichkeit ist es, Funktionen auf die Daten anzuwenden\n",
    "titanic['increased_age'] = titanic['age'].apply(lambda x: x + 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic.drop(columns=['increased_age'], inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wir können groupby verwenden, um die Daten zu gruppieren\n",
    "grouped_df = titanic.groupby('pclass')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir können nun mit dem gruppierten Dataframe `grouped_df` arbeiten"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "count_values = grouped_df.size()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "count_values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for name, group in grouped_df:    \n",
    "    print(name)\n",
    "    print(group)\n",
    "    print(\"*\"*50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "survival_rate = titanic.groupby('pclass')['survived'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "survival_rate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fehlende Werte/Datenpunkte"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* In vielen Fällen werden Daten nicht perfekt sein\n",
    "* Daten können:\n",
    "    * fehlerhaft/falsch sein bzw. mit Rauschen überlagert\n",
    "    * \"unmöglich\" sein (Zum Beispiel: männlich und schwanger)\n",
    "    * fehlen\n",
    "\n",
    "Es gibt verschiedene Möglichkeiten, wie diese Probleme behandelt werden können."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sanity Checks. Wir laden zuerst ein modifiziertes Dataset\n",
    "\n",
    "modified_titanic = pd.read_csv(path.replace(\"titanic\", \"modified_titanic\"), header=0, sep=',', index_col=False, decimal='.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Thorneycroft, Mr. Percival</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>376564</td>\n",
       "      <td>16.1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lindqvist, Mr. Eino William</td>\n",
       "      <td>male</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O 2. 3101285</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>First Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Head, Mr. Christopher</td>\n",
       "      <td>male</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113038</td>\n",
       "      <td>42.5000</td>\n",
       "      <td>B11</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London / Middlesex</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Lahtinen, Mrs. William (Anna Sylfven)</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>250651</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Minneapolis, MN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)</td>\n",
       "      <td>female</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11753</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>D19</td>\n",
       "      <td>S</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Boston, MA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>White, Mrs. John Stuart (Ella Holmes)</td>\n",
       "      <td>female</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17760</td>\n",
       "      <td>135.6333</td>\n",
       "      <td>C32</td>\n",
       "      <td>C</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, NY / Briarcliff Manor NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lurette, Miss. Elise</td>\n",
       "      <td>female</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17569</td>\n",
       "      <td>146.5208</td>\n",
       "      <td>B80</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Celotti, Mr. Francesco</td>\n",
       "      <td>male</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>343275</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Fischer, Mr. Eberhard Thelander</td>\n",
       "      <td>male</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350036</td>\n",
       "      <td>7.7958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Otter, Mr. Richard</td>\n",
       "      <td>male</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28213</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Middleburg Heights, OH</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            pclass  survived  \\\n",
       "0      Third Class         0   \n",
       "1      Third Class         1   \n",
       "2      First Class         0   \n",
       "3     Second Class         0   \n",
       "4      First Class         1   \n",
       "...            ...       ...   \n",
       "1304   First Class         1   \n",
       "1305   First Class         1   \n",
       "1306   Third Class         0   \n",
       "1307   Third Class         0   \n",
       "1308  Second Class         0   \n",
       "\n",
       "                                                  name     sex   age  sibsp  \\\n",
       "0                           Thorneycroft, Mr. Percival    male   NaN      1   \n",
       "1                          Lindqvist, Mr. Eino William    male  20.0      1   \n",
       "2                                Head, Mr. Christopher    male  42.0      0   \n",
       "3                Lahtinen, Mrs. William (Anna Sylfven)  female  26.0      1   \n",
       "4     Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)  female  45.0      1   \n",
       "...                                                ...     ...   ...    ...   \n",
       "1304             White, Mrs. John Stuart (Ella Holmes)  female  55.0      0   \n",
       "1305                              Lurette, Miss. Elise  female  58.0      0   \n",
       "1306                            Celotti, Mr. Francesco    male  24.0      0   \n",
       "1307                   Fischer, Mr. Eberhard Thelander    male  18.0      0   \n",
       "1308                                Otter, Mr. Richard    male  39.0      0   \n",
       "\n",
       "      parch             ticket      fare cabin embarked boat  body  \\\n",
       "0         0             376564   16.1000   NaN        S  NaN   NaN   \n",
       "1         0  STON/O 2. 3101285    7.9250   NaN        S   15   NaN   \n",
       "2         0             113038   42.5000   B11        S  NaN   NaN   \n",
       "3         1             250651   26.0000   NaN        S  NaN   NaN   \n",
       "4         0              11753   52.5542   D19        S    5   NaN   \n",
       "...     ...                ...       ...   ...      ...  ...   ...   \n",
       "1304      0           PC 17760  135.6333   C32        C    8   NaN   \n",
       "1305      0           PC 17569  146.5208   B80        C  NaN   NaN   \n",
       "1306      0             343275    8.0500   NaN        S  NaN   NaN   \n",
       "1307      0             350036    7.7958   NaN        S  NaN   NaN   \n",
       "1308      0              28213   13.0000   NaN        S  NaN   NaN   \n",
       "\n",
       "                               home.dest  \n",
       "0                                    NaN  \n",
       "1                                    NaN  \n",
       "2                     London / Middlesex  \n",
       "3                        Minneapolis, MN  \n",
       "4                             Boston, MA  \n",
       "...                                  ...  \n",
       "1304  New York, NY / Briarcliff Manor NY  \n",
       "1305                                 NaN  \n",
       "1306                              London  \n",
       "1307                                 NaN  \n",
       "1308              Middleburg Heights, OH  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "modified_titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Welche(n) Fehler vermuten wir hier? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Es gibt negative Werte für das Alter\n"
     ]
    }
   ],
   "source": [
    "# Testen, ob alter \"age\" negativ ist:\n",
    "negative_age = modified_titanic['age'] < 0\n",
    "\n",
    "if negative_age.any():\n",
    "    print(\"Es gibt negative Werte für das Alter\")\n",
    "else:\n",
    "    print(\"Alle Werte in der Spalte Alter sind nicht-negativ (positiv oder 0)\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       False\n",
       "1       False\n",
       "2       False\n",
       "3       False\n",
       "4       False\n",
       "        ...  \n",
       "1304    False\n",
       "1305    False\n",
       "1306    False\n",
       "1307    False\n",
       "1308    False\n",
       "Name: age, Length: 1309, dtype: bool"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Wie sieht das Objekt negative_age aus:\n",
    "negative_age"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Anzahl der negativen Werte für das Alter: 9\n"
     ]
    }
   ],
   "source": [
    "# Wir können auch die genaue Anzahl der negativen Werte anzeigen lassen\n",
    "negative_age_count = negative_age.sum()\n",
    "print(f\"Anzahl der negativen Werte für das Alter: {negative_age_count}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1309 entries, 0 to 1308\n",
      "Data columns (total 14 columns):\n",
      " #   Column     Non-Null Count  Dtype  \n",
      "---  ------     --------------  -----  \n",
      " 0   pclass     1309 non-null   object \n",
      " 1   survived   1309 non-null   int64  \n",
      " 2   name       1309 non-null   object \n",
      " 3   sex        1309 non-null   object \n",
      " 4   age        1037 non-null   float64\n",
      " 5   sibsp      1309 non-null   int64  \n",
      " 6   parch      1309 non-null   int64  \n",
      " 7   ticket     1309 non-null   object \n",
      " 8   fare       1308 non-null   float64\n",
      " 9   cabin      295 non-null    object \n",
      " 10  embarked   1307 non-null   object \n",
      " 11  boat       486 non-null    object \n",
      " 12  body       121 non-null    float64\n",
      " 13  home.dest  745 non-null    object \n",
      "dtypes: float64(3), int64(3), object(8)\n",
      "memory usage: 143.3+ KB\n"
     ]
    }
   ],
   "source": [
    "modified_titanic.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.series.Series'>\n",
      "RangeIndex: 1309 entries, 0 to 1308\n",
      "Series name: age\n",
      "Non-Null Count  Dtype  \n",
      "--------------  -----  \n",
      "1037 non-null   float64\n",
      "dtypes: float64(1)\n",
      "memory usage: 10.4 KB\n"
     ]
    }
   ],
   "source": [
    "modified_titanic['age'].info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nun müssen wir etwas mit diesen falschen/fehlerhaften Daten unternehmen. Wir können:\n",
    "\n",
    "* NaN oder anderen wiedererkennbaren Wert vergeben (0, -1) (Vorteile? Nachteile?)\n",
    "* Mittelwert vergeben (Vorteile? Nachteile?)\n",
    "* Werte der Zeilen geben, die sich darüber/darunter befinden (Vorteile? Nachteile?)\n",
    "* Diese Daten=Zeilen entfernen (Vorteile? Nachteile?)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NaN vergeben für die negativen Werte\n",
    "modified_titanic.loc[negative_age, 'age'] = None # negative_age = modified_titanic['age'] < 0 (von vorher)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Thorneycroft, Mr. Percival</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>376564</td>\n",
       "      <td>16.1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lindqvist, Mr. Eino William</td>\n",
       "      <td>male</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O 2. 3101285</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>First Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Head, Mr. Christopher</td>\n",
       "      <td>male</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113038</td>\n",
       "      <td>42.5000</td>\n",
       "      <td>B11</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London / Middlesex</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Lahtinen, Mrs. William (Anna Sylfven)</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>250651</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Minneapolis, MN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)</td>\n",
       "      <td>female</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11753</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>D19</td>\n",
       "      <td>S</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Boston, MA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>White, Mrs. John Stuart (Ella Holmes)</td>\n",
       "      <td>female</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17760</td>\n",
       "      <td>135.6333</td>\n",
       "      <td>C32</td>\n",
       "      <td>C</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, NY / Briarcliff Manor NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lurette, Miss. Elise</td>\n",
       "      <td>female</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17569</td>\n",
       "      <td>146.5208</td>\n",
       "      <td>B80</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Celotti, Mr. Francesco</td>\n",
       "      <td>male</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>343275</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Fischer, Mr. Eberhard Thelander</td>\n",
       "      <td>male</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350036</td>\n",
       "      <td>7.7958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Otter, Mr. Richard</td>\n",
       "      <td>male</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28213</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Middleburg Heights, OH</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            pclass  survived  \\\n",
       "0      Third Class         0   \n",
       "1      Third Class         1   \n",
       "2      First Class         0   \n",
       "3     Second Class         0   \n",
       "4      First Class         1   \n",
       "...            ...       ...   \n",
       "1304   First Class         1   \n",
       "1305   First Class         1   \n",
       "1306   Third Class         0   \n",
       "1307   Third Class         0   \n",
       "1308  Second Class         0   \n",
       "\n",
       "                                                  name     sex   age  sibsp  \\\n",
       "0                           Thorneycroft, Mr. Percival    male   NaN      1   \n",
       "1                          Lindqvist, Mr. Eino William    male  20.0      1   \n",
       "2                                Head, Mr. Christopher    male  42.0      0   \n",
       "3                Lahtinen, Mrs. William (Anna Sylfven)  female  26.0      1   \n",
       "4     Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)  female  45.0      1   \n",
       "...                                                ...     ...   ...    ...   \n",
       "1304             White, Mrs. John Stuart (Ella Holmes)  female  55.0      0   \n",
       "1305                              Lurette, Miss. Elise  female  58.0      0   \n",
       "1306                            Celotti, Mr. Francesco    male  24.0      0   \n",
       "1307                   Fischer, Mr. Eberhard Thelander    male  18.0      0   \n",
       "1308                                Otter, Mr. Richard    male  39.0      0   \n",
       "\n",
       "      parch             ticket      fare cabin embarked boat  body  \\\n",
       "0         0             376564   16.1000   NaN        S  NaN   NaN   \n",
       "1         0  STON/O 2. 3101285    7.9250   NaN        S   15   NaN   \n",
       "2         0             113038   42.5000   B11        S  NaN   NaN   \n",
       "3         1             250651   26.0000   NaN        S  NaN   NaN   \n",
       "4         0              11753   52.5542   D19        S    5   NaN   \n",
       "...     ...                ...       ...   ...      ...  ...   ...   \n",
       "1304      0           PC 17760  135.6333   C32        C    8   NaN   \n",
       "1305      0           PC 17569  146.5208   B80        C  NaN   NaN   \n",
       "1306      0             343275    8.0500   NaN        S  NaN   NaN   \n",
       "1307      0             350036    7.7958   NaN        S  NaN   NaN   \n",
       "1308      0              28213   13.0000   NaN        S  NaN   NaN   \n",
       "\n",
       "                               home.dest  \n",
       "0                                    NaN  \n",
       "1                                    NaN  \n",
       "2                     London / Middlesex  \n",
       "3                        Minneapolis, MN  \n",
       "4                             Boston, MA  \n",
       "...                                  ...  \n",
       "1304  New York, NY / Briarcliff Manor NY  \n",
       "1305                                 NaN  \n",
       "1306                              London  \n",
       "1307                                 NaN  \n",
       "1308              Middleburg Heights, OH  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "modified_titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.series.Series'>\n",
      "RangeIndex: 1309 entries, 0 to 1308\n",
      "Series name: age\n",
      "Non-Null Count  Dtype  \n",
      "--------------  -----  \n",
      "1028 non-null   float64\n",
      "dtypes: float64(1)\n",
      "memory usage: 10.4 KB\n"
     ]
    }
   ],
   "source": [
    "modified_titanic['age'].info() # nun sind in der Spalte \"age\" weniger Werte vorhanden (mehr Werte sind NaN)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ersetzen von negativen Alter mit Mittelwert\n",
    "mean_age = modified_titanic['age'].mean()\n",
    "modified_titanic.loc[negative_age, 'age'] = mean_age"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "29.885619357976655\n"
     ]
    }
   ],
   "source": [
    "print(mean_age)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Thorneycroft, Mr. Percival</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>376564</td>\n",
       "      <td>16.1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lindqvist, Mr. Eino William</td>\n",
       "      <td>male</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O 2. 3101285</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>First Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Head, Mr. Christopher</td>\n",
       "      <td>male</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113038</td>\n",
       "      <td>42.5000</td>\n",
       "      <td>B11</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London / Middlesex</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Lahtinen, Mrs. William (Anna Sylfven)</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>250651</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Minneapolis, MN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)</td>\n",
       "      <td>female</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11753</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>D19</td>\n",
       "      <td>S</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Boston, MA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>White, Mrs. John Stuart (Ella Holmes)</td>\n",
       "      <td>female</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17760</td>\n",
       "      <td>135.6333</td>\n",
       "      <td>C32</td>\n",
       "      <td>C</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, NY / Briarcliff Manor NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lurette, Miss. Elise</td>\n",
       "      <td>female</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17569</td>\n",
       "      <td>146.5208</td>\n",
       "      <td>B80</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Celotti, Mr. Francesco</td>\n",
       "      <td>male</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>343275</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Fischer, Mr. Eberhard Thelander</td>\n",
       "      <td>male</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350036</td>\n",
       "      <td>7.7958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Otter, Mr. Richard</td>\n",
       "      <td>male</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28213</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Middleburg Heights, OH</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            pclass  survived  \\\n",
       "0      Third Class         0   \n",
       "1      Third Class         1   \n",
       "2      First Class         0   \n",
       "3     Second Class         0   \n",
       "4      First Class         1   \n",
       "...            ...       ...   \n",
       "1304   First Class         1   \n",
       "1305   First Class         1   \n",
       "1306   Third Class         0   \n",
       "1307   Third Class         0   \n",
       "1308  Second Class         0   \n",
       "\n",
       "                                                  name     sex   age  sibsp  \\\n",
       "0                           Thorneycroft, Mr. Percival    male   NaN      1   \n",
       "1                          Lindqvist, Mr. Eino William    male  20.0      1   \n",
       "2                                Head, Mr. Christopher    male  42.0      0   \n",
       "3                Lahtinen, Mrs. William (Anna Sylfven)  female  26.0      1   \n",
       "4     Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)  female  45.0      1   \n",
       "...                                                ...     ...   ...    ...   \n",
       "1304             White, Mrs. John Stuart (Ella Holmes)  female  55.0      0   \n",
       "1305                              Lurette, Miss. Elise  female  58.0      0   \n",
       "1306                            Celotti, Mr. Francesco    male  24.0      0   \n",
       "1307                   Fischer, Mr. Eberhard Thelander    male  18.0      0   \n",
       "1308                                Otter, Mr. Richard    male  39.0      0   \n",
       "\n",
       "      parch             ticket      fare cabin embarked boat  body  \\\n",
       "0         0             376564   16.1000   NaN        S  NaN   NaN   \n",
       "1         0  STON/O 2. 3101285    7.9250   NaN        S   15   NaN   \n",
       "2         0             113038   42.5000   B11        S  NaN   NaN   \n",
       "3         1             250651   26.0000   NaN        S  NaN   NaN   \n",
       "4         0              11753   52.5542   D19        S    5   NaN   \n",
       "...     ...                ...       ...   ...      ...  ...   ...   \n",
       "1304      0           PC 17760  135.6333   C32        C    8   NaN   \n",
       "1305      0           PC 17569  146.5208   B80        C  NaN   NaN   \n",
       "1306      0             343275    8.0500   NaN        S  NaN   NaN   \n",
       "1307      0             350036    7.7958   NaN        S  NaN   NaN   \n",
       "1308      0              28213   13.0000   NaN        S  NaN   NaN   \n",
       "\n",
       "                               home.dest  \n",
       "0                                    NaN  \n",
       "1                                    NaN  \n",
       "2                     London / Middlesex  \n",
       "3                        Minneapolis, MN  \n",
       "4                             Boston, MA  \n",
       "...                                  ...  \n",
       "1304  New York, NY / Briarcliff Manor NY  \n",
       "1305                                 NaN  \n",
       "1306                              London  \n",
       "1307                                 NaN  \n",
       "1308              Middleburg Heights, OH  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "modified_titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.series.Series'>\n",
      "RangeIndex: 1309 entries, 0 to 1308\n",
      "Series name: age\n",
      "Non-Null Count  Dtype  \n",
      "--------------  -----  \n",
      "1037 non-null   float64\n",
      "dtypes: float64(1)\n",
      "memory usage: 10.4 KB\n"
     ]
    }
   ],
   "source": [
    "modified_titanic['age'].info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "modified_titanic['age'] = modified_titanic['age'].bfill() # Ersetzt alle fehldenden Werte der Spalte age mit dem nächsten vorhandenen Wert"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Thorneycroft, Mr. Percival</td>\n",
       "      <td>male</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>376564</td>\n",
       "      <td>16.1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lindqvist, Mr. Eino William</td>\n",
       "      <td>male</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O 2. 3101285</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>First Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Head, Mr. Christopher</td>\n",
       "      <td>male</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113038</td>\n",
       "      <td>42.5000</td>\n",
       "      <td>B11</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London / Middlesex</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Lahtinen, Mrs. William (Anna Sylfven)</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>250651</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Minneapolis, MN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)</td>\n",
       "      <td>female</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11753</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>D19</td>\n",
       "      <td>S</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Boston, MA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>White, Mrs. John Stuart (Ella Holmes)</td>\n",
       "      <td>female</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17760</td>\n",
       "      <td>135.6333</td>\n",
       "      <td>C32</td>\n",
       "      <td>C</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, NY / Briarcliff Manor NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lurette, Miss. Elise</td>\n",
       "      <td>female</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17569</td>\n",
       "      <td>146.5208</td>\n",
       "      <td>B80</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Celotti, Mr. Francesco</td>\n",
       "      <td>male</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>343275</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Fischer, Mr. Eberhard Thelander</td>\n",
       "      <td>male</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350036</td>\n",
       "      <td>7.7958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Otter, Mr. Richard</td>\n",
       "      <td>male</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28213</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Middleburg Heights, OH</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            pclass  survived  \\\n",
       "0      Third Class         0   \n",
       "1      Third Class         1   \n",
       "2      First Class         0   \n",
       "3     Second Class         0   \n",
       "4      First Class         1   \n",
       "...            ...       ...   \n",
       "1304   First Class         1   \n",
       "1305   First Class         1   \n",
       "1306   Third Class         0   \n",
       "1307   Third Class         0   \n",
       "1308  Second Class         0   \n",
       "\n",
       "                                                  name     sex   age  sibsp  \\\n",
       "0                           Thorneycroft, Mr. Percival    male  20.0      1   \n",
       "1                          Lindqvist, Mr. Eino William    male  20.0      1   \n",
       "2                                Head, Mr. Christopher    male  42.0      0   \n",
       "3                Lahtinen, Mrs. William (Anna Sylfven)  female  26.0      1   \n",
       "4     Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)  female  45.0      1   \n",
       "...                                                ...     ...   ...    ...   \n",
       "1304             White, Mrs. John Stuart (Ella Holmes)  female  55.0      0   \n",
       "1305                              Lurette, Miss. Elise  female  58.0      0   \n",
       "1306                            Celotti, Mr. Francesco    male  24.0      0   \n",
       "1307                   Fischer, Mr. Eberhard Thelander    male  18.0      0   \n",
       "1308                                Otter, Mr. Richard    male  39.0      0   \n",
       "\n",
       "      parch             ticket      fare cabin embarked boat  body  \\\n",
       "0         0             376564   16.1000   NaN        S  NaN   NaN   \n",
       "1         0  STON/O 2. 3101285    7.9250   NaN        S   15   NaN   \n",
       "2         0             113038   42.5000   B11        S  NaN   NaN   \n",
       "3         1             250651   26.0000   NaN        S  NaN   NaN   \n",
       "4         0              11753   52.5542   D19        S    5   NaN   \n",
       "...     ...                ...       ...   ...      ...  ...   ...   \n",
       "1304      0           PC 17760  135.6333   C32        C    8   NaN   \n",
       "1305      0           PC 17569  146.5208   B80        C  NaN   NaN   \n",
       "1306      0             343275    8.0500   NaN        S  NaN   NaN   \n",
       "1307      0             350036    7.7958   NaN        S  NaN   NaN   \n",
       "1308      0              28213   13.0000   NaN        S  NaN   NaN   \n",
       "\n",
       "                               home.dest  \n",
       "0                                    NaN  \n",
       "1                                    NaN  \n",
       "2                     London / Middlesex  \n",
       "3                        Minneapolis, MN  \n",
       "4                             Boston, MA  \n",
       "...                                  ...  \n",
       "1304  New York, NY / Briarcliff Manor NY  \n",
       "1305                                 NaN  \n",
       "1306                              London  \n",
       "1307                                 NaN  \n",
       "1308              Middleburg Heights, OH  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "modified_titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.series.Series'>\n",
      "RangeIndex: 1309 entries, 0 to 1308\n",
      "Series name: age\n",
      "Non-Null Count  Dtype  \n",
      "--------------  -----  \n",
      "1309 non-null   float64\n",
      "dtypes: float64(1)\n",
      "memory usage: 10.4 KB\n"
     ]
    }
   ],
   "source": [
    "modified_titanic['age'].info() # nun gibt es keine NaN Werte mehr. WICHTIG: Es wurde kein negatives Alter hier ersetzt sondern nur NaN Werte (bisherige negative Werte sind aber aktuell auf mean_age gesetzt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Erneutes Laden des manipulierten Datensets\n",
    "modified_titanic = pd.read_csv(path.replace(\"titanic\", \"modified_titanic\"), header=0, sep=',', index_col=False, decimal='.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Eine weitere sehr praktische Variante ist es, die Methode fillna() zu verwenden\n",
    "modified_titanic['age'] = modified_titanic['age'].fillna(mean_age)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Thorneycroft, Mr. Percival</td>\n",
       "      <td>male</td>\n",
       "      <td>29.885619</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>376564</td>\n",
       "      <td>16.1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lindqvist, Mr. Eino William</td>\n",
       "      <td>male</td>\n",
       "      <td>20.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O 2. 3101285</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>First Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Head, Mr. Christopher</td>\n",
       "      <td>male</td>\n",
       "      <td>42.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113038</td>\n",
       "      <td>42.5000</td>\n",
       "      <td>B11</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London / Middlesex</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Lahtinen, Mrs. William (Anna Sylfven)</td>\n",
       "      <td>female</td>\n",
       "      <td>26.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>250651</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Minneapolis, MN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)</td>\n",
       "      <td>female</td>\n",
       "      <td>45.000000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11753</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>D19</td>\n",
       "      <td>S</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Boston, MA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>White, Mrs. John Stuart (Ella Holmes)</td>\n",
       "      <td>female</td>\n",
       "      <td>55.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17760</td>\n",
       "      <td>135.6333</td>\n",
       "      <td>C32</td>\n",
       "      <td>C</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, NY / Briarcliff Manor NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lurette, Miss. Elise</td>\n",
       "      <td>female</td>\n",
       "      <td>58.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17569</td>\n",
       "      <td>146.5208</td>\n",
       "      <td>B80</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Celotti, Mr. Francesco</td>\n",
       "      <td>male</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>343275</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Fischer, Mr. Eberhard Thelander</td>\n",
       "      <td>male</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350036</td>\n",
       "      <td>7.7958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Otter, Mr. Richard</td>\n",
       "      <td>male</td>\n",
       "      <td>39.000000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28213</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Middleburg Heights, OH</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            pclass  survived  \\\n",
       "0      Third Class         0   \n",
       "1      Third Class         1   \n",
       "2      First Class         0   \n",
       "3     Second Class         0   \n",
       "4      First Class         1   \n",
       "...            ...       ...   \n",
       "1304   First Class         1   \n",
       "1305   First Class         1   \n",
       "1306   Third Class         0   \n",
       "1307   Third Class         0   \n",
       "1308  Second Class         0   \n",
       "\n",
       "                                                  name     sex        age  \\\n",
       "0                           Thorneycroft, Mr. Percival    male  29.885619   \n",
       "1                          Lindqvist, Mr. Eino William    male  20.000000   \n",
       "2                                Head, Mr. Christopher    male  42.000000   \n",
       "3                Lahtinen, Mrs. William (Anna Sylfven)  female  26.000000   \n",
       "4     Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)  female  45.000000   \n",
       "...                                                ...     ...        ...   \n",
       "1304             White, Mrs. John Stuart (Ella Holmes)  female  55.000000   \n",
       "1305                              Lurette, Miss. Elise  female  58.000000   \n",
       "1306                            Celotti, Mr. Francesco    male  24.000000   \n",
       "1307                   Fischer, Mr. Eberhard Thelander    male  18.000000   \n",
       "1308                                Otter, Mr. Richard    male  39.000000   \n",
       "\n",
       "      sibsp  parch             ticket      fare cabin embarked boat  body  \\\n",
       "0         1      0             376564   16.1000   NaN        S  NaN   NaN   \n",
       "1         1      0  STON/O 2. 3101285    7.9250   NaN        S   15   NaN   \n",
       "2         0      0             113038   42.5000   B11        S  NaN   NaN   \n",
       "3         1      1             250651   26.0000   NaN        S  NaN   NaN   \n",
       "4         1      0              11753   52.5542   D19        S    5   NaN   \n",
       "...     ...    ...                ...       ...   ...      ...  ...   ...   \n",
       "1304      0      0           PC 17760  135.6333   C32        C    8   NaN   \n",
       "1305      0      0           PC 17569  146.5208   B80        C  NaN   NaN   \n",
       "1306      0      0             343275    8.0500   NaN        S  NaN   NaN   \n",
       "1307      0      0             350036    7.7958   NaN        S  NaN   NaN   \n",
       "1308      0      0              28213   13.0000   NaN        S  NaN   NaN   \n",
       "\n",
       "                               home.dest  \n",
       "0                                    NaN  \n",
       "1                                    NaN  \n",
       "2                     London / Middlesex  \n",
       "3                        Minneapolis, MN  \n",
       "4                             Boston, MA  \n",
       "...                                  ...  \n",
       "1304  New York, NY / Briarcliff Manor NY  \n",
       "1305                                 NaN  \n",
       "1306                              London  \n",
       "1307                                 NaN  \n",
       "1308              Middleburg Heights, OH  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "modified_titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.series.Series'>\n",
      "RangeIndex: 1309 entries, 0 to 1308\n",
      "Series name: age\n",
      "Non-Null Count  Dtype  \n",
      "--------------  -----  \n",
      "1309 non-null   float64\n",
      "dtypes: float64(1)\n",
      "memory usage: 10.4 KB\n"
     ]
    }
   ],
   "source": [
    "modified_titanic['age'].info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Erneutes Laden des manipulierten Datensets\n",
    "modified_titanic = pd.read_csv(path.replace(\"titanic\", \"modified_titanic\"), header=0, sep=',', index_col=False, decimal='.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Zeilen entfernen, die negative Werte für das Alter enthalten, NaN Werte bleiben erhalten\n",
    "modified_titanic = modified_titanic[modified_titanic['age'].ge(0) | modified_titanic['age'].isnull()] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Thorneycroft, Mr. Percival</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>376564</td>\n",
       "      <td>16.1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lindqvist, Mr. Eino William</td>\n",
       "      <td>male</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O 2. 3101285</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>First Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Head, Mr. Christopher</td>\n",
       "      <td>male</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113038</td>\n",
       "      <td>42.5000</td>\n",
       "      <td>B11</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London / Middlesex</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Lahtinen, Mrs. William (Anna Sylfven)</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>250651</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Minneapolis, MN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)</td>\n",
       "      <td>female</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11753</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>D19</td>\n",
       "      <td>S</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Boston, MA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>White, Mrs. John Stuart (Ella Holmes)</td>\n",
       "      <td>female</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17760</td>\n",
       "      <td>135.6333</td>\n",
       "      <td>C32</td>\n",
       "      <td>C</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, NY / Briarcliff Manor NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lurette, Miss. Elise</td>\n",
       "      <td>female</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17569</td>\n",
       "      <td>146.5208</td>\n",
       "      <td>B80</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Celotti, Mr. Francesco</td>\n",
       "      <td>male</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>343275</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Fischer, Mr. Eberhard Thelander</td>\n",
       "      <td>male</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350036</td>\n",
       "      <td>7.7958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Otter, Mr. Richard</td>\n",
       "      <td>male</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28213</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Middleburg Heights, OH</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1300 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            pclass  survived  \\\n",
       "0      Third Class         0   \n",
       "1      Third Class         1   \n",
       "2      First Class         0   \n",
       "3     Second Class         0   \n",
       "4      First Class         1   \n",
       "...            ...       ...   \n",
       "1304   First Class         1   \n",
       "1305   First Class         1   \n",
       "1306   Third Class         0   \n",
       "1307   Third Class         0   \n",
       "1308  Second Class         0   \n",
       "\n",
       "                                                  name     sex   age  sibsp  \\\n",
       "0                           Thorneycroft, Mr. Percival    male   NaN      1   \n",
       "1                          Lindqvist, Mr. Eino William    male  20.0      1   \n",
       "2                                Head, Mr. Christopher    male  42.0      0   \n",
       "3                Lahtinen, Mrs. William (Anna Sylfven)  female  26.0      1   \n",
       "4     Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)  female  45.0      1   \n",
       "...                                                ...     ...   ...    ...   \n",
       "1304             White, Mrs. John Stuart (Ella Holmes)  female  55.0      0   \n",
       "1305                              Lurette, Miss. Elise  female  58.0      0   \n",
       "1306                            Celotti, Mr. Francesco    male  24.0      0   \n",
       "1307                   Fischer, Mr. Eberhard Thelander    male  18.0      0   \n",
       "1308                                Otter, Mr. Richard    male  39.0      0   \n",
       "\n",
       "      parch             ticket      fare cabin embarked boat  body  \\\n",
       "0         0             376564   16.1000   NaN        S  NaN   NaN   \n",
       "1         0  STON/O 2. 3101285    7.9250   NaN        S   15   NaN   \n",
       "2         0             113038   42.5000   B11        S  NaN   NaN   \n",
       "3         1             250651   26.0000   NaN        S  NaN   NaN   \n",
       "4         0              11753   52.5542   D19        S    5   NaN   \n",
       "...     ...                ...       ...   ...      ...  ...   ...   \n",
       "1304      0           PC 17760  135.6333   C32        C    8   NaN   \n",
       "1305      0           PC 17569  146.5208   B80        C  NaN   NaN   \n",
       "1306      0             343275    8.0500   NaN        S  NaN   NaN   \n",
       "1307      0             350036    7.7958   NaN        S  NaN   NaN   \n",
       "1308      0              28213   13.0000   NaN        S  NaN   NaN   \n",
       "\n",
       "                               home.dest  \n",
       "0                                    NaN  \n",
       "1                                    NaN  \n",
       "2                     London / Middlesex  \n",
       "3                        Minneapolis, MN  \n",
       "4                             Boston, MA  \n",
       "...                                  ...  \n",
       "1304  New York, NY / Briarcliff Manor NY  \n",
       "1305                                 NaN  \n",
       "1306                              London  \n",
       "1307                                 NaN  \n",
       "1308              Middleburg Heights, OH  \n",
       "\n",
       "[1300 rows x 14 columns]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "modified_titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 1300 entries, 0 to 1308\n",
      "Data columns (total 14 columns):\n",
      " #   Column     Non-Null Count  Dtype  \n",
      "---  ------     --------------  -----  \n",
      " 0   pclass     1300 non-null   object \n",
      " 1   survived   1300 non-null   int64  \n",
      " 2   name       1300 non-null   object \n",
      " 3   sex        1300 non-null   object \n",
      " 4   age        1028 non-null   float64\n",
      " 5   sibsp      1300 non-null   int64  \n",
      " 6   parch      1300 non-null   int64  \n",
      " 7   ticket     1300 non-null   object \n",
      " 8   fare       1299 non-null   float64\n",
      " 9   cabin      293 non-null    object \n",
      " 10  embarked   1298 non-null   object \n",
      " 11  boat       484 non-null    object \n",
      " 12  body       120 non-null    float64\n",
      " 13  home.dest  740 non-null    object \n",
      "dtypes: float64(3), int64(3), object(8)\n",
      "memory usage: 152.3+ KB\n"
     ]
    }
   ],
   "source": [
    "modified_titanic.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# andere Möglichkeit mit der Query Methode\n",
    "modified_titanic = pd.read_csv(path.replace(\"titanic\", \"modified_titanic\"), header=0, sep=',', index_col=False, decimal='.')\n",
    "\n",
    "modified_titanic = modified_titanic.query('age >= 0 or age.isnull()') # Achtung: lässt NaN Werte bestehen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Thorneycroft, Mr. Percival</td>\n",
       "      <td>male</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>376564</td>\n",
       "      <td>16.1000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lindqvist, Mr. Eino William</td>\n",
       "      <td>male</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O 2. 3101285</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>15</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>First Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Head, Mr. Christopher</td>\n",
       "      <td>male</td>\n",
       "      <td>42.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113038</td>\n",
       "      <td>42.5000</td>\n",
       "      <td>B11</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London / Middlesex</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Lahtinen, Mrs. William (Anna Sylfven)</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>250651</td>\n",
       "      <td>26.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Minneapolis, MN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)</td>\n",
       "      <td>female</td>\n",
       "      <td>45.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>11753</td>\n",
       "      <td>52.5542</td>\n",
       "      <td>D19</td>\n",
       "      <td>S</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Boston, MA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>White, Mrs. John Stuart (Ella Holmes)</td>\n",
       "      <td>female</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17760</td>\n",
       "      <td>135.6333</td>\n",
       "      <td>C32</td>\n",
       "      <td>C</td>\n",
       "      <td>8</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, NY / Briarcliff Manor NY</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>First Class</td>\n",
       "      <td>1</td>\n",
       "      <td>Lurette, Miss. Elise</td>\n",
       "      <td>female</td>\n",
       "      <td>58.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17569</td>\n",
       "      <td>146.5208</td>\n",
       "      <td>B80</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Celotti, Mr. Francesco</td>\n",
       "      <td>male</td>\n",
       "      <td>24.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>343275</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>London</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>Third Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Fischer, Mr. Eberhard Thelander</td>\n",
       "      <td>male</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>350036</td>\n",
       "      <td>7.7958</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>Second Class</td>\n",
       "      <td>0</td>\n",
       "      <td>Otter, Mr. Richard</td>\n",
       "      <td>male</td>\n",
       "      <td>39.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28213</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Middleburg Heights, OH</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1300 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            pclass  survived  \\\n",
       "0      Third Class         0   \n",
       "1      Third Class         1   \n",
       "2      First Class         0   \n",
       "3     Second Class         0   \n",
       "4      First Class         1   \n",
       "...            ...       ...   \n",
       "1304   First Class         1   \n",
       "1305   First Class         1   \n",
       "1306   Third Class         0   \n",
       "1307   Third Class         0   \n",
       "1308  Second Class         0   \n",
       "\n",
       "                                                  name     sex   age  sibsp  \\\n",
       "0                           Thorneycroft, Mr. Percival    male   NaN      1   \n",
       "1                          Lindqvist, Mr. Eino William    male  20.0      1   \n",
       "2                                Head, Mr. Christopher    male  42.0      0   \n",
       "3                Lahtinen, Mrs. William (Anna Sylfven)  female  26.0      1   \n",
       "4     Kimball, Mrs. Edwin Nelson Jr (Gertrude Parsons)  female  45.0      1   \n",
       "...                                                ...     ...   ...    ...   \n",
       "1304             White, Mrs. John Stuart (Ella Holmes)  female  55.0      0   \n",
       "1305                              Lurette, Miss. Elise  female  58.0      0   \n",
       "1306                            Celotti, Mr. Francesco    male  24.0      0   \n",
       "1307                   Fischer, Mr. Eberhard Thelander    male  18.0      0   \n",
       "1308                                Otter, Mr. Richard    male  39.0      0   \n",
       "\n",
       "      parch             ticket      fare cabin embarked boat  body  \\\n",
       "0         0             376564   16.1000   NaN        S  NaN   NaN   \n",
       "1         0  STON/O 2. 3101285    7.9250   NaN        S   15   NaN   \n",
       "2         0             113038   42.5000   B11        S  NaN   NaN   \n",
       "3         1             250651   26.0000   NaN        S  NaN   NaN   \n",
       "4         0              11753   52.5542   D19        S    5   NaN   \n",
       "...     ...                ...       ...   ...      ...  ...   ...   \n",
       "1304      0           PC 17760  135.6333   C32        C    8   NaN   \n",
       "1305      0           PC 17569  146.5208   B80        C  NaN   NaN   \n",
       "1306      0             343275    8.0500   NaN        S  NaN   NaN   \n",
       "1307      0             350036    7.7958   NaN        S  NaN   NaN   \n",
       "1308      0              28213   13.0000   NaN        S  NaN   NaN   \n",
       "\n",
       "                               home.dest  \n",
       "0                                    NaN  \n",
       "1                                    NaN  \n",
       "2                     London / Middlesex  \n",
       "3                        Minneapolis, MN  \n",
       "4                             Boston, MA  \n",
       "...                                  ...  \n",
       "1304  New York, NY / Briarcliff Manor NY  \n",
       "1305                                 NaN  \n",
       "1306                              London  \n",
       "1307                                 NaN  \n",
       "1308              Middleburg Heights, OH  \n",
       "\n",
       "[1300 rows x 14 columns]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "modified_titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 1300 entries, 0 to 1308\n",
      "Data columns (total 14 columns):\n",
      " #   Column     Non-Null Count  Dtype  \n",
      "---  ------     --------------  -----  \n",
      " 0   pclass     1300 non-null   object \n",
      " 1   survived   1300 non-null   int64  \n",
      " 2   name       1300 non-null   object \n",
      " 3   sex        1300 non-null   object \n",
      " 4   age        1028 non-null   float64\n",
      " 5   sibsp      1300 non-null   int64  \n",
      " 6   parch      1300 non-null   int64  \n",
      " 7   ticket     1300 non-null   object \n",
      " 8   fare       1299 non-null   float64\n",
      " 9   cabin      293 non-null    object \n",
      " 10  embarked   1298 non-null   object \n",
      " 11  boat       484 non-null    object \n",
      " 12  body       120 non-null    float64\n",
      " 13  home.dest  740 non-null    object \n",
      "dtypes: float64(3), int64(3), object(8)\n",
      "memory usage: 152.3+ KB\n"
     ]
    }
   ],
   "source": [
    "modified_titanic.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalisierung"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In vielen Fällen sind die gegebenen Daten nicht normalisiert. Dies kann erhebliche Nachteile mit sich bringen. Wann kann es nachteilig sein, Daten zu normalisieren? (Ausreißer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Die folgende Abbildung zeigt, warum Normalisierung so wichtig ist. "
   ]
  },
  {
   "attachments": {
    "Untitled-2.png": {
     "image/png": "[encoded data, 8936 characters]"
    },
    "Untitled.png": {
     "image/png": "[encoded data, 7740 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Untitled.png](attachment:Untitled.png)![Untitled-2.png](attachment:Untitled-2.png)\n",
    "\n",
    "(von https://python-graph-gallery.com/94-use-normalization-on-seaborn-heatmap/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Es gibt verschiedene Möglichkeiten, Daten zu normalisieren. Was sind die Vorteile und Nachteile? Wann eignen sich besondere Normalisierungsmethoden besonders gut bzw. schlecht? Was ist der Wertebereich von den neuen Daten?\n",
    "* Min-Max Normalisierung $x' = \\frac{x-\\min x}{\\max x-\\min x}$\n",
    "* z-Wert Normalisierung $x'=\\frac{x-\\mu}{\\sigma}$\n",
    "* Relative Skalierung: $x' = \\frac{x}{\\Sigma x}$ (Aufpassen auf negative Werte (Division durch 0?))\n",
    "* Logarithmische Transformation: $x' = \\log(x)$ bzw. $x'=\\log(x+1)$\n",
    "* $\\ell_2$-Normalisierung (Längennormalisierung) $x'=\\frac{x}{\\sqrt{\\Sigma_i x_i^2}}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Es kann aber das Normalisieren oft auch einfach nur eine Konvertierung der Werte in eine andere Einheit bedeuten."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Durch das Normalisieren geht oftmals der Bezug zu den Werten/Daten verloren, auf dies muss bei Visualisierungen oder späteren Datenverarbeitung, insbesondere in Machine Learning Modellen berücksichtigt werden."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.') # Achtung jetzt wieder normales Titanic Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# min-max normalization\n",
    "def min_max_normalization(df, columns):\n",
    "    for column in columns:\n",
    "        df[column] = (df[column] - df[column].min()) / (df[column].max() - df[column].min())\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns = ['age', 'fare']\n",
    "titanic = min_max_normalization(titanic, columns) # Die übergebenen Spalten sind nun im Bereich [-1, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>0.361169</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>0.412503</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>0.009395</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>0.295806</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>0.022964</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>0.295806</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>0.373695</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>0.295806</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>0.311064</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>0.295806</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>female</td>\n",
       "      <td>0.179540</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>0.028213</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>0.028213</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>male</td>\n",
       "      <td>0.329854</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>0.014102</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>male</td>\n",
       "      <td>0.336117</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>0.014102</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>male</td>\n",
       "      <td>0.361169</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>0.015371</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton   \n",
       "1          1         1                   Allison, Master. Hudson Trevor   \n",
       "2          1         0                     Allison, Miss. Helen Loraine   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)   \n",
       "...      ...       ...                                              ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni   \n",
       "1305       3         0                            Zabour, Miss. Thamine   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer   \n",
       "1307       3         0                              Zakarian, Mr. Ortin   \n",
       "1308       3         0                               Zimmerman, Mr. Leo   \n",
       "\n",
       "         sex       age  sibsp  parch  ticket      fare    cabin embarked boat  \\\n",
       "0     female  0.361169      0      0   24160  0.412503       B5        S    2   \n",
       "1       male  0.009395      1      2  113781  0.295806  C22 C26        S   11   \n",
       "2     female  0.022964      1      2  113781  0.295806  C22 C26        S  NaN   \n",
       "3       male  0.373695      1      2  113781  0.295806  C22 C26        S  NaN   \n",
       "4     female  0.311064      1      2  113781  0.295806  C22 C26        S  NaN   \n",
       "...      ...       ...    ...    ...     ...       ...      ...      ...  ...   \n",
       "1304  female  0.179540      1      0    2665  0.028213      NaN        C  NaN   \n",
       "1305  female       NaN      1      0    2665  0.028213      NaN        C  NaN   \n",
       "1306    male  0.329854      0      0    2656  0.014102      NaN        C  NaN   \n",
       "1307    male  0.336117      0      0    2670  0.014102      NaN        C  NaN   \n",
       "1308    male  0.361169      0      0  315082  0.015371      NaN        S  NaN   \n",
       "\n",
       "       body                        home.dest  \n",
       "0       NaN                     St Louis, MO  \n",
       "1       NaN  Montreal, PQ / Chesterville, ON  \n",
       "2       NaN  Montreal, PQ / Chesterville, ON  \n",
       "3     135.0  Montreal, PQ / Chesterville, ON  \n",
       "4       NaN  Montreal, PQ / Chesterville, ON  \n",
       "...     ...                              ...  \n",
       "1304  328.0                              NaN  \n",
       "1305    NaN                              NaN  \n",
       "1306  304.0                              NaN  \n",
       "1307    NaN                              NaN  \n",
       "1308    NaN                              NaN  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ebenso möglich ist die Normalisierung mittels `sklearn`. Es kann installiert werden mit `conda install scikit-learn`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "titanic[['age', 'fare']] = MinMaxScaler().fit_transform(titanic[['age', 'fare']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>0.361169</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>0.412503</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>0.009395</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>0.295806</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>0.022964</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>0.295806</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>0.373695</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>0.295806</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>0.311064</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>0.295806</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>female</td>\n",
       "      <td>0.179540</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>0.028213</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>0.028213</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>male</td>\n",
       "      <td>0.329854</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>0.014102</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>male</td>\n",
       "      <td>0.336117</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>0.014102</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>male</td>\n",
       "      <td>0.361169</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>0.015371</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton   \n",
       "1          1         1                   Allison, Master. Hudson Trevor   \n",
       "2          1         0                     Allison, Miss. Helen Loraine   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)   \n",
       "...      ...       ...                                              ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni   \n",
       "1305       3         0                            Zabour, Miss. Thamine   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer   \n",
       "1307       3         0                              Zakarian, Mr. Ortin   \n",
       "1308       3         0                               Zimmerman, Mr. Leo   \n",
       "\n",
       "         sex       age  sibsp  parch  ticket      fare    cabin embarked boat  \\\n",
       "0     female  0.361169      0      0   24160  0.412503       B5        S    2   \n",
       "1       male  0.009395      1      2  113781  0.295806  C22 C26        S   11   \n",
       "2     female  0.022964      1      2  113781  0.295806  C22 C26        S  NaN   \n",
       "3       male  0.373695      1      2  113781  0.295806  C22 C26        S  NaN   \n",
       "4     female  0.311064      1      2  113781  0.295806  C22 C26        S  NaN   \n",
       "...      ...       ...    ...    ...     ...       ...      ...      ...  ...   \n",
       "1304  female  0.179540      1      0    2665  0.028213      NaN        C  NaN   \n",
       "1305  female       NaN      1      0    2665  0.028213      NaN        C  NaN   \n",
       "1306    male  0.329854      0      0    2656  0.014102      NaN        C  NaN   \n",
       "1307    male  0.336117      0      0    2670  0.014102      NaN        C  NaN   \n",
       "1308    male  0.361169      0      0  315082  0.015371      NaN        S  NaN   \n",
       "\n",
       "       body                        home.dest  \n",
       "0       NaN                     St Louis, MO  \n",
       "1       NaN  Montreal, PQ / Chesterville, ON  \n",
       "2       NaN  Montreal, PQ / Chesterville, ON  \n",
       "3     135.0  Montreal, PQ / Chesterville, ON  \n",
       "4       NaN  Montreal, PQ / Chesterville, ON  \n",
       "...     ...                              ...  \n",
       "1304  328.0                              NaN  \n",
       "1305    NaN                              NaN  \n",
       "1306  304.0                              NaN  \n",
       "1307    NaN                              NaN  \n",
       "1308    NaN                              NaN  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "# genauso bei der z-Score Normalisierung\n",
    "\n",
    "def z_score_normalization(df, columns):\n",
    "    for column in columns:\n",
    "        df[column] = (df[column] - df[column].mean()) / df[column].std()\n",
    "    return df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns = ['age', 'fare']\n",
    "titanic = z_score_normalization(titanic, columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>-0.061133</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>3.439849</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>-2.009535</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>2.284729</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>-1.934376</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>2.284729</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>0.008247</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>2.284729</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>-0.338650</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>2.284729</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>female</td>\n",
       "      <td>-1.067134</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>-0.364022</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>-0.364022</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>male</td>\n",
       "      <td>-0.234581</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>-0.503693</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>male</td>\n",
       "      <td>-0.199891</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>-0.503693</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>male</td>\n",
       "      <td>-0.061133</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>-0.491135</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton   \n",
       "1          1         1                   Allison, Master. Hudson Trevor   \n",
       "2          1         0                     Allison, Miss. Helen Loraine   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)   \n",
       "...      ...       ...                                              ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni   \n",
       "1305       3         0                            Zabour, Miss. Thamine   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer   \n",
       "1307       3         0                              Zakarian, Mr. Ortin   \n",
       "1308       3         0                               Zimmerman, Mr. Leo   \n",
       "\n",
       "         sex       age  sibsp  parch  ticket      fare    cabin embarked boat  \\\n",
       "0     female -0.061133      0      0   24160  3.439849       B5        S    2   \n",
       "1       male -2.009535      1      2  113781  2.284729  C22 C26        S   11   \n",
       "2     female -1.934376      1      2  113781  2.284729  C22 C26        S  NaN   \n",
       "3       male  0.008247      1      2  113781  2.284729  C22 C26        S  NaN   \n",
       "4     female -0.338650      1      2  113781  2.284729  C22 C26        S  NaN   \n",
       "...      ...       ...    ...    ...     ...       ...      ...      ...  ...   \n",
       "1304  female -1.067134      1      0    2665 -0.364022      NaN        C  NaN   \n",
       "1305  female       NaN      1      0    2665 -0.364022      NaN        C  NaN   \n",
       "1306    male -0.234581      0      0    2656 -0.503693      NaN        C  NaN   \n",
       "1307    male -0.199891      0      0    2670 -0.503693      NaN        C  NaN   \n",
       "1308    male -0.061133      0      0  315082 -0.491135      NaN        S  NaN   \n",
       "\n",
       "       body                        home.dest  \n",
       "0       NaN                     St Louis, MO  \n",
       "1       NaN  Montreal, PQ / Chesterville, ON  \n",
       "2       NaN  Montreal, PQ / Chesterville, ON  \n",
       "3     135.0  Montreal, PQ / Chesterville, ON  \n",
       "4       NaN  Montreal, PQ / Chesterville, ON  \n",
       "...     ...                              ...  \n",
       "1304  328.0                              NaN  \n",
       "1305    NaN                              NaN  \n",
       "1306  304.0                              NaN  \n",
       "1307    NaN                              NaN  \n",
       "1308    NaN                              NaN  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Auch hierfür gibt es eine Funktion in `sklearn`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "# auch hier gibt es eine Funktion in sklearn\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "titanic[['age', 'fare']] = StandardScaler().fit_transform(titanic[['age', 'fare']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>-0.061162</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>3.441165</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>-2.010496</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>2.285603</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>-1.935302</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>2.285603</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>0.008251</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>2.285603</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>-0.338812</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>2.285603</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>female</td>\n",
       "      <td>-1.067644</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>-0.364161</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>-0.364161</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>male</td>\n",
       "      <td>-0.234693</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>-0.503886</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>male</td>\n",
       "      <td>-0.199987</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>-0.503886</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>male</td>\n",
       "      <td>-0.061162</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>-0.491323</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton   \n",
       "1          1         1                   Allison, Master. Hudson Trevor   \n",
       "2          1         0                     Allison, Miss. Helen Loraine   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)   \n",
       "...      ...       ...                                              ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni   \n",
       "1305       3         0                            Zabour, Miss. Thamine   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer   \n",
       "1307       3         0                              Zakarian, Mr. Ortin   \n",
       "1308       3         0                               Zimmerman, Mr. Leo   \n",
       "\n",
       "         sex       age  sibsp  parch  ticket      fare    cabin embarked boat  \\\n",
       "0     female -0.061162      0      0   24160  3.441165       B5        S    2   \n",
       "1       male -2.010496      1      2  113781  2.285603  C22 C26        S   11   \n",
       "2     female -1.935302      1      2  113781  2.285603  C22 C26        S  NaN   \n",
       "3       male  0.008251      1      2  113781  2.285603  C22 C26        S  NaN   \n",
       "4     female -0.338812      1      2  113781  2.285603  C22 C26        S  NaN   \n",
       "...      ...       ...    ...    ...     ...       ...      ...      ...  ...   \n",
       "1304  female -1.067644      1      0    2665 -0.364161      NaN        C  NaN   \n",
       "1305  female       NaN      1      0    2665 -0.364161      NaN        C  NaN   \n",
       "1306    male -0.234693      0      0    2656 -0.503886      NaN        C  NaN   \n",
       "1307    male -0.199987      0      0    2670 -0.503886      NaN        C  NaN   \n",
       "1308    male -0.061162      0      0  315082 -0.491323      NaN        S  NaN   \n",
       "\n",
       "       body                        home.dest  \n",
       "0       NaN                     St Louis, MO  \n",
       "1       NaN  Montreal, PQ / Chesterville, ON  \n",
       "2       NaN  Montreal, PQ / Chesterville, ON  \n",
       "3     135.0  Montreal, PQ / Chesterville, ON  \n",
       "4       NaN  Montreal, PQ / Chesterville, ON  \n",
       "...     ...                              ...  \n",
       "1304  328.0                              NaN  \n",
       "1305    NaN                              NaN  \n",
       "1306  304.0                              NaN  \n",
       "1307    NaN                              NaN  \n",
       "1308    NaN                              NaN  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Datenskalen und deren Transformation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Viele Datenskalen sind nicht numerisch. Dies erschwert in manchen Fällen die weitere Verarbeitung. Grundsätzlich unterscheidet man 3 Typen von Daten:\n",
    "1. Quantitative Daten\n",
    "2. Ordinale Daten\n",
    "3. Nominelle Daten"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Quantitative Daten</b>\n",
    "\n",
    "* Numerische Typen\n",
    "* Kann unterschieden werden in diskret und stetig\n",
    "* Haben normalerweise eine (physikalische) Einheit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Ordinale Daten</b>\n",
    "\n",
    "* Kategorien\n",
    "* Es gibt eine **sinnvolle Ordnung**\n",
    "* **Keine** **Relation**/Verhältnis, zBsp. Kleidungsgröße L ist <b>nicht</b> $n$-mal so groß wie Kleidungsgröße M\n",
    "* Beispiele: Kleidungsgrößen, Schulnoten, Job-Klassifizierung"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Nominale Daten</b>\n",
    "* Kategorien\n",
    "* **Keine** Reihenfolge/Ordnung\n",
    "* Beispiele: Geschlecht, Farbe, Name, $\\ldots$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>0.9167</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>2.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>25.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>female</td>\n",
       "      <td>14.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>male</td>\n",
       "      <td>26.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>male</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>7.8750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton   \n",
       "1          1         1                   Allison, Master. Hudson Trevor   \n",
       "2          1         0                     Allison, Miss. Helen Loraine   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)   \n",
       "...      ...       ...                                              ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni   \n",
       "1305       3         0                            Zabour, Miss. Thamine   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer   \n",
       "1307       3         0                              Zakarian, Mr. Ortin   \n",
       "1308       3         0                               Zimmerman, Mr. Leo   \n",
       "\n",
       "         sex      age  sibsp  parch  ticket      fare    cabin embarked boat  \\\n",
       "0     female  29.0000      0      0   24160  211.3375       B5        S    2   \n",
       "1       male   0.9167      1      2  113781  151.5500  C22 C26        S   11   \n",
       "2     female   2.0000      1      2  113781  151.5500  C22 C26        S  NaN   \n",
       "3       male  30.0000      1      2  113781  151.5500  C22 C26        S  NaN   \n",
       "4     female  25.0000      1      2  113781  151.5500  C22 C26        S  NaN   \n",
       "...      ...      ...    ...    ...     ...       ...      ...      ...  ...   \n",
       "1304  female  14.5000      1      0    2665   14.4542      NaN        C  NaN   \n",
       "1305  female      NaN      1      0    2665   14.4542      NaN        C  NaN   \n",
       "1306    male  26.5000      0      0    2656    7.2250      NaN        C  NaN   \n",
       "1307    male  27.0000      0      0    2670    7.2250      NaN        C  NaN   \n",
       "1308    male  29.0000      0      0  315082    7.8750      NaN        S  NaN   \n",
       "\n",
       "       body                        home.dest  \n",
       "0       NaN                     St Louis, MO  \n",
       "1       NaN  Montreal, PQ / Chesterville, ON  \n",
       "2       NaN  Montreal, PQ / Chesterville, ON  \n",
       "3     135.0  Montreal, PQ / Chesterville, ON  \n",
       "4       NaN  Montreal, PQ / Chesterville, ON  \n",
       "...     ...                              ...  \n",
       "1304  328.0                              NaN  \n",
       "1305    NaN                              NaN  \n",
       "1306  304.0                              NaN  \n",
       "1307    NaN                              NaN  \n",
       "1308    NaN                              NaN  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Welche Spalten sind hier quantitativ, ordinal bzw. nominal?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Warum kann das numerisieren von Daten wichtig sein? In welchen Fällen ergibt sich welcher Vorteil/Nachteil?\n",
    "Sollen nun die Daten \"ver-numerisiert\" werden, so gibt es folgende Möglichkeiten."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# wir verwenden nun folgende Methoden um die Daten zu quantifizieren. Dafür verwenden wir die Methoden von sklearn\n",
    "\n",
    "from sklearn.preprocessing import OrdinalEncoder, OneHotEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')\n",
    "titanic_ordinal = titanic.copy(deep=True)\n",
    "\n",
    "titanic_ordinal[['sex']] = OrdinalEncoder(dtype=int).fit_transform(titanic_ordinal[['sex']]) # \"sex\" ist nun ein \"0/1\" (binary) Attribut"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>0</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>1</td>\n",
       "      <td>0.9167</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>1</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>0</td>\n",
       "      <td>25.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>0</td>\n",
       "      <td>14.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>1</td>\n",
       "      <td>26.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>1</td>\n",
       "      <td>27.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>1</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>7.8750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  sex  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton    0   \n",
       "1          1         1                   Allison, Master. Hudson Trevor    1   \n",
       "2          1         0                     Allison, Miss. Helen Loraine    0   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton    1   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)    0   \n",
       "...      ...       ...                                              ...  ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni    0   \n",
       "1305       3         0                            Zabour, Miss. Thamine    0   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer    1   \n",
       "1307       3         0                              Zakarian, Mr. Ortin    1   \n",
       "1308       3         0                               Zimmerman, Mr. Leo    1   \n",
       "\n",
       "          age  sibsp  parch  ticket      fare    cabin embarked boat   body  \\\n",
       "0     29.0000      0      0   24160  211.3375       B5        S    2    NaN   \n",
       "1      0.9167      1      2  113781  151.5500  C22 C26        S   11    NaN   \n",
       "2      2.0000      1      2  113781  151.5500  C22 C26        S  NaN    NaN   \n",
       "3     30.0000      1      2  113781  151.5500  C22 C26        S  NaN  135.0   \n",
       "4     25.0000      1      2  113781  151.5500  C22 C26        S  NaN    NaN   \n",
       "...       ...    ...    ...     ...       ...      ...      ...  ...    ...   \n",
       "1304  14.5000      1      0    2665   14.4542      NaN        C  NaN  328.0   \n",
       "1305      NaN      1      0    2665   14.4542      NaN        C  NaN    NaN   \n",
       "1306  26.5000      0      0    2656    7.2250      NaN        C  NaN  304.0   \n",
       "1307  27.0000      0      0    2670    7.2250      NaN        C  NaN    NaN   \n",
       "1308  29.0000      0      0  315082    7.8750      NaN        S  NaN    NaN   \n",
       "\n",
       "                            home.dest  \n",
       "0                        St Louis, MO  \n",
       "1     Montreal, PQ / Chesterville, ON  \n",
       "2     Montreal, PQ / Chesterville, ON  \n",
       "3     Montreal, PQ / Chesterville, ON  \n",
       "4     Montreal, PQ / Chesterville, ON  \n",
       "...                               ...  \n",
       "1304                              NaN  \n",
       "1305                              NaN  \n",
       "1306                              NaN  \n",
       "1307                              NaN  \n",
       "1308                              NaN  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_ordinal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Im vorigen Beispiel wurden doppelte eckige Klammern verwendet, um ein Dataframe zu erhalten, ansonsten wird ein Series Objekt zurückgegeben, siehe folgenden Code\n",
    "a = titanic[['sex']]\n",
    "b = titanic['sex']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sex</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         sex\n",
       "0     female\n",
       "1       male\n",
       "2     female\n",
       "3       male\n",
       "4     female\n",
       "...      ...\n",
       "1304  female\n",
       "1305  female\n",
       "1306    male\n",
       "1307    male\n",
       "1308    male\n",
       "\n",
       "[1309 rows x 1 columns]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0       female\n",
       "1         male\n",
       "2       female\n",
       "3         male\n",
       "4       female\n",
       "         ...  \n",
       "1304    female\n",
       "1305    female\n",
       "1306      male\n",
       "1307      male\n",
       "1308      male\n",
       "Name: sex, Length: 1309, dtype: object"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Falls das Objekt gespeichert wird, kann es auch wieder zurück transformiert werden\n",
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')\n",
    "titanic_ordinal = titanic.copy(deep=True)\n",
    "\n",
    "OE = OrdinalEncoder(dtype=int)\n",
    "\n",
    "titanic_ordinal[['sex']] = OE.fit_transform(titanic_ordinal[['sex']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>0</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>1</td>\n",
       "      <td>0.9167</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>0</td>\n",
       "      <td>2.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>1</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>0</td>\n",
       "      <td>25.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>0</td>\n",
       "      <td>14.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>1</td>\n",
       "      <td>26.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>1</td>\n",
       "      <td>27.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>1</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>7.8750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  sex  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton    0   \n",
       "1          1         1                   Allison, Master. Hudson Trevor    1   \n",
       "2          1         0                     Allison, Miss. Helen Loraine    0   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton    1   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)    0   \n",
       "...      ...       ...                                              ...  ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni    0   \n",
       "1305       3         0                            Zabour, Miss. Thamine    0   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer    1   \n",
       "1307       3         0                              Zakarian, Mr. Ortin    1   \n",
       "1308       3         0                               Zimmerman, Mr. Leo    1   \n",
       "\n",
       "          age  sibsp  parch  ticket      fare    cabin embarked boat   body  \\\n",
       "0     29.0000      0      0   24160  211.3375       B5        S    2    NaN   \n",
       "1      0.9167      1      2  113781  151.5500  C22 C26        S   11    NaN   \n",
       "2      2.0000      1      2  113781  151.5500  C22 C26        S  NaN    NaN   \n",
       "3     30.0000      1      2  113781  151.5500  C22 C26        S  NaN  135.0   \n",
       "4     25.0000      1      2  113781  151.5500  C22 C26        S  NaN    NaN   \n",
       "...       ...    ...    ...     ...       ...      ...      ...  ...    ...   \n",
       "1304  14.5000      1      0    2665   14.4542      NaN        C  NaN  328.0   \n",
       "1305      NaN      1      0    2665   14.4542      NaN        C  NaN    NaN   \n",
       "1306  26.5000      0      0    2656    7.2250      NaN        C  NaN  304.0   \n",
       "1307  27.0000      0      0    2670    7.2250      NaN        C  NaN    NaN   \n",
       "1308  29.0000      0      0  315082    7.8750      NaN        S  NaN    NaN   \n",
       "\n",
       "                            home.dest  \n",
       "0                        St Louis, MO  \n",
       "1     Montreal, PQ / Chesterville, ON  \n",
       "2     Montreal, PQ / Chesterville, ON  \n",
       "3     Montreal, PQ / Chesterville, ON  \n",
       "4     Montreal, PQ / Chesterville, ON  \n",
       "...                               ...  \n",
       "1304                              NaN  \n",
       "1305                              NaN  \n",
       "1306                              NaN  \n",
       "1307                              NaN  \n",
       "1308                              NaN  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_ordinal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "# To demonstrate inverse transform\n",
    "titanic_ordinal[['sex']] = OE.inverse_transform(titanic_ordinal[['sex']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pclass</th>\n",
       "      <th>survived</th>\n",
       "      <th>name</th>\n",
       "      <th>sex</th>\n",
       "      <th>age</th>\n",
       "      <th>sibsp</th>\n",
       "      <th>parch</th>\n",
       "      <th>ticket</th>\n",
       "      <th>fare</th>\n",
       "      <th>cabin</th>\n",
       "      <th>embarked</th>\n",
       "      <th>boat</th>\n",
       "      <th>body</th>\n",
       "      <th>home.dest</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allen, Miss. Elisabeth Walton</td>\n",
       "      <td>female</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>24160</td>\n",
       "      <td>211.3375</td>\n",
       "      <td>B5</td>\n",
       "      <td>S</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>St Louis, MO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Allison, Master. Hudson Trevor</td>\n",
       "      <td>male</td>\n",
       "      <td>0.9167</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>11</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Miss. Helen Loraine</td>\n",
       "      <td>female</td>\n",
       "      <td>2.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mr. Hudson Joshua Creighton</td>\n",
       "      <td>male</td>\n",
       "      <td>30.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>135.0</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>Allison, Mrs. Hudson J C (Bessie Waldo Daniels)</td>\n",
       "      <td>female</td>\n",
       "      <td>25.0000</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>113781</td>\n",
       "      <td>151.5500</td>\n",
       "      <td>C22 C26</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Montreal, PQ / Chesterville, ON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1304</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Hileni</td>\n",
       "      <td>female</td>\n",
       "      <td>14.5000</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>328.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1305</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zabour, Miss. Thamine</td>\n",
       "      <td>female</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>2665</td>\n",
       "      <td>14.4542</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1306</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Mapriededer</td>\n",
       "      <td>male</td>\n",
       "      <td>26.5000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2656</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>304.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1307</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zakarian, Mr. Ortin</td>\n",
       "      <td>male</td>\n",
       "      <td>27.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>2670</td>\n",
       "      <td>7.2250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>C</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1308</th>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>Zimmerman, Mr. Leo</td>\n",
       "      <td>male</td>\n",
       "      <td>29.0000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>315082</td>\n",
       "      <td>7.8750</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1309 rows × 14 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      pclass  survived                                             name  \\\n",
       "0          1         1                    Allen, Miss. Elisabeth Walton   \n",
       "1          1         1                   Allison, Master. Hudson Trevor   \n",
       "2          1         0                     Allison, Miss. Helen Loraine   \n",
       "3          1         0             Allison, Mr. Hudson Joshua Creighton   \n",
       "4          1         0  Allison, Mrs. Hudson J C (Bessie Waldo Daniels)   \n",
       "...      ...       ...                                              ...   \n",
       "1304       3         0                             Zabour, Miss. Hileni   \n",
       "1305       3         0                            Zabour, Miss. Thamine   \n",
       "1306       3         0                        Zakarian, Mr. Mapriededer   \n",
       "1307       3         0                              Zakarian, Mr. Ortin   \n",
       "1308       3         0                               Zimmerman, Mr. Leo   \n",
       "\n",
       "         sex      age  sibsp  parch  ticket      fare    cabin embarked boat  \\\n",
       "0     female  29.0000      0      0   24160  211.3375       B5        S    2   \n",
       "1       male   0.9167      1      2  113781  151.5500  C22 C26        S   11   \n",
       "2     female   2.0000      1      2  113781  151.5500  C22 C26        S  NaN   \n",
       "3       male  30.0000      1      2  113781  151.5500  C22 C26        S  NaN   \n",
       "4     female  25.0000      1      2  113781  151.5500  C22 C26        S  NaN   \n",
       "...      ...      ...    ...    ...     ...       ...      ...      ...  ...   \n",
       "1304  female  14.5000      1      0    2665   14.4542      NaN        C  NaN   \n",
       "1305  female      NaN      1      0    2665   14.4542      NaN        C  NaN   \n",
       "1306    male  26.5000      0      0    2656    7.2250      NaN        C  NaN   \n",
       "1307    male  27.0000      0      0    2670    7.2250      NaN        C  NaN   \n",
       "1308    male  29.0000      0      0  315082    7.8750      NaN        S  NaN   \n",
       "\n",
       "       body                        home.dest  \n",
       "0       NaN                     St Louis, MO  \n",
       "1       NaN  Montreal, PQ / Chesterville, ON  \n",
       "2       NaN  Montreal, PQ / Chesterville, ON  \n",
       "3     135.0  Montreal, PQ / Chesterville, ON  \n",
       "4       NaN  Montreal, PQ / Chesterville, ON  \n",
       "...     ...                              ...  \n",
       "1304  328.0                              NaN  \n",
       "1305    NaN                              NaN  \n",
       "1306  304.0                              NaN  \n",
       "1307    NaN                              NaN  \n",
       "1308    NaN                              NaN  \n",
       "\n",
       "[1309 rows x 14 columns]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "titanic_ordinal"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')\n",
    "titanic_onehot = titanic.copy(deep=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "# OneHotEncoder\n",
    "encoder = OneHotEncoder(sparse_output=False, dtype=int)\n",
    "encoded_sex = encoder.fit_transform(titanic_onehot[['sex']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 0],\n",
       "       [0, 1],\n",
       "       [1, 0],\n",
       "       ...,\n",
       "       [0, 1],\n",
       "       [0, 1],\n",
       "       [0, 1]])"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "encoded_sex"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoded_sex_df = pd.DataFrame(encoded_sex, columns=encoder.get_feature_names_out(['sex'])) # erstellt die Spalten Sex_male und Sex_female"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoded_sex_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic_onehot.drop(columns=['sex'], inplace=True)\n",
    "titanic_onehot = pd.concat([titanic_onehot, encoded_sex_df], axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic_onehot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Verknüpfen von Dataframes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Es gibt mehrere Befehle, um Dataframes miteinander zu verknüpfen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`merge()`:2 Objekte verknüpfen und join über die Spalten.\n",
    "\n",
    "`join()`: 2 Objekte verknüpfen und Join über die Indizes.\n",
    "\n",
    "`concat()`: 2 oder mehr Dataframes werden miteinander verknüpft. (Bereits verwendet)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Merge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')\n",
    "\n",
    "homes = titanic[['home.dest']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "homes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic.drop(columns=['home.dest'], inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfmerged = titanic.merge(right=homes, left_index=True, right_index=True) # wenn nicht der Index verwendet wird, dann left_on='column_name' und right_on='column_name'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfmerged"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Join"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic = pd.read_csv(path, header=0, sep=',', index_col=False, decimal='.')\n",
    "\n",
    "homes = titanic[['home.dest']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "homes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic.drop(columns=['home.dest'], inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "titanic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Join\n",
    "dfjoined = titanic.join(homes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfjoined"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Nahezu) sämtliche bisher durchgemachte Schritte sind im Normalfall möglicherweise notwendig, um ein eine vernünftige Datenananlyse zu beginnen. Nicht jeder Schritt ist dabei bei jedem Dataset anwendbar, jedoch muss sich mit diesen Schritten auseinander gesetzt werden."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Weitere Möglichkeiten werden in den folgenden Wochen erläutert (Dimensionsreduzierung, Feature Selection, Smoothing methods, etc.). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Da es auch andere Formen von Datasets gibt (Bilder, etc. - werden später durchgemacht), gibt es dann noch ein paar Datenset-spezifische Vorverarbeitungsschritte."
   ]
  },
  {
   "attachments": {
    "image.png": {
     "image/png": "[encoded data, 377996 characters]"
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](attachment:image.png)\n",
    "\n",
    "(adaptiert von Seminar Künstliche Intelligenz 16. und 17. Juni St.Polten/Online)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dsai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
