{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div style=\"\n",
    "    border: 2px solid #4CAF50; \n",
    "    padding: 15px; \n",
    "    background-color: #f4f4f4; \n",
    "    border-radius: 10px; \n",
    "    align-items: center;\">\n",
    "\n",
    "<h1 style=\"margin: 0; color: #4CAF50;\">Weitere Datenformen</h1>\n",
    "<h2 style=\"margin: 5px 0; color: #555;\">DSAI</h2>\n",
    "<h3 style=\"margin: 5px 0; color: #555;\">Jakob Eggl</h3>\n",
    "\n",
    "<div style=\"flex-shrink: 0;\">\n",
    "    <img src=\"https://www.htl-grieskirchen.at/wp/wp-content/uploads/2022/11/logo_bildschirm-1024x503.png\" alt=\"Logo\" style=\"width: 250px; height: auto;\"/>\n",
    "</div>\n",
    "<p1> © 2024/25 Jakob Eggl. Nutzung oder Verbreitung nur mit ausdrücklicher Genehmigung des Autors.</p1>\n",
    "</div>\n",
    "<div style=\"flex: 1;\">\n",
    "</div>   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Teile dieses Notebooks (adaptiert) aus Hands-On AI 1 WS21; JKU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bisher haben wir uns hauptsächlich mit tabellarischen Daten beschäftigt. Dabei wurden insbesondere folgende Themen durchgemacht:\n",
    "* **Vorverarbeitungsschritte** (Normalisierung, Fehlende Werte, Data Cleaning, etc.)\n",
    "* **Statistiken**\n",
    "* **Visualisierungen**\n",
    "\n",
    "\n",
    "Nun beschäftigen wir uns mit **anderen Formen von Daten**, welche, je nach Anwendungsfall, auch in vielen Gebieten im Machine Learning auftreten."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bilder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bilder besitzen normalerweise 3 Dimensionen.\n",
    "\n",
    "Die 3 Dimension sind dabei $\\text{height}\\times\\text{length}\\times \\text{channels}$.\n",
    "\n",
    "Dabei ist die letzte Dimension (*channels*) zuständig für die Farben eines jedes einzelnen Pixels. (*RGB*, sprich *Rot*, *Grün* und *Blau*)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Python können wir uns diese Dimensionen anzeigen lassen. Dafür laden wir zuerst einmal ein Bild."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir benötigen die `cv2` library. Wir installieren sie mit `conda install -c conda-forge opencv`.\n",
    "\n",
    "Außerdem benötigen wir die `spacy` library. Wir installieren sie mit `conda install -c conda-forge spacy`. (siehe https://spacy.io/usage für andere Betriebssysteme, Prozessorarchitekturen und/oder Sprachen)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.image import imread\n",
    "import numpy as np\n",
    "from PIL import Image\n",
    "from scipy import ndimage\n",
    "import cv2\n",
    "import scipy\n",
    "from scipy.ndimage import gaussian_filter\n",
    "from sklearn.decomposition import PCA\n",
    "import IPython.display as ipd\n",
    "from scipy.io.wavfile import read\n",
    "from typing import Tuple, Union\n",
    "import spacy\n",
    "import glob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_image(image_path: str):\n",
    "    img = imread(image_path)\n",
    "    plt.imshow(img)\n",
    "    plt.axis('off') # turn off axis\n",
    "    plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_path = os.path.join(\"..\", \"resources\", \"Columbus.jpg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 199972 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_image(image_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In unserem Fall können wir auch die Dimensionen ausgeben. Dies passiert mit dem `imread(path).shape` Befehl."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3087, 2289, 3)\n"
     ]
    }
   ],
   "source": [
    "img = imread(image_path)\n",
    "shape = img.shape # height x width x color channels\n",
    "\n",
    "print(shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Eine weitere wichtige Eigenschaft ist die Farbtiefe von Bildern. Normalerweise haben wir eine $8$ Bit Farbtiefe pro Farbe. Sprich pro Channel sind $2^8=256$ verschiedene Farben möglich.\n",
    "\n",
    "In manchen Fällen haben Bilder aber auch eine $16$ Bit Farbtiefe. Hier gibt es dann $2^{16}=256\\cdot 256 = 65536$ verschiedene Farben pro Channel.\n",
    "\n",
    "Wie viele mögliche Farben gibt es also insgesamt? Und wie viele mögliche Bilder resultieren daraus?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RGB Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In unserem Fall besitzt das Bild 3 Kanäle mit einer Farbtiefe von jeweils 8 Bit. Wir können diese Kanäle nun etwas genauer untersuchen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum: 0\n",
      "Maximum: 234\n"
     ]
    }
   ],
   "source": [
    "channel = 'green' # 'green', 'blue'\n",
    "\n",
    "if channel == 'red':\n",
    "    channel = 0\n",
    "elif channel == 'green':\n",
    "    channel = 1\n",
    "elif channel == 'blue':\n",
    "    channel = 2\n",
    "else:\n",
    "    raise ValueError(f\"Invalid channel: {channel}\")\n",
    "\n",
    "img = imread(image_path)\n",
    "print(f\"Minimum: {np.min(img[:,:,channel])}\")\n",
    "print(f\"Maximum: {np.max(img[:,:,channel])}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Das RGB Model wird als **additives Farbmodel** bezeichnet, da die 3 Kanäle in **Summe** das gesamte Bild erzeugen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_image_channels_rgb(image_path: str):\n",
    "    \n",
    "    img = imread(image_path)\n",
    "    fig, ax = plt.subplots(1, 3, figsize=(15, 10))\n",
    "    channels = [0, 1, 2]\n",
    "    colors = ['red', 'green', 'blue']\n",
    "    \n",
    "    for channel, channel_name in zip(channels, colors):\n",
    "        tmp = np.zeros(img.shape, dtype='uint8')\n",
    "        tmp[:, :, channel] = img[:, :, channel]\n",
    "        ax[channel].imshow(tmp)\n",
    "        ax[channel].set_title(channel_name + ' channel')\n",
    "        ax[channel].axis(r'off')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 428416 characters]",
      "text/plain": [
       "<Figure size 1500x1000 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_image_channels_rgb(image_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_rgb_histograms(image_path: str, bins:int=255):\n",
    "    img = imread(image_path)\n",
    "    fig, ax = plt.subplots(1, 3, figsize=(15, 3))\n",
    "    colors = ['red', 'green', 'blue']\n",
    "    for i, color in enumerate(colors):\n",
    "        ax[i].hist(img[:, :, i].ravel(), bins=bins, histtype=r'stepfilled', density=True, color=color)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 24320 characters]",
      "text/plain": [
       "<Figure size 1500x300 with 3 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_rgb_histograms(image_path, bins=50) # default bins = 255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### RGBA Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In vielen Bildern gibt es einen weiteren Channel, welcher die *Transparenz* darstellt. Dabei steht das *A* im Modell für *alpha*. Die Werte sind dabei gleich wie bei den anderen 3 Kanälen. Wir können aber auch einen Prozentwert vergeben."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_image_rgba(image_path: str, alpha: float = 1.0):\n",
    "    with Image.open(image_path) as img:\n",
    "        img = img.convert(r'RGBA')\n",
    "        int_alpha = int(256 * alpha)\n",
    "        img.putalpha(int_alpha)\n",
    "        plt.imshow(img)\n",
    "        plt.axis('off')\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 156368 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_image_rgba(image_path, alpha=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graustufen"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Eine weitere Variante sind die sogenannten Schwarz-Weiß bzw. Graustufen Bilder. Dabei gibt es nur einen Channel, der die Helligkeit im Bild repräsentiert. Das Umwandeln in ein Graustufen Bild ist einfach, umgekehrt geht das ohne zusätzliche Information nicht. Wieso?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_image_grayscale(image_path: str):\n",
    "    with Image.open(image_path) as img:\n",
    "        img = img.convert(mode='1') # mode '1' for grayscale image.\n",
    "        plt.imshow(img)\n",
    "        plt.axis('off')\n",
    "        plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 112696 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_image_grayscale(image_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wo könnten Graustufen Bilder helfen?\n",
    "* Bild Klassifizierung (zBsp Auto/Motorrad Unterscheidung) Warum ist dies für Modelle eventuell vom Vorteil?\n",
    "* Speicherplatzreduzierung"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Augmentation (=Daten Erweiterung)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In weiterer wichtiger Punkt ist die sogenannte *Data Augmentation*, sprich die Datenerweiterung. Dabei geht es darum aus einem bestehenden Dataset weitere Daten zu produzieren."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Annahme, wir wollen nun einen Klassifizierer bauen, welcher zwischen Bildern, auf denen sich die Katze *Columbus* befindet, und Bildern, wo sich die Katze nicht befindet, unterscheiden kann. Dazu würden wir zum Trainieren dieses Modells jetzt noch viele weitere Fotos von Columbus brauchen (und auch viele ohne Columbus). Um den großen Aufwand der Datenbeschaffung zu reduzieren können wir nun Data Augmentation verwenden.\n",
    "\n",
    "(Wichtig: Hier soll der Klassifizierer **nicht** in einer großen Datenbank einfach Fotos vergleichen sondern er soll wirklich das Konzept einer Katze lernen und auch neue, noch nie gesehene, Fotos richtig zuordnen können in die Kategorien *Katze* und *Keine Katze*)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Möglichkeiten zur Daten Erweiterung:**\n",
    "* Rotation\n",
    "* Spiegeln\n",
    "* Zuschneiden\n",
    "* Sharpening/Blurring\n",
    "* Graustufen\n",
    "* Resizing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Rotation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Einfache Möglichkeit, weitere Daten zu generieren\n",
    "* Ob und wie sehr rotiert werden kann/soll hängt von der Anwendung ab (Katzenbilder vs. Autobilder: Autos werden 180° gedreht wenig Sinn machen, Katzen auch nur manchmal)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_rotated_image(image_path: str, angle):\n",
    "    img = imread(image_path)\n",
    "    img_rotated = ndimage.rotate(img, angle=angle)\n",
    "    plt.imshow(img_rotated)\n",
    "    plt.axis('off')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 164300 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_rotated_image(image_path, angle=45)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 199548 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_rotated_image(image_path, angle=180)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Spiegeln"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Sehr ähnlich (aber nicht gleich) zum Rotieren\n",
    "* Auch hier nicht bei jedem Bild anwendbar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_flipped_image(image_path: str, flipping: str):\n",
    "    img = imread(image_path)\n",
    "    if flipping == 'horizontal':\n",
    "        plt.imshow(img[:, ::-1, :])\n",
    "    elif flipping == 'vertical':\n",
    "        plt.imshow(img[::-1, :, :])\n",
    "    plt.gca().axis(r'off')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 199304 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_flipped_image(image_path, flipping=r'vertical')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 199280 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_flipped_image(image_path, flipping=r'horizontal')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Zuschneiden"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Weitere einfache Möglichkeit, Daten zu generieren\n",
    "* Kann (fast) beliebig angewendet werden (zu großes oder kleines Objekt unbrauchbar)\n",
    "* muss danach wieder die selben Dimensionen haben!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_cropped_image(image_path: str, left, top, width, height):\n",
    "    with Image.open(image_path) as img:\n",
    "        img_cropped = img.crop((left, top, left + width, top + height)) # \n",
    "        plt.imshow(img_cropped)\n",
    "        plt.gca().axis(r'off')\n",
    "        plt.show()\n",
    "    return img_cropped"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 194568 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "img_cropped = plot_cropped_image(image_path, left=1000, top=1000, width=300, height=300) # Macht dieses Bild Sinn?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(300, 300)\n"
     ]
    }
   ],
   "source": [
    "print(img_cropped.size) # Achtung: Shape ist nun anders als bei vorigen Bilder. Normalerweise wollen wir (lernen wir später/nächstes Jahr) gleiche Größe behalten -> siehe Resizing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Unschärfen/Schärfen (Blurring/Sharpening)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Da auch echte Fotos nicht immer gleich scharf/verschwommen sind, kann es auch hilfreich sein, die Fotos künstlich etwas Unscharf zu machen bzw. zu schärfen.\n",
    "Wichtig dabei ist:\n",
    "* Fotos, die zu unscharf sind, sind unbrauchbar\n",
    "* Ebenso sind \"zu scharfe\" Bilder möglicherweise nicht naturgetreu\n",
    "\n",
    "Zum Blurren verwenden wir als Methode *Gauss'sche Unschärfe* (Gaussian Blur), dabei wird ein Filter (in diesem Fall Modellierung einer Normalverteilung) über das Bild gelegt, welcher zu dem verschwommenen Bild führt. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_blurred_image(image_path: str, sigma: float):\n",
    "    img = imread(image_path)\n",
    "    blurred = gaussian_filter(img, sigma=(sigma, sigma, 1))\n",
    "    plt.imshow(blurred)\n",
    "    plt.gca().axis('off')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 132976 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_blurred_image(image_path, sigma=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sharpen_image(image_path, sharpening_filter=None, strength=2.0, grayscale:bool = False):\n",
    "    if grayscale:\n",
    "        image = cv2.cvtColor(imread(image_path), cv2.COLOR_RGB2GRAY)\n",
    "    else:\n",
    "        image = cv2.cvtColor(imread(image_path), cv2.COLOR_BGR2RGB)  # Added missing argument\n",
    "\n",
    "    if sharpening_filter is None:\n",
    "        sharpening_filter = strength * np.array([[0, -1, 0], [-1, 5, -1], [0, -1, 0]])\n",
    "\n",
    "    if grayscale:\n",
    "        sharpened_image = cv2.filter2D(image, -1, sharpening_filter)[1:-1, 1:-1]\n",
    "    else:\n",
    "        sharpened_image = np.zeros_like(image)\n",
    "        for i in range(3):\n",
    "            sharpened_image[..., i] = cv2.filter2D(image[..., i], -1, sharpening_filter)\n",
    "\n",
    "    sharpened_image = sharpened_image[1:-1, 1:-1]\n",
    "\n",
    "    if grayscale:\n",
    "        plt.imshow(sharpened_image, cmap='gray')\n",
    "    else:\n",
    "        plt.imshow(cv2.cvtColor(sharpened_image, cv2.COLOR_BGR2RGB))\n",
    "    plt.axis('off')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 217572 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sharpen_image(image_path, grayscale=False, strength=2) # Bemerkbar vor allem bei geraden Linien"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 118908 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sharpen_image(image_path, grayscale=True, strength=2) # Bemerkbar vor allem bei geraden Linien"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graustufen"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wurde bereits besprochen, ist aber ebenso in vielen Fällen ein essentieller Bestandteil der Data Augmentation Methodiken. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Resizing (Auflösung verringern)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Um den benötigten Speicherplatz zu verringern, werden oft Bilder herunterskaliert. Dies führt zu einfacheren Modellen, die schneller trainierbar sind (und normalerweise vergleichbare Performance bieten). Außerdem wird der Speicher auf der Festplatte bzw. der benötigte Speicher auf der GPU (oder RAM) verringert.\n",
    "\n",
    "Außerdem benötigen die meisten Modelle, welche Bilder verarbeiten, eine fixe Eingangsgröße. Sprich Sie können nur mit Bildern der gleichen Größe arbeiten."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def resize_image(image_path, target_width, target_height):\n",
    "    # Open an image file\n",
    "    with Image.open(image_path) as img:\n",
    "        resized_img = img.resize((target_width, target_height), Image.LANCZOS) # LANCZOS is the algorithm used for resizing\n",
    "        return resized_img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 40900 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 100)\n"
     ]
    }
   ],
   "source": [
    "resized_img = resize_image(image_path, target_width=100, target_height=100)\n",
    "plt.imshow(resized_img)\n",
    "plt.axis('off')\n",
    "plt.show()\n",
    "print(resized_img.size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sequentielle Daten:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Neben den tabellarischen Daten und den Bildern ist eine weitere wichtige Datenquelle jene, welche sich mathematisch durch Folgen beschreiben lässt. Sie zeichnet sich darin aus, dass die verschiedenen Datenpunkte eine Reihenfolge besitzen. Eine bekannte Schreibweise für diese Daten ist:\n",
    "$$(a_n)_{n\\in\\mathbb{N}} = a_1, a_2, a_3, \\ldots .$$\n",
    "Dabei steht der Subindex $i$ in $a_i$ für das $i$-te Element dieser Folge."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Es gibt hier auch mehrere Arten von sequentiellen Daten, wobei wir insbesondere die Tonspuren (Sound) und Daten in Textform genauer behandeln werden."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sequentielle Daten: Sound"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Daten hier sind sehr ähnlich zu tabellarischen Daten mit nur einem (numerischen) Feature. Tonaufnahmen sind natürlich im Original (Analog) eine Funktion $f(x)$. Wir können diese aber nicht messen, sondern nur punktuelle Aufnahmen von $f(x)$ (Digitalisierungsvorgang). Somit haben wir auch hiere eine Folge $(a_n)_{n\\in\\mathbb{N}}$, wobei der Index $i$ im $a_i$ nun für den Zeitpunkt $i$ steht. Der Zeitabstand $t_S$ zwischen den Datenpunkten $a_i$ und $a_{i+1}$ ist konstant und ist bestimmt durch die Sampling-Frequency (Messfrequenz) $f_S$. Normalerweise beträgt die Messfrequenz in etwa $f_S = 48\\, \\text{kHz}$, was bedeutet, dass der Abstand zwischen 2 Samples $t_S=f_S^{-1} \\approx 0.021\\, \\text{ms}$ beträgt.\n",
    "\n",
    "Ein konstanter Ton bedeutet außerdem, dass sich die Funktion $f(x)$ aus der Summe von Sinusfunktionen (mit verschiedenen Frequenzen $f_i$ und Amplituden $a_i$) zusammensetzt. Sprich\n",
    "$$f(x)=\\sum_{i}a_i\\sin(2\\pi f_i x).$$\n",
    "\n",
    "Dabei ändert die Frequenz den Ton, die Amplitude die Lautstärke."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir starten nun mit der Generierung von einer einzelnen Sinus-Funktion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_sine(freq: float, signal_length: float, sampling_rate: int):\n",
    "    t = np.linspace(0, signal_length, int(signal_length * sampling_rate), endpoint=False)\n",
    "    return np.sin(2 * np.pi * freq * t)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir können uns diese Welle nun grafisch darstellen lassen (und auch auditiv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_signal(points: np.ndarray, time: float, sampling_rate: int, show_sampling_points:bool=False):\n",
    "    n_samples = int(sampling_rate * time)\n",
    "    t = np.linspace(0, time, n_samples, endpoint=False)\n",
    "\n",
    "    plt.plot(t, points)\n",
    "    if show_sampling_points:\n",
    "        plt.plot(t, points, r'o')\n",
    "    \n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Anzahl der Punkte: 250\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 51932 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 63416 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "f = 20\n",
    "T = 0.5\n",
    "sampling_rate = 500\n",
    "\n",
    "signal = generate_sine(f, T, sampling_rate)\n",
    "print(f\"Anzahl der Punkte: {len(signal)}\")\n",
    "\n",
    "plot_signal(signal, T, sampling_rate)\n",
    "plot_signal(signal, T, sampling_rate, show_sampling_points=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Eine falsche (=zu kleine) Samplingrate führt zu einer Signalverfälschung, wie folgende Beispiele zeigen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_signal_with_sampling_rate(points: np.ndarray, time: float, sampling_rate: int):\n",
    "    \n",
    "    t1 = np.linspace(0, time, len(points))\n",
    "    plt.plot(t1, points)\n",
    "\n",
    "    n_samples = np.ceil(sampling_rate * time).astype(int)\n",
    "    t2 = [i / sampling_rate for i in range(n_samples)]\n",
    "\n",
    "    idx = np.round(np.linspace(0, (len(points) / time) * t2[-1], n_samples, endpoint=True)).astype(int)\n",
    "    sampled_points = points[idx]\n",
    "    plt.plot(t2, sampled_points, 'o')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 66564 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 67028 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 67648 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 69956 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 93960 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "T = 1\n",
    "f = 20\n",
    "sampling_rate = 500\n",
    "\n",
    "signal = generate_sine(f, T, sampling_rate)\n",
    "\n",
    "plot_signal_with_sampling_rate(signal, T, sampling_rate = 5)\n",
    "plot_signal_with_sampling_rate(signal, T, sampling_rate = 10)\n",
    "plot_signal_with_sampling_rate(signal, T, sampling_rate = 20)\n",
    "plot_signal_with_sampling_rate(signal, T, sampling_rate = 25)\n",
    "plot_signal_with_sampling_rate(signal, T, sampling_rate = 500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" >\n",
       "                    <source src=\"data:audio/wav;base64,[encoded data, 64060 characters]\" type=\"audio/wav\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Nun generieren wir einen Ton\n",
    "\n",
    "T = 1\n",
    "f = 440\n",
    "sampling_rate = 24000\n",
    "\n",
    "signal = generate_sine(f, T, sampling_rate)\n",
    "\n",
    "ipd.Audio(signal, rate=sampling_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 51884 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" >\n",
       "                    <source src=\"data:audio/wav;base64,[encoded data, 64060 characters]\" type=\"audio/wav\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Nun erstellen wir einen etwas komplizierteren Ton\n",
    "T = 1\n",
    "sampling_rate = 24000\n",
    "\n",
    "signal_1 = generate_sine(44, T, sampling_rate)\n",
    "signal_2 = generate_sine(240, T, sampling_rate)\n",
    "signal_3 = generate_sine(330, T, sampling_rate)\n",
    "\n",
    "signal = signal_1 + signal_2 + signal_3\n",
    "\n",
    "plot_signal(signal, T, sampling_rate)\n",
    "ipd.Audio(signal, rate=sampling_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Zuguterletzt laden wir jetzt noch eine Wav-Datei visualisieren sie und spielen sie ab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_wav_file(wav_file: str, time: int = None) -> Tuple[np.ndarray, int]:\n",
    "    file = read(wav_file)\n",
    "    signal_rate = file[0]\n",
    "    points = file[1]\n",
    "    # check for multiple channels (should only be stereo, i.e., 2 at max)\n",
    "    if len(points.shape) > 1:\n",
    "        points = points[:, 0]  # just take the first channel\n",
    "    if time:\n",
    "        length_of_signal = int(time * signal_rate)\n",
    "        if length_of_signal < len(points):\n",
    "            points = points[0:length_of_signal]\n",
    "    return points, signal_rate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 32748 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "                <audio  controls=\"controls\" >\n",
       "                    <source src=\"data:audio/wav;base64,[encoded data, 1105980 characters]\" type=\"audio/wav\" />\n",
       "                    Your browser does not support the audio element.\n",
       "                </audio>\n",
       "              "
      ],
      "text/plain": [
       "<IPython.lib.display.Audio object>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sound_path = os.path.join(\"..\", \"resources\", \"piano.wav\")\n",
    "T = 20\n",
    "signal, sampling_rate = read_wav_file(sound_path, time=T)\n",
    "\n",
    "t = np.linspace(0, T, len(signal))\n",
    "plt.plot(t, signal)\n",
    "plt.show()\n",
    "\n",
    "ipd.Audio(signal, rate=sampling_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Um im Machine Learning Bereich mit sequentiellen Daten zu arbeiten gibt es mehrere Möglichkeiten:\n",
    "* Recurrent Neural Networks ((x)LSTM, Transformer, etc.)\n",
    "* Features aus den Tonspuren berechnen und diese in ein normales Machine Learning Modell stecken"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Für den letzteren Punkt sehen wir uns nun die **Fourier Transformation** an.\n",
    "Sie zerlegt das Signal in die einzelnen Frequenzbestandteile, was *Spektrum* genannt wird. Wir verwenden dazu das `fft` Package von `scipy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "def apply_fourier_transform(points: np.ndarray):\n",
    "    points_fourier_transform = scipy.fft.rfft(points)\n",
    "    winsize = len(points_fourier_transform) * 2 - 2\n",
    "    freqs = scipy.fft.fftfreq(winsize)[:len(points_fourier_transform)]\n",
    "    data = 2 * np.abs(points_fourier_transform) / len(points)\n",
    "    return pd.Series(data, index=freqs)\n",
    "\n",
    "def plot_spectrum(ft_points: Union[np.ndarray, pd.Series], sampling_rate: int, max_freq: float = None):\n",
    "    if isinstance(ft_points, pd.Series):\n",
    "        ft_points = ft_points.values\n",
    "    winsize = len(ft_points) * 2 - 2\n",
    "    freqs = scipy.fft.fftfreq(winsize)[:len(ft_points)] * sampling_rate\n",
    "    if max_freq:\n",
    "        max_bin = np.searchsorted(freqs, max_freq)\n",
    "        freqs = freqs[:max_bin]\n",
    "        ft_points = ft_points[:max_bin]\n",
    "    plt.plot(freqs, ft_points)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 12292 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "f = 440\n",
    "T = 0.5\n",
    "sampling_rate = 24000\n",
    "\n",
    "signal = generate_sine(f, T, sampling_rate)\n",
    "\n",
    "fourier_transform = apply_fourier_transform(signal)\n",
    "plot_spectrum(fourier_transform, sampling_rate, max_freq=1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Nun erstellen wir einen etwas komplizierteren Ton\n",
    "T = 1\n",
    "sampling_rate = 24000\n",
    "\n",
    "signal_1 = generate_sine(180, T, sampling_rate)\n",
    "signal_2 = generate_sine(240, T, sampling_rate)\n",
    "signal_3 = generate_sine(330, T, sampling_rate)\n",
    "\n",
    "signal = signal_1 + signal_2 + signal_3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 14208 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fourier_transform = apply_fourier_transform(signal)\n",
    "plot_spectrum(fourier_transform, sampling_rate, max_freq=500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "vscode": {
     "languageId": "powershell"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 21196 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Wir können auch die Fourier Transformation für die .wav Datei von vorher bestimmen.\n",
    "sound_path = os.path.join(\"..\", \"resources\", \"piano.wav\")\n",
    "T = 20\n",
    "signal, sampling_rate = read_wav_file(sound_path, time=T)\n",
    "\n",
    "fourier_transform = apply_fourier_transform(signal)\n",
    "plot_spectrum(fourier_transform, sampling_rate, max_freq=5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Wichtig**:\n",
    "* Das Spektrum macht nur bei konstanten Tönen einen Sinn\n",
    "* In unserem Fall ändert sich aber der Ton laufend\n",
    "* Abhilfe schafft hier ein **Spektrogramm**, welches quasi für jeden Zeitpunkt ein Spektrum berechnet (werden wir jetzt nicht näher behandeln)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sequentielle Daten: Text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ein sehr aktuelles Thema in Zeiten von Large Language Models (LLM's) ist die Datenform als Text. Auch hier sind die Datenpunkte geordnet, was wiederum mit einer Folge $(a_n)_{n\\in\\mathbb{N}}$ modelliert werden kann. Dieses mal ist diese Folge aber keine Digitalisierung einer Funktion $f(x)$.\n",
    "\n",
    "Wir werden uns nun etwas genauer mit Textverarbeitung beschäftigen. Sie ist das Hauptthema im Gebiet *Natural Language Processing* (NLP)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wenn wir mit Text arbeiten, dann unterliegt dieser normalerweise einem Vokabular. Dieses basiert sehr stark auf der verwendeten Sprache. Außerdem ist dieses Vokabular sehr riesig, weswegen oft nur die $n$-meistverwendetsten Wörter genutzt werden (zBsp. $n=10000$).\n",
    "\n",
    "Mit dieser Idee könnt nun jedes Wort in diesem Vokabular als *One-Hot*-Vektor dargestellt werden."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Annahme, wir wollen nun die Wörter $\\{\\text{Columbus},\\text{ist},\\text{ein},\\text{lieber},\\text{Kater}\\}$ darstellen als One-Hot Vektoren."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "vocabulary = ['Columbus', 'ist', 'ein', 'lieber', 'Kater']\n",
    "sentence = 'Columbus ist ein lieber Hund'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_to_onehot(vocab: list, sentence: str):\n",
    "    words_sentence = sentence.split()\n",
    "    words_vocab = np.unique(vocab)\n",
    "    one_hot = np.zeros(shape=(len(words_sentence), len(words_vocab)), dtype=np.uint8)\n",
    "    for i, word in enumerate(words_sentence):\n",
    "        try:\n",
    "            one_hot[i, vocab.index(word)] = 1\n",
    "        except Exception as e:\n",
    "            print(e) # Wort ist nicht im vocabulary\n",
    "    return pd.DataFrame(data=one_hot, index=words_sentence)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'Hund' is not in list\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Columbus</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ist</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ein</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lieber</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Hund</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          0  1  2  3  4\n",
       "Columbus  1  0  0  0  0\n",
       "ist       0  1  0  0  0\n",
       "ein       0  0  1  0  0\n",
       "lieber    0  0  0  1  0\n",
       "Hund      0  0  0  0  0"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "convert_to_onehot(vocabulary, sentence)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Da es aber nicht recht intuitiv, bzw. schnell unübersichtlich wird, wenn wir zBsp. beginnen mit $10000$-dimensionalen One-Hot Vektoren zu rechnen ($n$ kann auch viel größer sein), verwendet (und trainiert) man sogenannte **Word Embeddings**.\n",
    "\n",
    "Diese sind kompakte Representationen von den Wörtern, in einem viel kleineren Raum. Sprich wir wandeln unsere Wörter von $\\{0,1\\}^n$ auf $[0,1]^m$ mit $m << n$ um.\n",
    "\n",
    "Dieser neue Raum soll dafür auch ein paar Eigenschaften haben:\n",
    "* Ähnliche Wörter sollen nahe beinander sein\n",
    "* \"Rechnen\" mit Wörtern ist möglich (Cosine Similarity oder König-Mann+Frau $\\approx$ Königin)\n",
    "\n",
    "\n",
    "Im generellen kann man sich Embeddings so vorstellen, wie wenn der Inhalt von Wörtern/Sätzen kompakt repräsentiert wird als $n$-dimensionaler Vektor."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wir verwenden nun so ein Modell, in diesem Fall `word2vec`.\n",
    "\n",
    "Das Embedding kann dann mit dem Befehl `python -m spacy download en_core_web_md` heruntergeladen werden. Wir machen dies im Notebook, könnten es aber genauso in der *miniconda* Konsole machen."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Möglicherweise muss auch die Library `pydantic` erneut installiert werden (war bei mir der Fall), idealerweise mit `pip install pydantic==1.7.4` (Erst Relevant für die Textverarbeitung am Ende des Notebooks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/bash: /home/tikaiz/miniconda3/envs/dsai/lib/libreadline.so.8: no version information available (required by /bin/bash)\n",
      "/bin/bash: /home/tikaiz/miniconda3/envs/dsai/lib/libreadline.so.8: no version information available (required by /bin/bash)\n",
      "/bin/bash: /home/tikaiz/miniconda3/envs/dsai/lib/libreadline.so.8: no version information available (required by /bin/bash)\n",
      "/bin/bash: /home/tikaiz/miniconda3/envs/dsai/lib/libreadline.so.8: no version information available (required by /bin/bash)\n",
      "Collecting en-core-web-md==3.8.0\n",
      "  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_md-3.8.0/en_core_web_md-3.8.0-py3-none-any.whl (33.5 MB)\n",
      "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m33.5/33.5 MB\u001b[0m \u001b[31m15.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[?25hInstalling collected packages: en-core-web-md\n",
      "Successfully installed en-core-web-md-3.8.0\n",
      "\u001b[38;5;2m✔ Download and installation successful\u001b[0m\n",
      "You can now load the package via spacy.load('en_core_web_md')\n"
     ]
    }
   ],
   "source": [
    "# download embedding \n",
    "!python -m spacy download en_core_web_md"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Nun laden wir dieses Embedding-Model\n",
    "word_embedding_german = spacy.load('en_core_web_md')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Nun können wir die Vektoren extrahieren von den vorigen Wörtern\n",
    "columbus = word_embedding_german('Columbus')\n",
    "_is = word_embedding_german('is')\n",
    "a = word_embedding_german('a')\n",
    "lovely = word_embedding_german('lovely')\n",
    "cat = word_embedding_german('cat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(300,)\n"
     ]
    }
   ],
   "source": [
    "# Größe eines Vektors\n",
    "print(columbus.vector.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-0.61506    0.45745    0.34629   -0.42685    0.84782   -0.41435\n",
      "  0.12422    0.13778    0.30993    1.2499    -1.2305    -0.20239\n",
      " -0.2944    -0.21408   -0.63358   -0.34638    0.30566    1.3755\n",
      "  0.024425   0.16306   -0.04864   -0.61891    0.11519   -0.48656\n",
      " -0.10342    0.23771   -0.51147    0.37534   -0.12923    0.29566\n",
      " -0.36304   -0.06235    0.098723   0.34361   -0.085307   0.42652\n",
      "  0.037085   0.31742   -0.74668    0.10692   -0.18141    0.40202\n",
      "  0.56771   -0.11344    0.24492    0.22647    0.19766   -0.2177\n",
      "  0.41926    0.048979  -0.13648    0.024594  -0.10012   -0.086374\n",
      " -0.12419    0.15195    0.20617    0.066046   0.25952    0.060823\n",
      " -0.30025    0.22363    0.39436    0.18966    0.10297    0.5493\n",
      " -0.16654    0.34562   -0.37017   -0.15314   -0.55936    0.11936\n",
      " -0.3041    -0.64677    0.80659    0.19804   -0.72554    0.3686\n",
      " -0.10039    0.061825  -0.35904    0.37567    0.27465    0.41907\n",
      " -0.15196    0.10567    1.5322     0.88811   -0.30164   -0.62204\n",
      "  0.015519  -0.44977   -0.070418  -1.3392     0.088872   0.13014\n",
      "  0.0333    -0.1848     0.36442   -0.080644   0.083077   0.12435\n",
      "  0.31706    0.42273    0.18776   -1.2395    -0.092028  -0.17125\n",
      " -0.51127   -0.47822    0.30558    0.37546    0.37755   -0.42921\n",
      "  0.021247   0.40339    0.60259    0.036205  -0.041036   0.32171\n",
      "  0.37171    0.040198  -0.18334   -0.0447    -0.24585    0.29742\n",
      "  0.2393     0.38284    0.38449    0.20371    0.33737   -0.77606\n",
      " -0.55508   -0.57942    0.14639   -0.1893     0.32992    0.064223\n",
      "  0.47045   -0.42607   -0.98109   -0.10846    0.51672    0.090351\n",
      "  0.021559   0.26452    0.1438     0.46015   -0.82069   -0.24497\n",
      "  0.27639   -0.70956   -0.42903   -0.26765    0.16444   -0.21051\n",
      "  0.19579    0.0090573  0.25458    0.28926    0.29481   -0.21562\n",
      "  0.37987    0.31905    0.014812   0.37282    0.29541    0.20963\n",
      "  0.50274    0.38258    0.60429   -0.15597   -0.33508    0.51761\n",
      "  0.26309   -0.21988   -0.19498    0.31704    0.18338    0.043678\n",
      " -0.18888    0.3809    -0.082466   0.3431     0.42513    0.71107\n",
      " -0.18484    0.15365   -0.66804   -0.11016   -0.087064   0.066326\n",
      "  0.22737   -0.44755   -0.47979    0.35158    0.035813   0.074965\n",
      "  0.65592   -0.14649   -0.22774   -0.68082   -0.21418   -0.75301\n",
      " -0.31699    0.073951  -0.36822    0.45779    0.070358  -0.37969\n",
      "  0.11242    0.66541    0.32094    0.49692   -0.67857    0.040064\n",
      " -0.032593   0.28999   -0.17695    0.10553    0.028307   0.079542\n",
      " -0.067165  -0.24853   -0.13014    0.26031   -0.42556   -0.078152\n",
      "  0.073664   0.2732    -0.25717    0.018035  -0.013583  -0.40746\n",
      " -0.47916    0.16565    0.19592   -0.25108   -0.014654   0.20064\n",
      " -0.071179  -1.0957     0.22794   -0.67024   -0.3586    -0.16928\n",
      "  0.52867    0.30426    0.27346   -0.87561    0.26439   -0.24421\n",
      "  0.074842  -0.18769    0.17039    0.062676  -0.3092     0.063599\n",
      "  0.46017    0.26915   -0.022005  -0.017233   0.15467    0.43704\n",
      " -0.060314  -0.63906    0.015476  -0.045431  -1.0311     0.29663\n",
      " -1.0004    -0.12976   -0.27376    0.50473    0.41841   -0.23903\n",
      "  0.4275    -0.51737    0.037586  -1.0034    -0.086042   0.21325\n",
      " -0.58561    0.57117    0.077634   0.36398    0.18921    0.0028133\n",
      "  0.48844    0.026449   0.1542    -0.25781   -0.18064    0.38123\n",
      "  0.087815  -0.24095    0.62373   -0.35313   -0.56627    0.52642  ]\n"
     ]
    }
   ],
   "source": [
    "print(columbus.vector)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wir können nun aber auch noch die Ähnlichkeit zwischen den Wörtern berechnen\n",
    "car = word_embedding_german('car')\n",
    "bike = word_embedding_german('bike')\n",
    "dog = word_embedding_german('dog')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.22796432673931122"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "car.similarity(bike)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.19304995238780975"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "car.similarity(dog)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Nun verwenden wir noch einige Beispielwörter und lassen uns diese später visualisieren\n",
    "beispiel_woerter = ['dog', 'cat', 'cow', 'jeans', 'dress', 'happy', 'nice', 'king', 'queen', 'woman', 'man', 'school', 'teacher', 'student']\n",
    "\n",
    "dataframe_embeddings = pd.DataFrame(data=[word_embedding_german(word).vector for word in beispiel_woerter], index=beispiel_woerter)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>...</th>\n",
       "      <th>290</th>\n",
       "      <th>291</th>\n",
       "      <th>292</th>\n",
       "      <th>293</th>\n",
       "      <th>294</th>\n",
       "      <th>295</th>\n",
       "      <th>296</th>\n",
       "      <th>297</th>\n",
       "      <th>298</th>\n",
       "      <th>299</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>dog</th>\n",
       "      <td>-0.72483</td>\n",
       "      <td>0.425380</td>\n",
       "      <td>0.025489</td>\n",
       "      <td>-0.398070</td>\n",
       "      <td>0.037463</td>\n",
       "      <td>-0.298110</td>\n",
       "      <td>-0.282790</td>\n",
       "      <td>0.293330</td>\n",
       "      <td>0.577750</td>\n",
       "      <td>1.22050</td>\n",
       "      <td>...</td>\n",
       "      <td>0.019095</td>\n",
       "      <td>-0.342830</td>\n",
       "      <td>-0.586020</td>\n",
       "      <td>0.009563</td>\n",
       "      <td>-0.085052</td>\n",
       "      <td>0.833120</td>\n",
       "      <td>0.319780</td>\n",
       "      <td>0.050317</td>\n",
       "      <td>-0.231590</td>\n",
       "      <td>0.281650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cat</th>\n",
       "      <td>-0.72483</td>\n",
       "      <td>0.425380</td>\n",
       "      <td>0.025489</td>\n",
       "      <td>-0.398070</td>\n",
       "      <td>0.037463</td>\n",
       "      <td>-0.298110</td>\n",
       "      <td>-0.282790</td>\n",
       "      <td>0.293330</td>\n",
       "      <td>0.577750</td>\n",
       "      <td>1.22050</td>\n",
       "      <td>...</td>\n",
       "      <td>0.019095</td>\n",
       "      <td>-0.342830</td>\n",
       "      <td>-0.586020</td>\n",
       "      <td>0.009563</td>\n",
       "      <td>-0.085052</td>\n",
       "      <td>0.833120</td>\n",
       "      <td>0.319780</td>\n",
       "      <td>0.050317</td>\n",
       "      <td>-0.231590</td>\n",
       "      <td>0.281650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cow</th>\n",
       "      <td>-0.95603</td>\n",
       "      <td>0.141260</td>\n",
       "      <td>-0.028987</td>\n",
       "      <td>0.000202</td>\n",
       "      <td>-0.094250</td>\n",
       "      <td>0.685630</td>\n",
       "      <td>0.252780</td>\n",
       "      <td>-0.293350</td>\n",
       "      <td>0.097644</td>\n",
       "      <td>1.30080</td>\n",
       "      <td>...</td>\n",
       "      <td>0.190610</td>\n",
       "      <td>-0.153820</td>\n",
       "      <td>-0.189270</td>\n",
       "      <td>-0.398270</td>\n",
       "      <td>-0.384530</td>\n",
       "      <td>0.194640</td>\n",
       "      <td>0.056178</td>\n",
       "      <td>0.157380</td>\n",
       "      <td>0.233600</td>\n",
       "      <td>0.034716</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>jeans</th>\n",
       "      <td>-0.64995</td>\n",
       "      <td>-0.120890</td>\n",
       "      <td>0.037206</td>\n",
       "      <td>-0.263410</td>\n",
       "      <td>0.286470</td>\n",
       "      <td>0.575020</td>\n",
       "      <td>0.486520</td>\n",
       "      <td>0.050770</td>\n",
       "      <td>0.127320</td>\n",
       "      <td>0.47011</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.344500</td>\n",
       "      <td>0.206400</td>\n",
       "      <td>0.464560</td>\n",
       "      <td>-0.509310</td>\n",
       "      <td>0.366480</td>\n",
       "      <td>0.418830</td>\n",
       "      <td>0.675610</td>\n",
       "      <td>0.488660</td>\n",
       "      <td>-0.261030</td>\n",
       "      <td>-0.498360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dress</th>\n",
       "      <td>-0.90833</td>\n",
       "      <td>0.187590</td>\n",
       "      <td>-0.483800</td>\n",
       "      <td>0.646020</td>\n",
       "      <td>0.571520</td>\n",
       "      <td>-0.632710</td>\n",
       "      <td>0.025627</td>\n",
       "      <td>-0.101550</td>\n",
       "      <td>0.063692</td>\n",
       "      <td>0.29793</td>\n",
       "      <td>...</td>\n",
       "      <td>0.651420</td>\n",
       "      <td>0.080100</td>\n",
       "      <td>0.450930</td>\n",
       "      <td>0.408230</td>\n",
       "      <td>0.307260</td>\n",
       "      <td>-0.350350</td>\n",
       "      <td>0.197770</td>\n",
       "      <td>-0.556100</td>\n",
       "      <td>0.013015</td>\n",
       "      <td>-0.367050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>happy</th>\n",
       "      <td>-0.71589</td>\n",
       "      <td>0.342390</td>\n",
       "      <td>-0.776760</td>\n",
       "      <td>0.262120</td>\n",
       "      <td>-0.103790</td>\n",
       "      <td>0.367260</td>\n",
       "      <td>0.467000</td>\n",
       "      <td>-0.144060</td>\n",
       "      <td>0.577090</td>\n",
       "      <td>2.59080</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.539270</td>\n",
       "      <td>-0.068958</td>\n",
       "      <td>-0.599250</td>\n",
       "      <td>0.022476</td>\n",
       "      <td>0.488180</td>\n",
       "      <td>-0.004069</td>\n",
       "      <td>0.412400</td>\n",
       "      <td>-0.311280</td>\n",
       "      <td>-0.192970</td>\n",
       "      <td>0.242070</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>nice</th>\n",
       "      <td>-0.73351</td>\n",
       "      <td>0.413920</td>\n",
       "      <td>-0.442500</td>\n",
       "      <td>-0.291270</td>\n",
       "      <td>-0.096179</td>\n",
       "      <td>0.097562</td>\n",
       "      <td>0.131510</td>\n",
       "      <td>-0.518250</td>\n",
       "      <td>0.106710</td>\n",
       "      <td>2.41440</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.304850</td>\n",
       "      <td>0.186480</td>\n",
       "      <td>-0.270480</td>\n",
       "      <td>-0.111110</td>\n",
       "      <td>0.126810</td>\n",
       "      <td>0.395380</td>\n",
       "      <td>-0.056084</td>\n",
       "      <td>-0.187770</td>\n",
       "      <td>-0.076822</td>\n",
       "      <td>-0.015507</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>king</th>\n",
       "      <td>-0.60644</td>\n",
       "      <td>-0.512050</td>\n",
       "      <td>0.006492</td>\n",
       "      <td>-0.291940</td>\n",
       "      <td>-0.565150</td>\n",
       "      <td>-0.115230</td>\n",
       "      <td>0.077274</td>\n",
       "      <td>0.335610</td>\n",
       "      <td>0.115930</td>\n",
       "      <td>2.35160</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.712360</td>\n",
       "      <td>0.294900</td>\n",
       "      <td>-0.214790</td>\n",
       "      <td>0.256580</td>\n",
       "      <td>-0.193580</td>\n",
       "      <td>1.105700</td>\n",
       "      <td>0.228620</td>\n",
       "      <td>0.218590</td>\n",
       "      <td>-0.190440</td>\n",
       "      <td>-0.102530</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>queen</th>\n",
       "      <td>-1.12090</td>\n",
       "      <td>-0.288090</td>\n",
       "      <td>0.123360</td>\n",
       "      <td>-0.216080</td>\n",
       "      <td>-0.098690</td>\n",
       "      <td>-0.241770</td>\n",
       "      <td>0.070938</td>\n",
       "      <td>-0.274450</td>\n",
       "      <td>-0.200250</td>\n",
       "      <td>1.33570</td>\n",
       "      <td>...</td>\n",
       "      <td>0.545290</td>\n",
       "      <td>0.087036</td>\n",
       "      <td>-0.527260</td>\n",
       "      <td>-0.328620</td>\n",
       "      <td>0.138660</td>\n",
       "      <td>0.366820</td>\n",
       "      <td>-0.053972</td>\n",
       "      <td>-0.009001</td>\n",
       "      <td>-0.197320</td>\n",
       "      <td>-0.127800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>woman</th>\n",
       "      <td>-0.62299</td>\n",
       "      <td>-0.313320</td>\n",
       "      <td>-0.014659</td>\n",
       "      <td>0.144490</td>\n",
       "      <td>-0.143000</td>\n",
       "      <td>-0.611890</td>\n",
       "      <td>0.009913</td>\n",
       "      <td>0.105850</td>\n",
       "      <td>-0.216510</td>\n",
       "      <td>1.18710</td>\n",
       "      <td>...</td>\n",
       "      <td>0.321240</td>\n",
       "      <td>0.064128</td>\n",
       "      <td>0.486100</td>\n",
       "      <td>-0.507630</td>\n",
       "      <td>0.155070</td>\n",
       "      <td>0.047091</td>\n",
       "      <td>-0.081585</td>\n",
       "      <td>-0.482580</td>\n",
       "      <td>0.049814</td>\n",
       "      <td>-0.402190</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>man</th>\n",
       "      <td>-0.60482</td>\n",
       "      <td>0.373990</td>\n",
       "      <td>0.284460</td>\n",
       "      <td>-0.016269</td>\n",
       "      <td>-0.118930</td>\n",
       "      <td>-0.197060</td>\n",
       "      <td>-0.351180</td>\n",
       "      <td>-0.065699</td>\n",
       "      <td>-0.218660</td>\n",
       "      <td>3.22520</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.284210</td>\n",
       "      <td>-0.209050</td>\n",
       "      <td>-0.240500</td>\n",
       "      <td>-0.156590</td>\n",
       "      <td>-0.381150</td>\n",
       "      <td>-0.340890</td>\n",
       "      <td>-0.672440</td>\n",
       "      <td>-0.136010</td>\n",
       "      <td>0.303780</td>\n",
       "      <td>-0.084864</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>school</th>\n",
       "      <td>-0.65605</td>\n",
       "      <td>0.483440</td>\n",
       "      <td>-0.083531</td>\n",
       "      <td>0.338010</td>\n",
       "      <td>0.095574</td>\n",
       "      <td>-0.056138</td>\n",
       "      <td>0.347310</td>\n",
       "      <td>0.291710</td>\n",
       "      <td>-0.429500</td>\n",
       "      <td>1.48490</td>\n",
       "      <td>...</td>\n",
       "      <td>0.432600</td>\n",
       "      <td>-0.057419</td>\n",
       "      <td>-0.026134</td>\n",
       "      <td>0.498960</td>\n",
       "      <td>0.313360</td>\n",
       "      <td>0.458440</td>\n",
       "      <td>-0.554270</td>\n",
       "      <td>-0.226080</td>\n",
       "      <td>0.265790</td>\n",
       "      <td>-0.154570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>teacher</th>\n",
       "      <td>-0.65605</td>\n",
       "      <td>0.483440</td>\n",
       "      <td>-0.083531</td>\n",
       "      <td>0.338010</td>\n",
       "      <td>0.095574</td>\n",
       "      <td>-0.056138</td>\n",
       "      <td>0.347310</td>\n",
       "      <td>0.291710</td>\n",
       "      <td>-0.429500</td>\n",
       "      <td>1.48490</td>\n",
       "      <td>...</td>\n",
       "      <td>0.432600</td>\n",
       "      <td>-0.057419</td>\n",
       "      <td>-0.026134</td>\n",
       "      <td>0.498960</td>\n",
       "      <td>0.313360</td>\n",
       "      <td>0.458440</td>\n",
       "      <td>-0.554270</td>\n",
       "      <td>-0.226080</td>\n",
       "      <td>0.265790</td>\n",
       "      <td>-0.154570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>student</th>\n",
       "      <td>-0.96802</td>\n",
       "      <td>-0.019952</td>\n",
       "      <td>-0.276190</td>\n",
       "      <td>-0.192280</td>\n",
       "      <td>0.194890</td>\n",
       "      <td>-0.077457</td>\n",
       "      <td>-0.423130</td>\n",
       "      <td>-0.284030</td>\n",
       "      <td>0.140500</td>\n",
       "      <td>2.65290</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.850620</td>\n",
       "      <td>0.823750</td>\n",
       "      <td>0.158610</td>\n",
       "      <td>0.100800</td>\n",
       "      <td>0.599730</td>\n",
       "      <td>-0.261220</td>\n",
       "      <td>0.307910</td>\n",
       "      <td>0.072593</td>\n",
       "      <td>-0.121590</td>\n",
       "      <td>0.130530</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>14 rows × 300 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             0         1         2         3         4         5         6    \\\n",
       "dog     -0.72483  0.425380  0.025489 -0.398070  0.037463 -0.298110 -0.282790   \n",
       "cat     -0.72483  0.425380  0.025489 -0.398070  0.037463 -0.298110 -0.282790   \n",
       "cow     -0.95603  0.141260 -0.028987  0.000202 -0.094250  0.685630  0.252780   \n",
       "jeans   -0.64995 -0.120890  0.037206 -0.263410  0.286470  0.575020  0.486520   \n",
       "dress   -0.90833  0.187590 -0.483800  0.646020  0.571520 -0.632710  0.025627   \n",
       "happy   -0.71589  0.342390 -0.776760  0.262120 -0.103790  0.367260  0.467000   \n",
       "nice    -0.73351  0.413920 -0.442500 -0.291270 -0.096179  0.097562  0.131510   \n",
       "king    -0.60644 -0.512050  0.006492 -0.291940 -0.565150 -0.115230  0.077274   \n",
       "queen   -1.12090 -0.288090  0.123360 -0.216080 -0.098690 -0.241770  0.070938   \n",
       "woman   -0.62299 -0.313320 -0.014659  0.144490 -0.143000 -0.611890  0.009913   \n",
       "man     -0.60482  0.373990  0.284460 -0.016269 -0.118930 -0.197060 -0.351180   \n",
       "school  -0.65605  0.483440 -0.083531  0.338010  0.095574 -0.056138  0.347310   \n",
       "teacher -0.65605  0.483440 -0.083531  0.338010  0.095574 -0.056138  0.347310   \n",
       "student -0.96802 -0.019952 -0.276190 -0.192280  0.194890 -0.077457 -0.423130   \n",
       "\n",
       "              7         8        9    ...       290       291       292  \\\n",
       "dog      0.293330  0.577750  1.22050  ...  0.019095 -0.342830 -0.586020   \n",
       "cat      0.293330  0.577750  1.22050  ...  0.019095 -0.342830 -0.586020   \n",
       "cow     -0.293350  0.097644  1.30080  ...  0.190610 -0.153820 -0.189270   \n",
       "jeans    0.050770  0.127320  0.47011  ... -0.344500  0.206400  0.464560   \n",
       "dress   -0.101550  0.063692  0.29793  ...  0.651420  0.080100  0.450930   \n",
       "happy   -0.144060  0.577090  2.59080  ... -0.539270 -0.068958 -0.599250   \n",
       "nice    -0.518250  0.106710  2.41440  ... -0.304850  0.186480 -0.270480   \n",
       "king     0.335610  0.115930  2.35160  ... -0.712360  0.294900 -0.214790   \n",
       "queen   -0.274450 -0.200250  1.33570  ...  0.545290  0.087036 -0.527260   \n",
       "woman    0.105850 -0.216510  1.18710  ...  0.321240  0.064128  0.486100   \n",
       "man     -0.065699 -0.218660  3.22520  ... -0.284210 -0.209050 -0.240500   \n",
       "school   0.291710 -0.429500  1.48490  ...  0.432600 -0.057419 -0.026134   \n",
       "teacher  0.291710 -0.429500  1.48490  ...  0.432600 -0.057419 -0.026134   \n",
       "student -0.284030  0.140500  2.65290  ... -0.850620  0.823750  0.158610   \n",
       "\n",
       "              293       294       295       296       297       298       299  \n",
       "dog      0.009563 -0.085052  0.833120  0.319780  0.050317 -0.231590  0.281650  \n",
       "cat      0.009563 -0.085052  0.833120  0.319780  0.050317 -0.231590  0.281650  \n",
       "cow     -0.398270 -0.384530  0.194640  0.056178  0.157380  0.233600  0.034716  \n",
       "jeans   -0.509310  0.366480  0.418830  0.675610  0.488660 -0.261030 -0.498360  \n",
       "dress    0.408230  0.307260 -0.350350  0.197770 -0.556100  0.013015 -0.367050  \n",
       "happy    0.022476  0.488180 -0.004069  0.412400 -0.311280 -0.192970  0.242070  \n",
       "nice    -0.111110  0.126810  0.395380 -0.056084 -0.187770 -0.076822 -0.015507  \n",
       "king     0.256580 -0.193580  1.105700  0.228620  0.218590 -0.190440 -0.102530  \n",
       "queen   -0.328620  0.138660  0.366820 -0.053972 -0.009001 -0.197320 -0.127800  \n",
       "woman   -0.507630  0.155070  0.047091 -0.081585 -0.482580  0.049814 -0.402190  \n",
       "man     -0.156590 -0.381150 -0.340890 -0.672440 -0.136010  0.303780 -0.084864  \n",
       "school   0.498960  0.313360  0.458440 -0.554270 -0.226080  0.265790 -0.154570  \n",
       "teacher  0.498960  0.313360  0.458440 -0.554270 -0.226080  0.265790 -0.154570  \n",
       "student  0.100800  0.599730 -0.261220  0.307910  0.072593 -0.121590  0.130530  \n",
       "\n",
       "[14 rows x 300 columns]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe_embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 27572 characters]",
      "text/plain": [
       "<Figure size 1000x1000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 2D Scatter Plot inklusive Beschriftung der Wörter mit Hilfe von PCA\n",
    "\n",
    "# Hier passieren ein paar Dinge, die wir noch nicht im Detail kennen gelernt haben. Wir werden unter anderem PCA noch im Sommersemester kennen lernen.\n",
    "\n",
    "pca = PCA(n_components=2)\n",
    "dataframe_embeddings_pca = pca.fit_transform(dataframe_embeddings)\n",
    "\n",
    "plt.figure(figsize=(10, 10))\n",
    "plt.scatter(dataframe_embeddings_pca[:, 0], dataframe_embeddings_pca[:, 1])\n",
    "for i, word in enumerate(beispiel_woerter):\n",
    "    plt.annotate(word, (dataframe_embeddings_pca[i, 0], dataframe_embeddings_pca[i, 1]))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Funktioniert es gut?\n",
    "\n",
    "Mögliche Gründe:\n",
    "* Zu kleine Dimension im Embedding Space\n",
    "* Schlecht trainiertes Embedding\n",
    "* Wörter sind vielleicht nicht so ähnlich wie geglaubt\n",
    "* Dimensionsreduzierung scheitert"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Abschließend können wir noch die Wörter in den Beispiel Wörtern finden, welche am ähnlichsten zu einem gegebenen Wort sind\n",
    "\n",
    "def find_similar_words(query: np.ndarray, data: pd.DataFrame):\n",
    "    norms = data.apply(np.square).sum(axis=1).apply(np.sqrt)\n",
    "    data = data.apply(lambda x: x / norms).dot(query).sort_values()\n",
    "    similar = data.index[:-4:-1].to_list()\n",
    "    distant = data.index[:3].to_list()\n",
    "    return similar, distant"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "word = 'cat'\n",
    "\n",
    "similar, distant = find_similar_words(word_embedding_german(word).vector, dataframe_embeddings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['dog', 'cat', 'cow']\n",
      "['student', 'teacher', 'school']\n"
     ]
    }
   ],
   "source": [
    "print(similar)\n",
    "print(distant)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `Glob` Package"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Abschließend wollen wir noch kurz diskutieren, wie man Dateien rekursiv in einem Ordner suchen kann. Dafür verwenden wir das `glob` Package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Alle Jupyter-Notebooks in dem Über-Übergeoardneten Ordner finden\n",
    "\n",
    "path = os.path.join(\"..\", \"..\", \"**\", \"*.ipynb\")\n",
    "file_list = glob.glob(path, recursive=True) # recursive = True hier sehr wichtig, ansonsten keine Dateien"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "../../01_Python_Jupyter/code/00/einfuehrung_jupyter.ipynb\n",
      "../../02_daten_tabellarisch/code/daten_tabellarisch_1.ipynb\n",
      "../../02_daten_tabellarisch/code/daten_tabellarisch_2.ipynb\n",
      "../../02_daten_tabellarisch/code/daten_tabellarisch_3.ipynb\n",
      "../../02_daten_tabellarisch/code/daten_tabellarisch_4.ipynb\n",
      "../../02_daten_tabellarisch/code/daten_tabellarisch_5.ipynb\n",
      "../../03_Visualisierungen/code/visualisierung_1.ipynb\n",
      "../../03_Visualisierungen/code/visualisierung_2.ipynb\n",
      "../../03_Visualisierungen/code/visualisierung_3.ipynb\n",
      "../../03_Visualisierungen/code/visualisierung_4.ipynb\n",
      "../../04_weitere_datenformen/code/datenformen_1.ipynb\n"
     ]
    }
   ],
   "source": [
    "for file in file_list:\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dsai",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
