EBM_Deep_Learning_on_Audio_08_Predicting_Class.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Learning in Audio Classification in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predicting Sick and Non-Sick Audio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "What's the Execution Plan?\n",
    "- The data is in the directory Dataset\n",
    "  - further in the directories: 'Train' 'Test' and 'Validation'\n",
    "- Each Set has two directories named by the dataset classes\n",
    "\n",
    "What's the dataset Size?\n",
    "- Its Big !!!\n",
    "\n",
    "Is it Big Data Problem?\n",
    "- Yes\n",
    "\n",
    "Do I have resources to use hadoop/aws?\n",
    "- no, I'm in Lockdown and limited time and knowledge is a concern for me!!\n",
    "\n",
    "What's the solution?\n",
    "- Have to use my old Intel i3 core :/ laptop to do some basic template\n",
    "- once I get internet access, I'll use the template to run on Google's Colab =')\n",
    "- After debugging, I'll increase the full dataset and re-run the program files for visualizaton, model training :O\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Importing Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.io import wavfile"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from python_speech_features import mfcc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Anaconda3\\lib\\site-packages\\h5py\\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
      "  from ._conv import register_converters as _register_converters\n",
      "Using TensorFlow backend.\n",
      "C:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n",
      "C:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n",
      "C:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n",
      "C:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n",
      "C:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n",
      "C:\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n",
      "  np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n"
     ]
    }
   ],
   "source": [
    "from keras.models import load_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import classification_report, confusion_matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### User Defined"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def build_predictions(audio_dir):\n",
    "    y_true = []\n",
    "    y_pred = []\n",
    "    \n",
    "    fn_prob = {}\n",
    "    \n",
    "    print('Extracting features from audio')\n",
    "    \n",
    "    for fn in tqdm(os.listdir(audio_dir)):\n",
    "        rate, wav = wavfile.read(os.path.join(audio_dir, fn))\n",
    "        Class = fn2class[fn]\n",
    "        c = classes.index(Class)\n",
    "        y_prob = []\n",
    "        \n",
    "        for i in range(0,wav.shape[0] - config.step, config.step):\n",
    "            sample = wav[i : i+config.step]\n",
    "            x = mfcc(sample, rate, numcep=config.nfeat, nfilt=config.nfilt, nfft=config.nfft)\n",
    "            \n",
    "            x = (x - config.min) / (config.max - config.min)\n",
    "            \n",
    "            if config.mode == 'conv':\n",
    "                x = x.reshape(1, x.shape[0], x.shape[1], 1)\n",
    "            elif config.mode == 'time':\n",
    "                x = np.expand_dims(x, axis = 0)\n",
    "            \n",
    "            y_hat = model.predict(x)\n",
    "            y_prob.append(y_hat)\n",
    "            y_pred.append(np.argmax(y_hat))\n",
    "            y_true.append(c)\n",
    "            \n",
    "        fn_prob[fn] = np.mean(y_prob, axis = 0).flatten()\n",
    "    return y_true, y_pred, fn_prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Config:\n",
    "    def __init__(self, mode= 'conv', nfilt=26, nfeat=13, nfft = 2048, rate = 16000):\n",
    "        self.mode = mode\n",
    "        self.nfilt = nfilt\n",
    "        self.nfeat = nfeat\n",
    "        self.nfft = nfft\n",
    "        self.rate = rate\n",
    "        self.step = int(rate/10)\n",
    "        self.model_path = os.path.join('models', mode + '.model')\n",
    "        self.p_path = os.path.join('pickles', mode + '.p')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Extraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['test', 'train', 'validation']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.listdir('Temp_Dataset/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of Classes in the Data Set: 2 Classes\n",
      "The classes of the dataset are   : not_sick , sick\n"
     ]
    }
   ],
   "source": [
    "classes = list(os.listdir('Temp_Dataset/validation/'))\n",
    "\n",
    "print(\"Number of Classes in the Data Set:\", len(classes), \"Classes\")\n",
    "print(\"The classes of the dataset are   :\", classes[0], \",\", classes[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating the dataframe with basic column names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Index: 0 entries\n",
      "Data columns (total 3 columns):\n",
      "Fname     0 non-null object\n",
      "Class     0 non-null object\n",
      "Length    0 non-null object\n",
      "dtypes: object(3)\n",
      "memory usage: 0.0+ bytes\n"
     ]
    }
   ],
   "source": [
    "column_names = ['Fname','Class', 'Length']\n",
    "df = pd.DataFrame(columns = column_names)\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dataset_directory = 'Dataset/validation/'\n",
    "dataset_directory = 'Temp_Dataset/train/'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of files in the directory 'not_sick' are 385\n",
      "Number of files in the directory 'sick' are 313\n"
     ]
    }
   ],
   "source": [
    "for c in list(classes):\n",
    "    print('Number of files in the directory \\'{}\\' are {}'.format(c,len(os.listdir(dataset_directory+c))))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "385it [00:01, 333.43it/s]\n",
      "313it [00:00, 340.42it/s]\n"
     ]
    }
   ],
   "source": [
    "for c in list(classes):\n",
    "    for n,f in tqdm(enumerate(os.listdir(dataset_directory+c))):\n",
    "        rate, signal = wavfile.read(dataset_directory+str(c)+\"/\"+str(f))\n",
    "        length = signal.shape[0]/rate\n",
    "        f_df = pd.DataFrame({\n",
    "            \"Fname\": str(f),\n",
    "            \"Class\": str(c),\n",
    "            \"Length\": length}, index = [n])\n",
    "        df = df.append(f_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 698 entries, 0 to 312\n",
      "Data columns (total 3 columns):\n",
      "Fname     698 non-null object\n",
      "Class     698 non-null object\n",
      "Length    698 non-null float64\n",
      "dtypes: float64(1), object(2)\n",
      "memory usage: 21.8+ KB\n"
     ]
    }
   ],
   "source": [
    "df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "fn2class = dict(zip(df.Fname, df.Class))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "698"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(fn2class)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "p_path = os.path.join('pickles', 'time.p')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(p_path, 'rb') as handle:\n",
    "    config = pickle.load(handle)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = load_model(config.model_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      "  0%|                                                  | 0/698 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting features from audio\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|████████████████████████████████████████| 698/698 [11:30<00:00,  1.01it/s]\n"
     ]
    }
   ],
   "source": [
    "y_true, y_pred, fn_prob = build_predictions('Temp_train')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7901531728665208"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "acc_score = accuracy_score(y_true=y_true, y_pred=y_pred)\n",
    "acc_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.90      0.70      0.79     52936\n",
      "          1       0.71      0.91      0.79     43034\n",
      "\n",
      "avg / total       0.81      0.79      0.79     95970\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_true=y_true, y_pred=y_pred))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.85      0.79      0.82     43035\n",
      "          1       0.75      0.81      0.78     33129\n",
      "\n",
      "avg / total       0.80      0.80      0.80     76164\n",
      "\n"
     ]
    }
   ],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_probs = []\n",
    "\n",
    "for i, row in df.iterrows():\n",
    "    y_prob = fn_prob[row.Fname]\n",
    "    y_probs.append(y_prob)\n",
    "    for c, p in zip(classes, y_prob):\n",
    "        df.at[i,c] = p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_predictions = [classes[np.argmax(y)] for y in y_probs]\n",
    "df['y_pred'] = y_predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv('predictions.csv', index = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}