Machine Learning Algorithms From Scratch

The sole purpose of this repository is to understand the concepts of machine learning algorithms - how they work under their hood.

The code was not designed to be used in production (poorly documented, has no unit tests) - for that we have Scikit-learn and other optimized ML frameworks.

Jupyter Notebooks

1. K-nearest neighbors implementation
2. Linear Regression implementation
3. Decision Trees investigation
4. Dimensionality reduction
   1. SVD investigation
   2. PCA implementation
5. Clustering
   1. K-Means implementation implementation

Resources

• MLcourse.ai
• Machine Learning in Action by Peter Harrington
• Machine Learning with PyTorch and Scikit-Learn by Sebastian Raschka
• Scikit-Learn documentation
• StatQuest videos
• ...

TO DO

• Add implementation with examples for the next algorithms:
  • Linear regression using SGD with regularization
  • Linear regression using formula through SVD
  • Logistic regression using SGD with regularization with logistic loss (labels 0/1)
  • Logistic regression using SGD with regularization (labels -1/1)
  • word2vec implementation with SGD optimization and the choice of training method (naive softmax, negative sampling)
  • Decision Tree implementation for regression and classification