ML Cheatsheet

Basics

  • Linear Regression
    • Introduction
    • Simple regression
      • Making predictions
      • Cost function
      • Gradient descent
      • Training
      • Model evaluation
      • Summary
    • Multivariable regression
      • Growing complexity
      • Normalization
      • Making predictions
      • Initialize weights
      • Cost function
      • Gradient descent
      • Simplifying with matrices
      • Bias term
      • Model evaluation
  • Gradient Descent
    • Introduction
    • Learning rate
    • Cost function
    • Step-by-step
  • Logistic Regression
    • Introduction
      • Comparison to linear regression
      • Types of logistic regression
    • Binary logistic regression
      • Sigmoid activation
      • Decision boundary
      • Making predictions
      • Cost function
      • Gradient descent
      • Mapping probabilities to classes
      • Training
      • Model evaluation
    • Multiclass logistic regression
      • Procedure
      • Softmax activation
      • Scipy example
  • Glossary

Math

  • Calculus
    • Introduction
    • Derivatives
      • Geometric definition
      • Taking the derivative
      • Step-by-step
      • Machine learning use cases
    • Chain rule
      • How It Works
      • Step-by-step
      • Multiple functions
    • Gradients
      • Partial derivatives
      • Step-by-step
      • Directional derivatives
      • Useful properties
    • Integrals
      • Computing integrals
      • Applications of integration
        • Computing probabilities
        • Expected value
        • Variance
  • Linear Algebra
    • Vectors
      • Notation
      • Vectors in geometry
      • Scalar operations
      • Elementwise operations
      • Dot product
      • Hadamard product
      • Vector fields
    • Matrices
      • Dimensions
      • Scalar operations
      • Elementwise operations
      • Hadamard product
      • Matrix transpose
      • Matrix multiplication
      • Test yourself
    • Numpy
      • Dot product
      • Broadcasting
  • Probability (TODO)
  • Statistics (TODO)
  • Notation
    • Algebra
    • Calculus
    • Linear algebra
    • Probability
    • Set theory
    • Statistics

Neural Networks

  • Concepts
    • Neural Network
    • Neuron
    • Synapse
    • Weights
    • Bias
    • Layers
    • Weighted Input
    • Activation Functions
    • Loss Functions
    • Optimization Algorithms
  • Forwardpropagation
    • Simple Network
      • Steps
      • Code
    • Larger Network
      • Architecture
      • Weight Initialization
      • Bias Terms
      • Working with Matrices
      • Dynamic Resizing
      • Refactoring Our Code
      • Final Result
  • Backpropagation
    • Chain rule refresher
    • Applying the chain rule
    • Saving work with memoization
    • Code example
  • Activation Functions
    • Linear
    • ELU
    • ReLU
    • LeakyReLU
    • Sigmoid
    • Tanh
    • Softmax
  • Layers
    • BatchNorm
    • Convolution
    • Dropout
    • Linear
    • LSTM
    • Pooling
    • RNN
  • Loss Functions
    • Cross-Entropy
    • Hinge
    • Huber
    • Kullback-Leibler
    • MAE (L1)
    • MSE (L2)
  • Optimizers
    • Adadelta
    • Adagrad
    • Adam
    • Conjugate Gradients
    • BFGS
    • Momentum
    • Nesterov Momentum
    • Newton’s Method
    • RMSProp
    • SGD
  • Regularization
    • Data Augmentation
    • Dropout
    • Early Stopping
    • Ensembling
    • Injecting Noise
    • L1 Regularization
    • L2 Regularization
  • Architectures
    • Autoencoder
    • CNN
    • GAN
    • MLP
    • RNN
    • VAE

Algorithms (TODO)

  • Classification
    • Bayesian
    • Boosting
    • Decision Trees
    • K-Nearest Neighbor
    • Logistic Regression
    • Random Forests
    • Support Vector Machines
  • Clustering
    • Centroid
    • Density
    • Distribution
    • Hierarchical
    • K-Means
    • Mean shift
  • Regression
    • Lasso
    • Linear
    • Ordinary Least Squares
    • Polynomial
    • Ridge
    • Splines
    • Stepwise
  • Reinforcement Learning

Resources

  • Datasets
  • Libraries
  • Papers
  • Other

Contributing

  • How to contribute
ML Cheatsheet
  • Docs »
  • Regularization
  • Edit on GitHub

Regularization¶

  • Data Augmentation
  • Dropout
  • Early Stopping
  • Ensembling
  • Injecting Noise
  • L1 Regularization
  • L2 Regularization

Techniques for combating overfitting and improving training.

Data Augmentation¶

Be the first to contribute!

Dropout¶

Be the first to contribute!

Early Stopping¶

Be the first to contribute!

Ensembling¶

Be the first to contribute!

Injecting Noise¶

Be the first to contribute!

L1 Regularization¶

Be the first to contribute!

L2 Regularization¶

Be the first to contribute!

References

[1]http://www.deeplearningbook.org/contents/regularization.html
Next Previous

© Copyright 2017. Revision 31739f24.

Built with Sphinx using a theme provided by Read the Docs.