Jeromy's Notes

Deep Learning Basics

1 min read

History

  1. 1958: Perceptron (linear model)
  2. 1959: Perceptron has limitation
  3. 1980: Multi-layer perceptron
    • Do not have sinificant difference from DNN today.
  4. 1986: Backpropagation
    • Usually more than 3 hidden layers is not helpful
  5. 1989: 1 hidden layer is “good enough”, why deep?
  6. 2006: RBM initialization (breakthrough)
  7. 2009: GPU
  8. 2011: Start to be popular in speech recognition
  9. win ILSVRC image competition

Fully Connect Feedforward Network

References

  1. Regression
  2. https://speech.ee.ntu.edu.tw/~tlkagk/courses/ML_2016/Lecture/DL (v2).pdf

Graph View