Deep learning is a subset of machine learning: a new take on learning from data that puts an emphasis on learning successive layers of increasingly meaningful representations. But what does the deep in deep learning mean?
"The deep in deep learning isn't a reference to any kind of deeper understanding achieved by the approach; rather, it stands for this idea of successive layers of representations."
– F. Chollet, Lead Developer of Keras
The depth of the model is indicative of how many layers of such representations we use. F Chollet suggested layered representations learning and hierarchical representations learning as better names for this. Another name could have been differentiable programming.
The term differentiable programming, coined by Yann LeCun, stems from the fact that what our deep learning methods have in common is not...