Part II
Deep Networks: Modern
This part of the book summarizes the state of modern deep learning as it is
used to solve practical applications.
Deep learning has a long history and many aspirations. Several proposed
approaches have yet to entirely bear fruit. Several ambitious goals have yet to be
realized. These less-developed branches of deep learning appear in the final part of
the book.
This part focuses only on those approaches that are essentially working tech-
nologies that are already used heavily in industry.
Modern deep learning provides a powerful framework for supervised learning.
By adding more layers and more units within a layer, a deep network can represent
functions of increasing complexity. Most tasks that consist of mapping an input
vector to an output vector, and that are easy for a person to do rapidly, can be
accomplished via deep learning, given sufficiently large models and sufficiently
large datasets of labeled training examples. Other tasks, that cannot be described
as associating one vector to another, or that are difficult enough that a person
would require time to think and reflect in order to accomplish the task, remain
beyond the scope of deep learning for now.
This part of the book describes the core parametric function approximation
technology that is behind nearly all modern practical applications of deep learning.
We begin by describing the feedforward deep network model that is used to
represent these functions. Next, we present advanced techniques for regularization
and optimization of such models. Scaling these models to large inputs such as high-
resolution images or long temporal sequences requires specialization. We introduce
the convolutional network for scaling to large images and the recurrent neural
network for processing temporal sequences. Finally, we present general guidelines
for the practical methodology involved in designing, building, and configuring an
application involving deep learning and review some of its applications.
These chapters are the most important for a practitioner—someone who wants
to begin implementing and using deep learning algorithms to solve real-world
problems today.