
INDEX
Noise-contrastive estimation, 612
Nonparametric model, 111
Norm, xiv, 36
Normal distribution, 60, 61, 122
Normal equations, 106, 106, 109, 228
Normalized initialization, 296
Numerical differentiation, see finite differ-
ences
Object detection, 443
Object recognition, 443
Objective function, 79
OMP-k, see orthogonal matching pursuit
One-shot learning, 530
Operation, 199
Optimization, 77, 79
Orthodox statistics, see frequentist statistics
Orthogonal matching pursuit, 23, 250
Orthogonal matrix, 39
Orthogonality, 38
Output layer, 162
Parallel distributed processing, 16
Parameter initialization, 294, 397
Parameter sharing, 247, 328, 364, 366, 379
Parameter tying, see Parameter sharing
Parametric model, 111
Parametric ReLU, 187
Partial derivative, 81
Partition function, 560, 597, 659
PCA, see principal components analysis
PCD, see stochastic maximum likelihood
Perceptron, 14, 23
Persistent contrastive divergence, see stochas-
tic maximum likelihood
Perturbation analysis, see reparametrization
trick
Point estimator, 119
Policy, 470
Pooling, 323, 672
Positive definite, 86
Positive phase, 460, 598, 600, 646, 658
Precision, 414
Precision (of a normal distribution), 60, 62
Predictive sparse decomposition, 515
Preprocessing, 444
Pretraining, 316, 520
Primary visual cortex, 355
Principal components analysis, 44, 143, 144,
480, 622
Prior probability distribution, 132
Probabilistic max pooling, 673
Probabilistic PCA, 480, 481, 623
Probability density function, 55
Probability distribution, 53
Probability mass function, 53
Probability mass function estimation, 100
Product of experts, 562
Product rule of probability, see chain rule
of probability
PSD, see predictive sparse decomposition
Pseudolikelihood, 607
Quadrature pair, 360
Quasi-Newton methods, 309
Radial basis function, 191
Random search, 425
Random variable, 53
Ratio matching, 610
RBF, 191
RBM, see restricted Boltzmann machine
Recall, 414
Receptive field, 329
Recommender Systems, 468
Rectified linear unit, 166, 187, 416, 498
Recurrent network, 23
Recurrent neural network, 369
Regression, 97
Regularization, 117, 117, 172, 222, 421
Regularizer, 116
REINFORCE, 679
Reinforcement learning, 25, 103, 470, 678
Relational database, 473
Relations, 473
Reparametrization trick, 678
Representation learning, 3
Representational capacity, 111
Restricted Boltzmann machine, 347, 450,
470, 578, 622, 646, 647, 661, 666,
779