Learning Deep Architectures for AI

Theoretical results suggest that in order to learn the kind of complicated

functions that can represent high-level abstractions (e.g., in

vision, language, and other AI-level tasks), one may need deep architectures.

Deep architectures are composed of multiple levels of non-linear

operations, such as in neural nets with many hidden layers or in complicated

propositional formulae re-using many sub-formulae. Searching

the parameter space of deep architectures is a difficult task, but learning

algorithms such as those for Deep Belief Networks have recently been

proposed to tackle this problem with notable success, beating the stateof-

the-art in certain areas. This monograph discusses the motivations

and principles regarding learning algorithms for deep architectures, in

particular those exploiting as building blocks unsupervised learning of

single-layer models such as Restricted Boltzmann Machines, used to

construct deeper models such as Deep Belief Networks.