Information Theoretic Learning

This book presents the first cohesive treatment of Information Theoretic Learning (ITL) algorithms to adapt linear or nonlinear learning machines both in supervised or unsupervised paradigms. ITL is a framework where the conventional concepts of second order statistics (covariance, L2 distances, correlation functions) are substituted by scalars and functions with information th...

(展开全部)

This book presents the first cohesive treatment of Information Theoretic Learning (ITL) algorithms to adapt linear or nonlinear learning machines both in supervised or unsupervised paradigms. ITL is a framework where the conventional concepts of second order statistics (covariance, L2 distances, correlation functions) are substituted by scalars and functions with information theoretic underpinnings, respectively entropy, mutual information and correntropy.

ITL quantifies the stochastic structure of the data beyond second order statistics for improved performance without using full-blown Bayesian approaches that require a much larger computational cost. This is possible because of a non-parametric estimator of Renyi’s quadratic entropy that is only a function of pairwise differences between samples. The book compares the performance of ITL algorithms with the second order counterparts in many engineering and machine learning applications.

Students, practitioners and researchers interested in statistical signal processing, computational intelligence, and machine learning will find in this book the theory to understand the basics, the algorithms to implement applications, and exciting but still unexplored leads that will provide fertile ground for future research.

José C. Principe is Distinguished Professor of Electrical and Biomedical Engineering, and BellSouth Professor at the University of Florida, and the Founder and Director of the Computational NeuroEngineering Laboratory. He is an IEEE and AIMBE Fellow, Past President of the International Neural Network Society, Past Editor-in-Chief of the IEEE Trans. on Biomedical Engineering and...

(展开全部)

José C. Principe is Distinguished Professor of Electrical and Biomedical Engineering, and BellSouth Professor at the University of Florida, and the Founder and Director of the Computational NeuroEngineering Laboratory. He is an IEEE and AIMBE Fellow, Past President of the International Neural Network Society, Past Editor-in-Chief of the IEEE Trans. on Biomedical Engineering and the Founder Editor-in-Chief of the IEEE Reviews on Biomedical Engineering. He has written an interactive electronic book on Neural Networks, a book on Brain Machine Interface Engineering and more recently a book on Kernel Adaptive Filtering, and was awarded the 2011 IEEE Neural Network Pioneer Award.

Front Matter Pages i-xxii
Information Theory, Machine Learning, and Reproducing Kernel Hilbert Spaces José C. Principe Pages 1-45
Renyi’s Entropy, Divergence and Their Nonparametric Estimators Dongxin Xu, Deniz Erdogmuns Pages 47-102
Adaptive Information Filtering with Error Entropy and Error Correntropy Criteria Deniz Erdogmus, Weifeng Liu Pages 103-140
Algorithms for Entropy and Correntropy Adaptation with Applications to Linear Systems Deniz Erdogmus, Seungju Han, Abhishek Singh Pages 141-179
Nonlinear Adaptive Filtering with MEE, MCC, and Applications Deniz Erdogmus, Rodney Morejon, Weifeng Liu Pages 181-218
· · · · · · (更多)

Front Matter Pages i-xxii
Information Theory, Machine Learning, and Reproducing Kernel Hilbert Spaces José C. Principe Pages 1-45
Renyi’s Entropy, Divergence and Their Nonparametric Estimators Dongxin Xu, Deniz Erdogmuns Pages 47-102
Adaptive Information Filtering with Error Entropy and Error Correntropy Criteria Deniz Erdogmus, Weifeng Liu Pages 103-140
Algorithms for Entropy and Correntropy Adaptation with Applications to Linear Systems Deniz Erdogmus, Seungju Han, Abhishek Singh Pages 141-179
Nonlinear Adaptive Filtering with MEE, MCC, and Applications Deniz Erdogmus, Rodney Morejon, Weifeng Liu Pages 181-218
Classification with EEC, Divergence Measures, and Error Bounds Deniz Erdogmus, Dongxin Xu, Kenneth Hild II Pages 219-261
Clustering with ITL Principles Robert Jenssen, Sudhir Rao Pages 263-298
Self-Organizing ITL Principles for Unsupervised Learning Sudhir Rao, Deniz Erdogmus, Dongxin Xu, Kenneth Hild II Pages 299-349
A Reproducing Kernel Hilbert Space Framework for ITL Jianwu Xu, Robert Jenssen, Antonio Paiva, Il Park Pages 351-384
Correntropy for Random Variables: Properties and Applications in Statistical Inference Weifeng Liu, Puskal Pokharel, Jianwu Xu, Sohan Seth Pages 385-413
Correntropy for Random Processes: Properties and Applications in Signal Processing Puskal Pokharel, Ignacio Santamaria, Jianwu Xu, Kyu-hwa Jeong, Weifeng Liu Pages 415-455
Back Matter Pages 457-515
· · · · · · (收起)