Lecture Forecast| Prof. Sergios Theodoridis brings a talk about Deep Neural Networks

Topic: Deep Neural Networks: A Nonparametric Bayesian View with Local Competition

Time & Date: May 9th ( Thursday), 11:00 am -12:00 pm

Venue: TC 206

Speaker: Sergios Theodoridis, Professor in National and Kapodistrian University of Athens, Greece and The Chinese University of Hong Kong, Shenzhen, China

Host: CUHK, Shenzhen and SRIBD

Abstract

 In this talk, a fully probabilistic approach to the design and training of deep neural networks will be presented. The framework is that of the nonparametric Bayesian learning. Both fully connected as well as convolutional networks will be discussed. The structure of the networks is not a-priori chosen. Adopting nonparametric priors for infinite binary matrices, such as the Indian Buffet Process (IBP), the number of weights as well as the number of nodes or number of kernels are estimated via the resulting posterior distributions. The training evolves around variational Bayesian arguments. 

Besides the probabilistic arguments that are followed for the inference of the involved parameters, the nonlinearities used are neither squashing functions not rectified linear units (ReLU). Instead, inspired by neuroscientific findings, the nonlinearities comprise units of probabilistically competing linear neurons, in line with what is known as the local winner-take-all  (LTWA) strategy. In each node, only one neuron fires to provide the output. Thus, neurons, in each node, perform a lateral communication, and the strongest one “survives”. Such rationale is closer to the way that our brain works and makes decisions.

The experiments, over a number of standard data bases, verify that highly efficient structures are obtained in terms of number of units, weights and kernels (in CNNs) as well as in terms of bit precision requirements at no sacrifice to performance, compared to previously published state of the art research.The presentation mainly focuses on the concepts and the rationale behind the method and less in the mathematical details.

About the Speaker

Sergios Theodoridis is currently Professor of Signal Processing and Machine Learning in the Department of Informatics and Telecommunications of the National and Kapodistrian University of Athens and he is the holder of a part time Chair at The Chinese University of Hong Kong, Shenzhen. His research interests lie in the areas of Adaptive Algorithms, Distributed and Sparsity-Aware Learning, Machine Learning and Pattern Recognition, Signal Processing and Learning for Bio- Medical Applications and Audio Processing and Retrieval.

He is the author of the book “Machine Learning: A Bayesian and Optimization Perspective” Academic Press, 2015, the co-author of the best-selling book “Pattern Recognition”, Academic Press, 4th ed. 2009, the co-author of the book “Introduction to Pattern Recognition: A MATLAB Approach”, Academic Press, 2010, the co-editor of the book “Efficient Algorithms for Signal Processing and System Identification”, Prentice Hall 1993, and the co-author of three books in Greek, two of them for the Greek Open University.

He is the co-author of seven papers that have received Best Paper Awards including the 2014 IEEE Signal Processing Magazine Best Paper award and the 2009 IEEE Computational Intelligence Society Transactions on Neural Networks Outstanding Paper Award. He is the recipient of the 2017 EURASIP Athanasios Papoulis Award, the 2014 IEEE Signal Processing Society Education Award and the 2014 EURASIP Meritorious Service Award. He  has served as a Distinguished Lecturer for the IEEE Signal Processing as well as the Circuits and Systems Societies. He was Otto Monstead Guest Professor, Technical University of Denmark, 2012, and holder of the Excellence Chair, Dept. of Signal Processing and Communications, University Carlos III, Madrid, Spain, 2011.

He has served as Editor-in-Chief for the IEEE Transactions on Signal Processing. He is Editor-in- Chief for the Signal Processing Book Series, Academic Press and co-Editor in Chief for the E- Reference Signal Processing, Elsevier. He has served as President of the European Association for Signal Processing (EURASIP), as a member of the Board of Governors for the IEEE Circuits and Systems (CAS) Society, as a member of the Board of Governors (Member-at-Large) of the IEEE SP Society and as a Chair of the Signal Processing Theory and Methods (SPTM) technical committee of IEEE SPS. He currently serves as Vice President IEEE Signal Processing Society. He is Fellow of IET, a Corresponding Fellow of the Royal Society of Edinburgh (RSE), a Fellow of EURASIP and a Fellow of IEEE.

* There will be a nice tea break after the lecture, all of you are warmly welcomed to join !