|
Information
Course beginning:
Febrary -
last week,
2023
Duration 13 weeks.
Lesson timetable
TBD .
|
Motivation
This course introduces
the Deep Neural Networks (DNN) architectures, the relative Deep Learning
Algorithms (DLAs), from now denoted to as Deep Learning (DL) and their
(main) applications
Objectives of the Course
The
educational objectives include the acquisition of the following skills:
1) knowledge and understanding of the problems related to the use of DL;
2) the ability to apply knowledge on DL in the most common problems
described in the course (knowledge and know-how), 3) development of
independent judgment regarding the possible optimal solution with DL of
a given problem, 4) the development of communication skills on the
topics covered in the course, 5) the ability to autonomous learning on
specialized texts.
Final
examination
The exam consists of a
discussion of an assigned project or home-work (max 24pt) and some
theoretical questions (max 6pt).
The home-work is assigned
to the student in the last week of the course, and typically the student
can choose the project from a list of possible topics.
The project can also be
done by a group of maximum 3 students. In this case the task of each
individual student must be well specified.
The final discussion of the project is done at the teacher's office by
appointment (at any period of the year, and usually Tuesday afternoons).
For the final exam the student must: 1) present a report (in the form of
a short scientific paper) on the project carried out; 2) give a short
presentation in which the results and the acquired skills are
highlighted.
Course syllabus (Tentative)
1. Introduction and Prerequisites:
a. Rewiev of Supervised and Unsupervised Learning
b. Stochastic optimization algorithms
c.
Regression and Classification Performance Metric
d.
Lab session: Python and NumPy
2. Deep Neural Networks
a. End-to-End Problems and DNNs’ Architectures
b. Backpropagation,
BP-variants and
Automatic Differentiation
c. Mechanisms of regularization
d.
Lab session: introduction to TensorFlow
3. Deep Recurrent Neural Networks
a. Elman Models
b. Convolutional NNs
c. Long Short-Time Memory and Gated NNs
d. Lab session: TensorFlow prediction and filtering
4. Generative models
a. Autoencoder (AE), variational AE.
b. Generative Adversarial Networks (GAN), Cycle GAN.
c. Deep Convolutional GANs.
d. Lab session: generative models.
5. Graph Neural Networks
a. Definition of GNNs
b.
Deep Learning for GNNs
c. Graph Convolutional NNs
d. Lab session: GNNs.
6. DL's Main Applications
a. Computer vision.
b. Sequence and text analysis.
c. Audio and Speech Signal
d.
Environment, Energy and Smart Grid
e.
......
f.
Lab sessions.
References
Text books and papers
-
A. Uncini, Introduction to Neural
Networks and Deep Learning, Lecture notes +
slides - ed. 2022.
-
A. Uncini, Mathematical Elements
for Machine Learning, Lecture
notes + slides - ed. 2022.
-
Aston Zhang, Zack C. Lipton, Mu Li,
Alex J. Smola, “Dive into Deep
Learning,” CoRR abs/2106.11342 (2021)
Other
other recommended text and papers
-
I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, MIT book E. 2018.
-
Aurélien Géron, Hands-on Machine Learning with Scikit-Learn, Keras & TensorFlow, 2nd Edition, Published by O’Reilly Media, 2019
-
Li Deng and Dong Yu, Deep Learning Methods and Applications, Foundations and Trends in Signal Processing 7:3-4, 2014
Further reading
books
-
S. Haykin, “Neural
Networks”, MacMillan College Publishing Company, NY, 2009.
-
Thomas Weise, “Global
Optimization Algorithms Theory and Applications”, University
of Kassel, http://www.it-weise.de/
-
R.O. Duda e P.E.
Hart, “Pattern Classification and Scene Analysis”, J.
Wiley & Sons, 1973 (MAT 68-1973-03IN, ING2 EL.0069).
-
J.-S.R. Jang,
C.-T. Sun, E. Mizutani, “Neuro-Fuzzy and Soft Computing”,
Prentice Hall, 1997.
-
A. Uncini, Fundamentals
of Adaptive Signal Processing - Springer, Febrary 2015.
|