Deep and Reinforcement Learning

Recommended Prerequisites

Artificial Intelligence; Machine Learning; Programming, ideally in Python; Good English, reading, writing and speaking skills.

Teaching Methods

During the lectures (T) the concepts, the theories, the algorithms will be presented and discussed. In the (PL) classes students will consolidate what was learned in T. The practical work will be done under the supervision of the teacher. Grading will be based on two components: (1) projects involving the techniques and/or a practical problem; (2) a written exam to assess students' knowledge about the subject of Avanced Machine Learning.

Learning Outcomes

After completing the curricular unit, it is expected that students acquire knowledge about advanced topics of computational learning and skills for the development of solutions involving deep learning networks and reinforcement learning models. In the end, they should be able to analyze, model, implement, train and execute:

 - fully connected, convolutional, sequential and recursive networks of graphs and "transformers"

 - model free q-learning and deep q-networks.

Moreover, state of the art topics will be covered showcasing the latest trends on topics, models and applications of Deep and reinforcement learning.

Students will consolidate their communication skills, analysis and synthesis, writing and speaking, and of working in group.

Work Placement(s)

No

Syllabus

1. Neural Network Training

1.1 Forward Propagation

1.2 Backpropagation and Chain Rule

1.3 Optimization and Optimizers

1.4 Forward Propagation Alternatives

1.5 Initialization, Normalisation and Regularization

2. Deep Learning

2.1. Deep Neural Networks

2.2. Convolutional Neural Networks

2.3. Sequence Models and Recurrent Neural Networks

2.3.1 RNNs, LSTMs, GRUs

2.3.2 Embeddings

2.3.3 Seq2seq - RNNs Encoder Decoder Networks

2.4. Transformers  

2.4.1 Attention mechanism

2.4.2 Self-Attention

2.4.3 Transformer Networks

2.4.4 Pre-training and Transfer Learning

2.5. Graph Neural Networks

3. Reinforcement Learning

3.1 Q-Learning

3.2 Deep Q-Networks

3.3 Policy-Gradient Methods

3.4 Actor-Critic Methods

4. State of the Art Topics.     

Head Lecturer(s)

João Nuno Gonçalves Costa Cavaleiro Correia

Assessment Methods

Assessment
Project: 40.0%
Exam: 60.0%

Bibliography

1. Christopher M. Bishop and Hugh Bishop, Deep Learning - Foundations and Concepts (2024). Springer 2024, ISBN 978-3-031-45467-7

2. Prince, S. J. (2023). Understanding Deep Learning. MIT Press.

3. Drori, I. (2022). The Science of Deep Learning. Cambridge University Press.

4. Banzhaf, W., Machado, P., & Zhang, M. (Eds.). (2023). Handbook of Evolutionary Machine Learning. Springer Nature Singapore. ISBN 9789819938148

5. Kamath, U., Graham, K., & Emara, W. (Eds.). (2022). Transformers for Machine Learning. A Deep Dive. Chapman & Hall. ISBN 9780367767341

6. Richard S. Sutton and Andrew G. Barto (2018), Reinforcement Learning: an introduction (2nd Edition), MIT Press.

7. Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016) , Deep Learning, MIT Press.