KPL4: Model Mechanics for Standard Feed-Forward Neural Networks

This is Key-Point Lecture 4 in a series of lectures prepared for a two-week introductory course in Machine Learning at the University of Cape Town, South Africa. The course is aimed at students with some background in statistical modelling, computing, and linear algebra. KPL4 covers model mechanics for neural networks. That is, the simple elements from which the feed-forward networks are constructed, the model structure itself, and details pertaining to the learning algorithm associated with the class.

This work by Etienne A.D. Pienaar is licensed under CC BY-NC-ND 4.0.

Relevant references:

Yaser S Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin. Learning From Data, volume 4.
AMLBook New York, NY, USA:, 2012.

Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning, volume 1.
Springer series in statistics New York, NY, USA:, 2001.

Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. An introduction to statistical
learning, volume 112. Springer, 2013.

Michael A Nielsen. Neural networks and deep learning. Determination Press, 2015.

Networks