Artificial Neural Networks, Machine Learning, Deep Thinking Training Course

DAY 1 - ARTIFICIAL NEURAL NETWORKS

Introduction and ANN Architecture.

• Comparing biological and artificial neurons.
• The structural model of an ANN.
• Activation functions applied in ANNs.
• Common categories of network architectures.

Mathematical Foundations and Learning Mechanisms.

• Reviewing vector and matrix algebra.
• Concepts in state-space.
• Principles of optimization.
• Learning via error correction.
• Learning based on memory.
• Hebbian learning principles.
• Competitive learning strategies.

Single Layer Perceptrons.

• Perceptron structure and learning processes.
• Pattern classifiers: an introduction to Bayes' classifiers.
• Utilizing the perceptron for pattern classification.
• The perceptron convergence theorem.
• Inherent limitations of the perceptron.

Feedforward ANNs.

• Architecture of Multi-layer feedforward networks.
• The Backpropagation algorithm.
• Backpropagation: training dynamics and convergence.
• Functional approximation through backpropagation.
• Practical considerations and design challenges in backpropagation.

Radial Basis Function Networks.

• Pattern separability and interpolation techniques.
• Theory of Regularization.
• Regularization techniques in RBF networks.
• Designing and training RBF networks.
• Approximation capabilities of RBF models.

Competitive Learning and Self-Organizing ANNs.

• General approaches to clustering.
• Learning Vector Quantization (LVQ).
• Algorithms and architectures for competitive learning.
• Self-organizing feature maps.
• Key characteristics of feature maps.

Fuzzy Neural Networks.

• Neuro-fuzzy integration.
• Foundations of fuzzy sets and logic.
• Designing fuzzy systems.
• Construction of fuzzy ANNs.

Practical Applications

• Discussion of select Neural Network applications, highlighting their benefits and challenges.

DAY 2 - MACHINE LEARNING

• The PAC Learning Framework
  • Guarantees for finite hypothesis sets – consistent scenarios
  • Guarantees for finite hypothesis sets – inconsistent scenarios
  • Broad Considerations
    • Deterministic vs. Stochastic scenarios
    • Bayes error and noise
    • Estimation and approximation errors
    • Model selection strategies
• Rademacher Complexity and VC Dimension
• The Bias-Variance tradeoff
• Regularization techniques
• Managing Over-fitting
• Validation processes
• Support Vector Machines
• Kriging (Gaussian Process Regression)
• PCA and Kernel PCA
• Self-Organization Maps (SOM)
• Kernel-induced vector spaces
  • Mercer Kernels and Kernel-induced similarity metrics
• Reinforcement Learning

DAY 3 - DEEP LEARNING

This module connects back to topics explored in Day 1 and Day 2

• Logistic and Softmax Regression
• Sparse Autoencoders
• Vectorization, PCA, and Whitening
• Self-Taught Learning
• Deep Network Architectures
• Linear Decoders
• Convolution and Pooling operations
• Sparse Coding
• Independent Component Analysis
• Canonical Correlation Analysis
• Demonstrations and Real-World Applications