Time and location: Tue, Thu 4:00 – 5:15pm, EPIC 3222

Instructor & TAs:		Razvan Bunescu		Youssef Ait Alama
Office:		Woodward 410G		Burson 239B
Office hours:		Tue, Thu 5:30 – 6:30pm		Mon, Wed 12:00 – 1:00pm
Email:		razvan.bunescu @ charlotte edu		yaitalam @ charlotte edu

1. Syllabus & Introduction with the simple Perceptron
   • Illustrative ML examples pdf / ipynb using sklearn and gym.
   • Python and NumPy examples from lecture on Aug 27 pdf / ipynb.
   • Notes from lecture on Aug 27: paper, board 1, and board 2.
   • Notes from lectures on Aug 29, Sep 3, and Sep 5.
   • Numpy examples from lectures on Sep3 & 5.
2. Programming with Python
   • Examples from session on Aug 22
   • Python tutorial
   • Python for Everybody
3. Basic linear algebra
   • Chapter 2 in DL textbook on Linear Algebra.
   • Paper notes from lecture on Sep 10.
   • Paper solutions for in-class quiz on Sep 10.
4. NumPy for linear algebra
   • Introduction to NumPy pdf / ipynb.
   • Tutorials on NumPy and SciPy.
     • Broadcasting explained.
   • Matplotlib tutorial and examples:
     • A non-convex surface: visualization in 3D and code.
     • Athens houses: visualization and code.
5. k-Nearest Neighbors for classification and regression
   • Notes from lectures on Sep 17 and Sep 19.
   • Feature Selection
6. Differentiation and optimization
   • Notes from lecture on Sep 24: one and two.
   • Notes from lecture on Sep 26: one, two, and three.
   • Symbolic differentiation with SymPy.
   • Automatic differentiation with Autograd.
7. Linear regression and ordinary least squares
   • Notes from lecture on Sep 26.
8. Gradient descent and least mean squares
   • Notes from lecture on Oct 1: one, two, three, and four.
   • Notes from lecture on Oct 8: one, two, three, and four.
   • An overview of gradient descent optimization algorithms, Sebastian Ruder, CoRR 2016
9. Curve fitting and regularization
   • Notes from lecture on Oct 10: one and two.
   • Notes from lecture on Oct 17: one and two.
   • Notes from lecture on Oct 22: one and two.
10. Kernel Perceptron and Averaged Perceptron
    • Notes from lecture on Oct 24: one and two.
    • LibSVM applet demonstrating kernels.
11. Probability theory
    • Notes from lecture on Oct 29: one, two, three, and four.
    • Notes from lecture on Oct 31: one and two.
12. Maximum Likelihood Estimation principle
    • Notes from lecture on Nov 5: one and two.
    • Sections 2.1 and 2.2 in Tom Mitchell's chapter on Estimating Probabilities
13. Naive Bayes
    • Notes from lecture on Nov 5: three and four.
    • Notes from lecture on Nov 7.
    • Chapter 4 from Speech and Language Processing, by Jurafsky and Martin, 2024.
14. Logistic regression
    • Multinomial softmax with temperature: notebook and pdf.
    • Notes from lecture on Nov 12: one and two.
    • Notes from lecture on Nov 14: one, two, three, and four.
    • Notes from lecture on Nov 19: one and two.
15. Computation graphs and automatic differentiation in PyTorch
    • Hand notes from lecture on Nov 21: one and two.
    • PyTorch examples and linear regression.
    • PyTorch tutorials.
16. Multilayer Perceptrons, Backpropagation, and Deep Learning
    • Hand notes from lecture on Nov 26: one and two.
    • Hand notes from lecture on Dec 3: one and two.
    • Chapter 7 on Neural Networks from the NLP textbook.
    • Andrew Ng's introduction to deep learning from 2012 IPAM Graduate Summer School: video and PDF slides
    • Andrej Karpathy: "Yes you should understand backprop"
17. Reinforcement learning

• Assignment 0 on basic programming in Python
  • Skeleton code and data
• Assignment 1 on Perceptrons using NumPy
  • Skeleton code and data
• Assignment 2 on kNNs using NumPy
  • Skeleton code and data
• Assignment 3 on Linear regression (normal equations)
  • Skeleton code and data
• Assignment 4 on Linear regression (gradient descent)
  • Skeleton code and data
• Assignment 5 on Polynomial Curve Fitting
  • Skeleton code and data
• Assignment 6 on Kernel Perceptron and Average Perceptron.
  • Skeleton code and data
• Assignment 7 on Probability Theory and Naive Bayes
  • Skeleton code and data
• Assignment 8 on Logistic Regression in NumPy and PyTorch.
  • Skeleton code and data
• Assignment 9 on Neural Networks in NumPy and PyTorch.
  • Theory
  • Implementation: (1) NNs in Numpy and (2) NNs in Pytorch.
    • Skeleton code and data

• Tips for choosing a project topic:
  • UT Austin
  • Stanford
• Project report guidelines

• A Course in Machine Learning book by Hal Daume III
• Machine Learning book by Tom Mitchell. McGraw Hill, 1997
• Supervised Machine Learning video lectures @ Coursera.
• Mathematics for Deep Learning and Data Science video lectures @ Coursera:
  • Linear Algebra module.
  • Calculus module.
  • Probability and Statistics module.

• Python programming:
  • Google's Crash Course on Python, free for UNCC students
  • W3 Schools' Python tutorial
  • Microsoft's Python for beginners
  • Python for Everybody
  • Python lecture
• Probability and statistics:
  • Basic probability theory (pp. 12-19) in Pattern Recognition and Machine Learning.
  • Chapter 3 in DL textbook on Probability and Information Theory.
  • Parts 1 and 2 in CS109: Probability for Computer Scientists course reader.
  • Seeing Theory: A visual introduction to probability and statistics, Kunin et al., 2018.
• Linear Algebra:
  • Chapter 2 in DL textbook on Linear Algebra.
  • Interactive Linear Algebra, Margalit and Rabinoff, Georgia Tech, 2017.
  • Chapter 2 on Linear Algebra in Mathematics for Machine Learning.
  • Inderjit Dhillon's Linear Algebra Background
  • Gilbert Strang's Introduction to Linear Algebra
  • Petersen et al.'s The Matrix Cookbook
  • Mike Brookes' Matrix Reference Manual
• Calculus:
  • Basic properties for derivatives, integrals, exponentials, and logarithms.
  • Chapter 4.3 in DL textbook on Numerical Computation.
  • Gilbert Strang's Calculus texbook.

• PyTorch Deep learning in Python
• PyTorch tutorial from University of Amsterdam.
• Scikit-learn Machine learning in Python
• Hugging Face Transformers