ITCS 4156: Introduction to Machine Learning
Spring 2021

---

Time and Location: Wed 5:30 – 8:15pm, Online
Instructor: Razvan Bunescu
Office: Woodward Hall 210F
Office Hours: Tue, Thu 4:00 – 5:00pm on Zoom, or by email appointment
Email: rbunescu @ uncc edu

Teaching Assistant: Priyanka Jadhav (pjadhav7 @ uncc edu)
Office Hours: Mon, Wed 2:00 – 3:00 pm on Zoom, or by email appointment

Textbook: There is no required textbook for this class. Slides and supplementary materials will be made available on the course website.

Supplementary Texts:

Pattern Recognition and Machine Learning by Christopher Bishop. Springer, 2007.

The Elements of Statistical Learning: Data Mining, Inference, and Prediction by T. Hastie, R. Tibshirani, & J. H. Friedman. Springer Verlag, 2009 - 2017

Reinforcement Learning: An Introduction by Richard S. Sutton and Andrew G. Barto. MIT Press, 2018

Machine Learning: The Art and Science of Algorithms that Make Sense of Data by Peter Flach, Cambridge University Press, 2012

A Course in Machine Learning by Hal Daume III

Machine Learning by Tom Mitchell. McGraw Hill, 1997

Course description:
This course will give an overview of the main concepts, techniques, and algorithms underlying the theory and practice of machine learning. The course will cover the fundamental topics of regression, classification, clustering, and reinforcement learning, and a number of corresponding learning models such as linear regression, perceptrons, logistic regression, neural networks, Naive Bayes, nearest neighbors, decision tres, k-Means, and Q-learning. The description of the formal properties of the algorithms will be supplemented with motivating applications in a wide range of areas including natural language processing, computer vision, bioinformatics, and music analysis.

Prerequisites:
The students are expected to be comfortable with programming and familiar with basic concepts in linear algebra, calculus, and statistics. Relevant background material is made available on the course website below.

Lecture notes¹:

1. Syllabus & Introduction
   • Hand notes: Jan 20 and Jan 27
2. Linear regression, Ridge regression, and Lasso
   • Hand notes: Feb 3
3. Linear algebra and optimization in NumPy and SciPy
   • Python lecture
   • Tutorials on NumPy and SciPy.
     • Broadcasting explained.
   • NumPy/SciPy examples and NumPy session.
   • Matplotlib tutorial and examples:
     • A non-convex surface: visualization in 3D and code.
     • Athens houses: visualization and code.
4. Gradient Descent algorithms
   • Hand notes: Feb 24
   • An overview of gradient descent optimization algorithms, Sebastian Ruder, CoRR 2016
5. Perceptrons and Kernels
   • Hand notes: Mar 10 and Mar 24
6. Support Vector Machines
7. Logistic regression, Maximum likelihood, and Maximum entropy
   • Hand notes: Apr 7
   • Intro to MLE and MAP estimation: Sections 2.1 and 2.2
   • A more gentle introduction to LR from the NLP class
8. Machine learning and optimization in PyTorch
   • PyTorch tutorials.
   • PyTorch examples and linear regression.
9. Neural Networks
   • Hand notes: Apr 21
10. Naive Bayes
    • Naive Bayes and Logistic Regression, chapter in Tom Mitchell, Machine Learning, 2017
    • Another introduction to NB from the NLP class
11. Nearest Neighbor methods
12. Decision Trees and Random Forests
    • Classification And Regression Trees (CART), slides by Alexandra Chouldechova @ CMU.
      • Decision trees in scikit-learn.
13. Clustering
    • Clustering algorithms in scikit-learn.

Homework assignments²:

1. Linear regression (normal equations):
   • Theory
   • Implementation
     • Skeleton code and data
2. Linear regression (gradient descent)
   • Skeleton code and data
3. Perceptron
   • Skeleton code and data
4. Kernel Perceptron
   • Skeleton code and data
5. Text Classification with Perceptrons
   • Skeleton code and data
6. Logistic regression in NumPy
   • Skeleton code and data
7. Logistic Regression in PyTorch
   • Skeleton code and data
8. Neural networks in PyTorch
   • Skeleton code and data
9. ML theory exercises

² The spam classification exercise and the flower and spiral datasets are adapted from homework assignments developed by Andrew Ng.


Background reading materials:

• Linear Algebra:
  • Gilbert Strang's Introduction to Linear Algebra
  • Strang's Video Lectures on Linear Algebra
  • Inderjit Dhillon's Linear Algebra Background
  • Petersen et al.'s The Matrix Cookbook
  • Mike Brookes' Matrix Reference Manual
• Calculus and differentiation:
  • Volume 1 of calculus book on OpenStax.
    • Section on differentiation rules.
  • Video lectures on differential calculus at Khan Academy.
• Probability and statistics:
  • James H. Martin's Introduction to probabilities
  • Jason Eisner's equestrian Introduction to probabilities
  • Introductory Statistics book on OpenStax.
  • Statistical Inference book, Casella and Berger, 2001.

Machine learning software:

• PyTorch Deep learning in Python
• Scikit-learn Machine learning in Python
• Weka Data mining in Java
• MALLET Java implementations of logistic regression, HMMs, linear chain CRFs, and other ML models.
• LibSVM applet demonstrating SVMs.