ITCS 6101/8101: Natural Language Processing
Spring 2026

---

Time and Location: Tue, Thu 5:30 – 6:45pm, Woodward 135

Instructor & TA:		Razvan Bunescu		Tonmoy Hasan
Office:		Woodward 410G		Cone 164
Office hours:		Tue, Thu 4:00 – 5:00pm		Mon, Wed 4:00 – 5:00pm
Email:		rbunescu @ charlotte edu		thasan1 @ charlotte edu

Textbook (PDF available online):

Speech and Language Processing (3rd edition draft), by Daniel Juraksfy and James E. Martin; draft released on Jan 6, 2026.

Course description:
Natural Language Processing (NLP) is a branch of Artificial Intelligence whose focus is on the development of computer systems that can process, understand, or communicate in natural language. The course will cover foundational techniques for developing state-of-the-art NLP models, such as Large Language Models (LLMs), as well as methods for understanding their behavior and limitations. On the application side, the course will introduce agentic AI workflow patterns for LLM-based applications. Students will reinforce core concepts and skills through homework assignments, research paper presentations, and a final project.

Prerequisites:
Introduction to Machine Learning (ITCS 3156 or 5356). Students are expected to be comfortable with programming in Python, data structures and algorithms (ITSC 2214), and basic machine learning techniques. Relevant background material will be made available on this website throughout the course.

Lecture notes:

1. Syllabus & Introduction
   • Chapter 1 from the textbook.
   • Example of PP-attachment ambiguity.
2. Tokenization
   • Notes from lecture on Jan 15.
   • Tokenization examples: PDF and notebook.
   • Chapter 2 on words and tokens.
   • Unicode and UTF-8 encoding.
3. ML review on logistic regression
   • Chapter 4 on logistic regression and text classification.
   • Basics of matrices and vectors (Jan 22): one, two, three, and four.
   • Notes from lecture on Jan 27: inference and training.
4. Word embeddings
   • Chapter 5 on vector semantics and embeddings.
5. ML review on neural networks
   • Chapter 6 on neural networks.
6. Language Modeling architectures
   • Chapter 7 on large language models.
7. RNNs and Attention
   • Chapter 13 on RNNs and LSTMs.
8. Transformers and Self-Attention
   • Chapter 8 in J & M on Transformers.
9. Pretraining, Supervised Fine-tuning, and Alignment
10. LLM application development through APIs
    • Connecting applications with tools and external resources
    • Efficient factual grounding using RAG and vector DBs
    • Developing and deploying multi-agent systems
11. Advanced topics in NLP
    • Repository of research articles for student presentations

Homework assignments¹:

• Assignment 1 on subword tokenization with BPE.
  • Skeleton code and data.
• Assignment 2 on word embeddings.
  • Skeleton code and data.
• Assignment 3 on text classification with NNs.
  • Skeleton code and data.
• Assignment 4 on summarization with RNNs.
  • Skeleton code and data.
  • Educational cluster instructions.
• Assignment 5 on Transformer-based models for NLP.
  • Skeleton code and data.
• Assignment 6 on post-training LLMs.
  • Skeleton code and data.
• Assignment 7 on Question Answering on semi-structured restaurant data using the chat completion API.
  • Skeleton code and data.
• Assignment 8 on Restaurant Chatbot by interfacing LLMs with external tools through MCP.
  • Skeleton code and data.

Final project:

• Resources and guidelines for the final project
  • Project suggestions
  • Project report guidelines

Paper presentations:

1. Paper title, authors, venue, year
   • Presenter(s)
2. Paper title, authors, venue, year
   • Presenter(s)
3. Paper title, authors, venue, year
   • Presenter(s)
4. ...

Background reading materials:

• Python programming:
  • Google's Crash Course on Python
  • W3 Schools' Python tutorial
  • Python for Everybody
• Mathematics for Computer Science textbook on zyBooks.
• Probability and statistics:
  • Basic probability theory (pp. 12-19) in Pattern Recognition and Machine Learning.
  • Chapter 3 in DL textbook on Probability and Information Theory.
  • Parts 1 and 2 in CS109: Probability for Computer Scientists course reader.
  • Seeing Theory: A visual introduction to probability and statistics, Kunin et al., 2018.
• Linear Algebra:
  • Chapter 2 in DL textbook on Linear Algebra.
  • Interactive Linear Algebra, Margalit and Rabinoff, Georgia Tech, 2017.
  • Chapter 2 on Linear Algebra in Mathematics for Machine Learning.
  • Inderjit Dhillon's Linear Algebra Background
  • Gilbert Strang's Introduction to Linear Algebra
  • Petersen et al.'s The Matrix Cookbook
  • Mike Brookes' Matrix Reference Manual
• Calculus:
  • Basic properties for derivatives, integrals, exponentials, and logarithms.
  • Chapter 4.3 in DL textbook on Numerical Computation.
  • Gilbert Strang's Calculus texbook.

Supplemental readings:

• Attention is all you need, Vaswani et al., NeurIPS 2017.
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, NAACL 2019.

Tools and packages:

• Natural language processing:
  • spaCy
  • brat rapid annotation tool
  • FAISS open-source library for AI similarity search.
  • LangExtract for LLM-powered information extraction.
• Machine learning:
  • PyTorch Deep learning in Python
  • PyTorch tutorial from University of Amsterdam.
  • Scikit-learn Machine learning in Python
  • Hugging Face ML models and datasets