ITCS 4101: Introduction to Natural Language Processing
Fall 2025

---

Time and Location: Tue, Thu 1:00 – 2:15pm, CHHS 380

Instructor & TA:		Razvan Bunescu		Youssef Ait Alama
Office:		Woodward 410G		Woodward 412
Office hours:		Tue, Thu 2:30 – 3:30pm		Mon, Wed 12:00 – 1:00pm
Email:		rbunescu @ charlotte edu		yaitalam @ charlotte edu

Textbook (PDF available online):

Speech and Language Processing (3rd edition draft), by Daniel Juraksfy and James E. Martin. 2025.

Course description:
Natural Language Processing (NLP) is an area of Artificial Intelligence whose focus is on the development of computer systems that process or generate natural language. This course will first introduce fundamental linguistic analysis tasks, including tokenization, syntactic parsing, semantic parsing, and coreference resolution. We will then study vector based representations of text, ranging from bag-of-words and TF-IDF to neural word and text embeddings. The course will survey machine learning models and techniques underlying modern NLP, including attention and Transformer-based language models, which will be used in a number of NLP applications such as sentiment classification, information extraction, or question answering. In parallel, the course will introduce standard frameworks for developing workflows where LLM-based agents connect with tools and communicate with other agents. Overall, the aim of this course is to equip students with an array techniques and tools that they can use to solve known NLP tasks, as well as new types of NLP problems.

Prerequisites:
Introduction to Machine Learning (ITCS 3156). Students are expected to be comfortable with programming in Python, data structures and algorithms (ITSC 2214), and basic machine learning techniques. Relevant background material will be made available on this website throughout the course.

Lecture notes:

1. Syllabus & Introduction
2. Python for programming, linear algebra, and visualization
   • Python lecture
   • Python tutorial
   • NumPy tutorial
   • Matplotlib tutorial
   • Brute force search: PDF and notebook.
3. Tokenization: From text to sentences and tokens
   • Chapter 2, sections on words, morphemes, rule-based tokenization, BPE, and corpora.
   • Tokenization examples: PDF and notebook.
   • Notes from lecture on Sep 2.
4. Regular expressions
   • Regular expressions in Python with re.
   • Regular expressions in UNIX with sed.
   • Notes from lecture on Sep 4.
   • Examples of regular expressions: pdf and code.
5. Text classification using Logistic Regression
   • Chapter 4, sections on sentiment analysis and evaluation measures.
   • Chapter 5 on logistic regression from the textbook.
6. LLMs: use scenarios, strengths and weaknesses
7. LLMs: application development with the chat completion API
   • Notebooks with examples using GPT, Llama, and Gemini.
8. LLMs: connecting applications with tools and external resources through MCP
9. LLMs: developing and deploying multi-agent systems
10. Word meanings; Sparse vs. dense representations of words
    • Chapter 6 on vector semantics and embeddings from the textbook.
    • Visualization of word embeddings with TensorFlow Embedding Projector.
11. N-grams and Neural models for Language Modeling and Sequence Processing
    • Chapter 7 on neural networks and neural language models from the textbook.
    • Chapter 8 on RNNs and LSTMs from the textbook.
    • All Our N-gram are Belong to You
12. Machine translation, Sequence-to-sequence models and Attention
13. Transformers and Pretrained Language Models
    • Chapter 9 in J & M on Deep Learning Architectures for Sequence Processing.
    • Chapter 10 in J & M on Transformers and Pretrained Language Models.
14. Tuning LLMs with SFT, DPO, and online RL
15. Biases vs. fairness and rationality in NLP models
    • Benchmarking Cognitive Biases in Large Language Models as Evaluators, Koo et al., Findings of ACL, 2024

Homework assignments¹:

• Assignment 0 on Python lists and strings.
  • Skeleton code.
• Assignment 1 on Word distributions.
  • Skeleton code and data.
• Assignment 2 on Wikipedia processing with regular expressions.
  • Skeleton code and data.
• Assignment 3 on Sentiment Analysis with Logistic Regression and engineered features.
  • Skeleton code and data.
• Assignment 4 on Question Answering on semi-structured restaurant data using the chat completion API with GPT, LLama, or Gemini.
  • Skeleton code and data.
• Assignment 5 on interfacing LLMs with external tools and resources.
  • Skeleton code and data.
• Assignment 6 on LLM-based multi-agent workflows.
  • Skeleton code and data.
• Assignment 7 on Vector Representations of Words.
  • Skeleton code and data.
• Assignment 8 on RNNs for Sentiment Classification.
  • Skeleton code and data.
  • Educational cluster instructions.
• Assignment 9 on Transformer-based models for NLP.
  • Skeleton code and data.
• Assignment 10 on post-training LLMs.
  • Skeleton code and data.

Background reading materials:

• Python programming:
  • Google's Crash Course on Python
  • W3 Schools' Python tutorial
  • Python for Everybody
• Probability and statistics:
  • Basic probability theory (pp. 12-19) in Pattern Recognition and Machine Learning.
  • Chapter 3 in DL textbook on Probability and Information Theory.
  • Parts 1 and 2 in CS109: Probability for Computer Scientists course reader.
  • Seeing Theory: A visual introduction to probability and statistics, Kunin et al., 2018.
• Linear Algebra:
  • Chapter 2 in DL textbook on Linear Algebra.
  • Chapter 2 on Linear Algebra in Mathematics for Machine Learning.
  • Inderjit Dhillon's Linear Algebra Background
  • Gilbert Strang's Introduction to Linear Algebra
  • Petersen et al.'s The Matrix Cookbook
  • Mike Brookes' Matrix Reference Manual
• Calculus:
  • Basic properties for derivatives, integrals, exponentials, and logarithms.
  • Chapter 4.3 in DL textbook on Numerical Computation.
  • Gilbert Strang's Calculus texbook.

Supplemental readings:

• Accurate methods for the statistics of surprise and coincidence, Ted Dunning, Computational Linguistics, 1993.
• Attention is all you need, Vaswani et al., NIPS 2017.
• Durably reducing conspiracy beliefs through dialogues with AI, Costello et al., Science, September 2024.
• AI can help humans find common ground in democratic deliberation, Tessler et al., Science, October 2024.

Tools and packages:

• Natural language processing:
  • spaCy
  • brat rapid annotation tool
• Machine learning:
  • PyTorch Deep learning in Python
  • PyTorch tutorial from University of Amsterdam.
  • Scikit-learn Machine learning in Python
  • Hugging Face ML models and datasets