Professor Weiqing Gu Harvey Mudd College Fall 2016
Readings should be done before class. Reading summaries (specification in info section) are due for all non-Murphy readings at the start of class.
| Monday | Thursday | |
|---|---|---|
| Aug. 29 | Linear Algebra and Matrix Calculus Review Convex OptimizationOverview I (notes from review) | |
| Supervised LearningSept. 5 | Introduction to Big Data. Linear Regression. Normal Equations and Optimization Techniques. Solving the Normal Equations efficiently (Cholesky Decomposition). Various forms of Linear regression. Read: Murphy 1.{all} (DEFINITELY READ BEFORE FIRST CLASS) Read: Murphy, 7.{1,...,5} | Probability Review (notes from review) (also Murphy 2.{all} is a great resource) |
| Sept. 12 | Classification. K-Nearest Neighbors. Logistic Regression. Exponential Family and Generalized Linear Models. Logistic Regression as a GLM. Read: Murphy, 8.{1,2,3,5} \ 8.{3.4,3.5}, 9.{1,2.2,2.4,3} Due: Homework 1 (first third) (solutions) | Convex OptimizationOverview II Tutorial on Scientific Python |
| Sept. 19 | Generalized Linear Models continued. Poisson Regression. Softmax Regression. Covariance matrix; Multivariate Gaussian Distribution. Marginalized Gaussian and the Schur Complement. Regularization continued. Big Data as a Regularizer. Read: Murphy 9.7, 4.{1,2,3,4,5,6} (important background) Due: Homework 1 (second third) | |
| Sept. 26 | Dimensionality Reduction; Spectral Decomposition, Singular Value Decomposition, and Principal Component Analysis. Generative Learning Algorithms. Gaussian Discriminant Analysis. Due: Final Project Proposal Due: Homework 1 (last third) | Scientific Computing Review / Help Session |
| Oct. 3 | Naive Bayes. L1 Regularization and Sparsity. Lasso. Support Vector Machines. Kernels. Read: Murphy 14.{1,2,3,4} \ 14.{4.4} Read: MapReduce: Simplified Data Processing on Large Clusters Due: Homework 2 (first half) (solutions) | Midterm Review |
| Unsupervised Learning Oct. 10 | Introduction to Unsupervised Learning. Clustering. K-Means. Mixture of Gaussians. Expectation-Maximization (EM) Algorithm. Read: Murphy 11.{1,2,3,4} \ 11.{4.6,4.9} Read: Pegasos: Primal Estimated sub-GrAdient SOlver for SVM Read: Random Features for Large-Scale Kernel Machines (deep insight) Due: Homework 2 (second half) | Constrained Optimization |
| Oct. 17 | Fall Break | MapReduce andDistributed Computation and Learning |
| Oct. 24 | Principal Component Analysis (PCA) Review. Kernel PCA. One Class Support Vector Machines. Read: Murphy 12.2.{0,1,2,3} 14.4.4 Read: Support Vector Method for Novelty Detection Due: Midterm. (solutions) (results) Due: Final Project Progress Report. | Evaluating Models; Deployment |
| Learning Theory Oct. 31 | Learning Theory. VC Dimension. Bias/Variance Trade-off. Union and Chernoff/Hoeffding Bounds. Read: Large-Scale Sparse Principal Component Analysis with Application to Text Data Read: On the Convergence Properties of the EM Algorithm Due: Homework 3 (first half) (solutions) | Applications of EM: Hidden Markov Models |
| Recommender Systems Nov. 7 | Introduction to Recommender Systems. Collaborative Filtering. Non-Negative Matrix Factorization. Using Non-Negative Matrix Factorization for Topic Modelling. Read: Murphy 27.6.2 Read: Netflix Update: Try This at Home Due: Homework 3 (second half) | |
| Graph MethodsNov. 14 | Graphs. Graph representations as data. The Laplacian and usage of Spectral (Eigenvalue-Eigenvector) information. Directed Graphical Models (Bayesian Networks). Conditional Independence. Naive Bayes as a Graphical Model. Plate Notation. Read: Murphy 10.{1,2,3,4,5,6} Due: Homework 4 (first half) (solutions) | XSEDE Demo |
| Bayesian LearningNov. 21 | Recap of Bayesian Reasoning. Bayesian Linear Regression (which we've already seen). Bayesian Logistic Regression. Intractable Integrals and Motivation for Approximate Methods. Read: Murphy 5.{1,2,3.0,3.2} 7.6, 8.4 Read: Bayesian Online Changepoint Detection Due: Homework 4 (second half) | Thanksgiving |
| Nov. 28 | Monte-Carlo Methods. Rejection Sampling. Importance Sampling. Intro to Markov-Chain Monte-Carlo. Gibbs Sampling. The Metropolis-Hastings Algorithm. Read: Murphy 23.{1,2,3,4} \ 23.4.3, 24.{1,2.(1,2,3,4), 3,4} \ 24.{3.7} Optional Read: Murphy 24.{5,6} Due: Homework 5 (all) (solutions) | Application: Identifying Rapidly Deteriorating Water Quality Locations With Gaussian Processes Some (~10) Final Project Presentations will be given one day this week. |
| Dec. 5 | Latent Dirichlet Allocation. Nonparametric Models. K-Nearest-Neighbors as a Nonparametric Model. Gaussian Processes. Dirichlet Processes and the infinite mixture of Gaussians. Read: Murphy 15.{1,2,3,4}, 25.2, 27.{1,2,3} Read: Gaussian Process Kernels for Pattern Discovery and Extrapolation Due: Final Project! | Final Project Presentations. We will distribute these across 2 days, where on the final day we will watch presentations of class and instructor chosen extraordinary projects. |
| Dec. 12 | Finals | Finals |