机器学习
机器学习
A 40-hour elective for first-year graduate students, designed to establish a foundation for research in pattern recognition and machine learning. The course uses Python.
Course Information
Statistical Learning
| Unit | Topic | Files |
|---|---|---|
| 01 | Machine Learning and Statistical Learning | 机器学习和统计学习 |
| 02 | Perceptron | 感知机 |
| 03 | k-Nearest Neighbors | k近邻算法 |
| 04 | Bayesian Classifiers | 贝叶斯分类器 |
| 06 | Logistic Regression and Maximum Entropy | Logistic回归与最大熵模型 |
| 07 | Support Vector Machines and Kernels | 支持向量机 |
| 09 | Expectation-Maximization | EM算法及其推广 · ELBO · ELBO + Jensen |
| 13 | Unsupervised Learning | 无监督学习概论 |
| 14 | Clustering | 聚类方法 |
| 15 | Singular Value Decomposition | 奇异值分解 · 线性代数基础 |
| 16 | Principal Component Analysis | 主成分分析 |
| Appendix | Distribution Measures | 分布度量 |
Supplementary materials:
- Statistical Learning Methods slides
- Notes, implementations, and exercises
- Algorithms from Hang Li’s textbook
Deep Learning
These lectures refer to the Dive into Deep Learning course and its online textbook.
| Unit | Topic | Files |
|---|---|---|
| 01 | Overview, Linear Algebra, and NDArray | 概述、线性代数和NDArray |
| 03 | Derivatives, Backpropagation, and Complexity | 导数、逆向传播和复杂度 |
| 04 | Linear Methods and Optimization | 线性方法、基础优化和层序回归 |
| 05 | Maximum Likelihood and Logistic Regression | 最大似然估计和逻辑回归 |
| 06 | Multilayer Perceptrons | 多层感知机 |
| 08 | Numerical Stability, Activations, and Hardware | 数值稳定性、激活函数和硬件 |
| 11 | Convolution and Pooling | 卷积和汇聚层 |
| 12 | LeNet, AlexNet, VGG, and NiN | LeNet、AlexNet、VGG和NiN |
| 13 | Inception, Batch Normalization, and Residual Networks | Inception、批量归一化和残差网络 |
| 15 | Image Augmentation, Fine-Tuning, and Style Transfer | 图像增广、微调和样式迁移 |
| 16 | Object Detection and Training Techniques | 目标检测、计算机视觉训练技巧 |
| 18 | Sequence Models | 序列模型 |
| 19 | Recurrent Neural Networks | 循环神经网络 |
| 20 | Advanced Recurrent Neural Networks | 高级循环神经网络 |
| 24 | Attention and Vision Transformers | 注意力机制 · ViT |
| 25 | Optimization | 优化问题 |
Language Models
Generative Models
- 预备知识
- Autoencoder
- Expectation-Maximization
- Variational Autoencoder
- Generative Adversarial Network
- Diffusion Model
3D Vision
Covered topics include ray tracing of volume densities, EWA volume splatting, NeRF, and 3D Gaussian Splatting.
Assignments
References
- Hang Li, Statistical Learning Methods, 2nd edition, Tsinghua University Press, 2019.
- Hang Li, Machine Learning Methods, Tsinghua University Press, 2022.
- Aston Zhang et al., Dive into Deep Learning, PyTorch edition, Posts & Telecom Press, 2023.
- Koki Saitoh, Deep Learning from Scratch, Posts & Telecom Press, 2018.