This course will look at how statistical computations are done, and develop students' abilities to write programs for statistical problems that aren't handled by standard packages. Students will program in the S language, which will be introduced at the start of the course. Topics will include the use of simulation to investigate the properties of statistical methods, matrix computations used to implement linear models, optimization methods used for maximum likelihood estimation, and numerical and Monte Carlo integration methods for Bayesian inference. The course will conclude with a look at some more specialized statistical algorithms, such as the EM algorithm for handling missing data and latent variables, and algorithms for propagating probabilities in graphical models.
This course is designed for graduate and senior undergraduate students in statistics, actuarial science, computer science, or other fields where statistical computation is important. Students should have a basic background in statistical methods (eg, at the level of STA302), and some prior experience with programming (eg, at the level of CSC108).
Course outline: Postscript, PDF.
Assignments:
Assignment 1: Postscript, PDF. Solution: S functions, Output and discussion.
Assignment 2: Postscript, PDF.
Solution to Part 1: S functions, Output and discussion, Plot in Postscript.
Solution to Part 2: S functions, Output and discussion.
Assignment 3: Postscript, PDF. Solution: S functions, output.
Assignment 4: Postscript, PDF. Here is the data file for this assignment.
Solution: S function, output.
Example programs:
Permutation test example: S program, output.
Cholesky decomposition example: S program.
Linear regresssion with Householder transformations: S program.
Tests of linear regression methods: S program, output.
Maximum likelihood estimation example: S program.
Trapezoidal integration for Bayesian inference: S program, output.
EM algorithm example: S program, output.
Gibbs sampling example: S program, some plots of output, for data of 3.9, 3.6, 3.7.
Non-credit exercises:
Exercise set 1, and its solution.
Exercise set 2.