STA 410/2102: Statistical Computation (Sep-Dec 2015)

Instructor:

Radford Neal, Office: SS6026A, Email: radford@stat.utoronto.ca

Office hours: Wednesdays 11:10-12:30, in SS6026A.

Lectures:

Tuesdays 1:10-2:00 and Thursdays 1:10-3:00, in MS 2170. First lecture September 15, last lecture December 8, no lecture on November 10 (fall break).

Evaluation:

66% Three assignments (22% each), due in class on October 22, November 19, and December 8.
34% Two 50-minute tests (17% each), held in lecture time on October 27 and December 8.

Graduate students in STA 2102 may have some additional questions to do on tests or assignments (which will be bonus questions for undergraduates).

Textbook:

Geof H. Givens and Jennifer A. Hoeting, Computational Statistics, 2nd edition, Wiley.

The textbook webpage has datasets, R code, and errata.

You should now have read Chapters 2, 4, 5, 6, and 7 (plus Chapter 1 as needed).

Computing:

Assignments will be done in R. Statistics Graduate students will use the Statistics research computing system. Undergraduates and graduate students from other departments will use CQUEST. You will probably automatically have an account on CQUEST if you're an undergraduate student in this course (you need to fill out a form if you're a grad student).

You can also use R on your laptop or home computer by downloading it for free from www.R-project.org.

You might also be interested in trying out my faster implementation of R, called pqR, available from pqR-project.org, although, it currently is distributed only in source form.

You might want to use the knitr package, and perhaps my settings of options for it: knitr-options.r.

Learning R:

At the r-project.org site, there is an Introduction to R.

You can also look at Hadley Wickham's online book on Advanced R.

You might also find it useful to look at the lecture slides and other material for my section (using R) of CSC 120 from last year.

Assignments:

Assignment 1: handout.
Solution: functions, script, knitr::spin output of script, discussion.

Assignment 2: handout, data. Note: For both problems, the M step should be doable analytically, with a reasonably simple formula giving the result. There should be no need for numerical maximization.
Solution: derivations and discussion, Problem 1: functions, script, output, Problem 2: functions, script, output, plot.

Assignment 3: handout.
Solution: derivations, functions, script, output, plots, discussion.

Things regarding tests:

Practice problems for test 1: questions, answers.
Practice problems for test 2: questions, answers.

Answers for test 1

Marks for test 1: On the test you will see written -X +Y in blue or in red, with X being the number of marks taken off for the main part, and Y being the number for the bonus/grad question. If it's in red, multiply X by 3/4 (an adjustment for different leniency by the two markers), and use that new X in the following formulas. If you are an undergrad student (in STA410), your mark is 100-X+Y. If you are a grad student (in STA2102), your mark is 0.9*(100-X)+Y.

Marks for test 2: The mark on test 2 was adjusted by adding 15% of the difference between the mark written and 100.

Examples:

Week1:
Simulating the distribution of the sample mean: function, script.
Illustration of underflow when computing likelihoods: script.
Using simulation to investigate effect of sample size: script.
Week2:
Finding zeros using bisection: functions.
Finding zeros with Newton iteration: functions.
Script using these functions: script.
Week3:
Multivariate Newton iteration: functions.
Maximum likelihood estimate for a Poisson regression model: functions, script (now updated to include "method of scoring").
Maximum likelihood estimation of Poisson mean from interval data with nlm: function.
Week5:
Integration using the midpoint rule: function (with tests), script (using knitr).
To use knitr, you must first install it with install.packages("knitr").
You can then use it with something like knitr::spin("midpoint-script.r").
This creates a file called midpoint-script.html which you can view in a browser.
Week 6:
Demo of transformation for integration: script.
Demo of multivariate integration: script.
Week 7:
EM for censored Poisson data: function, output of test.
EM for censored exponential data: function, output of test.
Week 8:
EM for mixture on binary data: function, test script.
Week 10:
Gibbs sampling for a simple random-effects model: R code.
Gibbs sampling for a latent Poisson process: R function.
Week 11:
Metropolis sampling for a simple Bayesian model: R code.

Web pages for past versions of the course:

Fall 2014 - Spring 2004 - Spring 2003 - Spring 2002 - Fall 2000 - Spring 2000