STA 414/2104: Project suggestions

Projects may be done individually or in groups of two. Groups of more than two are allowed only if you convince me that there is some special reason for them.

Please let me know if you are interested in one of these suggested projects, since only one group can do each one. You may also do a suitable project of your own devising, but you should talk to me first to make sure it's OK.

Projects involving research on methodology

Note that these are research ideas, for which I don't know the outcome - the main point of the project is to find out whether the idea is good or bad. I also don't know whether these ideas have been considered by someone else (part of the project would be to try to determine that).

Cross-validation modification of PLS

Partial Least Squares (PLS) finds one or more directions in input space based on the covariance of the inputs with the response. In PLS regression, the projections on these directions are then used as predictors in a model of the response. The use of the response both to find directions and to find regression coefficients might perhaps lead to overfitting. Can better results be obtained by splitting the training data into two parts, using one part to find the PLS directions, and the other part to find the regression coefficients? One might especially hope that the estimate of how well the model predicts will be more accurate this way. This project is no longer available, since it has been taken by a student

Shrinkage of class means in LDA

Logistic regression is often improved by shrinking the regression coefficients towards zero using some penalty. Can an analogous method be found for Linear Discriminant Analysis (LDA)? The idea would be to estimate the means for the classes using a penalty, which shrinks them towards either the overall mean of the data or the mean of the sample means for the classes (there would be a difference between these options if the classes are have unequal numbers of cases). The common covariance would then be estimated from deviations of cases from these estimated means. This project is no longer available, since it has been taken by a student

Comparison of methods for setting GP hyperparameters

As mentioned in class, one might set the hyperparameters for a Gaussian process model either by cross validation or by maximizing the probability of the training data. The latter is the "correct" Bayesian approach, but might possibly be less robust in situations where the Gaussian process model is not very good. The aim of this project would be to compare these methods, on simple artificial datasets, some for which we know that a Gaussian process is suitable, and some for which we know that a Gaussian process is not suitable (eg, a function with discontinuities). This project is no longer available, since it has been taken by a student