Notes on Matlab for Assignment 2:

Suppose you have the inputs and responses for training cases stored
in x and y as shown here:

>> x

x =

     3     8
     4     8
     9     7
     8     3
     2     3

>> y

y =

     0
     1
     0
     0
     1

If you have the stats toolbox, you can fit a logistic regression model
(with intercept) for y given x by maximum likelihood as follows:

>> glmfit(x,[y ones(size(y))],'binomial')

ans =

    4.6372
   -0.7888
   -0.2379

The "ones" is needed because glmfit wants a column giving the number
of "trials", which is always one the way we're expressing the problem.

If you don't have glmfit, you can use the following:

>> logLike = @(x, y, beta) sum(y.*(x*beta)-log(1+exp(x*beta)));
>> fminsearch(@(b) -logLike([ones(size(y)) x], y, b), zeros(size(x,2)+1, 1))

ans =

    4.6372
   -0.7888
   -0.2379

The "ones" here is to include an intercept in the model.

You can find the eigenvalues and eigenvectors of a matrix as follows
(I use X'*X as an example since it will be positive definite, as is
the case for covariance matrices):


>> X

X =

     4     3     2
     2     3     1
     1     5     5
     9     8     4

>> [vec val] = eig(X'*X)

vec =

   -0.3246   -0.7081    0.6271
    0.6918    0.2744    0.6679
   -0.6451    0.6506    0.4008


val =

    0.8828         0         0
         0   18.3286         0
         0         0  235.7886

The eigenvalues are on the diagonal of val.  The (right) eigenvectors
are the columns of vec.  The eigenvectors are chosen to have length
one, but that leaves an arbitrary sign (since negating everything
doesn't change the direction).