STA 4508: Topics in Likelihood Inference Fall 2018

Tuesdays 10-1. First class October 16.
On October 23 we meet in SS 6004. October 30 through November 20 we meet in SS 2110.

Topics

  1. Inference based on the likelihood function: derived quantities, limiting distributions, approximations to posterior distributions;
  2. Likelihood for semi-parametric and non-parametric models: proportional hazards regression, partially linear models, penalized likelihood;
  3. Composite likelihood: definition, summary statistics, asymptotic theory; applications
  4. Likelihood inference for p > n;
  5. Simulated likelihoods, indirect inference and approximate Bayesian computation

Running list of references and background reading

Review Papers

  • Reid, N. (2013) Aspects of likelihood inference Bernoulli 19, 1404-1418.
  • Reid, N. (2010) Likelihood Inference Wiley Interdisciplinary Reviews in Computational Statistics 5, 517-525.
    (I need to use Preview to view this, rather than Adobe.)
  • Reid, N. (2011) Likelihood International Encyclopedia of Statistical Science, Part 5, 455-459.
  • Reid, N. (2000) Likelihood. J. Am. Stat. Assoc., 95, 1335-1340.

Likelihood Basics

  • Davison, A.C. (2003) Statistical Models (SM) Cambridge University Press. -- Ch 4
  • Barndorff-Nielsen, O.E. and Cox, D.R. (1994) Inference and Asymptotics (BNC) Chapman and Hall. -- Ch 2.2
  • Cox, D.R. and Hinkley, D.V. (1974) Theoretical Statistics (CH)
    • Chapman and Hall. -- Ch 2.1 (i), (ii)
  • Cox, D.R. (2006) Principles of Statistical Inference (Cox) -- Ch.2.1

Marginal/conditional/profile/adjusted likelihood

  • Properties of profile likelihood: BNC -- Ch 3.4-7
  • Marginal and conditional likelihood functions: SM -- Ch 12.2,3
  • Adjustments to profile likelihood: SM -- 12.3; BNC -- 8.2; Cox & Reid (1987). JRSS B, 49, 1--39.
  • Laplace approximation: SM -- Ch 11.3; *Tierney & Kadane (1986), JASA 81, 82--86.

Semi-parametric and empirical likelihood

  • Davison, A.C. (2003) Statistical Models Ch.5.4; 10.8.
  • Cox, D.R. (1972). Regression models and life tables. JRSS B.
  • *Cox, D.R. (1975). Partial likelihood. Biometrika.
  • Murphy, S.A. and van der Waart, A. (2000). On profile likelihood. JASA
  • van der Waart, A. (1998). Asymptotic Statistics Ch.25.
  • Owen, A. (2000). Empirical Likelihood Ch.2.

Composite likelihood

  • Varin (2008) Advances in Statistical Analysis On composite marginal likelihoods. Available here
  • Varin, Reid and Firth (2011). Statistica Sinica An overview of composite likelihood methods. Available here
  • Cox and Reid (2004). Biometrika A note on pseudolikelihood constructed from marginal densities. Available here

Examples

  • Renard et al. (2004). Computational Statistics and Data Analysis A pairwise likelihood approach to estimation in multilevel probit models. Available here
  • Xu, X. and Reid, N. (2011) On the robustness of maximum composite likelihood, J. Statist. Plann. Inf., 141, 3047 -- 3054.
  • Xue et al. (2012) Nonconcave penalized composite conditional likelihood estimation of sparse Ising models. An.. Statist. 40, 1403--1429.
  • Ravikumar, P., Wainwright, M., Lafferty, J. High-dimensional Ising model selection using $\ell_1$-regularized logistic regression. Ann Statist 38, 1287-1319
  • Davison, A.C., Padoan, S. and Ribatet, M. (2012). Statistical modeling of spatial extremes. Statistical Science 27, 161-186
  • Varin, C. and Czado, C. (2010). A mixed autoregressive probit model for ordinal longitudinal data. Biostatistics 11, 127-138.
  • Henderson, R. and Shimakura, S. (2003). A serially correlated gamma frailty model for longitudinal count data. Biometrika 90, 335-366.

Misspecified Models

  • Kent, J. T. (1982). Robust properties of likelihood ratio tests. Biometrika 69, 19-27
  • White, H. (1982) Maximum likelihood estimation of misspecified models. Econometrica 50, 1--25.
  • Lindsay, B.G. (1988). Composite likelihood methods. Contemporary Mathematics 80, 221-239.
  • Liang, K.-Y. and Zeger, S.L. (1986) Longitudianl data analysis using generalized linear models. Biometrika 75, 13--22.
  • Gouerieroux, C. et al. (1993) Indirect inference. J. Appl. Econometrics 8 S85-S118
  • Smith, A.A. (2008). Indirect inference. in New Palgrave Dictionary of Economics

Approximate Bayesian Computation

  • Marin, J.-M., Pudlo, P., Robert, C.P., Ryder, R.J. (2012) Approximate Bayesian computational methods Stat Computing 22,1167--1180.

High-dimensional inference

  • Bühlmann, P., Kalisch, M. and Meier, L. (2014). High-Dimensional Statistics with a View Toward Applications in Biology.Annual Review of Statistics and its Application 1, 255--278.
  • Sartori, N. (2003). Modified profile likelihoods in models with stratum nuisance parameters. Biometrika 90, 533-549.
  • Portnoy, S. (1984). Asymptotic behavior of M-estimators of p Regression Parameters ... I. Ann. Statist. 12, 1298-1309.
  • Portnoy, S. (1985).Asymptotic behavior of M-estimators of p Regression Parameters ... II. Ann. Statist.13, 1403--1417.
  • Portnoy, S. (1988).Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity. Ann. Statist. 16, 356--66.

Report and Presentation

Choose a paper on a likelihood-related topic: suggestions include the papers above marked *, papers on the list below, or possibly one of the papers you chose for the Exercises in week 1.
Plan for a 15-20 minute presentation, either slides or blackboard. The report will be a typed note based on your presentation -- approximately five pages.
Please let me know by November 13 which paper you will review: provide a complete citation This handout has more details on the report outline.

  • Varin and Czado (2010). Biostatistics A mixed autoregressive probit model for ordinal longitudinal data. Available here
  • Hirosi, Y. (2011). Efficiency of profile likelihood in semi-parametric models. Ann. Inst. Stat. Math. 63, 1247-1275
  • Zeng, D. and Lin, D.Y. (2010). A general asymptotic theory for maximum likelihood estimation in semiparametric regression models with censored data.Stat. Sinica 20, 871-910
  • Bühlmann, P. (2013) Statistical significance in high-dimensional linear models Bernoulli 19, 1212-1242.
  • Portnoy, S. (1988). Asymptotic behaviour of likelihood methods for exponential families when the number of parameters tends to infinity. Ann. Statist. 16, 356-366.
  • Xue, L., Zou, H., Cai, T. (2012). Nonconcave penalized composite conditional likelihood estimation of sparse Ising models. Ann Statist. 40, 1403-1429.
  • Rue, H. Martino, S., Chopin, N. (2009). Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. JRSS B 71, 319-392.
  • Ravikumar, P., Wainwright, M., Lafferty, J. High-dimensional Ising model selection using $\ell_1$-regularized logistic regression. Ann Statist 38, 1287-1319.
  • Kuk, A. (2007). A hybrid pairwise likelihood method. Biometrika 97, 939-952.
  • Kent, J. T. (1982). Robust properties of likelihood ratio tests. Biometrika 69, 19-27.

November 20

November 13

November 6

  • Slides
  • Exercise for this week is to choose the paper that you will use for your report, to give me a complete citation for it and a one sentence description. And to make any changes or completions for Exercises from Oct. 23.

October 30

October 23

October 16