Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning [Hardcover]

This book surprised me. I was expecting a book filled with a discussion of mostly traditional multivariate techniques supplemented by a few chapters of more recent developments. Instead, I found a completely new and refreshing approach to statistics and data exploration that framed the classical regression approach to most issues as a special, limiting case of a broader view of data exploration and analysis.

Sections on random vectors and matrices, nonparametric density estimation, tree methods, ANI, support vector machines, random forests, bagging and boosting, latent variables, manifold learning, and other topics are discussed and explored in adequate depth for an introductory text. The book assumes you know matrix algebra and have had some exposure to probability distributions, and common multivariate methods, but it extends the discussion in areas that are usually only covered in separate advanced texts and research papers.

The book is a little light on Bayesian methods but some compromises had to be made considering the bulk of the range of new material discussed. I especially liked the broad array of examples from genetics, medicine, physics, and other application areas and the nice color graphs where needed. The references to Matlab, R, S-Plus and other standard math packages was much appreciated although I would have liked Mathematica to have been included as well.

Overall, this is a wonderful survey of a wide range of multivariate techniques and methods. I hope it gets incorporated in college grad and undergrad courses.

Traditional graduate level texts such as Ted Anderson's focus on the multivariate normal distribution and its statistical properties. So out of that we get MANOVA, Hotelling's T square, linear and quadratic discriminant analysis, principal component analysis, Wishart distributions and canonical correlations. As other reviewers have said this book is quite different. You don't see those topics as chapters in this book. In fact most of these topics are avoided. Izenman finds that with large dimensional data sets that come up in practice these classical techniques do not work very well. So he takes a more modern and "nonparametric' approach. Color adds to the attractiveness of the book although often not essential to the graphical description of the data.

The book begins with exploratory data analysis and extends it to the realm of data mining. The ability to do analysis like this on large data sets comes from the amazing advances in computer speed. Several important concepts are introduced in intuitive ways including pattern recognition and machine learning, prediction error, cross-validation and bootstrap, and overfitting of models.

Chapter is again aimed at the practical by emphaiszing data structure and data bases and by introducing data quality issues including data inconsistencies, outlying observations (which becomes more complicated in multivariate analysis as many directions in a multivariate space can be considered extreme), missing data, and common to today's research data containing many variables but only a few observations such as gene expression on microarrays and satellite images.

But great ideas are not always modern. Izenman points to the curse of dimensionality, a concept coined by Richard Belman back in 1961. Chapter 3 on random vectors and matrices is the one place where the multivariate normal theory is explicitly covered.

Chapter 4 is truly nonparametric and covers multivariate density estimation with instructional examples sprinkled throughout the chapter. Chapter 5 deals with multiple regression a very important and common technique that is described in many texts. Izenman starts with some historical perspective going back to work on least squares by Gauss, Laplace and Legendre where the determination of planetary orbits were modeled circa 1800 and the work of Galton on heredity and regression to the mean in the 1880s and 90s. He gets to all the classical work but also discussion prediction error, the bias of the apparent error rate for a model estimate and the use of cross-validation and the bootstrap as ways to remove large biases in estimates. Again teh techniques are demonstrated with real examples. He discusses some reasonable biased regression approaches including ridge regression, principal components regression and partial least squares regression. Some of these techniques are new even to me (an aging statistician). Since practical problems often involve many potential variables of which some may be unimportant or highly correlated with others, practical regression analysis often use variable selection techniques. Izenman explains the methods and the associated controversies with them. He then introduces some modern approaches not seen outside the research literature including regularized regression (Friedman's general penalized least square approach and the Tibshirani's lasso and Brieman's garotte). He also devotes a whole section to least angle regression developed by Efron, Hastie, Johnstone and Tibshirani.

Chapter 6 generalizes to multivariate regression which includes MANOVA and MANCOVA. Chapter 7 deals with linear dimensionality reduction which includes the classical principal component analysis, canonical variables and canonical correlation and generalizations and then moves to the not so commonly treated topic of project pursuit. In several of the chapters including chapter 7 software packages are listed that can be used to implement the techniques described in the chapter.

Chapter 8 introduces the classification problem with the classical approach of linear discriminant analysis which leads to the nonparametric approach in Chapter 9 sometimes called recursive partitioning but because of the fundamental book Classification and Regression Trees by Brieman, Olshen, Friedman and Stone the more common and popular term is tree-based methods. In Chapter 9 Izenman also includes extensions to these methods which include survival trees and Friedman's multivariate adaptive regression splines. Other approaches coming from the disciplines of artificial intelligence and computer science are the subjects of Chapters 10 and 11, neural netowrks and support vector machines respectively.

Chapter 12 covers unsupervised learning through techniques called cluster analysis methods. Chapter 13 covers multidimensional scaling (here color plays a useful role). Chapter 14 is called Committee Machines. This incorporated the great breakthroughs to improving classification algorithms; bootstrap aggregating which Breiman called "bagging" and boosting algorithms of Schapire and Freund in the early 1990s. Also random forests which introducing a randomization component to bagging also due to Brieman is also discussed.

Later chapters include nonlinear dimensionality reduction, exploratory factor analysis and ending up with a multivariate technique called correspondence analysis that got a lot of attention by the french school of statisticians but was largely ignored in the US for many years.

Aside from the many unique and modern topics discussed in this book what really sets it apart is the academic thoroughness from including a large bibliography of over 550 references, with bibliographic notes at the end of each chapter, illustrative and relevant examples expertly placed throughout the chapters, numerous homework exercises starting with Chapter 2 and a list of software tools for implementing the methods where applicable (every chapter from 7 through 17). As a statistician with interest in bootstrap methods I was particularly pleased with the heavy emphasis on the use of the bootstrap where it has been most successful and gratified that in addition to referencing the commonly referenced bootstrap texts Efron and Tibshirani(1993), Davison and Hinkley(1997) and Hall(1992), he also mentions Chernick (1999). Very little was left out on modern methods. The only things I can think of that are not included are the use of influence functions to detect multivariate outliers as described in Gnandesikan's text and the work of Pesarin and his colleagues on multivariate permutation tests.

This is not Johnson and Wichern or TW Anderson - Think Bishop (PRML) or Hastie, Tibshirani and Friedman (EoSL). We used this for a course last year and this is a great book - as opposed to Bishop which treats things form a Com. Sci. perspective or HTF which assumes a much higher level. One warning though - don't be turned off by the multivariate notation (Duh... Look at the title, of course), but once you master the early chapter on matrix theory and analysis, everything else is very readable.

› Go to Amazon.com to see all 7 reviews  4.6 out of 5 stars