Numerical Optimization [Hardcover]

This book by Nocedal and Wright has several attractive features. For one, it is probably the most "state-of-the-art" of the existing texts in optimization and as such covers most of the modern methods. It also has a nice section on LP (simplex as well as interior point methods) for someone interested in a course on optimization as opposed to NONLINEAR optimization (which is what I was looking for). Another strength is that it covers many of the algebra-related details very well. My only major complaint is that it seems to not get into any of the methods designed specifically for convex programs - these while admittedly less general are often very powerful. For example, there is NO mention even of Geometric Programming which has wide application in design. The convex simplex method also isn't mentioned anywhere. Finally,I wonder why there is no mention of the generalized reduced gradient (GRG) method.

All in all, a good book to own I think...

Within the range that this intends to cover, it is an outstnading reference. The first two chapters lay out the mathematical preliminaries, and get the book off to a fast start. The next four chapters discuss basic classes of algorithms for nonlinear optimization and choices of stopping criteria. This includes conjugate gradient methods adapted from the CG method for solving linear systems - since, in nearly all cases, non-linear optimization breaks down into iterations over locally linear approximations.

The emphasis thoughout is on practical algorithms and efficient computation. First and second derivatives are used heavily throughout this book, but symbolic differentiation of the nonlinear functions is usually unavailable. As a result, significant emphasis goes into approximation techniques, and into the common cases of sparse systems. Despite its heavily mathematical orientation, this really is a book about the practicalities of computation.

A bit further on, Nocedal and Wright get to the topic that brought me to this book in the first place: nonlinear least squares. As always, the presentation is clear but very dense. Other topics follow, including solutions of nonlinear equations (i.e. minimizing the error in approximating the exact solution), simplex and polynomial-order techniques for linear systems, and more.

This is a book for someone who's completely at home with differential calculus and linear algebra, and who's willing to spend time extracting the full meaning from terse descriptions. It's also for a reader who is comfortable translating dense notation into working numerical code - not a task to be undertaken lightly. That reader will be rewarded with wide-ranging and very practical discussions of many problems and the techniques used for each. As it says in the introduction, this doesn't address the whole world of optimization problems - combinatorics, discrete problems, and jagged search spaces are not the subject here. If, however, this book touches on your topic, you'll find it handled very well. This has my highest recommendation.

//wiredweird

The book does a very good job in teaching non-discrete mathematical programming techniques. But, it is not an introductory book. The reader is supposed to know linear algebra and numerical analysis to a certain extent. Most of the modern techniques are presented, but the layout is a little chaotic- the sequence of subjects could be made better. So, I would have preferred to give it 4.5 stars (which is impossible). However, that does not take away the fact that the book is excellent. I have used it primarily for modelling financial portfolios, and I am sure it can be used as a guide for other applications.

Conclusion: A little difficult, but well worth the time and money involved