Programming Collective Intelligence: Building Smart Web 2.0 Applications [Paperback]

This book is probably best for those of you who have read the theory, but are not quite sure how to turn that theory into something useful. Or for those who simply hunger for a survey of how machine learning can be applied to the web, and need a non-mathematical introduction.

My area of strength happens to be neural networks (my MS thesis topic was in the subject), so I will focus on that. In a few pages of the book, the author describes how the most popular of all neural networks, backpropagation, can be used to map a set of search terms to a URL. One might do this, for example, to try and find the page best matching the search terms. Instead of doing what nearly all other authors will do, prove the math behind the backprop training algorithm, he instead mentions what it does, and goes on to present python code that implements the stated goal.

The upside of the approach is clear -- if you know the theory of neural networks, and are not sure how to apply it (or want to see an example of how it can be applied), then this book is great for that. His example of adaptively training a backprop net using only a subset of the nodes in the network was interesting, and I learned from it. Given all the reading I have done over the years on the subject, that was a bit of a surprise for me.

However, don't take this book as being the "end all, be all" for understanding neural networks and their applications. If you need that, you will want to augment this book with writings that cover some of the other network architectures (SOM, hopfield, etc) that are out there. The same goes for the other topics that it covers.

In the end, this book is a great introduction to what is available for those new to machine learning, and shows better than any other book how it applies to Web 2.0. Major strengths of this book are its broad coverage, and the practicality of its contents. It is a great book for those who are struggling with the theory, and/or those who need to see an example of how the theory can be applied in a concise, practical way.

To the author: I expect this book will get a second edition, as the premise behind the book is such a good one. If that happens, perhaps beef up the equations a bit in the appendix, and cite some references or a bibliography for those readers interested in some more in depth reading about the theory behind all these wonderful techniques. (The lack of a bibliography is why I gave it 4 stars out of 5, I really think that those who are new to the subject would benefit greatly from knowing what sits on your bookshelf.)

Segaran has done an excellent job of explaining complex algorithms and mathematical concepts with clear examples and code that is both easy to read and useful. His coding style in Python often reads as clearly as pseudo-code in algorithm books. The examples give real-world grounding to abstract concepts like collaborative filtering and bayesian classification.

My favorite part is how he shows us code (gives it to us!) that goes out into the world, grabs masses of data and does interesting things with it. The use of a hierarchical clustering algorithm to dig into people's intrinsic desires in life as expressed in zebo is worth the price of the book alone. The graph that shows a strong connection between "wife", "kids", and "home" but a different connection between "husband", "children", and "job" is IMHO just fascinating.

Gems like that make this book worth reading cover to cover. After that it can happily hang out on your shelf as a reference anytime you need to build something to mine user data and extract the wisdom of crowds.

Have you ever wondered how some of those "collective intelligence" sites work? How Amazon can suggest books that you'll like based on your browsing history? How a search engine can rank and filter results? Toby Segaran does a very good job in revealing and teaching those types of algorithms in his book Programming Collective Intelligence: Building Smart Web 2.0 Applications. While I'm not ready to run out and build my own version of Facebook now, at least I can start to understand how sites like that are designed.

Contents:
Introduction to Collective Intelligence; Making Recommendations; Discovering Groups; Searching and Ranking; Optimization; Document Filtering; Modeling with Decision Trees; Building Price Models; Advanced Classification - Kernel Methods and SVMs; Finding Independent Features; Evolving Intelligence; Algorithm Summary; Third-Party Libraries; Mathematical Formulas; Index

In each of the chapters, Segaran takes a type of capability, be it decision-making or filtering, and shows how a programming language can be used to build that feature. His examples are all in Python, so it helps if you are already familiar with that language if you want to actually work with the code. But even if you don't know Python, the examples are clear and detailed enough that you can follow along and get the gist of what's happening. I personally think that it would help immensely if you had a background in mathematics and statistics. You can use the code here without having a detailed understanding of math, but I'm sure much of this would be more deeply appreciated if you already know about such things as Tanimoto similarity scores, Euclidean distances, or Pearson coefficients.

From my perspective (a non-Python programmer *without* the math background), I was more interested in understanding the overall picture about things like how ranking systems work or how recommendation engines are structured. While there was more detail than I needed (or understood), I still felt as if I accomplished my goal. I have a much greater appreciation for what companies like Google and Amazon have done in building web applications that allow the knowledge and wisdom of groups to be gathered and applied to my own preferences.

Statistical programmers will probably find years of entertainment here. :) "Normal" programmers will expand their horizons, too.