The Numerati [Hardcover]

Stephen Baker, a technology writer for Business Week, takes us into the world of data miners, forecasters, and matchmakers. The math whizzes who analyze our blogs for trends, create the ads that make us eager to buy, and analyze the chatter that could conceal signs of criminal activity--these are the Numerati. Baker gives us a chapter each on work, shopping, politics, spy vs. spy, healthcare, and even [...] (What does the length of your ring finger have to do with the kind of person you're attracted to? Read and find out.)

Some of it is "house-of-the-future" stuff--imagine, for instance, a floor tile that will alert the doctor when your aging parent's gait seems more hesitant than usual. According to Baker, experts watching old reruns of Michael J. Fox shows can detect characteristic signs years before he was diagnosed with Parkinson's disease.

And then there's the political game. With ever-more-insightful analysis, political math mavens have found that (thank god!) America is nowhere near as polarized as you would expect. Many a liberal Democrat lurks in the McMansion suburbs, and vice versa. But politics is tough--your grocery basket doesn't lie, but nobody wants to give the time of day to a pollster. How they craft the exact political messages that will get you to the voting booth might, oddly enough, be related to your shopping habits.

Shopping--now this is a chapter that should be of interest to every die-hard Amazon fan. Sophisticated algorithms designed to deduce your taste in novels or music can be frighteningly accurate (or, as my Quick Picks occasionally remind me, maddeningly stupid, but that's the topic for a different book). After finishing this chapter, I could think of half a dozen things my grocery store knows about me that I never told them. If they chose to sell their data to magazine publishers, say, we would surely be targeted for the cooking mags ("Look, this family buys at least four units of different fresh herbs a week, and their weight in extra-virgin olive oil every month"). They can tell we have a teenager in the house ("Lots of Clean&Clear products") and could probably guess how old within a year or two ("Look it up--when did they quit buying diapers?"). Any health insurer would be interested in knowing that we spend a lot in produce and seafood, and very little at the meat counter--but what about those frequent trips to the candy aisle? It's a false positive, I swear--they're for the snack bar at my office!

You should be a little frightened, and more than a little fascinated, by The Numerati.

[Edited to add: For a more detailed look at the doings of one of the Numerati, take a look at Click: What Millions of People Are Doing Online and Why it Matters, by Bill Tancer of Hitwise.]

I became interested in this book after reading the companion cover story in BusinessWeek. Although the stories and interviews were interesting, I thought the book fell short on connecting the math beyond the most basic concepts.

Baker admits he was a liberal arts major in college and doesn't pretend to fully understand the math behind the analysis. Obviously, an in-depth mathematical discussion would have been beyond the grasp of most readers and presumably the author. However, a little more detail on the methodologies beyond the simplistic descriptions would have given the book more substance and utility.

Data Mining and Data Warehousing have been around for many years. Retailers have used it extensively to understand their customers. Yet, Baker fails to discuss these established practices and compare them with this new emerging area.

Baker spends most of his book describing the people he interviews in a series of stories. The book is an easy read and is entertaining. If you read for entertainment and are interested in this subject, you will probably like this book. However, if you read for knowledge and are looking for a good, informative business book on this subject, it may disappoint you.

I would highly recommend reading Baker's book immediately before or after reading How to Measure Anything: Finding the Value of "Intangibles" in Business by Douglas Hubbard. Baker would probably consider Hubbard one of the "numerati". Both authors talk about some of the specifics of the analysis methods (but moreso Hubbard) and both talk about the general trends and impacts (but moreso Baker).

Like his table of contents (which is simply worker, shopper, voter, blogger, terrorist, patient, lover), Baker's book is sweeping if a bit terse in places. As a quant, I find Numerati an easy read with virtually no math but still enlightening even for the most quantitatively adept reader. There were several examples in Baker's book where I already knew of the mathod but had not heard of that application. He did some great research and covered a lot of topics in this giant and elaborate field of work.

My main concern for many management-level readers of this book is that in some cases Baker gives a reader just enough information to think they can apply it to a similar problem they have, falling into the "a little knowledge is a dangerous thing" trap. Again, this can be offset with a read of Hubbard's book. It might also have been helpful to talk about the rise of "crackpot rigour" in a world with lots of data and relatively few competent mathematical analysts (various "data mining" experts come to mind).

In all, its one of my favorite reads of the year. I felt like someone was finally casting light on my own obscure field.