Hadoop: The Definitive Guide [Paperback]

The second edition of the already fantastic Hadoop: The Definitive Guide adds the last few missing bits to the best Hadoop reference out there.

For those not familiar with the first edition, Hadoop: The Definitive Guide is exactly what it claims to be. If you're not already familiar with Hadoop, the first and second chapters (Meet Hadoop and MapReduce, respectively) take you through the basics in both concept as well as code. For those used to writing data processing applications, the rationale behind Hadoop and why it's useful are immediately apparent. If you've already been exposed to Hadoop, these chapters may be redundant but they're worth reading anyway the first time through.

The chapter on HDFS does a great job at explaining the underbelly of Hadoop's distributed file system including the Java APIs. The section on Hadoop IO is probably introduced a bit too early - Hadoop newbies probably don't care about compression and serialization prior to reading about map reduce - but excellent none the less in its detail. That said, you'll *really* want to go back and read it to understand the details of how compression codecs work after you learn more about map reduce.The "Writing a Map Reduce Application" chapter is probably the one existing users of Hadoop will skip. First timers will definitely get a lot out of a step by step walk through of a Java MR job from beginning to end.

The chapters on how map reduce works, types and formats (including input / output format details), and the advanced features (counters, sorting, the distributed cache, join libraries) are the ones you'll reread and reference constantly. The explanation, for instance, on how input splits are calculated demystifies the border between HDFS and the map reduce layer (and finally answers the question of "how does Hadoop know not to split in the middle of a record?"). Buy this book for these chapters, if not for the others.

The chapters on HBase, Pig, ZooKeeper, and Sqoop are excellent and, in some cases, the best reference on the topic to date.

There are enough corrections, updates, and new chapters that it's worth buying the second edition if you already have the first. For anyone new to Hadoop this is a must have. If you already use Hadoop the later chapters are what you're looking for; a deep explanation of not just "how," but "why."

Some reviewers have noted the discussion of deprecated APIs. This really isn't a flaw of the book, but of premature deprecation within Hadoop itself. The newer APIs didn't have all the features of the old and anyone writing production map reduce jobs would wind up needing a lot of those features. I think the author does a great job with a tough situation while still alerting the reader that newer APIs are on the horizon. Besides, the differences are so few that it's almost not worth mentioning. While APIs may change, the core design, execution model, and architecture of Hadoop haven't changed and this is the best book on the subject.

Hadoop's MapReduce and HBase went through a major API change right around the time this book was finishing up. Consequently, if you try to use the examples in the book as a guide while developing against either the Apache Hadoop latest release or against Cloudera's CDH3, you'll find a mountain of frustration in the form of deprecated or entirely deleted classes.

This book provides an excellent in-depth overview of all aspects of Hadoop with how-to examples that are easy to follow. It is well written, thorough and exactly what I needed to architect and build a Hadoop-based solution. Related technologies such as Hive, HBase, Sqoop, Pig and Zookeeper are also covered in decent depth.

Other reviewers gave poor reviews due to the APIs being not up to date, which I think is unfair. Those new APIs are still only available in early unstable Hadoop versions, so current developers are best served to use the earlier APIs. The book gives samples with new APIs and shows very clearly the API changes which are minor. The concepts are identical, but a few classes have been combined into a more cohesive "Context" class in the new APIs.

So, for example, to write a data record you call "context.collect(...);" rather than "output.collect(...);" with identical parameters. The structure of applications and the concepts are not changed. The changes to the syntax of Java calls is trivial and covered in the book very clearly. What is the big deal? Understanding the concepts is the most important thing and this book provides this very nicely.

I would recommend this book to anyone who is new to Hadoop and needs to learn it in depth.

› Go to Amazon.com to see all 8 reviews  3.6 out of 5 stars