Big Data: Principles and best practices of scalable realtime data systems Paperback – May 10 2015
Frequently Bought Together
Customers Who Bought This Item Also Bought
No Kindle device required. Download one of the Free Kindle apps to start reading Kindle books on your smartphone, tablet, and computer.
Getting the download link through email is temporarily not available. Please check back later.
To get the free app, enter your mobile phone number.
About the Author
Nathan Marz is currently working on a new startup. Previously, he was the lead engineer at BackType before being acquired by Twitter in 2011. At Twitter, he started the streaming compute team which provides and develops shared infrastructure to support many critical realtime applications throughout the company. Nathan is the creator of Cascalog and Storm, open-source projects which are relied upon by over 50 companies around the world, including Yahoo!, Twitter, Groupon, The Weather Channel, Taobao, and many more companies.
James Warren is an analytics architect at Storm8 with a background in big data processing, machine learning and scientific computing.
What Other Items Do Customers Buy After Viewing This Item?
Top Customer Reviews
Most Helpful Customer Reviews on Amazon.com (beta)
You may not agree with all that the authors offer or contend in this well-written "theory" text. But Nathan Marz’s Lambda Architecture is well worth serious consideration, especially if you are now trying to come up with more reliable and more efficient approaches to processing and mining Big Data. The writers' explanations of some of the power, problems, and possibilities of Big Data systems are among the clearest and best I have read.
"More than 30,000 gigabytes of data are generated every second, and the rate of data creation is only accelerating," Marz and Warren point out.
Thus, previous "solutions" for working with Big Data are now getting overwhelmed, not only by the sheer volume of information pouring in but by greater system complexities and failures of overworked hardware that now plague many outmoded systems.
The authors have structured their book to show "how to approach building a solution to any Big Data problem. The principles you’ll learn hold true regardless of the tooling in the current landscape, and you can use these principles to rigorously choose what tools are appropriate for your application.” In other words, they write, you will “learn how to fish, not just how to use a particular fishing rod.”
However, a particular Big Data architecture IS featured, as well: Marz's Lambda Architecture. It is, the two authors explain, "an architecture that takes advantage of clustered hardware along with new tools designed specifically to capture and analyze web-scale data. It describes a scalable, easy-to-understand approach to Big Data systems that can be built and run by a small team."
The Lambda Architecture has three layers: the batch layer, the serving layer, and the speed layer.
Not surprisingly, the book likewise is divided into three parts, each focusing on one of the layers:
In Part 1, chapters 4 through 9 deal with various aspects of the batch layer, such as building a batch layer from end to end and implementing an example batch layer.
Part 2 has two chapters that zero in on the serving layer. "The serving layer consists of databases that index and serve the results of the batch layer," the writers explain. "Part 2 is short because databases that don’t require random writes are extraordinarily simple.”
In Part 3, chapters 12 through 17 explore and explain the Lambda Architecture’s speed layer, which “compensates for the high latency of the batch layer to enable up-to-date results for queries.”
Marz and Warren contend that "[t]he benefits of data systems built using the Lambda Architecture go beyond just scaling. Because your system will be able to handle much larger amounts of data, you’ll be able to collect even more data and get more value out of it. Increasing the amount and types of data you store will lead to more opportunities to mine your data, produce analytics, and build new applications."
This book requires no previous experience with large-scale data analysis, nor with NoSQL tools. However, it helps to be somewhat familiar with traditional databases. Nathan Marz is the creator of Apache Storm and originator of the Lambda Architecture. James Warren is an analytics architect with a background in machine learning and scientific computing.
(My thanks to Manning for providing a review copy of this book.)
The only downside to the book is that the architecture and exosystem is so new that there's not really a lot of pragmatic solutions. For example, the theory describes a query layer that can merge the results of batch and real time processing for client applications. However, in real life there are no pragmatic solutions for doing this so you'd have to write your own.
It'll be interesting to see how the lambda architecture matures and to see future editions of this book. Hopefully, future editions will be as well written and have a better ecosystem for practice chapters.
The book is very organized. Introduction in chapter 1 will be the road map of the whole book. Motivating with a simple web application based on RDBMS, the author showed how the approach to scale it becomes undesirable. After enumerating a list of desired properties, he proposed Lambda architecture, an approach in contrast to fully incremental architecture (with RDBMS).
The Lambda architecture is partitioned into three layers:
1. batch layer that computes different views on big data
2. serving layer that answers user queries using views from the batch layer and speed layer.
3. speed layer that compensates an approximate answer over a period time when the batch layer is working on the complete answers.
In the remaining chapters, the author dive deep into the rationale and requirements of all the different pieces of Lambda Architecture.
To under the context of Lambda Architecture, also refer to the wikipedia for crticism.
Look for similar items by category
- Books > Computers & Technology > Computer Science > Software Engineering > Information Systems
- Books > Computers & Technology > Databases > Data Mining
- Books > Computers & Technology > Hardware
- Books > Computers & Technology > Internet & Social Media
- Books > Computers & Technology > Networking & Cloud Computing > Internet, Groupware, & Telecommunications > Groupware
- Books > Computers & Technology > Networking & Cloud Computing > Network Administration
- Books > Computers & Technology > Programming
- Books > Computers & Technology > Software > Business
- Books > Computers & Technology > Web Development > Programming
- Books > Computers & Technology > Web Development > User Generated Content