Effective awk Programming: Text Processing and Pattern Matching [Paperback]

Awk is one of those handy Unix tools with which you can easily impress people. Using a simple

/pattern/ { action }

syntax, you can construct powerful one-liners. Do you want to how much time in total you spent surfing the Internet? Here it is:

awk '/Connect time/ { s += $8 } END { print s }' /var/log/messages

It doesn't get much shorter in any other programming language, does it? Need to strip text of HTML tags? Need a frequency count of words in a text? Awk is the perfect tool for tasks like this. With its pattern-action structure, powerful regular expression mechanism, associative arrays and basic program flow control, it provides a powerful tool for manipulating flat text files. Even though other scripting languages may be richer in features, there exists a niche where Awk is just the right tool to do the job.

Arnold Robbins, the author of this book as well as of several other books on Awk, serves also as the maintainer of GNU Awk (gawk for short), the most influential version of Awk available today. With the version 3.10, released in 2001, GNU Awk became richer for a handful of new extensions over traditional Awk, most important among them are the TCP/IP networking and the support for internationalization. All new extensions are described in the book. How successful these new extensions will be is doubtful, however. Networking scripting niche is already well covered with Perl and Python, and internationalization doesn't really matter much in short throw-away scripts Awk is usually used for.

With all due respect to the creators of Awk and their book (Alfred V. Aho, Peter J. Weinberger, Brian W. Kernighan, The Awk Programming Language, Addison-Wesley, 1988), I have to say that "Effective Awk Programming" is probably the best Awk tutorial on the market today. If you are serious about learning Awk, you shouldn't be without it. If you are still hesitating whether it wouldn't be wiser investing those 28 USD elsewhere, here is chance to read it before you buy it: install GNU Awk 3.10, and the Texinfo source of the book comes with it. But sooner or later you will find O'Reilly RepKover binding too tempting...

This book is being picked up by the O'Reilly people. I have many of the O'Reilly books on my book shelf. There will be a third edition due out in July with Robbins as the author. I read though this book and thought it as good or better than the "Sed & awk" book that O'Reilly presently has out. Robbins is also an author on this book. I thought the book to be better than the "AWK programming language" by Aho, Kerninghan and Weinberger, the original authors of the AWK language. I recommend the book to beginers as I am.

This book explains both the awk language and how to run the awk utility. You should already be familiar with basic system commands, such as cat and ls, as well as basic shell facilities, such as input/output redirection and pipes. This book describes the awk language in general and also the particular implementation of awk called gawk. gawk runs on a broad range of Unix systems and has also been ported to Mac OS X, MS-DOS, Microsoft Windows, and VMS.

Many people are familiar with O'Reilly's book on sed and awk, but not this book. If you want to zero in on awk and its capabilities, this is really the better of the two books. It makes an excellent tutorial and reference for system administrators and anyone else that wants to use awk to extract and format text. The following is a description of the book from the context of the table of contents:

Chapter 1. The awk language and gawk - talks about the basics including how to run awk, when you should use awk, and starts you off with a few simple examples.

Chapter 2. Regular expressions - introduces regular expressions in general, and in particular the flavors supported by POSIX awk and gawk.

Chapter 3 Reading Input Files - describes how awk reads your data. It introduces the concepts of records and fields, as well as the getline command. I/O redirection is first described here.

Chapter 4. Printing Output - Besides basic and formatted printing, this chapter also covers I/O redirections to files and pipes, introduces the special filenames that gawk processes internally, and discusses the close built-in function.

Chapter 5. Expressions - describes expressions, which are the basic building blocks of awk patterns and actions.

Chapter 6. Patterns, Actions, and Variables - Each awk statement consists of a pattern with an associated action. This chapter describes how you build patterns and actions, what kinds of things you can do within actions, and awk's built-in variables.

Chapter 7. Arrays in awk - describes how arrays work in awk, how to use array elements, how to scan through every element in an array, and how to remove array elements. It also describes how awk simulates multidimensional arrays, as well as some of the less obvious points about array usage. The chapter finishes with a discussion of gawk's facility for sorting an array based on its indices.

Chapter 8. Functions - describes awk's built-in functions, which fall into three categories: numeric, string, and I/O. gawk provides additional groups of functions to work with values that represent time, do bit manipulation, and internationalize and localize programs.

Chapter 9. Internationalization with gawk - describes the underlying library gawk uses for internationalization, as well as how gawk makes internationalization features available at the awk program level. Having internationalization available at the awk level gives software developers additional flexibility - they are no longer required to write in C when internationalization is a requirement.

Chapter 10. Advanced Features of gawk - a "grab bag" of items that are otherwise unrelated to each other. First, a command-line option allows gawk to recognize nondecimal numbers in input data, not just in awk programs. Next, two-way I/O, discussed briefly in earlier parts of this book, is described in full detail, along with the basics of TCP/IP networking and BSD portal files. Finally, gawk can profile an awk program, making it possible to tune it for performance.

Chapter 11. Running awk and gawk - covers how to run awk, both POSIX-standard and gawk-specific command-line options, and what awk and gawk do with non-option arguments. It then proceeds to cover how gawk searches for source files, obsolete options and/or features, and known bugs in gawk. This chapter rounds out the discussion of awk as a program and as a language. While a number of the options and features described here were discussed in passing earlier in the book, this chapter provides the full details.

Chapter 12. A Library of awk Functions - One valuable way to learn a new programming language is to read programs in that language. To that end, this chapter and Chapter 13 provide a good-sized body of code for you to read, and hopefully, to learn from.

Chapter 13. Practical awk Programs - presents a potpourri of awk programs for your reading enjoyment. The first part describes how to run the programs presented in this chapter. The second presents awk versions of several common POSIX utilities. These are programs that you are hopefully already familiar with, and therefore, whose problems are understood. By reimplementing these programs in awk, you can focus on the awk-related aspects of solving the programming problem.

Chapter 14. Internetworking with gawk - describes gawk's networking features in depth, including a number of interesting examples and the reusable core of a gawk-based web server. The chapter is adapted from "TCP/IP Internetworking with gawk", by Jürgen Kahrs and Arnold Robbins, which is a separate document distributed with gawk.

Thus chapters 1-11 form a tutorial and reference on awk itself, and the last three chapters are additional material for reference or your own personal toolchest of programs. The last chapter is more of a curiosity than anything, since I don't know anyone who does internetworking with gawk, but it is still interesting material. I highly recommend this book if you plan to use awk extensively.

› Go to Amazon.com to see all 5 reviews  4.4 out of 5 stars