Network Monitoring and Analysis: A Protocol Approach to Troubleshooting [Paperback]

Did you ever wonder what in the world was happening under the hood of your network? Why things are running slowly, what causes print jobs to suddenly fail, or programs to unexpectedly terminate? Are you looking for something to do when users call up complaining that the server is slow? Do you want to get a better feel for how much traffic is actually getting through? If so, then this book is for you because it is about network monitoring and analysis—perhaps the least understood of all administrator activities.

For many, the question is not, why should I perform network monitoring; indeed, it seems a rather intuitive thing. The question is, when do I find time? Couple this with the fact that there is a learning curve, which must be met before the most useful information can be gleaned, and you have lots of resistance. In some respects it is like changing the oil in your car. You know it needs to be done; however, you do not want to get dirty.

The time to learn how to use the tool is not when the network is down, but when things are running well. Network monitoring throws open the door to your data communication stream allowing you to seize new vistas of understanding. We will share many insights with you and hint at possibilities for further exploration. In fact, you will find many ideas for setting up a regular monitoring and analysis program inside these covers. Some of the areas that will pay the greatest dividends are troubleshooting, optimization, and security concerns, each of which commands considerable attention.

In this book, we look at the protocols likely to be present on your network and describe many of the sources of traffic. Through this understanding, we arrive at a plan for fine-tuning our communication scenarios, and we offer solutions that work in real life settings. Once the traffic is characterized, we can reasonably predict the effect of adding additional services or computers onto the network. This gets us out of the reactive mode of chasing phantoms and goblins and allows us to take a proactive, balanced approach to network management. Through traffic prediction, we are able to determine infrastructure requirements and implement solutions before their need even becomes apparent to the users.

Networks are noisy places. As you look at network traces, one thing that stands out is the sheer volume of data that passes along the wire every second. It is amazing that more data is not lost. We will see several areas to reduce some of the noise. However, in network optimization, the mantra "nothing is free" holds especially true. As we tweak the operating system, we need to see what we are giving up in order to reduce some of the traffic. In many instances, we can make changes without giving up anything significant. At other times, it must be a carefully weighed decision, rooted firmly in a thorough understanding of your particular network configuration and the functionality provided by the specific service or setting. We offer advice and guidance with this determination, and thereby empower you to make the decisions needed to draw order from the ethereal chaos. This analysis is our task as we optimize the network.

Armed with a thorough understanding of the protocols, we can pick up ideas to reduce the traffic. One of the first things we learn is to eliminate superfluous services. As we will learn in this overview of the protocols, the traffic associated with additional protocols is not just the transport, but the services associated with the protocol. Each service talks to other services, advertises its presence, or in other ways makes itself known on the network. It is conceivable that by simply removing one or two services, a 5 to 10 percent reduction in network traffic can be achieved.

In order to use only the protocols needed on the network, we need to know what the protocols are and where they fit into the scheme of things. Therefore, in the first section of the book, we examine some of the common protocols in use, and look in depth at them to see how they work. With this insight, we can develop our optimization methodology. We see which ones we need, and which ones we do not. We will gain the confidence to run with only one protocol and avoid the temptation to "keep one for a backup." In addition to the flood of extraneous traffic, network communication slows to a crawl as programs make numerous attempts to find a shared protocol.

Although we cannot perform true baselining with the Microsoft Network Monitor tool (some of the other products currently available do this for us), we can get a good idea of our network utilization and thereby manually trend the pertinent statistics in a spreadsheet or database. As we turn off services, reduce protocols, and optimize all that remains, we can chart our progress. We will see the utilization percentages, broadcasts, and CRC errors all fall away like autumn leaves after the first rain. Armed with documentation collected in this step, we are in a good position to plan for expansion.

Whenever we add additional workstations, printers, servers, or services to our network we need to have a clear understanding of how the computing environment will be affected. As we plan for expansion, we need an idea of what and where the impact will be on the network itself. The traffic burden is likely to be far greater than a new machine simply talking to a server. If the new machine is performing file-sharing services, then it will be advertising its presence in some manner. When this computer hangs out an open-for-business shingle, how much traffic is going to be generated? What effect does it have on the rest of the segment? If this is a single segment, then what about the other machines sharing the wire? What will the impact be across the router, or at the switch? These are the things we need to look at and the kind of things we talk about in this section.

Network monitoring can be tremendously helpful as we fight the battle against hacker insurgents. Although it may be possible for them to slip into the network undetected—either through stealing passwords, or bypassing security altogether—it is impossible for them to hide their activities once inside. From this vantage point, the low-level network monitor can see everything. So how do we detect the hackers in our network neighborhood?

A rogue DHCP server is particularly nasty. The DHCP server sits on your network, receives the client request for an IP address, and then proceeds to hand out addresses on its own. They may or may not be legitimate for your network, or they may even hand out duplicate addresses causing no end of grief and heartache. In reality, Network monitoring is the best way to find a rogue DHCP server. Microsoft Network Monitor version 2 makes this sabotage even easier to detect.

Many years ago, the U.S. Navy realized that the best way to catch a submarine was to use another submarine. These silent deadly devices were purposefully designed to avoid detection and thus was born the class of submarines called the fast attack. In the same way, the only way to detect unauthorized sniffing is to use a network monitor. Nearly all tools in this class will assist you in finding clandestine sniffing. Network Monitor version 2 can even shut down unauthorized sniffers.

IP spoofing is a favorite hacker trick in which one computer masquerades as another by using the IP address of another machine and then responding to queries addressed to someone else. We can detect spoofing by firing up our favorite Network Monitoring tool. IP spoofing can also happen if routers are improperly configured. In this situation, a machine answers requests directed to a different machine with the same IP address. This can absolutely drive you crazy until you detect the spoofing.

Obviously, a bad Ethernet card is easy to detect. It just lies there, doing nothing but collecting dust—or you look for smoke. However, a card that thinks it is good, and that actually transmits and receives information from time to time, can be far more difficult to find. This is called chattering. The Ethernet card floods the network with bogus information causing all communication to bog down worse than I-75 in Cincinnati during rush hour. This can be detected using Network monitoring tools.

The old song "One bad app don't spoil a whole bunch of good" is not necessarily true. One bad app can affect every other program running on the network. Bad applications can manifest themselves in many different ways. They can look for support files that are not there, cause excessive lookups on the server, or even generate unnecessary traffic. We will look at several somewhat typical scenarios and develop a template you can use to look for other problems in this area.

Network monitoring excels at helping to solve perplexing connectivity problems. Obviously, if you are running TCP/IP, then you use ping to test basic communication between machines. But that is only the first step. When ping works and you still cannot talk to the server is the subject of this section.

The target audience is network administrators, system architects, technicians, and others who support Windows NT (although the book is useful to those not directly supporting a Windows NT network because the protocols are essentially the same no matter what platform they are running on). The book is also useful for those wishing to do supplemental reading while preparing for their MCSE, Cisco CCNA, or Comptia Network Plus certifications. It is therefore a moderately advanced book. We make no real assumptions either about knowledge of the protocols or experience with the products, as we will be discussing them. Exposure to the OSI model will help make the protocol sections go quicker, but we cover it as well. A basic knowledge of TCP/IP, DHCP, DNS, and WINS is helpful because they show up in some of the examples. If you want to perform network monitoring and analysis and/or wish to be able to troubleshoot and optimize your network communications, then this book is definitely for you.

In this book we approach Network Monitoring and Analysis from a protocol point of view. The tool we will use the most in our troubleshooting examples is Microsoft's Network Monitor (AKA Netmon). There are currently four different versions of this tool, and we compare each of them. Originally code-named bloodhound, Netmon has actually changed little since its initial release. To complicate matters, the interface is less than intuitive, and the online help files provide little about how to actually use the product.

We will close this gap and show you how to get the most out of this powerful tool. To this end, we illustrate typical usage scenarios, point out potential pitfalls, and then dive into real-world examples to drive home the utility of this program. Next, fresh from our review of the OSI model and the protocols themselves, we use our knowledge of protocol interlocking to release the full unbridled power of Network Monitor. Finally, we look at how the protocols talk to each other. With this information at our fingertips, we are able to understand what we are looking at in the frame fields. We become one with the network as we speak the language of our machines.

We show you how to use Network Monitor to analyze your network traffic, and how to troubleshoot utilizing this tool. We look at various optimization scenarios and give you lots of food for thought. After reading this book, you will look at your network in a new light. Of course, our end result is to be able to utilize existing tools to troubleshoot complex networks and shed light on these somewhat erstwhile entities.

Our approach is to an extent governed by our task, that is, we will move from the general to the specific. Our path will take us into some turbulent seas, but they are not uncharted waters. Indeed, with the foundation laid down in part one, we will have smooth sailing.

In order to properly perform network monitoring and analysis, we need to know what we are looking at. This is part of what keeps many of us from using these important tools. However, by looking at everyday protocols and examining the characteristics associated with them, we will be able to understand what it is we are looking at, and therefore be able to more effectively troubleshoot our networks. Chapters in this section include the following:

Chapter One: Basic Network Models begins with the Open Systems Interconnection Model and the modifications made by the IEEE 802 project. We also look at how packets are formed and the way in which protocols work with all this.

Chapter Two: The TCP/IP Protocol Suite provides an introduction to the senior protocol on the block. We will spend much of the book working with the transmission control protocol, the internet protocol, and all their relatives.

Chapter Three: The IPX/SPX Protocol Suite introduces both the Internet packet exchange protocol and the sequenced packet exchange protocol. We will look at how the packets are formed, as well as the role of the service advertising protocol and how it performs name resolution.

Chapter Four: The Server Message Block Protocol is central to network computer communications. We examine many of the commands as we prepare to interpret our traces. When we complete that task, we close out part one of our book.

In Part Two, we look at traffic from four different perspectives, and once this is done, we glean suggestions for reducing this traffic in each of the cases.

Chapter Five: Client Traffic looks at some of the sources of client traffic including that of browsing and attempting name resolution in order to communicate with other machines.

Chapter Six: Server Traffic discusses some of the sources of server traffic including that of directory replication, and responding to DNS queries.

Chapter Seven: Application Traffic discusses traffic related specifically to applications such as file and print, internet browsing, and even email programs.

Now we get to the fun stuff—a look at the tools of the trade. Microsoft has some good ones that are obtained in various ways, and in many respects are quite powerful. We begin our look at the tools by focusing on the Microsoft entry into this arena.

Chapter Eight: Microsoft's Network Monitor Family points out at least three different Microsoft Network Monitor tools out there—all called Netmon, and all a little different. In this section, we look at the tool, and the issues surrounding the tools, as well as hints for making the most out of these raw tools.

Ok, let's roll up our sleeves and apply our fine-tuned knowledge of protocols, and network monitoring tools to some real world problems. Armed with powerful network monitoring tools, we can solve complex problems in a single bound. Let's go troubleshooting.

Chapter Nine: Connectivity Problems looks at the age-old scenario, "I can't get logged in!" There are, of course, many permutations to this and we may occasionally see a workstation that cannot find the domain controller, obtain a DHCP lease, or maybe it just simply cannot connect to the server. Perhaps it is a password problem or other login issue. These issues simply cannot go undetected from a well-tuned network monitor. Unfortunately, some applications are not perfect on their ship date, and therefore get released to manufacturing prematurely. In many instances, these undocumented features are solved in later revisions of the code. But how are they detected? What are some of the clues that get you looking for fixes in the first place? Excessive broadcasts, slow network performance, and unallocated pages are all candidates for the probing ears of Netmon.

Chapter Ten: Security Issues can be looked at with our favorite sniffer. Rogue DHCP servers, unauthorized sniffing, and the like are discussed in this chapter.

On the CD-ROM we have copies of the capture files mentioned in the book to allow you to follow along with the examples and to delve more deeply into the ethereal abyss. We have created filters you can load into your Microsoft Network Monitor that you can use for different troubleshooting scenarios. In addition, there are sample batch files you can use to assist you in triggering unattended Netmon sessions using the Microsoft Windows NT scheduler service. These items are referred to in the text along with hints to allow you to obtain the full benefit from them.

Ed Wilson, MCSE + I, MCT, Master ASE, CCNA is a Senior Networking Specialist with Full Service Networking, a Microsoft Solution Provider Partner in Cincinnati, Ohio. His previous publications include chapters on Performance Monitoring, and Network Monitor in the MCSE for Dummies book NT Server in the Enterprise, Osborne McGraw Hill MCSE study guide for the Windows NT Workstation book chapters on troubleshooting and the registry, Osborne McGraw Hill MCSE study guide for Windows 98 chapter on troubleshooting, and the New Riders MCSE Training guide chapters on Setup, Installation, Troubleshooting and Exam Tips.