Network Bulls
www.networkbulls.com
Best Institute for CCNA CCNP CCSP CCIP CCIE Training in India
M-44, Old Dlf, Sector-14 Gurgaon, Haryana, India
Call: +91-9654672192
Cisco IOS has a number of ways to extract data about the state of the machine. Understanding the capabilities of the
operating system and how to use them effectively can reduce time-to-repair and the stress of a network outage.
IOS Filtering Tools
Most of the commands for pulling information from a router are familiar to anyone with Cisco IOS experience. Many
people are not familiar with the filtering techniques that enable a troubleshooter to quickly focus.
Some of these filters are command-specific. Consider show ip route, which is a familiar command. When used, this
command shows a complete routing table (as shown here):
Foard-rtr01#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is 10.100.1.1 to network 0.0.0.0
[ 22 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
172.0.0.0/8 is variably subnetted, 7 subnets, 2 masks
B 172.136.157.12/30 [20/0] via 10.1.254.246, 2d01h
S 172.99.120.2/31 [1/0] via 10.100.254.240
B 172.139.78.232/30 [20/0] via 10.1.254.246, 2d01h
B 172.136.88.20/30 [20/0] via 172.176.128.25, 5w2d
B 172.136.41.104/30 [20/0] via 172.176.128.25, 5w2d
B 172.137.230.128/30 [20/0] via 172.176.128.25, 6d18h
B 172.139.83.100/30 [20/0] via 172.176.128.25, 1w5d
172.16.0.0/32 is subnetted, 1 subnets
S 172.16.201.141 [1/0] via 10.100.254.240
192.168.0.0/30 is subnetted, 6 subnets
B 192.168.26.52 [20/0] via 10.1.254.246, 2d01h
B 192.168.241.236 [20/0] via 172.176.128.25, 5w2d
…
The output for this command can continue over many pages of information. One way to summarize this information is to
ask for a summary using show ip route summary.
Foard-rtr01#show ip route summary
IP routing table name is Default-IP-Routing-Table(0)
IP routing table maximum-paths is 32
Route Source Networks Subnets Overhead Memory (bytes)
connected 0 19 1216 2888
static 4 22 1664 3952
bgp 65100 19 385 25856 62428
External: 382 Internal: 22 Local: 0
internal 45 52740
Total 68 426 28736 122008
Removing Queue Size 0
[ 23 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
A second routing table filtering option is to ask for a selection of routes. Specifying an address, mask, and the keyword
longer-prefixes asks for anything that matches the prefix or any routes contained within the prefix. The following
example shows all the more-specific routes contained within the 10.1.254.0/24 block:
Foard-rtr01#show ip route 10.1.254.0 255.255.255.0 longer-prefixes
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route
Gateway of last resort is 10.100.254.240 to network 0.0.0.0
10.0.0.0/8 is variably subnetted, 241 subnets, 12 masks
C 10.1.254.244/30 is directly connected, Multilink31
C 10.1.254.246/32 is directly connected, Multilink31
B 10.1.254.252/30 [200/0] via 10.100.1.2, 1d09h
C 10.1.254.232/30 is directly connected, Multilink42
C 10.1.254.234/32 is directly connected, Multilink42
The options for filtering available for a given show command vary, so it’s a good idea to spend some time with the question
mark and understand the options available for areas of focus in your organization.
Generic filters can also be applied to all show commands. Show process cpu, which might be used to look for runaway
processes, can be used as an example. First, an example portion of output is shown:
Foard-rtr01#show process cpu
CPU utilization for five seconds: 14%/13%; one minute: 14%; five minutes: 14%
[ 24 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
1 292 6405 45 0.00% 0.00% 0.00% 0 Chunk Manager
2 296 639947 0 0.00% 0.00% 0.00% 0 Load Meter
3 0 1 0 0.00% 0.00% 0.00% 0 chkpt message ha
4 0 1 0 0.00% 0.00% 0.00% 0 EDDRI_MAIN
5 1600592 326740 4898 0.00% 0.04% 0.00% 0 Check heaps
6 2016 28869 69 0.00% 0.00% 0.00% 0 Pool Manager
7 0 2 0 0.00% 0.00% 0.00% 0 Timers
8 0 2 0 0.00% 0.00% 0.00% 0 ATM AutoVC Perio
9 0 2 0 0.00% 0.00% 0.00% 0 ATM VC Auto Crea
10 0 53330 0 0.00% 0.00% 0.00% 0 IPC Dynamic Cach
11 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager
12 20 3199682 0 0.00% 0.00% 0.00% 0 IPC Periodic Tim
13 12 3199682 0 0.00% 0.00% 0.00% 0 IPC Deferred Por
14 0 4 0 0.00% 0.00% 0.00% 0 IPC Seat Manager
15 0 1 0 0.00% 0.00% 0.00% 0 IPC BackPressure
16 387428 57716731 6 0.07% 0.01% 0.00% 0 EnvMon
…
The pipe (|) character is used to filter output by passing it through logic such as include, exclude, begin, and section.
Output is matched against a regular expression.
Following is a table of common regular expression characters.
Character Usage Example
^ Begins with ^Fast matches lines that begin with FastEthernet.
$ Ends with FastEthernet0/0$ matches lines that end with FastEthernet0/0.
[ 25 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
. Any character Ethernet./. matches Ethernet 0/0, FastEthernet0/1, and Ethernet ?.
| Or FastEthernet 0/0|1 matches either FastEthernet0/0 and FastEthernet0/1.
_ Matches beginning, end, or braces _Ethernet_ matches any line that includes the word “Ethernet.”
A show command piped to include will display any line of output that matches the regular expression. In the following
example, the pipe is used to look for any line that includes the text “IP Input”.
Foard-rtr01#show process cpu | include IP Input
87 2755292 47045037 58 0.07% 0.07% 0.07% 0 IP Input
The running configuration is another place to see piping work. In the following example, piping to begin starts the output
at the telnet ports. This is a lot easier that using the space key to work through a large configuration:
Foard-rtr01#show running-configuration | begin vty
line vty 0 4
exec-timeout 20 0
password 7 0401001C02010D4106
logging synchronous
transport input ssh
transport output telnet ssh
line vty 5 15
exec-timeout 20 0
password 7 0401001C02010D4106
logging synchronous
transport input ssh
transport output telnet ssh
!
ntp source Loopback0
[ 26 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
ntp master 5
ntp update-calendar
ntp server 172.31.55.2
ntp peer 10.1.1.123 key 1 source Loopback0
end
In the preceding example, piping to begin also includes all the text after the part of interest. Piping to section shows the
indented commands under a line that matches the regular expression. In the following example, the sections found under
the keyword vty are shown:
Foard-rtr01#show running-config | section vty
line vty 0 4
exec-timeout 20 0
password 7 045C021302284D4906
logging synchronous
transport input ssh
transport output telnet ssh
line vty 5 15
exec-timeout 20 0
password 7 14101B1E010D2B2C2B
logging synchronous
transport input ssh
transport output telnet ssh
The pipe symbol is also used as an OR within a regular expression, as shown in the next examples. Normally, show ip
interface brief summarizes all the interfaces found on a router. Some routers have a large number of interfaces, making
even this simplified display cumbersome. In the following text, some of the interfaces are grouped into multilinks and
others are turned off. Finding the detail you need is complicated by the long and confusing output:
Foard-rtr01#show ip interface brief
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 10.87.1.1 YES NVRAM up up
[ 27 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
NOTE:
Piping output can be a
great way to focus on
relevant details, but show
running-configuration |
section is a lot to type,
particularly repeatedly.
The alias command can
make this easier. In
configuration mode,
create a shortened
version of a command as
shown next.
rtr01(config)#alias
exec srs show
running-configuration
| section
Now “srs” is the shortened
version of the long
and cumbersome
command. Type srs vty
to see the same output as
the example.
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
FastEthernet0/0.2 10.76.2.2 YES NVRAM up up
FastEthernet0/0.3 10.76.3.2 YES NVRAM up up
FastEthernet0/0.4 10.76.4.2 YES NVRAM up up
FastEthernet0/0.5 10.76.5.2 YES NVRAM up up
FastEthernet0/0.6 10.76.6.2 YES NVRAM up up
FastEthernet0/0.7 10.76.7.2 YES NVRAM up up
FastEthernet0/0.8 10.76.8.2 YES NVRAM up up
FastEthernet0/0.12 10.76.12.2 YES NVRAM up up
FastEthernet0/0.120 10.76.12.130 YES NVRAM up up
FastEthernet0/0.1000 10.76.0.2 YES NVRAM up up
FastEthernet0/1 unassigned YES NVRAM administratively down down
GigabitEthernet0/1 unassigned YES NVRAM administratively down down
FastEthernet0/2 unassigned YES NVRAM administratively down down
GigabitEthernet0/2 unassigned YES NVRAM administratively down down
GigabitEthernet0/3 unassigned YES NVRAM administratively down down
Serial1/0 unassigned YES NVRAM administratively down down
Serial1/0.402 unassigned YES unset administratively down down
Serial1/0.404 10.1.254.237 YES NVRAM administratively down down
Serial1/1 unassigned YES NVRAM administratively down down
Serial1/2 unassigned YES NVRAM administratively down down
Serial1/3 unassigned YES NVRAM administratively down down
Serial1/4 unassigned YES NVRAM administratively down down
Serial1/5 unassigned YES NVRAM administratively down down
Serial1/6 unassigned YES NVRAM administratively down down
Serial1/7 unassigned YES NVRAM administratively down down
Serial2/0:0 unassigned YES NVRAM up up
Serial2/1:0 unassigned YES NVRAM up up
Serial2/2:0 unassigned YES NVRAM up up
Serial3/0 unassigned YES NVRAM up up
[ 28 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
Serial3/0.100 172.16.128.26 YES NVRAM up up
Serial3/1 unassigned YES NVRAM down down
Serial4/0:0 unassigned YES NVRAM down down
Serial4/1:0 unassigned YES NVRAM down down
Serial4/2:0 unassigned YES NVRAM down down
Serial4/3:0 unassigned YES NVRAM down down
Serial4/4:0 unassigned YES NVRAM up up
Serial4/5:0 unassigned YES NVRAM up up
Serial4/6:0 unassigned YES NVRAM up up
Serial4/7:0 unassigned YES NVRAM up up
Serial6/0:0 unassigned YES NVRAM down down
Serial6/1:0 unassigned YES NVRAM down down
Serial6/2:0 unassigned YES NVRAM down down
Serial6/3:0 unassigned YES NVRAM down down
Serial6/4:0 unassigned YES NVRAM up up
Serial6/5:0 unassigned YES NVRAM up up
Serial6/6:0 unassigned YES NVRAM up up
Serial6/7:0 unassigned YES NVRAM up up
SSLVPN-VIF0 unassigned NO unset up up
Multilink20 10.1.254.249 YES NVRAM down down
Multilink31 10.1.254.245 YES NVRAM up up
Multilink42 10.1.254.233 YES NVRAM up up
Loopback0 10.1.1.1 YES NVRAM up up
Loopback1 10.254.253.94 YES NVRAM up up
To condense the output to the active parts, the following example pipes the output to exclude any lines with the words
“unassigned” or “administratively.” Notice how much this simplifies the display:
Foard-rtr01# show ip interface brief | exclude unassigned|administratively
Interface IP-Address OK? Method Status Protocol
[ 29 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
FastEthernet0/0 10.87.1.1 YES NVRAM up up
FastEthernet0/0.2 10.76.2.2 YES NVRAM up up
FastEthernet0/0.3 10.76.3.2 YES NVRAM up up
FastEthernet0/0.4 10.76.4.2 YES NVRAM up up
FastEthernet0/0.5 10.76.5.2 YES NVRAM up up
FastEthernet0/0.6 10.76.6.2 YES NVRAM up up
FastEthernet0/0.7 10.76.7.2 YES NVRAM up up
FastEthernet0/0.8 10.76.8.2 YES NVRAM up up
FastEthernet0/0.12 10.76.12.2 YES NVRAM up up
FastEthernet0/0.120 10.76.12.130 YES NVRAM up up
FastEthernet0/0.1000 10.76.0.2 YES NVRAM up up
Serial3/0.100 172.176.128.26 YES NVRAM up up
Multilink20 10.1.254.249 YES NVRAM down down
Multilink31 10.1.254.245 YES NVRAM up up
Multilink42 10.1.254.233 YES NVRAM up up
Loopback0 10.1.1.1 YES NVRAM up up
Loopback1 10.254.253.94 YES NVRAM up up
A second example shows the OR capability by piping the output of show process cpu to include lines that start with
CPU or include the words IP Input:
Foard-rtr01#show process cpu | inc ^CPU|IP Input
CPU utilization for five seconds: 14%/13%; one minute: 14%; five minutes: 14%
87 2755772 47054573 58 0.07% 0.07% 0.07% 0 IP Input
Output Redirection
In addition to filtering output, IOS also enables show command output to be redirected. Redirecting output enables an
administrator to collect information for archiving or to share with other troubleshooters and save it as a text file.
[ 30 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
NOTE:
The alias command can
make this easier. In
configuration mode,
create a shortened
version of a command as
shown here.
Router(config)#alias
exec ii show ip
interface brief |
exclude
unassigned|administratively
Now ii is the shortened
version of the long and
cumbersome command.
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
Output can be piped to a file using either redirect or tee. Redirect just creates the file, whereas tee also displays the
content in session. Any filesystem supported by that router is supported, so output can be pointed at flash, tftp, ftp, http,
and other destinations.
The syntax to use this function is
Show command | redirect file
Show command | tee file
The next examples show the running configuration being piped to TFTP. In the first example, the output is redirected. The
second example tees the output so that it builds the TFTP file and displays on screen.
Foard-rtr01#show running-configuration | redirect tftp://tftp/Foard-rtr01-shrun.txt
Translating “tftp”...domain server (10.186.2.30) [OK]
Foard-rtr01#show running-configuration | tee tftp://tftp/Foard-rtr01-shrun.txt
!
Building configuration...
Current configuration : 22291 bytes
…
IOS Troubleshooting Tools
Ping and traceroute are the most obvious tools available in IOS to test the network.
Ping tests connectivity and is so commonly used that even end users are passingly familiar with it. A ping response shows
that a working path between two end points exists. End systems sometimes have firewalls that prevent response, but
generally ping is a reasonable first test of network connectivity:
Foard-rtr01#ping 10.186.1.1
Type escape sequence to abort.
[ 31 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
Sending 5, 100-byte ICMP Echos to 10.186.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/9/12 ms
Exclamation marks show a response, but there is a lot of information besides the most obvious part. First, pay attention to
the pattern of the response. Alternating success and failure (!.!.!) is a classic sign of a load balancing problem, where one
path succeeds and the other fails. Second, pay attention to the response time. Many applications depend on quick
response. Voice, for instance, assumes a round-trip time of less than 150 ms. The response time can also clue the troubleshooter
to utilization issues. If the response time is much larger than usual that might indicate a heavy traffic load and
queuing. If you notice that the minimum and maximum times vary widely, this could also be a sign of queuing because of
a heavy load.
Ping can do a lot more than that simple test, however. Privileged mode supports an extended ping that enables every
aspect of ping to be controlled. This opens up many more tests that can be accomplished with the humble command.
The following example below an extended ping. Notice that the command ping—with no destination specified—is
entered in privileged mode. The example sends five pings of 100 bytes, then five of 200 bytes, continuing to 1500 byte
pings. The DF bit (do not fragment) is set. A similar ping might be used if you suspect that an intermediate link didn’t
support the same size MTU as the source and destination. A more detailed explanation of the command is found after the
example:
Foard-rtr01#ping
Protocol [ip]:
Target IP address: 10.186.1.1
Repeat count [5]:
Datagram size [100]:
Timeout in seconds [2]:
Extended commands [n]: y
Source address or interface: loopback0
Type of service [0]:
[ 32 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
Set DF bit in IP header? [no]: y
Validate reply data? [no]:
Data pattern [0xABCD]:
Loose, Strict, Record, Timestamp, Verbose[none]:
Sweep range of sizes [n]: y
Sweep min size [36]: 100
Sweep max size [18024]: 1500
Sweep interval [1]: 100
Type escape sequence to abort.
Sending 75, [100..1500]-byte ICMP Echos to 10.186.1.1, timeout is 2 seconds:
Packet sent with a source address of 10.1.1.1
Packet sent with the DF bit set
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!
Success rate is 100 percent (75/75), round-trip min/avg/max = 8/10/12 ms
Remember that defaults are shown in square brackets. Selecting all the defaults is similar to a normal ping.
Sometimes testing involves repeatedly pinging (for instance, when you believe that an interface is flapping up and down).
An extended ping with a repeat count of 99999 can be used to interactively test the network over a period of time.
Pings can be set to different packet sizes through the Datagram Size variable. The router can automate testing a range of
sizes. To do so, use the extended commands and choose to sweep a range of sizes.
If a router is asked to forward a packet that is larger than the MTU of the transmitting link, the router normally breaks the
packet into smaller pieces. Setting the DF bit instructs receiving routers to discard the traffic rather than fragment it.
Using different size packets and setting the DF bit allows testing MTU. When the MTU limit is reached, all subsequent
pings will be dropped.
[ 33 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
Another nice testing technique is to change the source interface. Pings are normally sourced from the transmitting interface.
Using an internal interface as the source shows that the receiving device and the intermediate routers understand
how to route back to that prefix.
A final idea is to try different Type of Service settings. Many networks now carry voice, video, and prioritized data. Voice
is commonly set to ToS 5, so pinging using ToS 5 enables a peek into how the QoS settings are functioning.
Like ping, there is an extended version of traceroute. It has a few of the same capabilities, with one other significant
testing ability. Traceroute in IOS uses UDP, and extended traceroute enables setting the UDP port. This can be used to
test application performance for applications that use UDP, such as voice. This is important when trying to diagnose the
affects of firewalls and access-lists.
An example extended traceroute is shown next. The only choice specified in the example is to use UDP port 16000:
Newton-rtr01#traceroute
Protocol [ip]:
Target IP address: 10.200.1.1
Source address:
Numeric display [n]:
Timeout in seconds [3]:
Probe count [3]:
Minimum Time to Live [1]:
Maximum Time to Live [30]:
Port Number [33434]: 16000
Loose, Strict, Record, Timestamp, Verbose[none]:
Type escape sequence to abort.
Tracing the route to 10.200.1.1
[ 34 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
In the same way the UDP port connectivity can be probed with traceroute, telnet can be used to test TCP ports. Telnet
does not offer many options, but by changing the target port, different network services can be tested. The following
examples show that email and the web server respond on the appropriate ports:
Foard-rtr01#telnet www.example.com 25
Translating “www.example.com”...domain server (10.1.2.2) [OK]
Trying www.example.com (172.16.0.25, 25)... Open
220 www.example.com ESMTP Postfix
Foard-rtr01#telnet www.example.com 80
<ctrl-c>
HTTP/1.1 400 Bad Request
Content-Type: text/html
Date: Fri, 4 Sep 2009 17:14:29 GMT
Connection: close
Content-Length: 35
<h1>Bad Request (Invalid Verb)</h1>
Hardware Diagnostics
The commands examined so far have dealt with network issues, but sometimes the problem is within the IOS device.
Several commands describe the functional state of an IOS device.
If network hardware is suspected, a good place to start troubleshooting is to understand the external environment. The
show environment all command displays information about the temperature within the device and the state of the power
supplies. Especially when troubleshooting remotely it is easy to forget power and air conditioning, but problems in either
area can lead to device malfunction:
[ 35 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
Foard-rtr01#sh environment all
Power Supplies:
Power Supply 1 is AC Power Supply. Unit is on.
Power Supply 2 is AC Power Supply. Unit is on.
Temperature readings:
NPE Inlet measured at 25C/77F
NPE Outlet measured at 27C/80F
I/O Cont Inlet measured at 25C/77F
I/O Cont Outlet measured at 28C/82F
CPU Die measured at 43C/109F
Voltage readings:
+3.30 V measured at +3.30 V
+1.50 V measured at +1.49 V
+2.50 V measured at +2.50 V
+1.80 V measured at +1.79 V
+1.20 V measured at +1.20 V
VDD_CPU measured at +1.28 V
VDD_MEM measured at +2.50 V
VTT measured at +1.25 V
+3.45 V measured at +3.43 V
-11.95 measured at -12.17 V
+5.15 V measured at +4.96 V
+12.15 V measured at +12.18 V
Envm stats saved 0 time(s) since reload
[ 36 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
A complete and accurate inventory is another part of troubleshooting. Of course, this information is much more useful if
obtained before a problem occurs and connectivity drops! By comparing the inventory to previous inventories, it is possible
to recognize differences (caused, presumably, by hardware failure). If the organization has a Cisco SmartNet maintenance
contract, the serial number and part-number information is necessary to obtain spares:
Foard-rtr01#show inventory
NAME: “Chassis”, DESCR: “Cisco 7206VXR, 6-slot chassis”
PID: CISCO7206VXR , VID: , SN: 24323096
NAME: “NPE-G2 0”, DESCR: “Cisco 7200 Series Network Processing Engine NPE-G2”
PID: NPE-G2 , VID: V03 , SN: JAS1456B4EC
NAME: “disk2”, DESCR: “256MB Compact Flash Disk for NPE-G2”
PID: MEM-NPE-G2-FLD256 , VID: , SN:
NAME: “module 0”, DESCR: “I/O Dual FastEthernet Controller”
PID: C7200-I/O-2FE/E , VID: , SN: 21753008
NAME: “disk0”, DESCR: “Cisco 7200 I/O PCMCIA Flash Disk, 48M”
PID: MEM-I/O-FLD48M , VID: , SN:
NAME: “disk1”, DESCR: “Cisco 7200 I/O PCMCIA Flash Disk, 48M”
PID: MEM-I/O-FLD48M , VID: , SN:
NAME: “module 1”, DESCR: “Serial”
PID: PA-8T-V35= , VID: , SN: 49010448
NAME: “module 2”, DESCR: “4 port, software configurable Multichannel T1/E1 with TDM Port Adapter”
PID: PA-MCX-4TE1 , VID: , SN: JAS1680Y0EM
[ 37 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
NAME: “module 3”, DESCR: “Enhanced 2 port T3/E3 clear channel PA”
PID: PA-2T3/E3-EC , VID: V01 , SN: JAS249200K5
NAME: “module 4”, DESCR: “8 port, software configurable Multichannel T1/E1 without TDM Port Adapter”
PID: PA-MC-8TE1+ , VID: , SN: JAS1689A2MM
NAME: “module 6”, DESCR: “8 port, software configurable Multichannel T1/E1 without TDM Port Adapter”
PID: PA-MC-8TE1+ , VID: , SN: JAS1689A2BV
NAME: “Power Supply 1”, DESCR: “Cisco 7200 AC Power Supply”
PID: PWR-7200-AC , VID: , SN:
NAME: “Power Supply 2”, DESCR: “Cisco 7200 AC Power Supply”
PID: PWR-7200-AC , VID: , SN:
A lack of memory can also cause a network issue. The show memory command displays the state of memory on a
device; focus on the Free column to determine if enough is available. Another sign of memory issues is %SYS-2-
MALLOCFAIL messages:
Foard-rtr01#show memory
Head Total Used Free Lowest Largest
Processor 6319860 818832732 74864300 743968432 742841100 727580236
I/O 38000000 67108864 11964260 55144604 55137712 54643068
Transient 37000000 16777216 58244 16718972 16226680 16718696
…
Hardware issues can also manifest themselves on the interfaces. Show controller can show some information about the
interface—serial interfaces in particular report things such as cable information here. Show interface (shown next)
displays a good deal of information about the state of the interface. In particular, pay attention to four measurements:
[ 38 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
n Input queue drops: Signify that the router had more traffic than it could process. Some amount of drops is excusable,
but drops could be related to CPU oversaturation. Double-check the processor with the show processes cpu
command.
n Output queue drops: Usually mean that the line is congested.
n Input errors: These errors show duplex errors, interface problems, and CRC errors.
n Output errors: Usually related to duplex issues.
Foard-rtr01#show interface
FastEthernet0/0 is up, line protocol is up
Hardware is i82543 (Livengood), address is 000a.f3f7.9808 (bia 000a.f3f7.9808)
Description: enter port #
Internet address is 10.100.1.1/16
MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
reliability 255/255, txload 32/255, rxload 14/255
Encapsulation 802.1Q Virtual LAN, Vlan ID 1., loopback not set
Keepalive set (10 sec)
Full-duplex, 100Mb/s, 100BaseTX/FX
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of “show interface” counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 5517000 bits/sec, 2571 packets/sec
5 minute output rate 12927000 bits/sec, 2550 packets/sec
1326060749 packets input, 711066620 bytes
Received 45468700 broadcasts, 0 runts, 0 giants, 0 throttles
[ 39 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
148 input errors, 0 CRC, 0 frame, 0 overrun, 148 ignored
0 watchdog
0 input packets with dribble condition detected
1191821108 packets output, 2981100223 bytes, 0 underruns
2 output errors, 0 collisions, 4 interface resets
5634739 unknown protocol drops
0 babbles, 0 late collision, 0 deferred
2 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
Working with External Tools
The IOS troubleshooting capabilities are supplemented by external network management tools. Cisco IOS devices
support these tools and in many cases supply detailed information to the management system. This section describes the
methods used to coordinate with these tools.
Packet Sniffing
Packet capture from a laptop or specialized device enables low-level vision into the exact traffic flowing over a link.
Capturing traffic can show errors and underlying protocol traffic. The issue with packet capture is that switches do not
forward all traffic out all ports, so it is difficult to find a port from which to see all traffic.
SPAN (Switched Port Analyzer) is a tool within IOS switches to direct copies of packets to a capture port. SPAN is
configured by identifying a source port or VLAN from which traffic should be copied. SPAN is then pointed to an output
port, to which a packet capture tool is attached. SPAN can capture traffic on a switch and output to a trunked VLAN. A
second switch can then capture the VLAN and output it to a port. This configuration is called remote SPAN (RSPAN).
[ 40 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
The generic configuration of SPAN is
Monitor session [session number] [source|destination] [interface|vlan]
The following example shows the configuration used when suspicious device is on port F0/1 and a packet capture tool is
plugged into port F0/24:
Monitor session 1 source interface f0/1
Monitor session 1 destination interface f0/24
Router IP Traffic Export (RITE) is similar to SPAN but used by routers to capture traffic to a monitoring port. The following
example demonstrates capturing ten percent of the interesting traffic on f0/1 and exporting it to a device with a given MAC:
(config)# ip traffic-export profile rite
(config-rite)# interface FastEthernet 0/1
(config-rite)# bidirectional
(config-rite)# mac-address 00a.8aab.90a0
(config-rite)# incoming access-list my_acl
(config-rite)# outgoing sample one-in-every 10
(config)# interface FastEthernet0/0
(config-if)# ip traffic-export apply rite
RITE can also be used to export the traffic to a file on the router. From there it can be copied off for inspection on a PC:
traffic-export interface fastethernet0/0 copy tftp:
Netflow
Netflow collects summaries of traffic information and transmits the summary to a Netflow collector. Netflow is enabled
on each monitored interface. Netflow supports a version 5 and version 9; this should be set to match the requirements of
your network management system. Finally, Netflow exports information to a target IP address. The commands to accomplish
these actions are shown here:
[ 41 ]
© 2010 Pearson Education, Inc. All rights reserved. This publication is protected by copyright. Please see page 69 for more details.
CCNP TSHOOT 642-832 Quick Reference by Brent Stewart
www.CareerCert.info
CHAPTER 3
Troubleshooting Tools
(config-if)#ip flow ingress
(config)#ip flow-export version [5|9]
(config)#ip flow-export destination [ip-address]
In addition to using a monitoring system to track Netflow, an administrator can also peek into the current flows using
show ip cache flow.
SNMP and EEM
SNMP is another monitoring protocol. Whereas Netflow tracks traffic, SNMP can monitor any type of event or statistic
from the device. SNMP is supported by most network monitoring systems. The router also has a tool to react to events
through embedded event manager (EEM).
SNMP is set up by identifying a server and listing the events to be monitored. If snmp-server enable traps is used
without specifying specific events, all traps are monitored:
(config)#snmp-server host [ip-address]
(config)#snmp-server enable traps
EEM enables custom reactions to events and acts as a supplement to SNMP. Events can be triggered by any SNMP event
and actions can include (among others) SNMP, Syslog, IOS commands, and email messages.
A simple example EEM applet is shown next. This applet logs a Syslog message and outputs a message to the console in
reaction to an administrator entering configuration mode:
Event manager applet CONFIG-STARTED
Event cli pattern “configure terminal” sync on skip no occurs 1
Action 1.0 syslog priority critical msg “Configuration mode was entered”
Action 2.0 syslog priority informational msg “Change control policies apply. Authorized access only.”
No comments:
Post a Comment