Router Performance Issues Cisco CCSP Coaching Institute in Delhi Gurgaon

Network Bulls
www.networkbulls.com
Best Institute for CCNA CCNP CCSP CCIP CCIE Training in India
M-44, Old Dlf, Sector-14 Gurgaon, Haryana, India
Call: +91-9654672192

 “Advanced Cisco Catalyst Switch Troubleshooting,” discussed how performance
issues on a Cisco Catalyst switch can be the source of network problems. Similarly, a
router performance issue can impact user data flowing through the network.
Also, as an administrator you might notice sluggish response to Telnet sessions you attempt
to establish with a router or longer-than-normal ping response times. Such symptoms
might indicate a router performance issue.
This section investigates three potential router issues, each of which might result in poor
router performance. These three issues are
■ Excessive CPU utilization
■ The packet switching mode of a router
■ Excessive memory utilization
Excessive CPU Utilization
A router’s processor (that is, CPU) utilization escalating to a high level but only remaining
at that high level for a brief time could represent normal behavior. However, if a router’s
CPU utilization continually remains at a high level, network performance issues might result.
Aside from latency that users and administrators can experience, a router whose CPU
is overtaxed might not send routing protocol messages to neighboring routers in a timely
fashion. As a result, routing protocol adjacencies can fail, resulting in some networks becoming
unreachable.
Key
Topic
www.CareerCert.info
www.CareerCert.info
www - CareerCert - info
246 CCNP TSHOOT 642-832 Official Certification Guide
Processes That Commonly Cause Excessive CPU
Utilization
One reason that the CPU of a router might be overloaded is that the router is running a
process that is taking up an unusually high percentage of its CPU resources. Following are
four such processes that can result in excessive CPU utilization.
■ ARP Input process: The ARP Input process is in charge of sending Address Resolution
Protocol (ARP) requests. This process can consume an inordinate percentage of
CPU resources if the router has to send numerous ARP requests.
One configuration that can cause such a high number of ARP requests is having a default
route configured that points to a broadcast network. For example, perhaps a
router had the ip route 0.0.0.0 0.0.0.0 fa 0/1 command entered in global configuration
mode. Such a configuration can cause an ARP to be sent to all IP addresses available
off of that broadcast interface that are not reachable via a better route. From a
security perspective, numerous ARP requests can result from an attacker performing
a ping sweep of a subnet.
■ Net Background process: An interface has a certain number of buffers available to
store packets. These buffers are sometimes referred to as the queue of an interface. If
an interface needs to store a packet in a buffer but all the interface buffers are in use,
the interface can pull from a main pool of buffers that the router maintains. The
process that allows an interface to allocate one of these globally available buffers is
Net Background. If the throttles, ignored, and overrun parameters are incrementing
on an interface, the underlying cause might be the Net Background process consuming
too many CPU resources.
■ IP Background process: The IP Background process handles an interface changing
its state. A state change might be an interface going from an Up state to a Down state,
or vice versa. Another example of state change is an interface’s IP address changing.
Therefore, anything that can cause repeated state changes, such as bad cabling, might
result in the IP Background process consuming a high percentage of CPU resources.
■ TCP Timer process: The TCP Timer process runs for each TCP router connection.
Therefore, many connections can result in a high CPU utilization by the TCP Timer
process.
Cisco IOS Commands Used for Troubleshooting High
Processor Utilization
Table 8-5 offers a collection of show commands that can be valuable when troubleshooting
high CPU utilization on a router.

Table 8-5 Commands for Troubleshooting High CPU Utilization
Command Description
show arp Displays the ARP cache for a router. If several entries are in the
Incomplete state, you might suspect a malicious scan (for example,
a ping sweep) of a subnet.
show interface interfaceid
Displays a collection of interface statistics. If the throttles, overruns,
or ignore counters continually increment, you might suspect
that the Net Background process is attempting to allocate
buffer space for an interface from the main buffer pool of the
router.
show tcp statistics Provides information about the number of TCP segments a
router sends and receives, including the number of connections
initiated, accepted, established, and closed. A high number of
connections can explain why the TCP Timer process might be
consuming excessive CPU resources.
show processes cpu Displays average CPU utilization over 5-second, 1-minute, and
5-minute intervals, in addition to listing all the router processes
and the percentage of CPU resources consumed by each of
those processes.
show processes cpu
history
Displays a graphical view of CPU utilization over the past 60
seconds, 1 hour, and 3 days. This graphical view can indicate if
an observed high CPU utilization is a temporary spike in utilization
or if the high CPU utilization is an ongoing condition.
Example 8-19 shows sample output from the show arp command. In the output, only a
single instance exists of an Incomplete ARP entry. However, a high number of such entries
can suggest the scanning of network resources, which might indicate malicious reconnaissance
traffic.
Example 8-19 show arp Command Output
R2#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.3.3.2 61 0009.b7fa.d1e0 ARPA Ethernet0/0
Internet 10.3.3.1 - 00d0.06fe.9ea0 ARPA Ethernet0/0
Internet 192.168.1.50 0 Incomplete ARPA
Example 8-20 shows sample output from the show interface interface-id command. Note
the throttles, overrun, and ignored counters. If these counters continue to increment, the
Net Background process might be consuming excessive CPU resources while it allocates
buffers from the main buffer pool of the router.
www.CareerCert.info
www.CareerCert.info
www - CareerCert - info
248 CCNP TSHOOT 642-832 Official Certification Guide
Example 8-20 show interface interface-id Command Output
R2#show interface e0/0
Ethernet0/0 is up, line protocol is up
Hardware is AmdP2, address is 00d0.06fe.9ea0 (bia 00d0.06fe.9ea0)
Internet address is 10.3.3.1/24
MTU 1500 bytes, BW 10000 Kbit, DLY 1000 usec,
reliability 255/255, txload 1/255, rxload 1/255
Encapsulation ARPA, loopback not set
Keepalive set (10 sec)
ARP type: ARPA, ARP Timeout 04:00:00
Last input 00:00:02, output 00:00:02, output hang never
Last clearing of “show interface” counters never
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
5 minute input rate 0 bits/sec, 1 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
2156 packets input, 164787 bytes, 0 no buffer
Received 861 broadcasts, 0 runts, 0 giants, 0 throttles
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
0 input packets with dribble condition detected
2155 packets output, 212080 bytes, 0 underruns
0 output errors, 0 collisions, 7 interface resets
0 babbles, 0 late collision, 0 deferred
0 lost carrier, 0 no carrier
0 output buffer failures, 0 output buffers swapped out
Example 8-21 shows sample output from the show tcp statistics command. If the output
indicates numerous connections, the TCP Timer process might be consuming excessive
CPU resources while simultaneously maintaining all those connections.
Example 8-21 show tcp statistics Command Output
R2#show tcp statistics
Rcvd: 689 Total, 0 no port
0 checksum error, 0 bad offset, 0 too short
474 packets (681 bytes) in sequence
0 dup packets (0 bytes)
0 partially dup packets (0 bytes)
0 out-of-order packets (0 bytes)
0 packets (0 bytes) with data after window
0 packets after close
0 window probe packets, 0 window update packets
1 dup ack packets, 0 ack packets with unsend data
479 ack packets (14205 bytes)
Sent: 570 Total, 0 urgent packets

1 control packets (including 0 retransmitted)
562 data packets (14206 bytes)
0 data packets (0 bytes) retransmitted
0 data packets (0 bytes) fastretransmitted
7 ack only packets (7 delayed)
0 window probe packets, 0 window update packets
0 Connections initiated, 1 connections accepted, 1 connections established
0 Connections closed (including 0 dropped, 0 embryonic dropped)
0 Total rxmt timeout, 0 connections dropped in rxmt timeout
0 Keepalive timeout, 0 keepalive probe, 0 Connections dropped in keepalive
Example 8-22 shows sample output from the show processes cpu command. The output
in this example indicates a 34 percent CPU utilization in the past 5 seconds, with 13 percent
of CPU resources being spent on interrupts. The output also shows the 1-minute
CPU utilization average as 36 percent and the 5-minute average as 32 percent. Individual
processes running on the router are also shown, along with their CPU utilization levels.
Note the ARP Input, Net Background, TCP Timer, and IP Background processes referred
to in this section.
Example 8-22 show processes cpu Command Output
R2#show processes cpu
CPU utilization for five seconds: 34%/13%; one minute: 36%; five minutes: 32%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
...OUTPUT OMITTED...
12 4 69 57 0.00% 0.00% 0.00% 0 ARP Input
13 0 1 0 0.00% 0.00% 0.00% 0 HC Counter Timer
14 0 5 0 0.00% 0.00% 0.00% 0 DDR Timers
15 12 2 6000 0.00% 0.00% 0.00% 0 Entity MIB API
16 4 2 2000 0.00% 0.00% 0.00% 0 ATM Idle Timer
17 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A’detect
18 0 3892 0 0.00% 0.00% 0.00% 0 GraphIt
19 0 2 0 0.00% 0.00% 0.00% 0 Dialer event
20 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd
21 132 418 315 0.00% 0.00% 0.00% 0 Net Background
22 0 15 0 0.00% 0.00% 0.00% 0 Logger
...OUTPUT OMITTED...
46 0 521 0 0.00% 0.00% 0.00% 0 SSS Test Client
47 84 711 118 0.00% 0.00% 0.00% 0 TCP Timer
48 4 3 1333 0.00% 0.00% 0.00% 0 TCP Protocols
49 0 1 0 0.00% 0.00% 0.00% 0 Socket Timers
50 0 15 0 0.00% 0.00% 0.00% 0 HTTP CORE
www.CareerCert.info
www.CareerCert.info
www - CareerCert - info
250 CCNP TSHOOT 642-832 Official Certification Guide
51 12 5 2400 0.00% 0.00% 0.00% 0 PPP IP Route
52 4 5 800 0.00% 0.00% 0.00% 0 PPP IPCP
53 273 157 1738 0.00% 0.00% 0.00% 0 IP Background
54 0 74 0 0.00% 0.00% 0.00% 0 IP RIB Update
...OUTPUT OMITTED...
Example 8-23 shows sample output from the show processes cpu history command. The
graphical output produced by this command is useful in determining if a CPU spike is
temporary or if it is an ongoing condition.
Example 8-23 show processes cpu history Command Output
R2#show processes cpu history
R2 01:06:21 AM Monday Mar 1 1993 UTC
111113333311111 33333 1111122222 222222
100
90
80
70
60
50
40
30
20
10
0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per second (last 60 seconds)
2
2112411211332216331212211212212222221111111111111111111111
100
90
80
70
60
50
40
30 *
20 *
10 #

0....5....1....1....2....2....3....3....4....4....5....5....
0 5 0 5 0 5 0 5 0 5
CPU% per minute (last 60 minutes)
* = maximum CPU% # = average CPU%
8
0
100
90
80 *
70 *
60 *
50 *
40 *
30 *
20 *
10 *
0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
0 5 0 5 0 5 0 5 0 5 0 5 0
CPU% per hour (last 72 hours)
* = maximum CPU% # = average CPU%
Understanding Packet Switching Modes
In addition to the high CPU utilization issues previously discussed, a router’s packet
switching mode can impact router performance. Before discussing the most common
switching modes, realize that the way a router handles packets (or is capable of handling
packets) largely depends on the router’s architecture. Therefore, for real-world troubleshooting,
please consult the documentation for your router to determine how it implements
packet switching.
In general, however, Cisco routers support the following three primary modes of packet
switching:
■ Process switching
■ Fast switching
■ Cisco Express Forwarding
Packet switching involves the router making a decision about how a packet should be forwarded
and then forwarding that packet out of the appropriate router interface.
Operation of Process Switching
When a router routes a packet (that is, performs packet switching), the router removes the
packet’s Layer 2 header, examines the Layer 3 addressing, and decides how to forward the
packet. The Layer 2 header is then rewritten (which involves changing the source and destination
MAC addresses and computing a new cyclic redundancy check [CRC]), and the

Incoming Packets Outgoing Packets
CPU
Control Plane
Ingress
Interface
Egress
Interface
Data Plane
Packet Flow
Packet Flow
Figure 8-4 Data Flow with Process Switching
packet is forwarded out of the appropriate interface. With process switching, as illustrated
in Figure 8-4, the router’s CPU becomes directly involved with packet switching decisions.
As a result, the performance of a router configured for process switching can suffer
significantly.
An interface can be configured for process switching by disabling fast switching on that
interface. The interface configuration mode command used to disable fast switching is no
ip route-cache.
Operation of Fast Switching
Fast switching uses a fast cache maintained in a router’s data plane. The fast cache contains
information about how traffic from different data flows should be forwarded. As seen in
Figure 8-5, the first packet in a data flow is process-switched by a router’s CPU. After the
router determines how to forward the first frame of a data flow, that forwarding information
is stored in the fast cache. Subsequent packets in that same data flow are forwarded
based on information in the fast cache, as opposed to being process-switched. As a result,
fast switching reduces a router’s CPU utilization more than process switching does.
Fast switching can be configured in interface configuration mode with the command ip
route-cache.
Operation of Cisco Express Forwarding
As described in Chapter 5, Cisco Express Forwarding (CEF) maintains two tables in the
data plane. Specifically, the Forwarding Information Base (FIB) maintains Layer 3 forwarding
information, whereas the Adjacency Table maintains Layer 2 information for next hops
listed in the FIB.
www.CareerCert.info
www.CareerCert.info
www - CareerCert - info
Chapter 8: Troubleshooting BGP and Router Performance Issues 253
Incoming Packets Outgoing Packets
Egress
Interface
Data Plane
Packet #1
in a Data Flow
Packet #1
in a Data Flow
Subsequent
Packets in a
Data Flow
Subsequent
Packets in a
Fast Data Flow
Cache
Forwarding
Information
Ingress
Interface
CPU
Control Plane
Figure 8-5 Data Flow with Fast Switching
Using these tables, populated from a router’s IP routing table and ARP cache, CEF can efficiently
make forwarding decisions. Unlike fast switching, CEF does not require the first
packet of a data flow to be process-switched. Rather, an entire data flow can be forwarded
at the data plane, as seen in Figure 8-6.
Incoming Packets Outgoing Packets
Egress
Interface
Data Flow Data Flow
Ingress
Interface
IP Routing CPU ARP Cache
Table
Control Plane
Data Plane
Adjacency
Table
FIB
CEF Data
Structures
Layer 2 Information
Layer 3 Information
Figure 8-6 Data Flow with Cisco Express Forwarding

254 CCNP TSHOOT 642-832 Official Certification Guide
Key
Topic
Table 8-6 Commands for Troubleshooting a Router’s Packet Switching Modes
Command Description
show ip interface interface_id Displays multiple interface statistics, including information
about the packet switching mode of an
interface.
show ip cache Displays the contents of fast cache from a router if
fast switching is enabled.
show processes cpu | include IP Input Displays information about the IP input process on
a router. The CPU utilization for this process might
show a high value if the CPU of a router is actively
engaged in process-switching traffic.
show ip cef Displays the contents of a router FIB.
show ip cef adjacency egress-interfaceid
next-hop-ip-address detail
Displays destinations reachable via the combination
of the specified egress interface and next-hop
IP address.
show adjacency detail Provides information contained in the adjacency
table of a router, including protocol and timer information.
show cef not-cef-switched Displays information about packets the router forwards
using a packet switching mechanism other
than CEF.
On many router platforms, CEF is enabled by default. If it is not, you can globally enable
it with the ip cef command. Alternatively, you can enable CEF for a specific interface with
the interface configuration mode command ip route-cache cef.
Troubleshooting Packet Switching Modes
Table 8-6 provides a selection of commands you can use when troubleshooting the packet
switching modes of a router.
Example 8-24 shows sample output from the show ip interface interface-id command.
The output indicates that fast switching is enabled on interface Fast Ethernet 0/0. The reference
to flow switching being disabled refers to the Cisco IOS NetFlow feature, which
you can use to collect traffic statistics. CEF switching is also enabled.
Example 8-24 show ip interface interface-id Command Output
R4#show ip interface fa 0/0
FastEthernet0/0 is up, line protocol is up
...OUTPUT OMITTED...

IP fast switching is enabled
IP fast switching on the same interface is disabled
IP Flow switching is disabled
IP CEF switching is enabled
IP CEF Fast switching turbo vector
IP multicast fast switching is enabled
IP multicast distributed fast switching is disabled
IP route-cache flags are Fast, CEF
...OUTPUT OMITTED...
Example 8-25 shows sample output from the show ip cache command. If fast switching is
enabled and CEF is disabled, a router begins to populate its fast cache. This command
shows the contents of a router’s fast cache.
Example 8-25 show ip cache Command Output
R4#show ip cache
IP routing cache 3 entries, 588 bytes
12 adds, 9 invalidates, 0 refcounts
Minimum invalidation interval 2 seconds, maximum interval 5 seconds,
quiet interval 3 seconds, threshold 0 requests
Invalidation rate 0 in last second, 0 in last 3 seconds
Last full cache invalidation occurred 04:13:57 ago
Prefix/Length Age Interface Next Hop
10.8.8.4/32 00:00:07 FastEthernet0/1 10.8.8.4
10.8.8.6/32 00:00:10 FastEthernet0/1 10.8.8.6
192.168.0.0/24 00:00:10 FastEthernet0/0 10.3.3.1
Example 8-26 shows sample output from the show processes cpu | include ip input command.
In the output, the IP input process was using only 0.08 percent of its router’s CPU
capacity during the last 5-second interval. However, a high percentage value might indicate
that a router was performing process switching, where the CPU was directly involved
in packet switching.
Example 8-26 show processes cpu | include IP Input Command Output
R4#show processes cpu | include IP Input
63 3178 7320 434 0.08% 0.06% 0.04% 0 IP Input
Example 8-27 shows sample output from the show ip cef command. The output contains
the contents of the FIB for a router. Note that if a next-hop of the network prefix is set to
receive, that network is local to the router, whereas attached indicates that the network is
directly connected to the router.
Example 8-27 show ip cef Command Output
R4#show ip cef
Prefix Next Hop Interface
0.0.0.0/0 drop Null0 (default route handler entry)
0.0.0.0/32 receive
10.1.1.0/24 10.3.3.1 FastEthernet0/0
10.1.1.2/32 10.3.3.1 FastEthernet0/0
10.3.3.0/24 attached FastEthernet0/0
10.3.3.0/32 receive
10.3.3.1/32 10.3.3.1 FastEthernet0/0
10.3.3.2/32 receive
10.3.3.255/32 receive
10.4.4.0/24 10.3.3.1 FastEthernet0/0
10.5.5.0/24 10.3.3.1 FastEthernet0/0
10.7.7.0/24 10.3.3.1 FastEthernet0/0
10.7.7.2/32 10.3.3.1 FastEthernet0/0
10.8.8.0/24 attached FastEthernet0/1
10.8.8.0/32 receive
10.8.8.1/32 receive
10.8.8.4/32 10.8.8.4 FastEthernet0/1
10.8.8.5/32 10.8.8.5 FastEthernet0/1
10.8.8.6/32 10.8.8.6 FastEthernet0/1
10.8.8.7/32 10.8.8.7 FastEthernet0/1
10.8.8.255/32 receive
192.168.0.0/24 10.3.3.1 FastEthernet0/0
224.0.0.0/4 drop
224.0.0.0/24 receive
255.255.255.255/32 receive
Example 8-28 shows sample output from the show ip cef adjacency egress-interface-id
next-hop-IP-address detail command. This command shows the IP addresses that the
router knows how to reach using the specified combination of next-hop IP address and
egress interface. In this example, 10.8.8.6 is the IP address of a host and not a router.
Therefore, no other IP addresses are known to have a next-hop IP address of 10.8.8.6 with
an egress interface of Fast Ethernet 0/1.
Example 8-28 show ip cef adjacency egress-interface-id next-hop-IP-address
detail Command Output
R4#show ip cef adjacency fa 0/1 10.8.8.6 detail
IP CEF with switching (Table Version 25), flags=0x0
25 routes, 0 reresolve, 0 unresolved (0 old, 0 new), peak 0
25 leaves, 21 nodes, 25640 bytes, 90 inserts, 65 invalidations
0 load sharing elements, 0 bytes, 0 references
universal per-destination load sharing algorithm, id 24360DB1

5(2) CEF resets, 1 revisions of existing leaves
Resolution Timer: Exponential (currently 1s, peak 1s)
0 in-place/0 aborted modifications
refcounts: 5702 leaf, 5632 node
Table epoch: 0 (25 entries at this epoch)
Adjacency Table has 5 adjacencies
10.8.8.6/32, version 10, epoch 0, cached adjacency 10.8.8.6
0 packets, 0 bytes
via 10.8.8.6, FastEthernet0/1, 0 dependencies
next hop 10.8.8.6, FastEthernet0/1
valid cached adjacency
Example 8-29 shows sample output from the show adjacency detail command. When you
see a particular adjacency listed in the FIB, you can issue this command to confirm the
router has information about how to reach that adjacency.
Example 8-29 show adjacency detail Command Output
R4#show adjacency detail
Protocol Interface Address
IP FastEthernet0/0 10.3.3.1(19)
32 packets, 1920 bytes
00D006FE9EA00009B7FAD1E00800
ARP 03:53:01
Epoch: 0
IP FastEthernet0/1 10.8.8.6(5)
4 packets, 264 bytes
0008A3B895C40009B7FAD1E10800
ARP 03:53:35
Epoch: 0
...OUTPUT OMITTED...
Example 8-30 shows sample output from the show cef not-cef-switched command. Even
though CEF is enabled, some traffic might still be switched through another packetswitching
path, perhaps because a feature was enabled that required the processor of the
router (identified as RP in the output) to handle specific traffic types. The show cef notcef-
switched command shows information about such traffic that was not CEF-switched.
Example 8-30 show cef not-cef-switched Command Output
R4#show cef not-cef-switched
CEF Packets passed on to next switching layer
Slot No_adj No_encap Unsupp’ted Redirect Receive Options Access Frag
RP 0 0 0 0 6676 0 0 0
www.CareerCert.info
www.CareerCert.info
www - CareerCert - info
258 CCNP TSHOOT 642-832 Official Certification Guide
Now that you have reviewed the different packet-switching options for a router, you can
better analyze how a router is forwarding specific traffic. Following is a list of troubleshooting
steps you can follow if you suspect that network traffic is being impacted by a performance
problem on one of the routers along the path from the source to the destination.
Step 1. Use the traceroute command to determine which router along the path is
causing excessive delay.
Step 2. After you identify a router that is causing unusually high delay, use the show
processes cpu command to see the CPU utilization of that router and identify
any processes that might be consuming an unusually high percentage of
the CPU.
Step 3. Use the show ip route ip-address command to verify that the router has a
route to the destination IP address.
Step 4. Use the show ip cef command to determine whether all the router interfaces
are configured to use CEF.
Step 5. Use the show ip cef ip-address 255.255.255.255 command to verify that
CEF has an entry in its FIB that can reach the specified IP address. Part of the
output from this command will be the next-hop adjacency to which traffic
should be forwarded, along with the egress interface used to send traffic to
that next-hop.
Step 6. Issue the show adjacency interface-id detail command to verify that CEF has
an entry in its adjacency table for the egress interface identified in Step 5.
Step 7. With the show ip arp command, you can then confirm that the router knows
the MAC address associated with the next-hop IP address shown in the output
from Step 6.
Step 8. You can then connect to the next-hop device and verify that the MAC address
identified in Step 7 is indeed correct.
You can repeat these steps on the next-hop device or on another router whose response
time displayed in the output from Step 1 is suspect.
Excessive Memory Utilization
Much like a PC, router performance can suffer if it lacks sufficient available memory. For
example, perhaps you install a version of Cisco IOS on a router, and that router does not
have the minimum amount of memory required to support that specific Cisco IOS image.
Even though the router might load the image and function, its performance might be
sluggish.
Common Memory Troubleshooting Targets
Assuming a router does have the recommended amount of memory for its installed Cisco
IOS image, consider the following as potential memory utilization issues:
■ Memory leak: When a router starts a process, that process can allocate a block of
memory. When the process completes, the process should return its allocated mem-
Key
Topic
Key
Topic
www.CareerCert.info
www.CareerCert.info
www - CareerCert - info
Chapter 8: Troubleshooting BGP and Router Performance Issues 259
ory to the router’s pool of memory. If not all the allocated memory is returned to the
router’s main memory pool, a memory leak occurs. Such a condition usually results
from a bug in the Cisco IOS version running on the router, requiring an upgrade of
the router’s Cisco IOS image.
Example 8-31 shows sample output from the show memory allocating-process totals
command. This command can help identify memory leaks. The output shows information
about memory availability on a router after the Cisco IOS image of the
router has been decompressed and loaded.
Example 8-31 show memory allocating-process totals Command Output
R4#show memory allocating-process totals
Head Total(b) Used(b) Free(b) Lowest(b)
Largest(b)
Processor 83D27480 67463064 15347168 52115896 50311080 50127020
I/O 7C21800 4057088 2383016 1674072 1674072 1674044
Allocator PC Summary for: Processor
PC Total Count Name
0x809D7A30 1749360 180 Process Stack
0x80A7F664 918024 10 Init
0x81CEF6A0 882576 4 pak subblock chunk
0x81C04D9C 595344 54 TCL Chunks
0x800902A4 490328 6 MallocLite
...OUTPUT OMITTED...
The Head column in the output refers to the address (in hexadecimal) of the memory
allocation chain. The Total column is the total of used bytes and free bytes, which are
individually shown in their own columns. The Lowest column shows the lowest
amount of free memory (in bytes) that has been available since the router last booted.
The Largest column indicates the largest block of available memory. Following this
summary information, the output shows detailed memory allocation information for
each process running on a router.
■ Memory allocation failure: A memory allocation failure (which produces a MALLOCFAIL
error message) occurs when a process attempts to allocate a block of memory
and fails to do so. One common cause for a MALLOCFAIL error is a security
issue. For example, a virus or a worm that has infested the network can result in a
MALLOCFAIL error. Alternatively, a MALLOCFAIL error might be the result of a
bug in the router’s version of Cisco IOS. You can use the Cisco Bug Toolkit (available
from http://www.cisco.com/cgi-bin/Support/Bugtool/launch_bugtool.pl) to research
any such known issues with the version of Cisco IOS running on a router.
■ Buffer leak: Similar to a memory leak, in which a process does not return all of its allocated
memory to the router upon terminating, a buffer leak occurs when a process
www.CareerCert.info
www.CareerCert.info
www - CareerCert - info
260 CCNP TSHOOT 642-832 Official Certification Guide
does not return a buffer to the router when the process has finished using the buffer.
Consider the output of the show interfaces command seen in Example 8-32.
Example 8-32 Identifying a Wedged Interface
R4#show interfaces
...OUTPUT OMITTED...
Input queue: 76/75/780/0 (size/max/drops/flushes); Total output drops: 0
Queueing strategy: fifo
Output queue: 0/40 (size/max)
...OUTPUT OMITTED...
Notice the numbers 76 and 75 highlighted in the output. These values indicate that
an input queue of the interface has a capacity of 75 packets, and that queue currently
has 76 packets. This is an oversubscription of the queue space. An interface in this
condition is called a wedged interface. In such a condition, the router does not forward
traffic coming into the wedged interface.
The show buffers command can also be helpful in diagnosing a buffer leak. To illustrate,
consider the output of the show buffers command shown in Example 8-33.
Example 8-33 show buffers Command Output
Key
Topic R4#show buffers
Buffer elements:
1118 in free list (500 max allowed)
570 hits, 0 misses, 1119 created
Public buffer pools:
Small buffers, 104 bytes (total 71, permanent 50, peak 71 @ 00:21:43):
53 in free list (20 min, 150 max allowed)
317 hits, 7 misses, 0 trims, 21 created
0 failures (0 no memory)
Middle buffers, 600 bytes (total 49, permanent 25, peak 49 @ 00:21:43):
5 in free list (10 min, 150 max allowed)
122 hits, 8 misses, 0 trims, 24 created
...OUTPUT OMITTED...
This output indicates that the router has 49 middle buffers, but only 5 of those 49
buffers are available. Such a result might indicate a process allocating buffers but failing
to deallocate them. Like a memory leak, a buffer leak might require updating the
Cisco IOS image of a router.

Excessive BGP Memory Use
Earlier in this chapter you learned about troubleshooting BGP. If a router is running BGP,
be aware that BGP runs multiple processes and can consume significant amounts of router
memory. The show processes memory | include BGP command, as shown in Example 8-
34, can show you how much memory is being consumed by the various BGP processes of
a router. If BGP is consuming a large percentage of your router memory, you might consider
filtering out unneeded BGP routes, upgrading the memory on that router, or running
BGP on a different platform that has more memory.
Example 8-34 show processes memory | include BGP Command Output
R1#show processes memory include BGP
77 0 16960 0 10068 0 0 BGP Router
108 0 0 0 6892 0 0 BGP I/O
112 0 0 0 9892 0 0 BGP Scanner
Depending on the router platform, your router might have multiple line cards with different
amounts of memory available on each line card. The show diag command can help
you isolate a specific line card that is running low on memory, perhaps because that line
card is running BGP.