So, say we have an interface that’s dropping packets. Not too many, but the ideal number is zero, so we’d like to see them in the graphs in our NMS (which is based on rrdtool as all decent NMSes are). We use rrdgraph to show packets out in black and packets that should have gone out, but were discarded instead in purple:

rrdtool graph images/router-errors-unscaled.png \
--title "unscaled" \
--vertical-label 'Pkts/Second' \
--start end-2day \
--end -1hr \
--width 800 \
--height 250 \
--imgformat PNG \
--interlace \
DEF:ifOutUcastPkts=router.rrd:ifOutUcastPkts:AVERAGE \
DEF:ifOutNUcastPkts=router.rrd:ifOutNUcastPkts:AVERAGE \
DEF:ifOutDiscards=router.rrd:ifOutDiscards:AVERAGE \
CDEF:ifOutPkts=ifOutUcastPkts,ifOutNUcastPkts,+ \
LINE1:ifOutPkts#003300:ifOutPkts/sec \
LINE1:ifOutDiscards#990099:ifOutDiscards/sec\\n \
GPRINT:ifOutPkts:AVERAGE:"Avg ifOutPkts %1.2lf\\n" \
GPRINT:ifOutDiscards:MAX:"Max ifOutDiscards %1.2lf"

That produces a graph like this:

(Click on the thumbnails to get the full graphs.)

We can read the average rate of discarded packets in the graph key at the bottom, and there are tiny little blips in the purple line that represents discards , but we don’t have a strong visual cue that something is off.

One possible solution is to scale up the discard values relative to the total packets. A factor of 100 ought to do it. Then we’ll use the –right-axis options to rrdgraph to label the right-hand y-axis accordingly.

We add this CDEF to provide the scaling (the LINE1 etc commands will need to be altered accordingly; you’ll see those in the final snippet):

CDEF:scaled_ifOutDiscards=ifOutDiscards,100,*

That gives us a graph that looks like this:

Note that it now looks like we’re dropping up to 70 packets/second – we still have to read the stats in the key at the bottom of the graph. So let’s get the secondary y-axis correctly labeled & scaled, with the following commands:

--right-axis-label 'Discards/Second'
--right-axis 0.01:0

–right-axis-label prints the specified text along the right-hand axis.
–right-axis [scale:shift] scales and/or shifts the tickmarks on the right axis relative to the left axis. In this case, the new values we’re displaying are 100X the original values, so we need to scale our axis accordingly: 0.01. More simply: left/right = 1/100. We don’t need to start at a value other than 0, so we set the shift value to 0.

Hmmm…rrdtool has automatically converted our values to milli-units. (Note the lower-case m in the labels.) Let’s fix that with the –right-axis-format command:

--right-axis-format %1.1lf

And that’s all there is to it!

The final rrdgraph command looks like this:
rrdtool graph images/router-right-axis-format.png \
--title "right-axis-format" \
--vertical-label 'Pkts/Second' \
--right-axis-label 'Discards/Second' \
--right-axis 0.01:0 \
--right-axis-format %1.1lf \
--start end-2day \
--end -1hr \
--width 800 \
--height 250 \
--imgformat PNG \
--interlace \
DEF:ifOutUcastPkts=router.rrd:ifOutUcastPkts:AVERAGE \
DEF:ifOutNUcastPkts=router.rrd:ifOutNUcastPkts:AVERAGE \
DEF:ifOutDiscards=router.rrd:ifOutDiscards:AVERAGE \
CDEF:scaled_ifOutDiscards=ifOutDiscards,100,* \
CDEF:ifOutPkts=ifOutUcastPkts,ifOutNUcastPkts,+ \
LINE1:ifOutPkts#003300:ifOutPkts/sec \
LINE1:scaled_ifOutDiscards#990099:ifOutDiscards/sec\\n \
GPRINT:ifOutPkts:AVERAGE:"Avg ifOutPkts %1.2lf\\n" \
GPRINT:ifOutDiscards:MAX:"Max ifOutDiscards %1.2lf"

Part 1: Set up your log host.

Step 1: before editing any of the files discussed below, be sure to back them up, e.g.:
cp /etc/syslog.conf /etc/syslog.conf.dontmessthisup

Step 2: edit /etc/syslog.conf to include this:
#router logging
local6.debug                    /var/log/cisco.log

This means “send all messages from facility local6, with a priority of debug or greater, to /var/log/cisco.log”.

(Note that the default facility for Cisco is local7; if you want/need to use the Cisco default, change the above accordingly.)

Step 3: create the log file I specified above:
sudo touch /var/log/cisco.log

Step 4: make syslog listen to messages from remote machines:
edit /etc/default/syslogd to include the -r option:
SYSLOGD="-r"

Step 5: restart the syslog daemon:
sudo /etc/rc2.d/S10sysklogd restart
(or look up the process id using ps -ef, and kill -HUP it; or pkill -1 syslogd…  your choice.)

Step 6: test it:
logger -p local6.debug "is this working?"

cat /var/log/cisco.log, you should see the line above.

Now, we have a problem: we also see the message in some of the other log files in /etc/syslog.conf (such as /var/log/syslog, /var/log/messages, and /var/log/debug).
We don’t want the messages from the router mixed in with the system messages (pet peeve alert)!  Edit /etc/syslog.conf to include exceptions for local6 anywhere we have an *.[whatever], like so:

*.*;local6.none;\
auth,authpriv.none        -/var/log/syslog

Restart the syslog daemon again.

Test it:
for each in debug info notice warn err crit alert emerg panic
do
logger -p local6.${each} "this should only go to cisco.log - ${each}"
echo done with ${each}
done

Check /var/log/cisco.log, /var/log/syslog, /var/log/debug, and /var/log/messages – messages should only be in cisco.log.

—
Part 2: Configure your router to send messages to the log host.
config t
logging [ip address of your ubuntu box]
logging facility local6
logging history [severity]
logging on

Your version of IOS may require different commands. Have fun with that.

<0-7>          Logging severity level
emergencies    System is unusable                (severity=0)
alerts         Immediate action needed           (severity=1)
critical       Critical conditions               (severity=2)
errors         Error conditions                  (severity=3)
warnings       Warning conditions                (severity=4)
notifications  Normal but significant conditions (severity=5)
informational  Informational messages            (severity=6)   <--
debugging      Debugging messages                (severity=7)

Normally I stick with informational (sev=6); debugging may spew a huge amount of info.

Compare the logging buffer on your router (“sh logging”) with the file on your log server;  messages since you made the change should also be going to the server.  If not, make sure you can reach the log server from the router, and that port 514 isn’t blocked anywhere.

—
Part 3: Rotation
Once that’s working, set up the log rotation on your log host.
Add this to /etc/logrotate.conf, below the “system-specific logs may be configured here”
/var/log/cisco.log {
missingok
compress
notifempty
daily
rotate 7
size 500k
}

You may need to tinker around with these settings depending on how many messages you’re getting a day.