How to use Perl to connect to your devices and save the day

Today I want to write a little bit a about a non-CCIE topic, but a useful one if you are into the network administration arena, and that is “how perl (or scripting in general) can save your neck/network

Almost a year ago I had a problem, it was a Cisco IOS bug that made the PE devices lose IP connectivity with their CPE neighbors on a VRF (this was a cable environment). To make the story short, we had a case with Cisco TAC and we finally discovered how the problem appeared. At that time my skills on EEM where very small (right now they are not too great either), and the TAC wasn’t able to create an EEM script to detect and workaround the issue either. Cisco needed to find out exactly why this was happening and build a patch and I needed to build a workaround for the connectivity problems. I had to use my not-so-amazing perl knowledge to create a script that could be launched from a monitoring device to detect, solve the problem (clear the adjacency table) and gather some information for the TAC (the script was run every 5 minutes with cron).

I want to share with all of you what I did because it may come in hand on a difficult situation, you just need a device where you can run perl (Linux and Unix system are the most easy environments to do this). I’m going to show just a small part of the script (it end up being a very long script), for educational purpouses but with just a little bit of programming knowledge you might be able to do interesting things.

Perl, is well known for his amazing library (http://search.cpan.org), I used NET::Telnet for this, but it’s up to you to use anything else; Perl motto: TMTOWTDI (pronounced: Tim Toady) “There’s more than one way to do it“. On Debian finding the library is as easy as:

box:~#apt-cache search perl lib net telnet

And in the results you should see:

libnet-telnet-perl – Script telnetable connections
libnet-telnet-cisco-perl – Additional functionality to automate Cisco management

box:~#aptitude install libnet-telnet-perl

With that you are ready to go, the next step is to build your script. I’m going to take a section of my script, but functional to explain some basics and how you can use it.

Note: The intention here its not to do the most beautiful or perl-ish code, but to have a script that is functional and easy to understand.

#!/usr/local/bin/perl --

# Here we call the library that is going to allow us to be able to
# "telnet" to the remote device, from our script
use Net::Telnet ();

# The username and password used to connect to the remote device
my $username = 'cisco';
my $password = 'cisco';

my $status_log = "/tmp/log_status";

my %info;
# A simple switch, just to be able to see if the issue was detected at the end of the script
my $found = 0;

# Some date formating
my $now = localtime();
my ( $sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localtime();
my $timestamp = sprintf "%4d%02d%02d.%02d%02d", 1900+$year, $mon+1, $mday, $hour, $min, $sec;

# The script used and external file to define the PE device (the one we are connecting to),
# the VRF that where affected by the bug and the IP of the remote CPE device
# This file had one line for each combination in the format: PE_IP;VRF_NAME;CPE_IP
# Example:
# 192.168.0.1;vrf_test1;10.0.0.1
# 192.168.0.2;vrf_test2;10.0.1.1
# So here we open the file with connection information, if its not possible the script dies:
open(CONNECT, "<", "/tmp/PE-VRF-CPE")
    or die "Can't open the file with devices information: $!\n";

# We open a the log file where I'm going to dump script status information (define upstair).
open FLOG, ">>", $log;

# If the file is correct, for earch line in the file we go and build a hash called info.
# We split each line using the ";" character (as I explained upstaris)
while (  < CONNECT > ) {
   my ($pe,$vrf,$cpeip) = split(';', $_);
   push( @{ $info->{"$pe"}->{"$vrf"} }, $cpeip );
}
close(CONNECT);

#
# This is the important part of the script and the one you may find more useful
# Net::Telnet, allow us to connect, run remote commands and obtain outputs
#

# For each PE, we are going to open a connection
for my $pe (sort keys %$info) {
   $t = new Net::Telnet (Timeout => 15, Prompt => '/[%#>]$/');
   # The actual connection is made, and we look for a valid prompt
   $t->open($pe);
   # using the Authentication information that we define before
   $t->login($username, $password);
   # We increase the buffer because some of the show's needed by the TAC where ver long
   $t->max_buffer_length("50000000");
   $t->timeout("120");
   # as this is all auto, we are not going to be there to hit a key,
   # so the terminal lengh is set to 0 (all output is dumped at once).
   # As you can see this is the very first command actually runned
   $t->cmd("terminal len 0");
   # For each VRF and CPE we look into the routing table, looking for routes to the CPE with two
   # destination bundle interface (this was part of the bug)
   while(($vrf, $cpe) = each(%{$info->{ $pe }})) {
      foreach my $ce (@$cpe) {
         chomp($ce); # munch munch
         # Here we look for the route pointing to the CPE under the VRF
         @lines = $t->cmd("show ip route vrf $vrf $ce");
         # If we found two or more destination bundle interfaces, then we had the issue
         if ( grep(/Bundle/, @lines) >= 2) {
            # Status information is send to the status log
            print FLOG $now, " PANIC More than One Bundle on PE:$pe VRF:$vrf CPE:$ce\n";
            # And we call the function to gather information for the TAC
            DebugGather ( $vrf, $cpe, $pe );
            # To fix the issue we needed to clear the adjacency table,
            # so we did it after gathering all we need
            $t->cmd("clear adjacen");
         }
      }
   }
   # [...]
   # The script originally continued, but to make it short I cutted it here.
}

# If after all we dont find the issue, we log it to the status log:
print FLOG "$now -> No problem detected \n" if ( not $found );

close FLOG;

# This was the function used to gather information for the tac, that we called on the loop

# We call this function everytime we found the error
sub DebugGather {
  my ( $vrf, $cpe, $pe ) = @_;
  $found = 1;
  # If we found the issue the script runs all of this outputs (this are just a few of them),
  # each command needed is saved in "slot" of the array debug_commands
  my @debug_commands = (
                   "show ip cef vrf $vrf detail",
                   "show adjacency $cpe internal",
                   "show ip cef vrf $vrf $cpe internal",
                   "show log",
                   "show cable tech",
                 );
  # For each incident found we created a separate file, so all the information could
  # could be handled in a better way by the Cisco TAC
  open (FDEBUG, ">>", "/tmp/debug_$vrf.$timestamp");
  print FDEBUG "\n-----------------------------------\n";
  print FDEBUG "** [$now] ** Start of debug info DEBUG ($vrf) $cpe $pe)\n";
  # Each commands needed is runned here:
  foreach my $commands (@debug_commands) {
     # The output of each line end up in the array
     @lines = $t->cmd("$commands");
     # In this section I did some output manipulation that I dont find useful for the post,
     # so we just dump the information into the file
     foreach my $line (@lines) {
        print FDEBUG "$line";
     }
  }
  close FDEBUG;
}

So, this is just a small example of what you can do with Perl and some small programming. If you look at the previous code, it may seem like a long code, but if you remove all my comments it’s not so bad.

Remember, If you take into account the amazing amount of Perl Libs available at CPAN, you can build anything you need to monitor or react to problems in your network; from SNMP to complex scripts (including gathering information for the TAC and clearing adyacencies ;) ).

Daniel Rodriguez

Daniel is the co-founder of Network Faculty. He holds the following certifications: CCNA, CCNP, CCIP, LPIC Level 2 and Security+. He loves Perl Programming, Linux, IPv6 and Multicast deployments. When not doing network related study/work, he is training for triathlons and other endurance sports.

More Posts - Website - Twitter - Facebook

Creative Commons License
The How to use Perl to connect to your devices and save the day by CCIE Blog, unless otherwise expressly stated, is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Tags: ,

5 Responses to "How to use Perl to connect to your devices and save the day"

  • Jochen Bartl says:
  • jomih says:
  • Jose Leitao says:
    • Jochen Bartl says:
Leave a Comment

*

Notify me of followup comments via e-mail. You can also subscribe without commenting.