Expect scripting best practices that I learned so far

I use expect scripting for logging into remote machine and do some stuff and get back result as success or failure.

Here are my learning's..

1. Always setup keys at local and remote machines for logging in.

2. Setup your own prompt text at remote shell (> or # or $)

3. Use wait method to get exit value

1. REGEX matching:

If the remote machine's prompt is color coded with "$" in the prompt string, then for effective regex pattern match use follwing,

set prompt ".*(>|%|#|\\$).*"

expect -re $prompt

2. To know exit status of last command/script, add the following at the end of expect script,

lassign [wait -nowait] pid spawnid os_error_flag value

if {$os_error_flag == 0} {
    puts "exit status: $value"
} else {
    puts "errno: $value"
}

exit $value

3. Syntax issues:

While using "if" statements curly braces has to be in same line,

if {condition} {

   statement

}

While using expect_out, "no space" in between arguments(comma and string),

$expect_out(0,string)

4. Here document

If you ever has to use nested spawn (login to a shell again login to another shell), think of using here document 

set command "/opt/bin/another_shell << EOF\n"

append command "execute_command\n"

append command "exit\n"

append command "EOF"

send "$command\n" 

5. Solving issue of expect skipping conditional "send" statement

So, I had a part of script like,

set prompt ".*(>|%|#|\\$).*"

expect -re $prompt

if {[info exists args(-a)]} {
send "$args(-c) $args(-a)\n"
} else {
send "$args(-c)\n"
}

expect -re $prompt

send "echo \"returncode :$?\"\n"

exit 0

In the above snippet, "send" statements inside "if .. else" did not execute always. it gets skipped so many times.

Fix1: Adding delay (sleep 1) above last expect statement.

This worked well. Anyway I wanted a more elegant solution.

Fix2: using "$?"

if {[info exists args(-a)]} {
send "$args(-c) $args(-a);echo $?\n"
} else {
send "$args(-c);echo $?\n"
}

expect -re "0\r\n$prompt"

exit 0

Here the issue is, expect need to look for two lines 

0

PROMPT#

So, I added "\r\n" after "0" then the prompt statement.

Power of python dictionary

Scenario: Copy logs that were created after certain timestamp to a destination directory.
Catch: there are different types of logs - Alarms, Notifications, Errors etc.

Here is my initial ugly code,

#!/usr/bin/python
import glob
import os
import datetime, time
import math
import shutil

class logCollector:
    def __init__(self):
        self.toLogpath = "/home/appuser/automation/logs" 
        self.applogPath = "/cluster/storage/no-backup/coremw/var/log/saflog"
    def getFileTime(self,path):
        _mtime = math.floor(os.path.getmtime(path))
        return _mtime
    def getLogsAfter(self, filePath, start):
        logFiles = glob.glob(filePath)
        fileList = list()
        for file in logFiles:
                fileTime = self.getFileTime(file)
                if float(fileTime) >= math.floor(start):
                    fileList.append(file)
        return fileList
    def getNotificationLogAfter(self, start):
        filePath = self.applogPath+"/*otif*log"
        return self.getLogsAfter(filePath, start)
    def getAlarmLogAfter(self, start):
        filePath = self.applogPath+"/*larm*log"
        return self.getLogsAfter(filePath, start)
    def getErrorLogAfter(self, start):
        filePath = self.applogPath+"/*Err*log"
        return self.getLogsBWsecs(filePath, start)
    def copyTraces(self, time):
        start_time=time
        fileList = list()
            fileList.extend(self.getNotificationLogAfter(start_time))
            fileList.extend(self.getAlarmLogAfter(start_time))
            fileList.extend(self.getErrorLogAfter(start_time))
        for logfile in fileList:
            if os.path.isfile(logfile):
                shutil.copy(logfile, self.toLogpath)
            else:
                print(logfile+" is not found!!!")


After Learning dictionary in python,

#!/usr/bin/python
import glob
import os
import shutil
import math

class myTraceCollector:
        destination = "/home/appuser/automation/logs"
        appTraces ={
                        "Alarm Logs":"/cluster/storage/no-backup/coremw/var/log/saflog/*larm*log",
                        "Notification Logs":"/cluster/storage/no-backup/coremw/var/log/saflog/*otif*log",
                        "Error Logs":"/cluster/storage/no-backup/coremw/var/log/saflog/*Err*log"
                }

        def getTracesAfter(self, time):
                for fileType, filePath in self.appTraces.iteritems():
                        print fileType
                        tracesList = glob.glob(filePath)
                        for file in tracesList:
                                fileTime = os.path.getmtime(file)
                                if float(fileTime) >= math.floor(time):
                                        shutil.copy(file, self.destination)
                                        print file+" copied"

It is wow! experience for me...

Python, Automation, Problems, Solutions

Let me share, few of the problems that I faced during python automation here,

Handling Passwords across Python and Expect:

Scenario - Login to Hypervisor manager (where VNF is deployed) and reset (power off and on) a particular VM. Meanwhile monitor VNF node for status change. After that VM comes back to service perform assertions for setting test verdict.

Issue: password to VM manager must be captured using python and be sent to expect script as argument. password should not be printed / echoed anywhere. (catch: password may contain special characters)

Solution:
import getpass

pwd = getpass.getpass() # input: my#pwd

=> So password will not be echoed - partly solved

special characters must be escaped before send that as argument to expect script

pwd=re.escape(pwd)

cmd = "expect my_script.exp "+pwd  #expect my_script.exp my\#pwd
os.system(cmd)

=> Fully solved.

The Importance of "exit 0" in shell script:

 Scenario:
Python calls a shell script to kill a process. And based on exit status the Python takes further steps.

Problem:
When the processes that are being killed are interdependent, some time "no such process" error is thrown. So a non-zero exit status is returned to Python and It prints killing NOT successful, eventhough intended killing has been done.

Shell script:

#!/bin/bash

ps -ef | grep xxx | xargs -i kill {}

Python ():

import subprocess

def killme():
  p = subprocess.Popen(['./script.sh'], stdout=subprocess.PIPE)
  out, err = p.communicate()
  if p.returncode !=0:
    print "Killing NOT successful"
  print "Killed"

killme()


Solution:

Added exit 0 in shell script

#!/bin/bash

ps -ef | grep xxx | xargs -i kill {}
exit 0

Using Object of a class as an attribute of another class:

class1.py

class class1:
  def __init__(self):
    self.variable = "One"

class2.py

class class2:
  def __init__(self, obj):
    self.obj = obj

test.py

from class1 import class1
from class2 import class2

c1 = class1()
c2 = class2(c1)
c1.variable = "two"
print c2.obj.variable

#two

Grep LookAhead and LookBehind expressions

LookAhed ?=
 

LookBehind ?<=

?!
 

?<!

Biulding my own Framework for System Testing

    It happened in my career as well. I also developed yet another framework for automating system testing testcases.

First of all, I went through few blogs/presentations/posts to see If I can use any existing frameworks. There were so many content for Selenium and other web-based test frameworks. But my situation called for developing a new one. So, I built this on a basic idea. This idea has been repeated for numerous times in numerous websites. It goes as follows,

As you can see, It is a very simple yet powerful and widely used design. Here, the Automation Server holds all my scrips (Perl, Expect, Shell, Python ..etc) and few of these scripts runs on SUT and other few runs on AS itself. Actual test scenarios runs on SUT and logs are collected, then copied to Reports and Logs Server. URLs of these logs and other information are updated in database. When the test is completed, you get a URL to see the report. The presentation server takes care of tabulating and presenting the reports. In fact this Reports and Logs Server is a NFS mounted on Presentation Server.

But wait, who will initiate the test? There comes Jenkins!

Yes, you are right. Above framework is a part of our Continuous Integration architecture. Anyway, it was interesting to setup my own framework, though the same has been done by millions of other people.

Performance (patience) testing

    I was a happy functional tester who used to create and run automation scripts, collect the logs, analyse it and report the results. And then came the shift in career. I was posted as a performance tester in a reputed MNC.

Initial months were horrible, those were the times when my patience got tested heavily. Yes! performance testers needs to have more patience and focus. Let me tell you what difference I felt due to this shift in job profile.

As a functional tester in telecom domain, I used to write and run, for instance, a script that simulates direct dial voice call. And the SUT will write the logs of that single call. Moreover, my TTCN simulator also would write its own logs and report the mismatches in message flows. So, my focus would be on message flows and various parameters of each message. I was able to run quite a few test cases in a good day.

Now, here comes a typical week as a performance tester. I need to setup at-least 90 TTCN simulators with similar call flows. And I need to focus on how to make the system bleed due to load. Call flows and message parameters are no more concern now. They were certified by functional testers that those are fine. So, a full day gets consumed by environment setup (if something goes wrong then two days). And if something went wrong in the setup, I used to turn towards the system logs. It will slap me with a blob of details. Yes! the log will contain details of lakhs of calls. Then, it will easily take at-least an hour to analyse the log.

Here is where my learning starts again. Now, I see the "grep" tool of Linux as my saviour. My learnings may be small, but it shows how ignorant I was previously. For instance, we know the following command,

    # grep "pattern" *
file1: pattern match
file2: pattern match

You may want to use AWK again to filter the column level details. Here the filenames that are printed in each pattern match may be an overhead for you. So, you may use,

    # grep -h "pattern" *
pattern match
pattern match

Anyone can find this (https://www.gnu.org/software/grep/manual/grep.html) URL and gain more knowledge. However it will be complete when it is used in a context. So, I have started this blog to share my learnings (even if it is small or big or tiny) and experiences as a Software Professional. And please share yours too!! I would be grateful for that!