Expect scripting best practices that I learned so far

I use expect scripting for logging into remote machine and do some stuff and get back result as success or failure.

Here are my learning's..

1. Always setup keys at local and remote machines for logging in.

2. Setup your own prompt text at remote shell (> or # or $)

3. Use wait method to get exit value

1. REGEX matching:

If the remote machine's prompt is color coded with "$" in the prompt string, then for effective regex pattern match use follwing,

set prompt ".*(>|%|#|\\$).*"

expect -re $prompt

2. To know exit status of last command/script, add the following at the end of expect script,

lassign [wait -nowait] pid spawnid os_error_flag value

if {$os_error_flag == 0} {
    puts "exit status: $value"
} else {
    puts "errno: $value"
}

exit $value

3. Syntax issues:

While using "if" statements curly braces has to be in same line,

if {condition} {

   statement

}

While using expect_out, "no space" in between arguments(comma and string),

$expect_out(0,string)

4. Here document

If you ever has to use nested spawn (login to a shell again login to another shell), think of using here document 

set command "/opt/bin/another_shell << EOF\n"

append command "execute_command\n"

append command "exit\n"

append command "EOF"

send "$command\n" 

5. Solving issue of expect skipping conditional "send" statement

So, I had a part of script like,

set prompt ".*(>|%|#|\\$).*"

expect -re $prompt

if {[info exists args(-a)]} {
send "$args(-c) $args(-a)\n"
} else {
send "$args(-c)\n"
}

expect -re $prompt

send "echo \"returncode :$?\"\n"

exit 0

In the above snippet, "send" statements inside "if .. else" did not execute always. it gets skipped so many times.

Fix1: Adding delay (sleep 1) above last expect statement.

This worked well. Anyway I wanted a more elegant solution.

Fix2: using "$?"

if {[info exists args(-a)]} {
send "$args(-c) $args(-a);echo $?\n"
} else {
send "$args(-c);echo $?\n"
}

expect -re "0\r\n$prompt"

exit 0

Here the issue is, expect need to look for two lines 

0

PROMPT#

So, I added "\r\n" after "0" then the prompt statement.

Python, Automation, Problems, Solutions

Let me share, few of the problems that I faced during python automation here,

Handling Passwords across Python and Expect:

Scenario - Login to Hypervisor manager (where VNF is deployed) and reset (power off and on) a particular VM. Meanwhile monitor VNF node for status change. After that VM comes back to service perform assertions for setting test verdict.

Issue: password to VM manager must be captured using python and be sent to expect script as argument. password should not be printed / echoed anywhere. (catch: password may contain special characters)

Solution:
import getpass

pwd = getpass.getpass() # input: my#pwd

=> So password will not be echoed - partly solved

special characters must be escaped before send that as argument to expect script

pwd=re.escape(pwd)

cmd = "expect my_script.exp "+pwd  #expect my_script.exp my\#pwd
os.system(cmd)

=> Fully solved.

The Importance of "exit 0" in shell script:

 Scenario:
Python calls a shell script to kill a process. And based on exit status the Python takes further steps.

Problem:
When the processes that are being killed are interdependent, some time "no such process" error is thrown. So a non-zero exit status is returned to Python and It prints killing NOT successful, eventhough intended killing has been done.

Shell script:

#!/bin/bash

ps -ef | grep xxx | xargs -i kill {}

Python ():

import subprocess

def killme():
  p = subprocess.Popen(['./script.sh'], stdout=subprocess.PIPE)
  out, err = p.communicate()
  if p.returncode !=0:
    print "Killing NOT successful"
  print "Killed"

killme()


Solution:

Added exit 0 in shell script

#!/bin/bash

ps -ef | grep xxx | xargs -i kill {}
exit 0

Using Object of a class as an attribute of another class:

class1.py

class class1:
  def __init__(self):
    self.variable = "One"

class2.py

class class2:
  def __init__(self, obj):
    self.obj = obj

test.py

from class1 import class1
from class2 import class2

c1 = class1()
c2 = class2(c1)
c1.variable = "two"
print c2.obj.variable

#two