See Also

Switch Testing Framework Library

Business Challenge

The customer is a FC switches development company. The company is one of the leading innovator and provider of data center networking solutions. One of the departments in the company is SQA (System Quality Assurance). The goal of the project is to create a library to help SQA engineers perform their daily operational activities (such as, configure and maintain storage area network topologies, control each switch and whole fabrics, monitor/log device outputs, run test cases and analyze fabric response).
The following modern topologies are supported by the library:
  • Flat SAN Topologies
  • Mesh SAN Topologies
  • Core-Edge SAN Topologies

Solution

The proposed solution is a software framework that supports fabric-wide test automation. It provides a complete solution for rapid test automation development in SAN and metaSAN environment. It is designed for end-to-end system testers and at the same time its framework can be used to develop a wide range of testing automation applicable for unit testing, stress testing and functional testing.

Created library centralizes validation and verification methods enabling them to be shared across many test cases and providing increased test coverage. Framework reduces the time required to create new test cases and increases the number of data points which can be validated and verified. It also reduces the time required to run test cases with parallel test case processing.

The library provides the following functionality:
  • connecting to the fabric components (switches) and executing commands via CLI;
  • parsing output from diagnostic commands to get information about initial switch status and configuration;
  • simulating actions executed on switches to represent changes in initial data;
  • comparing the runtime status values to the expected values;
  • reporting errors if problems are found.
There are many test cases designed to use the library following these steps:
  • gathering initial data;
  • executing actions on fabric;
  • gathering runtime data;
  • executing validating modules, inside a module;
  • comparing the runtime data to initial data modified by action simulating procedure;
  • reporting problems that are found.
Testing is performed for the entire SAN (storage area network) gathering information from each FC switch using CLI.
Required verification modules cover the following test areas:
  • Common functionality
  • Zoning
  • Routing services
  • Fabric services
  • Access Gateway
  • Port Functionality
  • QoS
  • Stress Testing
  • Interoperability
  • High Availability
  • FICON
  • FC Router
  • FCoverIP
  • Statistical monitoring
  • Encryption
  • SAN Performance

Architecture

The Distributed Model consists of two parts: Broker and Agent. Every part is a superstructure over the library and provides distributed execution of its procedures without modifying the code. Therefore DM is not affected by changes in the library code. DM distributes the library functions between several computers. One of computers should be running Broker part while others Agent parts.
Broker library manages execution of modules by several independent Agents:
  • making connections to Agents;
  • sending data and commands to Agents;
  • getting results from Agents.
Broker library handles verification modules calls and allows all other calls go through the library.
Agent functions are:
  • listening for requests from Broker;
  • executing requested verification modules;
  • sending execution results to Broker.
Template and runtime data is supposed to be shared over NFS.

Tools and Technologies

  • Programming languages: TCL/Tk 8.4, expect 1.2
  • Tools: Subversion SVN, Mantis, DevTeam
  • Operating systems: Solaris, Debian, Ubuntu, Windows XP/Cygwin
  • Technologies: SAN, FC, FCoIP, FCoE, Access Gateway, AD, iSCSI, FCR

Benefits

The system allows user to create simple test scripts to automatically verify the state of the fabric for the correct behavior. Once created, a test script can be executed on various fabric topologies without any modification to the test script itself. A test script being executed provides comprehensive testing coverage by means of complete results verification. Upon completion, each test case run generates a detailed report of error analysis including all verification checkpoints.