TESTING FRAMEWORKS

A testing framework is a set of guidelines or rules used for creating and designing test cases. A framework is comprised of a combination of practices and tools that are designed to help QA professionals test more efficiently.

These guidelines could include coding standards, test-data handling methods, object repositories, processes for storing test results, or information on how to access external resources.

Benefits of a Test Automation Framework

Utilizing a framework for automated testing will increase a team’s test speed and efficiency, improve test accuracy, and will reduce test maintenance costs as well as lower risks. They are essential to an efficient automated testing process for a few key reasons:

·        Improved test efficiency

·        Lower maintenance costs

·        Minimal manual intervention

·        Maximum test coverage

·        Re-usability of code

Types of Automated Testing Frameworks

There are six common types of test automation frameworks and these include:

·        Linear Automation Framework

·        Modular Based Testing Framework

·        Library Architecture Testing Framework

·        Data-Driven Framework

·        Keyword-Driven Framework

·        Hybrid Testing Framework

1.      Linear Automation Framework

With a linear test automation framework, also referred to as a record-and-playback framework, testers don’t need to write code to create functions and the steps are written in a sequential order. In this process, the tester records each step such as navigation, user input, or checkpoints, and then plays the script back automatically to conduct the test.

Advantages

·        There is no need to write custom code, so expertise in test automation is not necessary.

·        This is one of the fastest ways to generate test scripts since they can be easily recorded in a minimal amount of time.

·        The test workflow is easier to understand for any party involved in testing since the scripts are laid out in a sequential manner.

·        This is also the easiest way to get up and running with automated testing, especially with a new tool. Most automated testing tools today will provide record-and-playback features, so you also won’t need to plan extensively with this framework.

Disadvantages

·        The scripts developed using this framework aren’t reusable. The data is hardcoded into the test script, meaning the test cases cannot be re-run with multiple sets and will need to be modified if the data is altered.

·        Maintenance is considered a hassle because any changes to the application will require a lot of rework. This model is not particularly scalable as the scope of testing expands.

2.      Modular Based Testing Framework

Implementing a modular framework will require testers to divide the application under test into separate units, functions, or sections, each of which will be tested in isolation. After breaking down the application into individual modules, a test script is created for each part and then combined to build larger tests in a hierarchical fashion. These larger sets of tests will begin to represent various test cases.

A key strategy in using the modular framework is to build an abstraction layer, so that any changes made in individual sections won’t affect the overarching module.

Advantages

·        If any changes are made to the application, only the module and it’s associated individual test script will need to be fixed, meaning you won’t have to tinker with the rest of the application and can leave it untouched.

·        Creating test cases takes less effort because test scripts for different modules can be reused.

Disadvantages

·        Data is still hard-coded into the test script since the tests are executed separately, so you can’t use multiple data sets.

·        Programming knowledge is required to set up the framework.

3.      Library Architecture Testing Framework

The library architecture framework for automated testing is based on the modular framework, but has some additional benefits. Instead of dividing the application under test into the various scripts that need to be run, similar tasks within the scripts are identified and later grouped by function, so the application is ultimately broken down by common objectives. These functions are kept in a library which can be called upon by the test scripts whenever needed.

Advantages

·        Utilizing this architecture will lead to a high level of modularization, which makes test maintenance and scalability easier and more cost effective.

·        This framework has a higher degree of reusability because there is a library of common functions that can be used by multiple test scripts.

Disadvantages:

·        Test data is still hard coded into the script. Therefore, any changes to the data will require changes to the scripts.

·        Technical expertise is needed to write and analyze the common functions within the test scripts.

·        Test scripts take more time to develop.

4.      Data-Driven Framework

Using a data-driven framework separates the test data from script logic, meaning testers can store data externally. Very frequently, testers find themselves in a situation where they need to test the same feature or function of an application multiple times with different sets of data. In these instances, it’s critical that the test data not be hard-coded in the script itself, which is what happens with a Linear or Modular-based testing framework.

Setting up a data-driven test framework will allow the tester to store and pass the input/output parameters to test scripts from an external data source, such as Excel Spreadsheets, Text Files, CSV files, SQL Tables, or ODBC repositories.

The test scripts are connected to the external data source and told to read and populate the necessary data when needed.

Advantages

·        Tests can be executed with multiple data sets.

·        Multiple scenarios can be tested quickly by varying the data, thereby reducing the number of scripts needed.

·        Hard-coding data can be avoided so any changes to the test scripts do not affect the data being used and vice versa.

·        You’ll save time by executing more tests faster.

Disadvantages

·        You’ll need a highly-experienced tester who is proficient in various programming languages to properly utilize this framework design.

·        Setting up a data-driven framework takes a significant amount of time.

5.      Keyword-Driven Framework

In a keyword-driven framework, each function of the application under test is laid out in a table with a series of instructions in consecutive order for each test that needs to be run. In a similar fashion to the data-driven framework, the test data and script logic are separated in a keyword-driven framework, but this approach takes it a step further.

With this approach, keywords are also stored in an external data table, making them independent from the automated testing tool being used to execute the tests. Keywords are the part of a script representing the various actions being performed to test the GUI of an application. These can be labeled as simply as click or login.

Advantages

·        Minimal scripting knowledge is needed.

·        A single keyword can be used across multiple test scripts, so the code is reusable.

·        Test scripts can be built independent of the application under test.

Disadvantages

·        The initial cost of setting up the framework is high.

·        It is time-consuming and complex.

·        You need an employee with good test automation skills.

·        Keywords can be a hassle to maintain when scaling a test operation. You will need to continue building out the repositories and keyword tables.

6.      Hybrid Test Automation Framework

As with most testing processes today, automated testing frameworks have started to become integrated and overlap with one another. As the name suggests, a hybrid framework is a combination of any of the previously mentioned frameworks set up to leverage the advantages of some and mitigate the weaknesses of others.

Every application is different, and so should the processes used to test them. As more teams move to an agile model, setting up a flexible framework for automated testing is crucial. A hybrid framework can be more easily adapted to get the best test results.