Framework for Automation of Cloud-Application Testing using Selenium (FACTS)

A R T I C L E I N F O A B S T R A C T Article history: Received: 10 December, 2019 Accepted: 08 January, 2020 Online: 22 January, 2020 A framework is an amalgamation of integrated tools, libraries, utilities and its coordination to interact with the other automation components. The motive of designing a test automation framework is to implement uniform standards towards automation throughout the organization to achieve the desired outcomes. Test automation framework is required to maintain the standardization of the activities performed to manipulate the operations (addition, modification and deletion) on the test scripts and functions easily, to provide the scalability and reliability. Tools, process and testing team are the three key player of test automation. The proposed framework is based on the consideration of test environment (includes language binding, IDE, automation tools and testing framework). Selenium WebDriver is used as web application automation framework to execute test across multiple browsers that supports multiple operating system with variant programming language. Selection of WebDriver automation tool for this framework is due to their internal architecture to directly communicate with browser for fast execution. The combination of test automation tools (i.e. Java, Eclipse, Selenium and TestNG) makes JEST for designing test automation framework. Both Positive and negative test must be performed to verify functionalities of applications to handle unusual exceptions.


Introduction
Software testing is an evaluating as well as mitigating process, as it evaluates the functionalities of application to meet out the specific requirements of software product, at the same time it also identify the defects in product to mitigate the risk of software failure by find bugs and errors in the program.
"Framework for Automation of Cloud-application Testing using Selenium (FACTS)" is almost self-explanatory, though this paper may also explore the terminology explicitly. This paper may discuss about, why do we require test automation framework, how the software testing framework may organize the test automation, how do we select best appropriate software testing framework, what are the challenges being faced by the software automation team to design and deploy the test automation framework, what are the potential benefits of using software testing framework.
Why selenium for automation? Selenium is powerful open source tool for test automation framework independent of language binding & development platform that support functional testing across the browsers.
A framework is a combination of various forms of principles, guidelines, processes, concepts and tools etc. followed by testers to take advantage of it, to automate the application. Test automation framework decides the success or failure of any software automation project. Test automation is testing process of automating test with the help of automation tools to find out defects in the application. Test automation reduces test execution time and testing cost along with the less human interaction with the system. Test script has been used to execute test multiple times, when required. Test script must consider, what to automate or what not to be, like functionalities that are going to be change in near future or not, whether script is applicable for regression testing or not, whether test cases takes more time to execute/execute with errors or operated over huge amount of input data etc.

Test Automation Framework -An Understanding
Test automation framework is set of guidelines/rules followed while writing test script to generate test cases as per the ASTESJ ISSN: 2415-6698 requirement of test scenarios. A framework is an amalgamation of integrated tools, libraries, utilities and its coordination to interact with the other automation components to execute the test scripts and to capture the outcomes in systematic and beneficial manner to test a specific product. If a testing activity automated without having a test automation framework, it is obvious to raise the question that "how reliable the method would be? And how quickly they can produce the same output?" The motive of designing a test automation framework is to implement uniform standards/policies towards automation throughout the organization to achieve the desired result. A test automation framework can be considered effective if the result of test automation is easily understandable to end user, otherwise executing high volume of test cases are of little or no use for end user. The automated test generated report must include the point where the application gets failure along with test data being used for automation.

Why do we require test automation framework?
Test automation framework is required to maintain the standardization of the activities performed to manipulate the operations (addition, modification and deletion) on the test scripts and functions easily, with the aim to provide the scalability and reliability. If a tester automate testing activities without considering a framework, it is obvious to ask how reliable the method would be. Can the tester re-generate the same result? An automation framework sort out all such problems by providing a standard guideline to leverage reusable components to attain the desired & reliable results rapidly. There are various factors for which an automation framework required [1].
• The Ease of use (Reusability of test script) to maintain the test script. • To maintain reliability and consistency follow specific guidelines. • To maintain scalability with changing requirement.
• To improve the efficiency of testing activities • To provide easily understandable reports of automated results.

How do we select best appropriate test automation tool?
Selecting best appropriate tool for automation is one of the basic tasks for organization. The impact of selecting best automation tool may greatly affect the automation process which may provide sophistication in automation. While selecting tools for automation, certain conditions must keep in mind which makes sense for automation, like how frequently the repetitive test may occur, how often the regression testing is required etc. There are certain criteria to evaluate and select the automation tools [2] as mentioned below.
• Whether the tools fulfill the testing requirements. Is it compatible with project environment & technology that support objects in application? • Which testing level and types does the tools support like (system, functional, performance) testing. Select tools which support maximum testing type with the ability to automate complex requirements. • Whether the tools support all major browsers/OS or portable devices in which the application runs? • Whether the estimated budget for purchasing automation tool affordable for organization? • Whether the organization has skilled resources that can understand the language the tools support? • Does the tool have powerful mechanism for graphical reporting?
While selecting a tool comprehensive understanding of its integration, reporting and ease of use with other automation tool must be executed. The standard automation testing framework should be application & tool independent & loosely coupled, it means that if the organization decides to migrate from one automation tool to other, it need not required to create a new test automation framework from scratch.  [3] 1) Environment to be tested: While designing a test automation framework all the test related tools, equipment, scripts and procedures must be verified and validated to ensure the system is stable to perform the desired task. This would be the first step towards the environment setup for the test automation. Until the test environment is not supposed to work accordingly the test automation may not be cost effective. Thus all the coordinating tools that support the environment must be stable and work together smoothly as a whole prior to start the test automation.

Components of test automation framework
2) Validate supporting tools: While writing test scripts various supporting tools are required to organize the test. All such type of tools are properly investigated and independently monitored for expected contributory result as a unit, like test automation tool, bandwidth monitoring tools used for traffic management, other supporting tools such as requirement analysis, configuration management tools, test factory, defect tracking. A traffic monitoring and controlling activity of network is important to improve performance, efficiency and security of network resources. Network accommodates different kind of data traffic, identification of such traffic or congestion may lead the network administrator to optimize the network. Various types of network traffics are [4]. Configuration management tools are used to implement the changes (add, remove and update) occur during the test automation framework design to all associated system by implementing and enforcing to adopt such possible change. Test factory is an extensive use of test automation that minimizes the cost of testing and error rate along with supporting rapid scalability. Defect tracking is tracking the logged defects from beginning to end by testing, inspection or from customer feedback in a product with the aim to provide latest version of the product to fix the defect.

3) Drivers & Library:
Different browsers require their own driver to communicate and control with the WebDriver. Some of the browsers are OS dependent but Firefox and Chrome are easily available across the platform. Download and unpack the drivers and add the location to you system PATH variable. The browser driver interacts with the respective browser by establishing a secure connection. While a test script is executed with WebDriver environment various activities performed in the background like: • For every selenium command an HTTP request is generated to communicate with the browser driver. • Drivers receive the HTTP request trough HTTP Server.
• HTTP Server decides the execution of instruction on the browser. • As HTTP Server receives the execution status, it automates the script. Browser automation library has been used to interact with other task required for automation. Library works as basic building block to establish a successful test environment to execute the test script by providing the required jar files, libraries, drivers and other supportive files.
Steps to add the Selenium Library to the project: • Download & extract current release of Java Selenium library.
• Open the Eclipse IDE, go to the "Project→Properties" • New dialog box open, Click on " Java Build Path" in the left pan, the to the register "Libraries" • Click on "Add External JARs" in the right pan.
• Move to selenium library downloaded folder to select all .jar files by clicking "Open" button. • Repeat this for all .jar files in the sub folder libs as well.
Once the library is configured with the project it can be reusable with the different tests, such utilities can perform different tasks for different test cases.

4) Organize Framework:
Selenium is automated testing framework for web applications. To organize the framework following major steps can be taken into consideration [5]. •

5) Testing Team:
Successful test automation depends on efficient testing tools, standard testing process and skilled testing team to execute assigned responsibilities. Tools, process and testing team are the three key player of test automation. Software testing team is mainly engaged for test automation to utilize dedicated automation testing resources. The test team ensures that the success of testing and automation effort based on the roles, duties and skill they provide.
✓ Manual Tester -Design and develop manual scenarios from requirement analysis. ✓ Automation Tester -Design and develop automation script. ✓ Lead Automator-Design and develop project automation architecture. Provide training and guidelines to testing team and collaborate to achieve the testing goal. ✓ Automation Environment Expert-Design, configure and update test environment and automation software. Apart from the above role and responsibility testing team manages test platform, test tools, test cases and also provides tutorial to the users.

Characteristics of test automation framework
• If test automation framework is independent of Operating System, Programming language, tools and browsers, it would be considered as standard framework.
• The framework must be easily understandable without having programming skills.
• The framework must be easily maintainable and extendable.
• The Framework must execute test cases automatically, which includes test scenarios, test data, report generation, error handling and reporting final result.
• The framework must be able to execute tests without human intervention.
• Every test executed, must assign status of either Pass or Fail, failed test cases must include the failure data, the point of failure and other short description for failure.
• The framework should have meaningful logging and reporting structure.

Types of automation framework [6]
1) Linear Scripting Framework -This framework is also known as 'Record and Playback' framework as it generates the test scripts based on simulated record and playback method. It is not useful to run with multiple data set i.e. lack of reusability. This framework is suitable for small size applications. It works for individual test cases because it records the test script for particular scenarios.
2) Modular Testing Framework -In this framework, the complete application is divided into number of smaller modules and test script is generated individually. Finally the results of individual modules are compiled together by the master script to achieve the required target.
3) Keyword Driven Testing Framework -This framework is based on the keywords or actions for executing functions or methods. The test script is totally keyword/action dependent stored in tabular format in excel sheet, so it is also known as

5) Hybrid Driven Testing
Framework -This framework is combination of two or more framework to leverage the strengths and benefits of each of the framework individually and to produce a better test environment overall.

6) Behavior Driven Development Testing Framework -The
Purpose of using this framework is to share the process and tools and to communicate amongst stakeholders, with the aim to deliver quality product. BDD test cases are being written in simple language that can be easily understandable, which contains the details of how application behaves? BDD provides the platform for both technical and non-technical team to work together. A test case written in BDD follows the format 'Given-When-Then'. Given a certain scenarios, When an action takes place, Then this should be the outcome.

Related Works
Selenium WebDriver is powerful tool for test automation that directly interacts with web browsers, it follows the page object pattern to access the web page to reuse the test script to write the test cases. The page object pattern is used to reduce the coupling between web pages and test cases, which improves test suite maintainability [7]. UI locators are used to locate the web elements (Text box, button, checkbox, etc.) on the web page. Different identifiers are used to locate the web elements like (ID, link, XPath, link Test etc.), ID Locators have better result than XPath Locator, Combination of ID locator or XPath Locator with Link Text have improved the result but still not conclusive [7].
WebDriver tool can be used to write test cases easily and efficiently, it can helpful to develop and analyze the test cases using screenshot property, at the same time it is easily maintainable, reusable and extendable with the new requirement. Since the selenium WebDriver does not support any built in mechanism to generate test report, for that purpose TestNG is used as a testing framework. TestNG is compatible to perform unit, functional and integration testing. The author has generated a customized test report to analyze the failure test cases using screenshot [8].
WebDriver is cross platform testing framework that provides dynamic characteristics of web page, it means page reload is not required, if page elements gets changed. Selenium WebDriver execute faster as it directly interacts, run and control browsers using own control engine [9]. Selenium WebDriver is not equipped with reporting mechanism, TestNG generate report on behalf of selenium result. Such report contains execution time, method and detailed status of test script of running test suite "Passed or Failed" [10]. Comparisons of automation tools [11] 4. Framework Design for Automation of Cloud-Application Testing Using Selenium

Proposed Work
The framework (FACTS), we have designed is based on the consideration of test environment (includes language binding, IDE, automation tools and testing framework), types of Application under Test (AUT) with the compatibility of web browsers. The architecture of the proposed framework ( Figure 2) includes following tools, functions, method and libraries etc. When we execute a selenium command (driver.getTitle();) an HTTP request will be created to communicate with particular browser driver, the browser driver then uses HTTP service to get HTTP requests to establish the selenium command. All the built-in methods of corresponding drivers can only be accessible after configuring the driver.
• @Test Methods: Why do we use @ annotation before writing test method. Annotations in selenium are used to bind the associated method going to execute. Test annotation @ always used before every method, if this test annotation '@' is not prefixed before the test method, that method would not be considered as part of test code to be executed. @Test indicates that the annotated method is a part of test case serves as the main method [12]. Some other annotations used with Negative data is used to test the ability of program to handle unusual, exceptional or unexpected input with its corresponding outcome [13].

Conclusion
In-depth planning, process and efforts are expected while design and develop a test automation framework. Selecting right automation tools plays a pivotal role to accomplish the automation task. Selenium WebDriver has been used to test web application end-to-end. WebDriver is flexible to integrate with variant development environment, tools (test, reporting and automation), OS, browsers and methods to design a test automation framework. Various types of test automation frameworks are available to design and develop test automation. The proposed framework (FACTS) is designed to work with Data Driven Framework as well as Hybrid Driven Testing Framework. A Standard automation tool is expected to execute its integration with other automation tools and reporting mechanism. Positive and negative test must be performed to verify the functionalities of application and to check the ability of program to handle unusual exceptions respectively.