Generate ReportNG report using Maven

Earlier I had covered in my blog about generating TestNg-xslt report for your TestNG test execution using Maven. Following is the link to it:

http://blog.varunin.com/2012/05/generate-testng-xslt-report-using-maven.html

In this blog I will write about how to generate a ReportNG report for your TestNG test execution using maven. Following is a pom.xml file that can be used to execute your tests in a maven project and also generate a ReportNG report after execution.

Run the following command to run your test cases and generate the ReportNG report.

mvn clean test

Or just

mvn test

Once run by default the execution report can be found under the folder named "html" under the surefire report directory inside "target" folder of your project folder.

Read More

Generate TestNG-xslt report using Maven

Earlier I had covered in my blog on how to generate a TestNG-xslt report for your TestNG test-case execution using Ant. Following is the link to it:
http://blog.varunin.com/2010/05/generating-selenium-reports-using.html

As many of the recent projects are now using maven as their build tool ,test-cases are also required to integrate itself to the existing build. In this blog I will mention about how to generate a TestNG-xslt report for your TestNG test-case execution using Maven.
Following is the "pom.xml" file for executing TestNG test-cases and to generate a "TestNG-xslt" report for the same:

Run the following command in terminal/command prompt to run your test cases and generate the TestNG-xslt report. The report will be generated under folder named "testng-xslt-report" inside the "target" folder under your build directory.

mvn clean test site org.reportyng:reporty-ng:1.2:reportyng

Note: In case the "test" task of mvn fails the report task of xslt will not be executed. You can apply any of the strategies mentioned in the following url to achieve the same as part of a single execution:

http://stackoverflow.com/questions/4174696/making-maven-run-all-tests-even-when-some-fail

Else 2 different commands have be run for test execution and report generation separately.

To know more about the TestNG-xslt configuration parameters please refer to my earlier blog post:
http://blog.varunin.com/2010/05/generating-selenium-reports-using.html

Read More

Selenium Grid with WebDriver

I had earlier covered in my blog on how to execute your RC cases in selenium grid. But with the commencement of Selenium-2/Webdriver, grid setup has been changed. In the following blog I will cover how to set-up your grid and the changes that will be required for easy execution of cases.


Following things are required:
1. Selenium 2.xx version server jar and Java library. The latest one can be downloaded from the link: Selenium-2
2. Java 1.6 and above
3. TestNg jar . You can download it from the link: TestNG
4. Eclipse with TestNG plugin installed(optional)
5. ChromeDriver. Can be downloaded from: ChromeDriver


The Test Code
Following is an example of test-class that have a test case to search google for "testing" and verifying after clicking it on a link.

In the above class I am using the TestNG "Parameter" property to provide different data set to the "BeforeClass" method "beforeClass". The beforeClass method accepts two properties "browser" and a "port".

These values are used for initialization of driver and in-turn for initialization of selenium object. In the above code I am using the "WebDriverBackedSelenium" class for creation of the selenium object, so that its easy for guys who had worked on Selenium-1 to understand the code. If you want the code to be purely WebDriver, you can directly use the "driver" object for defining your test-cases.

The main part in this test case is how the driver object is being created:

DesiredCapabilities capability= new DesiredCapabilities();
capability.setBrowserName(browser);
WebDriver driver= new RemoteWebDriver(new URL("http://localhost:".concat(port).concat("/wd/hub")), capability);

The above code creates an object of DesiredCapability and then set the browser value to it. Now using this "capability" object I am creating the webdriver object using the "RemoteWebDriver" class. This tells the selenium on which browser the test-case needs to run and where the server is located. In this example I am assuming the server to be running locally. In case it is on different system the "localhost" needs to be re-placed with the ip of the said system.

TestNG configuration

For parallel execution you need to use the TestNG configuration. Following is an "testng.xml" file for the above said test class. The said configuration executes the test-cases across different browser.

<suite name="Selenium TestNG Suite" parallel="tests"
    thread-count="5">

    <test name="Selenium TestNG - 1">
        <parameter name="browser" value="firefox" />
        <parameter name="port" value="4444" />
        <classes>
            <class name="com.test.testng.Google" />
        </classes>
    </test>
    <test name="Selenium TestNG - 2">
        <parameter name="browser" value="chrome" />
        <parameter name="port" value="4444" />
        <classes>
            <class name="com.test.testng.Google" />
        </classes>
    </test>

</suite>

In the above configuration, I am configuring TestNG to run "tests" in parallel. Also there are two different tests inside a suite. For each test a different "browser" parameter value has been configured.

Selenium-Grid server

Now start your grid using the following commands. Run each command in a seperate terminal or command prompt by going to the directory containing your selenium-server-standalone-2.x.x.jar. In the following example I am using the 2.7.0 version of selenium.

For Hub:

java -jar selenium-server-standalone-2.7.0.jar -role hub

For a firefox based node:

java -jar selenium-server-standalone-2.7.0.jar -role webdriver -hub http://localhost:4444/grid/register -port 5556 -browser browserName=firefox

For google-chrome based node:

java -Dwebdriver.chrome.driver=/path/to/chromedriver -jar selenium-server-standalone-2.7.0.jar -role webdriver -hub http://localhost:4444/grid/register -port 5555 -browser browserName=chrome

Before running  the above command  you need to provide the chrome-driver path to the property "-Dwebdriver.chrome.driver".

Now run your testng.xml from eclipse by selecting it -> Right click -> Run as -> TestNG suite(this will work only if you have TestNG plugin installed in your eclipse.)

Or you can choose other ways to execute "testng.xml" like from command prompt, using ant or maven.

Once you run the above testng.xml. TestNG will execute the cases from the Google class on grid across different browsers firefox and google-chrome as configured.

You can do configurations for environment like OS (Linux,Windows), browser, browser-version and all and then you can run you cases on particular type of environment by configuring them accordingly. More details can be found at the following URL:
http://code.google.com/p/selenium/wiki/Grid2

Read More

Data-Driven testing using Junit and TestNG

Most of the guys who are into automation may be knowing the term Data-Driven testing. But the word will still be new for some fresh faces in the field of automation. In this blog I will explain what is Data-Driven testing and will give an example of Data-driven testing using Junit and TestNG frameworks.

Data-Driven testing as the name suggests is a test driven by the Data. For ex. You have a user detail form where you need to enter details of multiple users and save them. Here if you have 5 different user data available and you have to write automation cases for these, you may end up writing 5 different automation test cases(one for each user data).

If you apply a Data-Driven approach you will end up with only one test-case, that is filling the form with user data and then submitting it. The test case will get executed based on the data provided to it. In this case it will be 5 and hence the test case will get executed 5 times with different data-set.

The advantage of using a Data-driven approach is that you reduce your effort in writing/maintaing test-cases for your different type of data. In case of additions or deletion of new/old entries , you just have to change the data and not your actual test-case.

Following I will mention a Data-Driven approach for searching on google with different data using Junit and TestNg frameworks:


Using Junit:

Using TestNg:

The main difference in the above two functions is that you provide a Paramaterized option to the class in Junit and supply data to the constructor of the said class. Where as in TestNG you do the same at the test-method level.

Its simple to do data-driven testing in TestNG framework as you can provide a different data providing function for each test-method, but the same is not possible in Junit.

Read More

Factory Class in TestNg

There are many functions provided by TestNG and you can use them in different ways I will mention in this blog about the @Factory implementation.

@ Factory

A Factory will allow to create test dynamically. And whenever we define a Factory it should always return an object array “Object[]”. All the test methods under the Class object which is created and returned using the Factory method is automatically executed by the Factory method. For ex.

public class FactoryClass {

@Factory
public Object[] createTest(){
   Object[] res = new Object[3];
   for(int i=1;i<=3;i++){
   res[i-1]=new main.java.FactoryImplementation(i);
  }
   return res;
}
}


public class FactoryImplementation {
int c_instance;
public FactoryImplementation(int instance){
   this.c_instance=instance;
  }
@Test
public void printMethod(){
   System.out.println("Instance Num is "+ c_instance);
  }
}

Whenever you execute the above two classes together as a TestNg test, the Output will be as follows:

Instance Num is 1

Instance Num is 2

Instance Num is 3

You only have to execute the Class named “FactoryClass”. Test-NG will find the Factory method inside it and will execute it. For each loop inside the Factory method an object of FactoryImplementation Class is created. On creation of the FactoryImplementation object and Instance value is set using the constructor. While creation of FactoryImplementation object the Test-NG automatically searches for all the “Test” methods inside the FactoryImplementation class and add them to its test execution list.

Another thing for you to notice is that each object created have a unique “c_instance” value set while object creation and the same value is used in the “printMethod”.

The Factory method can be used in different ways. One of the case may be when you want to execute a certain number of test inside a class for different values that can increase or decrease based on a condition.

This can be achieved by creation of the object of test class based on the number of values.

In the above class I am creating the object of the FactoryImplementation class inside a "for" loop. In a similar way you can iterate the for loop for number of different values for which the test needs to be run. And then initializing the test class object with such values and returning the Object array after storing them in the array. Something like the following:

@Factory

public Object[] createInstance(){
ArrayList ar = new ArrayList< Object>();
Object[] res =null;
for(int i=1;i<=5;i++){
ar.add(new main.java.FactoryImplementation(i+10));
}
res= new Object[ar.size()];
res=ar.toArray();
return res;
}

In the above method I am initializing the ArrayList which is dynamic in nature so depending on the no. of iterations I can increase the Array size and then store the same to the Object array and return it for execution.

Read More

DataProvider in TestNG

There are many functions provided by TestNG and you can use them in different ways one of them I will mention in this blog.

@DataProvider

A DataProvider provide data to the function which depends on it. And whenever we define a DataProvider it should always return a double object array “Object[][]”. The function which calls dataprovider will be executed based on no. of array returned from the DataProvider. For ex.

@Test(dataProvider="data")
public void printMethod(String s){
  System.out.println(s);
 }

@DataProvider(name="data")
public Object[][] dataProviderTest(){
return new Object[][]{{"Test Data 1"},{"Test Data 2"},{"Test Data 3"}};
}

The Output will be:
Test Data 1
Test Data 2
Test Data 3

As you can see that the Test method “printMethod ” gets called 3 times depending upon the data that was provided by the DataProvider.
The DataProvider can be used for getting data from some file or database according to test requirements.

Following I will mention two ways to use DataProvider:
For ex.
You need to get data from a file and print each line to the console. For doing that you had written some File Processing API that will return a List of the data read from file.
You can iterate on the List at the DataProvider level as well as at the test method level. Both I am mentioning below.

1.
@DataProvider(name = "data")
 public Object[][] init1() {
  List list = fileObject.getData();
   Object[][] result=new Object[list.size()][];
   int i=0;
   for(String s:list){
    result[i]=new Object[]{new String(s)};
    i++;
   }
 return result;
 }
@Test(dataProvider="data")
public void runTest1(String s){
  System.out.println("Data "+s);
 }

In this Implementation we are iterating over the List at the DataProvider level and storing it to another Object[][] result and returning the result.
This implementation will call the test method depending upon the List size.

2.

@DataProvider(name = "data")
public Object[][] init() {
  List list = fileObject.getData();
   
 return new Object[][]{{list}};
 }


@Test(dataProvider="data")
public void runTest(List list){
  for(String s:list){
   System.out.println(“Data” + s);
  }
 }

In this Implementation we are returning the List itself to the test method and method will iterate over the List data. Both implementation can be used based upon test requirements.

Output of both the implementation shown above remains the same. The only thing that changes is the way you return the data and Iterate.

Read More

Generating a Junit Report for TestNG execution

Now a days everyone is using TestNG in place of Junit and as part of Selenium execution.

TestNg provides a lot of added features. One of the good thing it provides is test execution results in the form of XML. We can use this XML file for other type of report format generation. One of them is Junit reporting.

In this blog I will mention how to generate a Junit report for a TestNG execution.
Below ant target will generate the Junit report for the TestNG execution results.

In the above ant target you need to specify the path to the TestNG results xml file in the "dir" attribute under the "fileset" tag.

You need to mention the path where you want to generate the Junit report in the "todir" attribute under the "report" tag.

Also you can mention whether you want the Junit report with frames or without frames in the "format" attribute under the "report" tag.

Read More

Generating selenium reports using TestNG-xslt through Ant

TestNG-xslt generates user friendly reports using the TestNG results output (testng-results.xml). Its uses the pure XSL for report generation and Saxon as an XSL2.0 implementation.

Most of the material is taken from the original site.
 http://code.google.com/p/testng-xslt/

Alternative link to download. Please open the said link and download it by going to File -> Download option.

I will tell in this blog how to implement this report for your project. This implementation will tell you how to generate the testng-xslt report using ant. If your current project does not use ant build then you can use ant only for the report generation purpose.

If you dont know ant please check the Apache ant website http://ant.apache.org/.

For generating testng-xslt report for your project do the following:
1. Download the testng-xslt
2. Unzip and copy the testng-results.xsl from the testng-xslt folder(testng-xslt-1.1\src\main\resources) to your own project folder.
3. Now copy the saxon library from (testng-xslt-1.1\lib\saxon-8.7.jar)to your project lib folder.
4. Modify your build.xml of ant and add the following target to it.

<project name="test" basedir=".">
    <property name="LIB" value="${basedir}/libs" />
    <property name="BIN" value="${basedir}/bin" />
    <path id="master-classpath">
        <pathelement location="${BIN}" />
        <fileset dir="${LIB}">
            <include name="**/*.jar" />
        </fileset>
    </path>
    
    <target name="testng-xslt-report">
        <delete dir="${basedir}/testng-xslt">
        </delete>
        <mkdir dir="${basedir}/testng-xslt">
        </mkdir>
        <xslt in="${basedir}/test-output/testng-results.xml" style="${basedir}/testng-results.xsl" out="${basedir}/testng-xslt/index.html">
            <param expression="${basedir}/testng-xslt/" name="testNgXslt.outputDir" />

            <param expression="true" name="testNgXslt.sortTestCaseLinks" />

            <param expression="FAIL,SKIP,PASS,CONF,BY_CLASS" name="testNgXslt.testDetailsFilter" />

            <param expression="true" name="testNgXslt.showRuntimeTotals" />

            <classpath refid="master-classpath">
            </classpath>
        </xslt>
    </target>
</project>

The XSL transformation can be configured using the parameters described below.

• testNgXslt.outputDir - Sets the target output directory for the HTML content. This is mandatory and must be an absolute path. If you are using the Maven plugin this is set automatically so you don't have to provide it.
• testNgXslt.cssFile - Specifies and alternative style sheet file overriding the default settings. This parameter is not required.
• testNgXslt.showRuntimeTotals - Boolean flag indicating if the report should display the aggregated information about the methods durations. The information is displayed for each test case and aggregated for the whole suite. Non-mandatory parameter, defaults to false.
• testNgXslt.reportTitle - Use this setting to specify a title for your HTML reports. This is not a mandatory parameter and defaults to "TestNG Results".
• testNgXslt.sortTestCaseLinks - Indicates whether the test case links (buttons) in the left frame should be sorted alphabetically. By default they are rendered in the order they are generated by TestNG so you should set this to true to change this behavior.
• testNgXslt.chartScaleFactor - A scale factor for the SVG pie chart in case you want it larger or smaller. Defaults to 1.
• testNgXslt.testDetailsFilter - Specified the default settings for the checkbox filters at the top of the test details page. Can be any combination (comma-separated) of: FAIL,PASS,SKIP,CONF,BY_CLASS

You need to provide the testng-xslt stylesheet the TestNG results xml(testng-results.xml) , the path to the style sheet testng-results.xsl and the output index.html path.

Also dont forget to add the saxon library to your target classpath else you will get an error. In my case it is the master-classpath.

Noe run the ant target for report generation (in my case "testng-xslt-report") and check the ouput folder configured by you for testng-xslt report.

Read More

Using TestNG for parallel execution on Selenium Grid

Following I will mention how to integrate TestNG with Selenium-Grid code and use it for parallel execution of test cases.
For this you create a Base class where you define the following two functions.


protected void startSession(String seleniumHost, int seleniumPort, String browser, String webSite)
throws Exception {

startSeleniumSession(seleniumHost, seleniumPort, browser, webSite);
session().setTimeout(TIMEOUT);

}

protected void closeSession() throws Exception {

closeSeleniumSession();

}

The above two functions are for the starting and closing the session in Selenium Grid. The references of it can be found in my earlier posts.

Now import the testng annotation class “org.testng.annotations.*” onto the Base Class.

Add @BeforeClass before the “startSession” method and @AfterClass before the “closeSession” method.

Now write your test cases in a way that all dependent test cases are in one class.

So whenever you execute the testNG it will call the “startSession” and “closeSession” before and after each test class execution. The “startSession” will create a Selenium session for test execution and return it to the test class. Where as the “closeSession” will close the session.

For parallel execution define inside the testng.xml file the attributes “suite” tag as “parallel="classes" thread-count="2" ”.
This will tell TestNG that parallel execution needs to be done for classes and max. Thread count for it is “2”.

Define 2 different test classes both extending the Base Class which defines the “startSession” and “closeSession” methods. Start a Selenium Grid Hub and register two Selenium remote control to the Hub. Now try running the package containing these classes using the TestNG.

You will see 2 different tests running in parallel in Selenium Grid each for Selenium Remote Control registered to the Hub.
The “startSeleniumSession” inside the “startSession” method create a unique selenium session for each of the test class and use that session for test class execution. Creating a session,registering the session with a Remote control and maintaining the session is done by Selenium Grid where as using TestNG framework we define how many sessions we need to create and how the test execution should be. So using the features of both TestNG and Selenium Grid we achieve Parallel execution of our automation testcases.

Currently here I had suggested to use @Before/AfterClass annotations in Base Class. You can use any of the other available @Before/After annotations available in TestNG depending upon your tests.

Read More

What is TestNG? And whats the difference between Junit and TestNG

TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as:

• Annotations.

• Flexible test configuration.

• Support for data-driven testing (with @DataProvider).

• Support for parameters.

• Allows distribution of tests on slave machines.

• Powerful execution model (no more TestSuite).

• Supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc...).

• Dependent methods for application server testing.

If you want to learn TestNG you can download it and see its documentation at “http://testng.org/doc/index.html”.

TestNG is extensively used in Selenium automation and Code testing instead of Junit due to its advantages over it.

Using TestNG Selenium automation becomes simplified and also the Parallel execution using Selenium Grid becomes easy.

TestNG have a lot of features in it and hence I wont be covering every thing in my blog. But will only talk about advantages of TestNG over Junit. I will be discussing about different ways of using TestNg functions in my coming blogs.

Advantages of TestNG over Junit

1. In Junit we have to declare @BeforeClass and @AfterClass which is a constraint where as in TestNG there is no constraint like this.

2. Additional Levels of setUp/tearDown level are available in TestNG like @Before/AfterSuite,@Before/AfterTest and @Before/AfterGroup

3. No Need to extend any class in TestNG.

4. There is no method name constraint in TestNG as in Junit. You can give any name to the test methods in TestNG

5. In TestNG we can tell the test that one method is dependent on another method where as in Junit this is not possible. In Junit each test is independent of another test.

6. Grouping of testcases is available in TestNG where as the same is not available in Junit.

7. Execution can be done based on Groups. For ex. If you have defined many cases and segregated them by defining 2 groups as Sanity and Regression. Then if you only want to execute the “Sanity” cases then just tell TestNG to execute the “Sanity” and TestNG will automatically execute the cases belonging to the “Sanity” group.

8. Also using TestNG your selenium testcase execution can be done in parallel.

Read More