Saturday, March 20, 2010

Ant script for generating a Junit report for Selenium Test execution

We know the advantages of using Selenium RC. But Selenium RC by default does not provide us with any reports for the test execution done through it. We need to explicitly implement a reporting framework for test reporting in RC. For implementing a Reporting framework you can write your own custom code or can use an already existing reporting framework available.

There are mainly two reporting framework available for Selenium RC they are Junit and Test-NG. There are two more frameworks Report-NG and testng-xslt but they are implemented over the Test-NG framework.

In this blog I will mention about generating a Junit Report for your Selenium RC test execution using Ant script. I had written a “build.xml” for Ant for executing Selenium cases and generating a Junit report for the said selenium execution. I will be discussing about the same in this blog. If someone doesn’t have an idea about Ant can follow my steps and can generate a Junit report for Selenium Execution.


Explaining the “build.xml”

  1. In Build.xml I am mainly defining 5 properties

o RELEASE_ROOT this has been set to the Java Project folder.

o SRC which defines the source folder of the project

o LIB which defines the lib folder of the project

o BIN which defines the binary folder where the classes needs to be pasted after compiling

o REPORT folder which defines the report folder where the test execution reports will be generated.

  1. There are 4 targets or tasks of ant inside the build.xml

o “init’ which deletes the binary folder and creates it again.

o “compile” which depends upon the “init” task and which compiles the java files under the source folder and paste the classes onto the binary folder.

o “run-single-test” which depends on “compile” and runs only a single Selenium Junit test mentioned in it. This task also generates a Junit html report under the report/html folder of Java project after Selenium Test execution is done.

o “run-batch-test” which depends on “compile” and runs multiple Selenium Junit test mentioned in it. This task also generates a Junit html report under the report/html folder of Java project after Selenium Test execution is done.

  1. In each of the compile, run-single-test and run-batch-test I am setting the classpath of the binary folder, the library folder and the source folder in case of “run-batch-test”.
  2. In targets “run-single-test” and “run-batch-test” I am using the “junit” task of test execution. This task provides two types of execution

o Single class execution which I had used in “run-single-test” target. There is a subtask “test” of “junit” that can used for single test class execution. It is written as follows:


<test name="test.TestClass" haltonfailure="no" todir="${REPORT}/xml" outfile="TEST-result">

<formatter type="xml" />

</test>


Here “name” is the test class name. “todir” is the directory where initial xml report needs to be generated. “outfile” is the xml file name for initial result xml file. “formatter” is the initial file format type.


o Batch test class execution which I had used in “run-batch-test” target. There is a subtask “batchtest” of “junit” that can used for single test class execution. It is written as follows:


<formatter type="xml" usefile="true" />

<batchtest fork="yes" todir="${REPORT}/xml">

<fileset dir="${SRC}">

<include name="**/Test*.java" />

</fileset>

</batchtest>


Here I am telling the “batchtest” what all test classes to include using the “fileset” and “include” task. Along with this if you want you can use the “exclude” task to exclude particular type of classes. In the following example the Junit will include all the test class starting from “Test” and exclude all the test class having “Base” word in their name.


<formatter type="xml" usefile="true" />

<batchtest fork="yes" todir="${REPORT}/xml">

<fileset dir="${SRC}">

<include name="**/Test*.java" />

<exclude name="**/*Base*.java" />

</fileset>

</batchtest>


5. For generating the test report I am using the “junitreport” task of ant. It takes all the xml file from the “report/xml” and generates an html report at “report/html”.


Setting up a Java Project for test execution using Ant.


In this I am just telling how to setup a basic java project for using the build.xml file attached with this blog and running the selenium test using Ant.


Preconditions:

1. Install Ant from apache.

2. Install Java JDK (not JRE) and add its bin folder to your “PATH” environment variable of your system. For checking JDK is mapped check whether "javac" is recognized in your system by typing "javac" in your command prompt.

3. Download the Selenium RC.

4. Test cases are already converted to Selenium Junit cases.


Steps:

  1. Create a Java project in your Java IDE. I am using Eclipse so I will refer to it.
  2. By default there will be the “src” folder inside your Java Project. Right click on the “src” folder and create a new package under it say”test”.
  3. Now copy your Selenium Junit classes to this new “test” package.
  4. Create a new folder called “lib” under your Java Project.
  5. Copy selenium-java-client-driver.jar, junit-4.5.jar(if you have another version of Junit 4 you can use that) and ant-junit.jar(can be found under the “lib” folder of your ant installation) to your “lib” folder of Java Project.
  6. Now create a folder called “build” under your Java Project and copy the “build.xml” file attached with this blog to the build folder you created.
  7. Now open the build.xml file for edit.
  8. Search for “run-single-test” in build.xml. Go to the “<test” under the “run-single-test” target .Replace the values for “name=” variable with your Selenium Junit class name. For ex if your class name is TestSelenium.java and it is under package “test” under “src” then replace the value with “test.TestSelenium” then save the build.xml file.
  9. Now start the Selenium RC server.
  10. Now go to the build folder of the Java Project using command prompt and run the command “ant run-single-test”
  11. It should execute your Selenium cases on selenium server and generate an html report onto the “report/html” folder of your Java project.


Similarly you can modify the “run-batch-test” target of build.xml and execute multiple Selenium Tests together.

Download build.xml from Here

71 comments :

Naga said...

Hi Varan thanks for the psot...i successfully ran one script and it generated report...but i did not understand hwo to run multiple tests...in run-batch-test where i need to modify..please tell me...thanks in advance

Parag said...

Hi Varun,
Thanks for this help,
Can you please tell me how can i save these test result in differant directory and maintain the version up system for every execution?

-Parag

Mariyam said...

Hi Varun,
We have succesfullly executed the single test script. However, we are facing problems while try to run batch test .We ahve succesfully build all the files in the Test folder but ther are not being executed automatically as in one script test. Pls help!

Thnx!
Mariyam

shikhi said...

Can anyone answer to Mariyam's question? we are still stuck in batch running ...

Thanks,
Shikhi

Varun said...

Sorry by mistake I had sent a mail to him.
Today after your comment i saw the mail-id.
Following were my comments:
You need to check the classpath. If u are getting a class not found exception.
If this not the case ,please send the ex eption u r getting.

Please send a mail to mr.varun.menon@gmail.com in case of more details. It will be easy for me.

Praveen said...

i am getting this error " Ant is not recognized as an internal or external command, operable program or batch file" in dos command

Varun Menon said...

@Praveen-You haven' installed ant properly into your system.
Just add your ant bin path to the "Path" variable of your systems environment variable. And open a new command prompt and then check.

Praveen said...

Hi Varun ,
Can we Customize the summary report??

swapnil_kude said...
This comment has been removed by the author.
swapnil_kude said...

Hi Varun,
I got the same error that was posted by 'Praveen'. So I done as you directed to him, but getting the compilation error as:

javac: invalid source release: 1.6

Then I edited the 'build' file as a javac source="1.5" srcdir="${SRC}" fork="true".....

Then i got lots of error with selenium around 93 error were there.

Please guide me.

Varun Menon said...

If its selenium or java related. Then you need to check your code. Without seeing the error I cant comment on anything :).

swapnil_kude said...

Varun,
Below is the error when I run the command “ant run-single-test”
``````````````````````````````````````````
compile:
[javac] Compiling 2 source files to C:\Documents and Settings\swapnil.NBITS\workspace\HHS\bin1
[javac] javac: invalid source release: 1.6
[javac] Usage: javac
[javac] where possible options include:
[javac] -g Generate all debugging info
[javac] -g:none Generate no debugging info
[javac] -g:{lines,vars,source} Generate only some debugging info
[javac] -nowarn Generate no warnings
[javac] -verbose Output messages about what the compiler is doing
[javac] -deprecation Output source locations where deprecated APIs are used
[javac] -classpath Specify where to find user class files
[javac] -cp Specify where to find user class files
[javac] -sourcepath Specify where to find input source files
[javac] -bootclasspath Override location of bootstrap class files
[javac] -extdirs Override location of installed extensions
[javac] -endorseddirs Override location of endorsed standards path
[javac] -d Specify where to place generated class files
[javac] -encoding Specify character encoding used by source files
[javac] -source Provide source compatibility with specified release
[javac] -target Generate class files for specific VM version
[javac] -version Version information
[javac] -help Print a synopsis of standard options
[javac] -X Print a synopsis of nonstandard options
[javac] -J Pass directly to the runtime system

BUILD FAILED
C:\Documents and Settings\swapnil.NBITS\workspace\HHS\build\build.xml:20: Compile failed; see the compiler error output for details.

Total time: 1 second

Saurabh said...

Hi varun,

getting error that symbol: variable selenium
Location: class LoginTestCase

Cannot find symbol selenium.open()

Varun Menon said...

@Saurabh: Cant reply to you based on above comments. Can you tell me exactly what u are trying to do and what is your error. You can send a mail to me at mr.varun.menon@gmail.com with your error stack and java file(if you can).

Saurabh said...

Thanks for quick response Varun but i resolved the problem.

Saurabh said...

Download given build.xml and modify accordingly but when run the build from given path then it will not launch the browser to run the test case.

should i do any modification in the build.xml ?

Varun Menon said...

@saurabh: Yes you need to modified the build.xml file based on your project and the java file you need to execute.

Saurabh said...

Hello Varun do you have any sample build.xml to figure out what kind of changes should i do ?

Varun Menon said...

@saurabh: I dnt have such build.xml file with me. But as I had mentioned in the steps you need to change the Java file name in target. Also you need to check the SRC, LIB variables set on top of the build.xml file.

Varun Menon said...

@Saurabh: I had modified the attached build.xml file. Please check it out. I had added my comments on each line. Hope it helps.

rk said...

Hi, I am not able to download the build.xml file anymore.

Can varun/someone in the group upload the file once again.. please

RK.

seleniumtest said...

Hi Varan, thanks for the post. I successfully ran one script and it generated report. I want to customize the report, add two more columns to the report. Please help me on this.

GK

Varun Menon said...

@RK: Now you will be able to download the build.xml file directly.
@seleniumtest/GK: Please write in detail about your issue/requirement.

seleniumtest said...

Hi Varun,

I have downloaded the build.xml file and successfully generate report using ANT + Selenium. I would like to add two more columns(1. Expected value, 2. Actual value) to the report. To customize ANT report, which xsl file to update or where this xsl file exist in ANT folder.

Thanks,
GK

Santhosh said...

Hi Varun,

I got the following error please help me to find the solution
java.lang.ClassNotFoundException: com.testscripts.loginpage.java at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Unknown Source)
-

Thanks
sandy

Saurabh said...

hi varun if i want to start server from eclipse itself and then generate html report how can i achieve this ?

Saurabh said...

Followed above steps but it is not launch firefox browser when i execute command ant run-single-test after running selenium server ?

Saurabh said...

Hello Varun,

Successfully run the test using above steps. Thanks.

But if i created one property file and parsing those property file parameters in my .java program and when i run same command it is not working.

Any other value need to add in build.xml file?

Ryan said...

Varun,

Thanks you very much for this helpful tutorial. I am wondering what is the best way to specify the order the tests run in, while running the batch, I noticed it is alphabetical, is there another way other than alphabetical?

santhosh said...

Hi Varun,

Can you please tell me that how can i execute a single test case instead of entire test class.

In Build.xml file, we need to mention that the class path in tag.So what should i need to change to execute any single testcase in a particular TestClass.

Can you please let me know that if you want more details.

Thanks
Santhosh.

Varun Menon said...

@santosh: I had seen received a mail today by you asking, whether we can execute a particular method inside the Junit test class using ant.
The answer is: No. But you can use eclipse to do it.
@saurabh: Your query is pretty old now. Hopefully it may had got solved. Sorry for not responding as I was very busy with my project.

d75b6e48-beca-11e0-b0c0-000bcdcb471e said...

This is simply superb. Great post!

Katha Download said...

Hi Varun,

I have followed all your steps mentioned above, however when i run i got the error message as below
Could you please let me know what is wrong?

C:\Selenium\DDF\test\build>ant run-single-test
Buildfile: build.xml

init:
[delete] Deleting directory C:\Selenium\DDF\test\bin1
[mkdir] Created dir: C:\Selenium\DDF\test\bin1

compile:
[javac] Compiling 1 source file to C:\Selenium\DDF\test\bin1
[javac] C:\Selenium\DDF\test\src\test\MotorCycle.java:9: cannot find symbol
[javac] symbol: class SeleneseTestCase
[javac] public class MotorCycle extends SeleneseTestCase {
[javac] ^
[javac] C:\Selenium\DDF\test\src\test\MotorCycle.java:16: cannot find symbol

[javac] symbol : variable selenium
[javac] location: class test.MotorCycle
[javac] selenium = new DefaultSelenium("localhost", 4444, "*chro
me", "https://secure.tradevent.com.au/");
[javac] ^
[javac] C:\Selenium\DDF\test\src\test\MotorCycle.java:17: cannot find symbol

[javac] symbol : variable selenium
[javac] location: class test.MotorCycle
[javac] selenium.start();
[javac] ^
[javac] C:\Selenium\DDF\test\src\test\MotorCycle.java:29: cannot find symbol

[javac] symbol : variable selenium
[javac] location: class test.MotorCycle
[javac] selenium.stop();
[javac] ^
[javac] 4 errors

BUILD FAILED
C:\Selenium\DDF\test\build\build.xml:20: Compile failed; see the compiler error
output for details.

Total time: 1 second

Thanks,
Virendra

Katha Download said...

Please note: Test Case is working fine if i manually run through Eclipse..

Varun Menon said...

@katha/virendra: Is your java selenium jar placed under the lib folder of the project. Also check whether the lib folder location is correctly set in the build.xml file.

Madhu said...

Copy selenium-java-client-driver.jar, junit-4.5.jar(if you have another version of Junit 4 you can use that) and ant-junit.jar(can be found under the “lib” folder of your ant installation) to your “lib” folder of Java Project.
and add selenium-server-standalone-2.7.0.jar also to the lib folder

sruthi said...

Hi All,
can any one provide the steps of how to install ANT.

Varun Menon said...

@sruthi - Try using google search. Thank u. If u are unable to install it still, ill help u.

Unknown said...
This comment has been removed by the author.
Unknown said...

@Varun: Hi.
Thanks for this great post.
This is really helpful
I am running selenium (Java) script and in that i am taking the values from excel file.
The problem is that,
It is giving me following error
->The system cannot find the path specified
->java.io.FileNotFoundException: data\poi-test.xls (The system cannot find the path specified).
Actually i have data folder in which i placed my excel files.
So how can i set the path in Build.xml file.
Please help me.

Regards,
Gaurav

Varun Menon said...

@Unknown- You need to include those files while compilation using.

or copy them to the bin folder by using the ant copy task.

shashank said...

Hi Varun,

Very good steps you explained which is helpful for all beginner of selenium............

nithin Sriram said...

Hi varun

I was trying to take the screenshot of failed testcases

My code is as below

@Override
public void onTestFailure(ITestResult result) {
File file = new File("");

Reporter.setCurrentTestResult(result);
System.out.println(file.getAbsolutePath());
Reporter.log(file.getAbsolutePath());

Reporter.log("screenshot saved at "+file.getAbsolutePath()+"\\reports\\"+result.getName()+".jpg");
Reporter.log(" ");
BaseClass.selenium.captureScreenshot(file.getAbsolutePath()+"\\reports\\"+result.getName()+".jpg");
Reporter.setCurrentTestResult(null);

but i want some clarity in the below mentioned line
BaseClass.selenium.captureScreenshot(file.getAbsolutePath()+"\\reports\\"+result.getName()+".jpg");

Its throwing an error at BaseClass

Pls help me.

Regards
Nithin S G

confused said...

Mind blowing article. Thanks a ton!! keep up the good work!!

Elangomani M said...

Hi Varun...
It is fantastic blog... Very good ideas I have seen here... Now I am doing automation testing in my company. So I would like to have tips from you for load testing using selenium... I am using selenium IDE & RC... If possible to do load testing at least for 15 users, that's enough to me... Please reply me asap.. Thanks in advance...

Varun Menon said...
This comment has been removed by the author.
Varun Menon said...

Selenium is a functional testing tool and not a load testing tool. Still I had seen people doing load testing using selenium. I haven't used selenium for load testing but in case you need to use it , use html unit driver as driver object.

Swapy said...

Hi Varun,
Thanks for great blog.
I am using the Selenium Web driver "selenium-server-standalone-2.25.0" I was trying to build by using your build XMl with ant 1.8 & junit 4 framework.
It successfully run through eclipse and shows the message that 'BUILD SUCCESSFUL' but no any report is generated and when I execute the same through command prompt “ant run-single-test” it gives tones of error.

Please help.

Here is the error log.
-------------------------
Buildfile: C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\build\build.xml

init:
[delete] Deleting directory C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\bin1
[mkdir] Created dir: C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\bin1

compile:
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\build\build.xml:20: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 10 source files to C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\bin1
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:5: package org.junit does not exist
[javac] import org.junit.*;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:6: package org.junit does not exist
[javac] import static org.junit.Assert.*;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:8: package org.openqa.selenium does not exist
[javac] import org.openqa.selenium.*;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:9: package org.openqa.selenium.firefox does not exist
[javac] import org.openqa.selenium.firefox.FirefoxDriver;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:15: cannot find symbol
[javac] symbol : class WebDriver
[javac] location: class com.Add_sub_with_login
[javac] private WebDriver driver;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:64: cannot find symbol
[javac] symbol : class By
[javac] location: class com.Add_sub_with_login
[javac] private boolean isElementPresent(By by) {
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Admin_login.java:5: package org.junit does not exist
[javac] import org.junit.*;
.
.
.
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\LoginDataDriven.java:30: cannot find symbol
[javac] symbol : class Test
[javac] location: class com.LoginDataDriven
[javac] @Test
[javac] ^
[javac] 100 errors

BUILD FAILED
C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\build\build.xml:20: Compile failed; see the compiler error output for details.

Varun Menon said...

You should use "selenium-java-.jar" and not "selenium-server".

vikas said...

Hi varun,

Relative path of the file given in .java file is not recognized while running through ant.

relative path given in .java file is as follows:
return "././Config/ConfigParams.xml"; (this xml file has test data information of our project)

while running the script using TestNG, it is running fine (i.e., it is accessing ConfigParams.xml file)

While running with ant after building the project,it is not accessing ConfigParams.xml file.

suppose if i hard code the file path: return "C:/YGIB/Config/ConfigParams.xml"
Then ant is recognizing ConfigParams.xml file and script is running sucessfully

i am running the ant via command prompt.
please let me know what i am doing wrong here

Manoj Pahuja said...

Hi Varun,

This thing just worked for me with few minor changes and I wanted to thank you for this, I really appreciate this article as I was looking for some thing like this for last couple of days and this article saved my day.

Thanks
Manoj

Yuda said...
This comment has been removed by the author.
Yuda said...
This comment has been removed by the author.
Yuda said...
This comment has been removed by the author.
shahrukh said...
This comment has been removed by the author.
Rajdeep said...

Hi Varun,

How you describe the steps is very understandable. I was confused at first about the report generation. But after reading your article I can able to run my code and can create report easily.

Thank you.

Anjali Gupta said...

Hi Varun ,

I need to generate a Junit report using ant.I am not using selenium .I am doing automation in Seetest(a mobile automation tool),whose frame work is in jUnit.
using ant I need to generate report and send to email address.
How to do it without selenium ?

Your help will be really helpful.

Regards
Anjali

Varun Menon said...

Use the following code inside your ant target:

<junitreport todir="${REPORT}">
<!-- This section generated the Junit report in html to "html" folder from the xml file. -->
<fileset dir="${REPORT}/xml">
<include name="TEST*.xml"/>
</fileset>
<report format="frames" todir="${REPORT}/html"/>
</junitreport>

Here ${REPORT} is the report directory that points to your local system folder.
${REPORT}/xml is the directory where all the report xml files generated after executing the Junit tests are placed.
${REPORT}/html is the directory where the junit html report is generated and placed.

Sachin Asthana said...

Hi, I successfully ran one script and it generated report...but it did not work for run multiple tests..Please update to how to run multiple testcases.

Also i don't want to override previous report . please suggest how to generate separate report every time based on datetime stamp name (as report filename format)

MIchael Barlow said...

Hey thank you for this great blog. I completed all of the directions but I continue to get this error:
BUILD FAILED
\workspace\Antproject\Build\build.xml:8: You must specify value, location or refid with the name attribute

Varun Menon said...

Unable to find the folder may be. Try providing the absolute path and check.

Prashanth T.V said...

Hi Varun,
Nice writeup.
I have question when I run ANT and generate Junit reports
I see that the package name is shown as "None" and the class name is shown as Blank. Can you let me know as to what the issue is.

Regards
Prashanth

Varun Menon said...

Is your tests getting executed? If yes does ant considers the correct directory for collecting report xml?

Prashanth T.V said...
This comment has been removed by the author.
Prashanth T.V said...

Hi Varun, Yes, my tests are getting executed and the ant considers the correct directory for the report.xml. The only issue us that the package name is shown as "None" and the class name is shown as Blank.
Please find the Ant target for generating Junit report. Can you please let me know if there is something wrong with this.




















Prashanth T.V said...

Varun, for some reason, I'm not able to paste the Ant target for generating Junit reports here for your reference. Is there any other way, I could share the same.

Regards
Prashanth

abhishek gupta said...

When i ran build.xml it generates html report but second time when i again ran build.xml , test report remains the same. where i can get the updated test report which i run second time.

Varun Menon said...

You may require to delete the report folder and recreate it for every new build

jjjjjjjuki said...

Hi varun,
When i run the script,i found following error :
compile:
[javac] C:\Users\Ht\workspace\com.shine.test\build\build.xml:20: warning: 'i
ncludeantruntime' was not set, defaulting to build.sysclasspath=last; set to fal
se for repeatable builds

Please tell me what is it?
I googled it but unable to find suitable answer.
Please help me.

peterson said...

Your information about Selenium scripts is really interesting. Also I want to know the latest new techniques which are implemented in selenium. Can you please update it in your website?
Selenium training Chennai

Vishnu Varma said...

Hi Varun,

I have successfully done Single Run Test. I am struggling on batch run test where i have two packages with 2 class files each. How to write a batch file? and explain it as steps.

Post a Comment