Saturday, March 20, 2010
There are mainly two reporting framework available for Selenium RC they are Junit and Test-NG. There are two more frameworks Report-NG and testng-xslt but they are implemented over the Test-NG framework.
In this blog I will mention about generating a Junit Report for your Selenium RC test execution using Ant script. I had written a “build.xml” for Ant for executing Selenium cases and generating a Junit report for the said selenium execution. I will be discussing about the same in this blog. If someone doesn’t have an idea about Ant can follow my steps and can generate a Junit report for Selenium Execution.
Explaining the “build.xml”
- In Build.xml I am mainly defining 5 properties
o RELEASE_ROOT this has been set to the Java Project folder.
o SRC which defines the source folder of the project
o LIB which defines the lib folder of the project
o BIN which defines the binary folder where the classes needs to be pasted after compiling
o REPORT folder which defines the report folder where the test execution reports will be generated.
- There are 4 targets or tasks of ant inside the build.xml
o “init’ which deletes the binary folder and creates it again.
o “compile” which depends upon the “init” task and which compiles the java files under the source folder and paste the classes onto the binary folder.
o “run-single-test” which depends on “compile” and runs only a single Selenium Junit test mentioned in it. This task also generates a Junit html report under the report/html folder of Java project after Selenium Test execution is done.
o “run-batch-test” which depends on “compile” and runs multiple Selenium Junit test mentioned in it. This task also generates a Junit html report under the report/html folder of Java project after Selenium Test execution is done.
- In each of the compile, run-single-test and run-batch-test I am setting the classpath of the binary folder, the library folder and the source folder in case of “run-batch-test”.
- In targets “run-single-test” and “run-batch-test” I am using the “junit” task of test execution. This task provides two types of execution
o Single class execution which I had used in “run-single-test” target. There is a subtask “test” of “junit” that can used for single test class execution. It is written as follows:
<test name="test.TestClass" haltonfailure="no" todir="${REPORT}/xml" outfile="TEST-result">
<formatter type="xml" />
</test>
Here “name” is the test class name. “todir” is the directory where initial xml report needs to be generated. “outfile” is the xml file name for initial result xml file. “formatter” is the initial file format type.
o Batch test class execution which I had used in “run-batch-test” target. There is a subtask “batchtest” of “junit” that can used for single test class execution. It is written as follows:
<formatter type="xml" usefile="true" />
<batchtest fork="yes" todir="${REPORT}/xml">
<fileset dir="${SRC}">
<include name="**/Test*.java" />
</fileset>
</batchtest>
Here I am telling the “batchtest” what all test classes to include using the “fileset” and “include” task. Along with this if you want you can use the “exclude” task to exclude particular type of classes. In the following example the Junit will include all the test class starting from “Test” and exclude all the test class having “Base” word in their name.
<formatter type="xml" usefile="true" />
<batchtest fork="yes" todir="${REPORT}/xml">
<fileset dir="${SRC}">
<include name="**/Test*.java" />
<exclude name="**/*Base*.java" />
</fileset>
</batchtest>
5. For generating the test report I am using the “junitreport” task of ant. It takes all the xml file from the “report/xml” and generates an html report at “report/html”.
Setting up a Java Project for test execution using Ant.
In this I am just telling how to setup a basic java project for using the build.xml file attached with this blog and running the selenium test using
Preconditions:
1. Install Ant from apache.
2. Install Java JDK (not JRE) and add its bin folder to your “PATH” environment variable of your system. For checking JDK is mapped check whether "javac" is recognized in your system by typing "javac" in your command prompt.
3. Download the Selenium RC.
4. Test cases are already converted to Selenium Junit cases.
Steps:
- Create a Java project in your Java IDE. I am using Eclipse so I will refer to it.
- By default there will be the “src” folder inside your Java Project. Right click on the “src” folder and create a new package under it say”test”.
- Now copy your Selenium Junit classes to this new “test” package.
- Create a new folder called “lib” under your Java Project.
- Copy selenium-java-client-driver.jar, junit-4.5.jar(if you have another version of Junit 4 you can use that) and ant-junit.jar(can be found under the “lib” folder of your ant installation) to your “lib” folder of Java Project.
- Now create a folder called “build” under your Java Project and copy the “build.xml” file attached with this blog to the build folder you created.
- Now open the build.xml file for edit.
- Search for “run-single-test” in build.xml. Go to the “<test” under the “run-single-test” target .Replace the values for “name=” variable with your Selenium Junit class name. For ex if your class name is TestSelenium.java and it is under package “test” under “src” then replace the value with “test.TestSelenium” then save the build.xml file.
- Now start the Selenium RC server.
- Now go to the build folder of the Java Project using command prompt and run the command “ant run-single-test”
- It should execute your Selenium cases on selenium server and generate an html report onto the “report/html” folder of your Java project.
Similarly you can modify the “run-batch-test” target of build.xml and execute multiple Selenium Tests together.
Download build.xml from Here
71 comments :
Hi Varan thanks for the psot...i successfully ran one script and it generated report...but i did not understand hwo to run multiple tests...in run-batch-test where i need to modify..please tell me...thanks in advance
Hi Varun,
Thanks for this help,
Can you please tell me how can i save these test result in differant directory and maintain the version up system for every execution?
-Parag
Hi Varun,
We have succesfullly executed the single test script. However, we are facing problems while try to run batch test .We ahve succesfully build all the files in the Test folder but ther are not being executed automatically as in one script test. Pls help!
Thnx!
Mariyam
Can anyone answer to Mariyam's question? we are still stuck in batch running ...
Thanks,
Shikhi
Sorry by mistake I had sent a mail to him.
Today after your comment i saw the mail-id.
Following were my comments:
You need to check the classpath. If u are getting a class not found exception.
If this not the case ,please send the ex eption u r getting.
Please send a mail to mr.varun.menon@gmail.com in case of more details. It will be easy for me.
i am getting this error " Ant is not recognized as an internal or external command, operable program or batch file" in dos command
@Praveen-You haven' installed ant properly into your system.
Just add your ant bin path to the "Path" variable of your systems environment variable. And open a new command prompt and then check.
Hi Varun ,
Can we Customize the summary report??
Hi Varun,
I got the same error that was posted by 'Praveen'. So I done as you directed to him, but getting the compilation error as:
javac: invalid source release: 1.6
Then I edited the 'build' file as a javac source="1.5" srcdir="${SRC}" fork="true".....
Then i got lots of error with selenium around 93 error were there.
Please guide me.
If its selenium or java related. Then you need to check your code. Without seeing the error I cant comment on anything :).
Varun,
Below is the error when I run the command “ant run-single-test”
``````````````````````````````````````````
compile:
[javac] Compiling 2 source files to C:\Documents and Settings\swapnil.NBITS\workspace\HHS\bin1
[javac] javac: invalid source release: 1.6
[javac] Usage: javac
[javac] where possible options include:
[javac] -g Generate all debugging info
[javac] -g:none Generate no debugging info
[javac] -g:{lines,vars,source} Generate only some debugging info
[javac] -nowarn Generate no warnings
[javac] -verbose Output messages about what the compiler is doing
[javac] -deprecation Output source locations where deprecated APIs are used
[javac] -classpath Specify where to find user class files
[javac] -cp Specify where to find user class files
[javac] -sourcepath Specify where to find input source files
[javac] -bootclasspath Override location of bootstrap class files
[javac] -extdirs Override location of installed extensions
[javac] -endorseddirs Override location of endorsed standards path
[javac] -d Specify where to place generated class files
[javac] -encoding Specify character encoding used by source files
[javac] -source Provide source compatibility with specified release
[javac] -target Generate class files for specific VM version
[javac] -version Version information
[javac] -help Print a synopsis of standard options
[javac] -X Print a synopsis of nonstandard options
[javac] -J Pass directly to the runtime system
BUILD FAILED
C:\Documents and Settings\swapnil.NBITS\workspace\HHS\build\build.xml:20: Compile failed; see the compiler error output for details.
Total time: 1 second
Hi varun,
getting error that symbol: variable selenium
Location: class LoginTestCase
Cannot find symbol selenium.open()
@Saurabh: Cant reply to you based on above comments. Can you tell me exactly what u are trying to do and what is your error. You can send a mail to me at mr.varun.menon@gmail.com with your error stack and java file(if you can).
Thanks for quick response Varun but i resolved the problem.
Download given build.xml and modify accordingly but when run the build from given path then it will not launch the browser to run the test case.
should i do any modification in the build.xml ?
@saurabh: Yes you need to modified the build.xml file based on your project and the java file you need to execute.
Hello Varun do you have any sample build.xml to figure out what kind of changes should i do ?
@saurabh: I dnt have such build.xml file with me. But as I had mentioned in the steps you need to change the Java file name in target. Also you need to check the SRC, LIB variables set on top of the build.xml file.
@Saurabh: I had modified the attached build.xml file. Please check it out. I had added my comments on each line. Hope it helps.
Hi, I am not able to download the build.xml file anymore.
Can varun/someone in the group upload the file once again.. please
RK.
Hi Varan, thanks for the post. I successfully ran one script and it generated report. I want to customize the report, add two more columns to the report. Please help me on this.
GK
@RK: Now you will be able to download the build.xml file directly.
@seleniumtest/GK: Please write in detail about your issue/requirement.
Hi Varun,
I have downloaded the build.xml file and successfully generate report using ANT + Selenium. I would like to add two more columns(1. Expected value, 2. Actual value) to the report. To customize ANT report, which xsl file to update or where this xsl file exist in ANT folder.
Thanks,
GK
Hi Varun,
I got the following error please help me to find the solution
java.lang.ClassNotFoundException: com.testscripts.loginpage.java at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Unknown Source)
-
Thanks
sandy
hi varun if i want to start server from eclipse itself and then generate html report how can i achieve this ?
Followed above steps but it is not launch firefox browser when i execute command ant run-single-test after running selenium server ?
Hello Varun,
Successfully run the test using above steps. Thanks.
But if i created one property file and parsing those property file parameters in my .java program and when i run same command it is not working.
Any other value need to add in build.xml file?
Varun,
Thanks you very much for this helpful tutorial. I am wondering what is the best way to specify the order the tests run in, while running the batch, I noticed it is alphabetical, is there another way other than alphabetical?
Hi Varun,
Can you please tell me that how can i execute a single test case instead of entire test class.
In Build.xml file, we need to mention that the class path in tag.So what should i need to change to execute any single testcase in a particular TestClass.
Can you please let me know that if you want more details.
Thanks
Santhosh.
@santosh: I had seen received a mail today by you asking, whether we can execute a particular method inside the Junit test class using ant.
The answer is: No. But you can use eclipse to do it.
@saurabh: Your query is pretty old now. Hopefully it may had got solved. Sorry for not responding as I was very busy with my project.
This is simply superb. Great post!
Hi Varun,
I have followed all your steps mentioned above, however when i run i got the error message as below
Could you please let me know what is wrong?
C:\Selenium\DDF\test\build>ant run-single-test
Buildfile: build.xml
init:
[delete] Deleting directory C:\Selenium\DDF\test\bin1
[mkdir] Created dir: C:\Selenium\DDF\test\bin1
compile:
[javac] Compiling 1 source file to C:\Selenium\DDF\test\bin1
[javac] C:\Selenium\DDF\test\src\test\MotorCycle.java:9: cannot find symbol
[javac] symbol: class SeleneseTestCase
[javac] public class MotorCycle extends SeleneseTestCase {
[javac] ^
[javac] C:\Selenium\DDF\test\src\test\MotorCycle.java:16: cannot find symbol
[javac] symbol : variable selenium
[javac] location: class test.MotorCycle
[javac] selenium = new DefaultSelenium("localhost", 4444, "*chro
me", "https://secure.tradevent.com.au/");
[javac] ^
[javac] C:\Selenium\DDF\test\src\test\MotorCycle.java:17: cannot find symbol
[javac] symbol : variable selenium
[javac] location: class test.MotorCycle
[javac] selenium.start();
[javac] ^
[javac] C:\Selenium\DDF\test\src\test\MotorCycle.java:29: cannot find symbol
[javac] symbol : variable selenium
[javac] location: class test.MotorCycle
[javac] selenium.stop();
[javac] ^
[javac] 4 errors
BUILD FAILED
C:\Selenium\DDF\test\build\build.xml:20: Compile failed; see the compiler error
output for details.
Total time: 1 second
Thanks,
Virendra
Please note: Test Case is working fine if i manually run through Eclipse..
@katha/virendra: Is your java selenium jar placed under the lib folder of the project. Also check whether the lib folder location is correctly set in the build.xml file.
Copy selenium-java-client-driver.jar, junit-4.5.jar(if you have another version of Junit 4 you can use that) and ant-junit.jar(can be found under the “lib” folder of your ant installation) to your “lib” folder of Java Project.
and add selenium-server-standalone-2.7.0.jar also to the lib folder
Hi All,
can any one provide the steps of how to install ANT.
@sruthi - Try using google search. Thank u. If u are unable to install it still, ill help u.
@Varun: Hi.
Thanks for this great post.
This is really helpful
I am running selenium (Java) script and in that i am taking the values from excel file.
The problem is that,
It is giving me following error
->The system cannot find the path specified
->java.io.FileNotFoundException: data\poi-test.xls (The system cannot find the path specified).
Actually i have data folder in which i placed my excel files.
So how can i set the path in Build.xml file.
Please help me.
Regards,
Gaurav
@Unknown- You need to include those files while compilation using.
or copy them to the bin folder by using the ant copy task.
Hi Varun,
Very good steps you explained which is helpful for all beginner of selenium............
Hi varun
I was trying to take the screenshot of failed testcases
My code is as below
@Override
public void onTestFailure(ITestResult result) {
File file = new File("");
Reporter.setCurrentTestResult(result);
System.out.println(file.getAbsolutePath());
Reporter.log(file.getAbsolutePath());
Reporter.log("screenshot saved at "+file.getAbsolutePath()+"\\reports\\"+result.getName()+".jpg");
Reporter.log(" ");
BaseClass.selenium.captureScreenshot(file.getAbsolutePath()+"\\reports\\"+result.getName()+".jpg");
Reporter.setCurrentTestResult(null);
but i want some clarity in the below mentioned line
BaseClass.selenium.captureScreenshot(file.getAbsolutePath()+"\\reports\\"+result.getName()+".jpg");
Its throwing an error at BaseClass
Pls help me.
Regards
Nithin S G
Mind blowing article. Thanks a ton!! keep up the good work!!
Hi Varun...
It is fantastic blog... Very good ideas I have seen here... Now I am doing automation testing in my company. So I would like to have tips from you for load testing using selenium... I am using selenium IDE & RC... If possible to do load testing at least for 15 users, that's enough to me... Please reply me asap.. Thanks in advance...
Selenium is a functional testing tool and not a load testing tool. Still I had seen people doing load testing using selenium. I haven't used selenium for load testing but in case you need to use it , use html unit driver as driver object.
Hi Varun,
Thanks for great blog.
I am using the Selenium Web driver "selenium-server-standalone-2.25.0" I was trying to build by using your build XMl with ant 1.8 & junit 4 framework.
It successfully run through eclipse and shows the message that 'BUILD SUCCESSFUL' but no any report is generated and when I execute the same through command prompt “ant run-single-test” it gives tones of error.
Please help.
Here is the error log.
-------------------------
Buildfile: C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\build\build.xml
init:
[delete] Deleting directory C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\bin1
[mkdir] Created dir: C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\bin1
compile:
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\build\build.xml:20: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 10 source files to C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\bin1
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:5: package org.junit does not exist
[javac] import org.junit.*;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:6: package org.junit does not exist
[javac] import static org.junit.Assert.*;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:8: package org.openqa.selenium does not exist
[javac] import org.openqa.selenium.*;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:9: package org.openqa.selenium.firefox does not exist
[javac] import org.openqa.selenium.firefox.FirefoxDriver;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:15: cannot find symbol
[javac] symbol : class WebDriver
[javac] location: class com.Add_sub_with_login
[javac] private WebDriver driver;
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Add_sub_with_login.java:64: cannot find symbol
[javac] symbol : class By
[javac] location: class com.Add_sub_with_login
[javac] private boolean isElementPresent(By by) {
[javac] ^
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\Admin_login.java:5: package org.junit does not exist
[javac] import org.junit.*;
.
.
.
[javac] C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\src\com\LoginDataDriven.java:30: cannot find symbol
[javac] symbol : class Test
[javac] location: class com.LoginDataDriven
[javac] @Test
[javac] ^
[javac] 100 errors
BUILD FAILED
C:\opt\eclipse\eclipse-ganymede\workspace\Alepo_Test\build\build.xml:20: Compile failed; see the compiler error output for details.
You should use "selenium-java-.jar" and not "selenium-server".
Hi varun,
Relative path of the file given in .java file is not recognized while running through ant.
relative path given in .java file is as follows:
return "././Config/ConfigParams.xml"; (this xml file has test data information of our project)
while running the script using TestNG, it is running fine (i.e., it is accessing ConfigParams.xml file)
While running with ant after building the project,it is not accessing ConfigParams.xml file.
suppose if i hard code the file path: return "C:/YGIB/Config/ConfigParams.xml"
Then ant is recognizing ConfigParams.xml file and script is running sucessfully
i am running the ant via command prompt.
please let me know what i am doing wrong here
Hi Varun,
This thing just worked for me with few minor changes and I wanted to thank you for this, I really appreciate this article as I was looking for some thing like this for last couple of days and this article saved my day.
Thanks
Manoj
Hi Varun,
How you describe the steps is very understandable. I was confused at first about the report generation. But after reading your article I can able to run my code and can create report easily.
Thank you.
Hi Varun ,
I need to generate a Junit report using ant.I am not using selenium .I am doing automation in Seetest(a mobile automation tool),whose frame work is in jUnit.
using ant I need to generate report and send to email address.
How to do it without selenium ?
Your help will be really helpful.
Regards
Anjali
Use the following code inside your ant target:
<junitreport todir="${REPORT}">
<!-- This section generated the Junit report in html to "html" folder from the xml file. -->
<fileset dir="${REPORT}/xml">
<include name="TEST*.xml"/>
</fileset>
<report format="frames" todir="${REPORT}/html"/>
</junitreport>
Here ${REPORT} is the report directory that points to your local system folder.
${REPORT}/xml is the directory where all the report xml files generated after executing the Junit tests are placed.
${REPORT}/html is the directory where the junit html report is generated and placed.
Hi, I successfully ran one script and it generated report...but it did not work for run multiple tests..Please update to how to run multiple testcases.
Also i don't want to override previous report . please suggest how to generate separate report every time based on datetime stamp name (as report filename format)
Hey thank you for this great blog. I completed all of the directions but I continue to get this error:
BUILD FAILED
\workspace\Antproject\Build\build.xml:8: You must specify value, location or refid with the name attribute
Unable to find the folder may be. Try providing the absolute path and check.
Hi Varun,
Nice writeup.
I have question when I run ANT and generate Junit reports
I see that the package name is shown as "None" and the class name is shown as Blank. Can you let me know as to what the issue is.
Regards
Prashanth
Is your tests getting executed? If yes does ant considers the correct directory for collecting report xml?
Hi Varun, Yes, my tests are getting executed and the ant considers the correct directory for the report.xml. The only issue us that the package name is shown as "None" and the class name is shown as Blank.
Please find the Ant target for generating Junit report. Can you please let me know if there is something wrong with this.
Varun, for some reason, I'm not able to paste the Ant target for generating Junit reports here for your reference. Is there any other way, I could share the same.
Regards
Prashanth
When i ran build.xml it generates html report but second time when i again ran build.xml , test report remains the same. where i can get the updated test report which i run second time.
You may require to delete the report folder and recreate it for every new build
Hi varun,
When i run the script,i found following error :
compile:
[javac] C:\Users\Ht\workspace\com.shine.test\build\build.xml:20: warning: 'i
ncludeantruntime' was not set, defaulting to build.sysclasspath=last; set to fal
se for repeatable builds
Please tell me what is it?
I googled it but unable to find suitable answer.
Please help me.
Your information about Selenium scripts is really interesting. Also I want to know the latest new techniques which are implemented in selenium. Can you please update it in your website?
Selenium training Chennai
Hi Varun,
I have successfully done Single Run Test. I am struggling on batch run test where i have two packages with 2 class files each. How to write a batch file? and explain it as steps.
Post a Comment