TestNG Listeners In Selenium Webdriver For Automated Testing

The TestNG tutorial on ToolsQA has covered most of the essential topics in TestNG. By important, I mean the concepts and features which we use most often while working in TestNG. Although, as a tester, you can go ahead and experiment with every feature available and use the combination of them to make your testing more efficient. Every feature is essential, and we use it for the corresponding situation. With this, we will move towards the end of this course with one topic, which is used very extensively in TestNG. It is called TestNG listeners, and it is another type of listener (in general) apart from WebDriver listeners. This tutorial will guide you through the following topics, keeping the interest of TestNG listeners:

What are TestNG Listeners?
Types of TestNG Listeners
ITestListener in TestNG
- How to implement ITestListener in TestNG?
IReporter Listeners in TestNG
- How to implement IReporter Listeners in TestNG?
ISuite Listeners in TestNG
- How to implement ISuite Listener in TestNG?
IInvokedMethod Listener in TestNG
- How to implement IInvokedMethod Listeners in TestNG?
IHookable Listeners in TestNG
IConfigurationListener in TestNG
IConfigurable Listeners in TestNG
IAnnotation Transformer Listener in TestNG
IExecution Listener in TestNG
IMethodInterceptor Listeners in TestNG

What are TestNG Listeners?

The first thing that comes to mind by reading the term "listeners" is that it must be listening to something in the code and being a "good listener", it does. TestNG listeners are the piece of code that listens to the events occurring in the TestNG. If the event matches the event for which we want the listener to listen, it executes the code, which ultimately results in modifying the default behavior of TestNG. For example, we want to print the exception error onto the reports only if the test fails. Here, we can apply a TestNG listener that will listen to the event of "failing of test case" and when it does, it will log the error.

To save the unnecessary utilization of the resources, we cannot let the listeners listen to the code all the time. Therefore, the listeners in TestNG are activated either before the test execution or before (noticing that test execution takes the majority share of testing time). TestNG Listeners are applied as interfaces in the code because "Listeners" is a "class" in TestNG. TestNG provides us with loads of listeners. We will discuss these in the next section.

Types Of Listeners In TestNG

TestNG provides a bunch of listeners as a part of its testing environment. These listeners are as follows:

ITestListener
IReporter
ISuiteListener
IInvokedMethod
IHookable
IConfigurationListener
IConfigurableListener
IAnnotationTransformer
IExecution
IMethodInterceptor

Remember that although all of these listeners are important, we don't use them at the same percentage by the testing community. It depends on the situation that the tester is facing in real practical experiences. In this tutorial, I have demonstrated the most popular TestNG listeners (with code) and explained a few unpopular but important ones (without code). You can practice them as a part of becoming a complete tester. They will surely be helpful in your testing journey.

ITestListener In TestNG

ITestListener is the most used listener in TestNG with Selenium webdriver. The ITestListener implements since it is an interface, and the class in which we implement the listener overrides the ITestListener methods. ITestListener listens to specific events (depending on its methods) and executes the code written inside the method. With ITestListener in TestNG, we can also log the events onto the reports using the Selenium web driver. The ITestListener contains the following methods:

onStart: This method invokes when the test class is instantiated and before executing any test method.

Syntax: void onStart(ITestContext context);

onFinish: This method invokes when all the test methods have run, and calling of all of their configuration methods happens.

Syntax: void onFinish(ITestContext context);

onTestStart: This method invokes every time a test method is called and executed.

Syntax: void onTestStart(ITestResult result);

onTestSuccess: This method is invoked every time a test case passes (succeeds).

Syntax: void onTestSuccess(ITestResult result);

onTestFailure: This method invokes every time a test case fails.

Syntax: void onTestFailure(ITestResult result);

onTestSkipped: This method invokes every time a test skips.

Syntax: void onTestSkipped (ITestResult result);

onTestFailedButWithinSuccessPercentage: This method invokes when the test method fails as a whole but has passed a certain success percentage, which is defined by the user.

Syntax: void onTestFailedButSuccessPercentage (ITestResult result);

Now in the above syntaxes, you must be wondering about the words ITestContext and ITestResult. So, the term 'ITestResult' is an interface that describes the result of the test. Therefore 'result' has been passed as its instance in the syntax. Whereas ' ITestContext ' is a class that defines an instance 'context', which contains all the information about the test run. We can use this information to pass to our listeners, and they can proceed with their queries.

How to implement ITestListener in TestsNG?

This section will explain step by step how to use listeners in TestNG to invoke various functions. It is important to note here that Listeners can implement in two ways in TestNG:

At the class level: Annotating listeners on each class in the test code.
At the suite level: Define the class names to implement listeners in the TestNG XML file.

So first, we will go with implementing the listeners at the class level.

Implementing ITestListener At Class Level

To implement ITestListener, we need to create a class with all the methods that we want to override. Let's call it ListenersTestng.java

import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class ListenersTestNG implements ITestListener {
	public void onStart(ITestContext context) {	
		System.out.println("onStart method started");
	}

	public void onFinish(ITestContext context) {
		System.out.println("onFinish method started");
	}
	
		public void onTestStart(ITestResult result) {
			System.out.println("New Test Started" +result.getName());
		}
		
		public void onTestSuccess(ITestResult result) {
			System.out.println("onTestSuccess Method" +result.getName());
		}

		public void onTestFailure(ITestResult result) {
			System.out.println("onTestFailure Method" +result.getName());
		}

		public void onTestSkipped(ITestResult result) {
			System.out.println("onTestSkipped Method" +result.getName());
		}

		public void onTestFailedButWithinSuccessPercentage(ITestResult result) {
			System.out.println("onTestFailedButWithinSuccessPercentage" +result.getName());
		}
}

Now that our Listeners class sets, we need to implement the listeners in another class whose events it will listen. Create a new file TestNG.java which may look like the following:

import org.openqa.selenium.WebDriver;
import org.testng.SkipException;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.Assert;

@Listeners(ListenersTestNG.class)
public class TestNG {
	WebDriver driver = new FirefoxDriver();
	
	@Test  //Success Test
	public void CloseBrowser() {
		driver.close();
		Reporter.log("Driver Closed After Testing");
	}
	
	@Test //Failed Test
	public void OpenBrowser() {
	        
	        driver.get("https://www.demoqa.com");
	        String expectedTitle = "Free QA Automation Tools For Everyone";
	        String originalTitle = driver.getTitle();
	        Assert.assertEquals(originalTitle, expectedTitle, "Titles of the website do not match");
  }
	private int i = 1;

	@Test (successPercentage = 60, invocationCount = 3) //Failing Within Success
	public void AccountTest() {
			if(i < 2)
				Assert.assertEquals(i , i);
		i++;
	}
	
	@Test  // Skip Test
	public void SkipTest() {
		throw new SkipException("Skipping The Test Method ");
	}
}

These test cases are easy to understand, and if you have gone through the rest of the tutorial of TestNG and Selenium, the methods and TestNG annotations used will not bother.

Understand that Listeners, when used at the class level, are used as annotations in TestNG in the form @Listeners. Whatever method triggers the Listeners' methods, it will execute the code written within that particular test method. For example, when the SkipTest method runs in the above code, it triggers the onTestSkipped method in the TestNG listener, which executes the code written in onTestSkipped. Execute the above code to see the listener's response. testng listeners itestlistener output

All the test cases that should be triggered have shown their output on the console. Great, now we need to think that what would happen if we have multiple classes in our test code? In practical testing scenarios, it does happen a lot. Implementing the listeners at the class level will cause a lot of distress; adding and managing the listeners. Therefore, in such a case of multiple classes, we adopt the way of implementing the listeners at the suite level in the TestNG XML file.

Implementing ITestListener At Suite Level

To implement the ITestListener at the suite level, remove the @Listener annotation from the test code, and add the following to the XML file.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name="ListenerExecution">
<listeners>
    <listener class-name = "ListenersTestNG"></listener>
    </listeners>

    <test name="ExecutingListeners">
        <classes>
            <class name="TestNG"></class>
        </classes>
    </test>
    
    
</suite>

Note: The file that contains the listener code comes under <listener class-name> tag.

Run the TestNG suite, and it will produce the same output as before. You can mention as many classes under the classes tag as you wish.

IReporter Listeners In TestNG

The IReporter interface in the TestNG Listener provides us with a medium to generate custom reports (i.e., customize the reports generated by TestNG). The interface contains a method called generateReport() which we invoke when all the suites have run. This method takes three arguments:

xmlSuite: This is the list of suites that exist in the XML file for execution.
suites: This is an object which contains all the information about the test execution and suite related information. This information may include the package names, class names, test method names, and the test execution results.
outputDirectory: It contains the path to the output folder where the generated reports will be available.

How to implement IReporter Listener in TestNG?

The following example shows the code written for the Listener class. Here, we will be logging the number of tests that have passed, the number of tests that have failed, and the number of skipped tests.

import java.util.List;
import java.util.Map;

import org.testng.IReporter;
import org.testng.ISuite;
import org.testng.ISuiteResult;
import org.testng.ITestContext;
import org.testng.xml.XmlSuite;

public class ListenersTestNG implements IReporter{
   @Override
   public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites,
      String outputDirectory) {
      
      //Iterate over every suite assigned for execution
      for (ISuite suite : suites) {
            
         String suiteName = suite.getName();
         Map<String, ISuiteResult> suiteResults = suite.getResults();
         for (ISuiteResult sr : suiteResults.values()) {
            ITestContext tc = sr.getTestContext();
            System.out.println("Passed tests for suite '" + suiteName +
               "' is:" + tc.getPassedTests().getAllResults().size());
            System.out.println("Failed tests for suite '" + suiteName +
               "' is:" + tc.getFailedTests().getAllResults().size());
            System.out.println("Skipped tests for suite '" + suiteName +
               "' is:" + tc.getSkippedTests().getAllResults().size());
         }
      }
   }
}

In the above tests, we are re-iterating through the object suites that contain all the information about it. Since it contains much information, we are storing the test execution results in a map and then iterating over the objects of the map and printing the required details.

The TestNG file would like follows:

import org.testng.Assert;
import org.testng.annotations.Test;

public class TestNG {
   @Test
   public void firstMethod() {
      Assert.assertTrue(true);
   }
	  
   @Test
   public void secondMethod() {
      Assert.assertTrue(false);
   }
	  
   @Test(dependsOnMethods = {"firstMethod"})
   public void thirdMethod() {
      Assert.assertTrue(true);
   }
}

The formation of the above code is such that it passes or fails the test without any logic forcefully. Execute the TestNG XML file to see the report output: ireporter listener testng output

The report contains the account for passed, failed, and skipped tests. The tester can use this interface in any way they want to generate custom reports in TestNG.

ISuiteListener In TestNG

ISuiteListener is another type of listener provided in TestNG. The ISuiteListener works on the suite level. Additionally, it listens to the event of the start of a suite execution and end of the suite execution. ISuiteListener then runs the methods only before the start of the suite and at the end. It contains two methods:

onStart: This method invokes before the test suite execution starts.

Syntax: void onStart(ISuite suite);

onFinish: This method invokes after the test suite execution ends.

Syntax: void onFinish(ISuite suite);

One should note that if the parent suite contains child suites as well, then the child suites are executed before the parent suite. It helps to retrieve the combined results from child suites reflecting onto the parent suites.

How to implement ISuiteListener in TestNG?

To demonstrate the same property discussed above, we will create two child suites and execute them through their parent suite.

The following is our first Listener file called Listener1.java.

import org.testng.ISuite;
import org.testng.ISuiteListener;

public class ListenersTestNG implements ISuiteListener {
	public void onStart(ISuite suite) {	
		System.out.println("onStart function started "  + suite.getName());
	}
	public void onFinish(ISuite suite) {
		System.out.println("onFinish function started "  + suite.getName());
	}	
}

Now we will declare two TestNG classes to execute from the child suites. The first class is TestNG.java with the following code:

import org.testng.annotations.AfterSuite;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.Test;

public class TestNG {
	
	@BeforeSuite
	public void bsuite()
	{
	   System.out.println("BeforeSuite method started");	
	}
	
	
	@Test
	public void test()
	{
		System.out.println("Test Method started");
	}
	
	@AfterSuite
	public void asuite()
	{
	   System.out.println("AfterSuite method started");	
	}

}

Create another class with the name TestNG2.java and copy-paste the above-given code to the file with a few minor changes that will help us identify that the method belongs to the second class.

import org.testng.annotations.AfterSuite;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.Test;

public class TestNG2 {
	
	@BeforeSuite
	public void bsuite()
	{
	   System.out.println("BeforeSuite method of TestNG2 started");	
	}
	
	
	@Test
	public void test()
	{
		System.out.println("Test Method of TestNG2 started");
	}
	
	@AfterSuite
	public void asuite()
	{
	   System.out.println("AfterSuite method of TestNG2 started");	
	}

}

Now that we have our Java files ready, we need to code our XML files. Since we are going ahead with two child suite and one parent suite, we are required to create three XML files.

Let's name the first child class as xmlFile1.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="First Test Suite">

 <test name="isuitetest">
   <classes>
     <class name="TestNG"/>
   </classes>
 </test>

</suite>

Let's name the second child class as xmlFile2.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Second Test Suite">

 <test name="isuitetest2">
   <classes>
     <class name="TestNG2"/>
   </classes>
 </test>
</suite>

Now, let's create the parent class, which we will execute. This file is testng.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">

<suite name="parentsuite">
    
    <listeners>
        <listener class-name="ListenersTestNG"></listener>
    </listeners>
    
    <suite-files>
        <suite-file path="xmlFile1.xml"> </suite-file>
        <suite-file path="xmlFile2.xml"> </suite-file>
    </suite-files>
</suite>

In this parent file, we are mentioning the child suites in the tag <suite-files>. We expect them to execute before the parent class. isuitelistener output testng

The ISuiteListener has executed as expected with both of its functions.

IInvokedMethod Listeners in TestNG

IInvokedMethod Listener performs some tasks before and after the methods execute. We generally use it for setting up configurations or cleanup etc. But it depends on the tester only. IInvokeListener invokes before and after every test method available in the test code, unlike ITestListener. The IInvokedMethod listener contains two methods:

beforeInvocation(): This method invokes before every method.

Syntax: void beforeInvocation (IInvokeMethod method, ITestResult testresult);

afterInvocation(): This method is invoked after every method.

Syntax: void beforeInvocation (IInvokeMethod method, ITestResult testresult);

How to implement IInvokedMethod Listener in TestNG?

For executing this code, our listener class would like the following:

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;

public class ListenersTestNG implements IInvokedMethodListener {

	public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
		
		 System.out.println("Before Invocation Method Started For: " + method.getTestMethod().getMethodName() + "of Class:" + testResult.getTestClass());		
	}

	public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
	
		 System.out.println("After Invocation Method Started For: " + method.getTestMethod().getMethodName() + "of Class:" + testResult.getTestClass());
		
	}
}

The TestNG class where the implementation of the listener happens is TestNG.java. The code is as follows:

import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(value=ListenersTestNG.class)
public class TestNG {	
	@Test
	public void firstTest()
	{
		System.out.println("My First Test");
	}
	
	@Test
	public void secondTest()
	{
		System.out.println("My Second Test");
	}
	
	 @BeforeClass
	    public void bclass() {
	        System.out.println("Before Class Method Started");
	    }
	 
	    @AfterClass
	    public void aclass() {
	        System.out.println("After Class Method Started");
	    }
}

Execute this TestNG test case and notice that the afterInvocation and beforeInvocation methods invoke after and before every test method, respectively. iinvoked listener testng

Alright, so these were the most used listeners in TestNG. We also used the rest of the listeners. However, they are not so popular, and therefore, I will not be demonstrating the code for the same. But, just so that they are essential to know, I will explain them in detail.

IHookable Listeners In TestNG

IHookable listener is used to performing some tasks before running a method such as JAAS authentication or setting permissions for the test methods etc. Additionally, the IHookable listener interface runs its run() method instead of the @Test method whenever found during the execution. This @Test method is although passed on and encapsulated in an IHookCallBack object, which can be run by invoking the runTestMethod function from the IHookable interface as in IHookable.runTestMethod().

IConfigurationListener In TestNG

The IConfigurationListener in TestNG triggers only when the configuration methods pass, fail, or skip. The IConfiguration listener has the following methods:

onConfigurationSuccess: The success of the configuration method invokes this method.

Syntax: void onConfigurationSuccess(ITestResult itr);

onConfigurationFailure: The failure of the configuration method invokes this method.

Syntax: void onConfigurationFailure(ITestResult itr);

onConfigurationSkip: This method is invoked when the configuration method is skipped.

Syntax: void onConfigurationSkip(ITestResult itr);

IConfigurable Listeners In TestNG

The IConfiguration listener provides us with a run() method. This method invokes instead of each configuration method that we find. The invocation of the configuration method, however, happens when we call the callback() function through the IConfigureCallBack parameter.

Syntax: void run(IConfigureCallBack callback, ITestResult testresult);

IAnnotationTranformer Listeners in TestNG

The IAnnotationTranformer listener, as its name suggests, transforms the annotation provided inside the test case code. These annotations transform through the "transform" method available in the listener. This method will detect the annotation and then transform it according to the code written inside the method. The transform method contains the following parameters:

annotation: This is the annotation that was read from your test class.
testClass: I1f the annotation were found in a class, this parameter would then represent the class (otherwise null).
testMethod: If the annotation was found on the method, this parameter would then represent the method ( otherwise null).
testConstructor: If the annotation were found on a constructor, this parameter would then represent that constructor (otherwise null).

It is important to note that at least one of the parameters should contain a value other than null, i.e., all the parameters cannot be null.

Syntax:

void transform( ITestAnnotation annotation, java.lang.Class testClass, java.lang.reflect.Constructor testConstructor, java.lang.reflect.method testMethod);

IExecutionListener in TestNG

The IExecutionListener listener monitors the start and the end of a test or a suite. The IExecutionListener contains two methods:

onExecutionStart: This method invokes before the test start.
onExecutionFinish: This method invokes after all the tests/suites have finished the execution.

IMethodInterceptor Listener in TestNG

IMethodInterceptor listener alters or modifies the list of tests that will be executed by TestNG. By using the IMethodInterceptor listener, we run the intercept method, which returns the list of methods that will be run by TestNG. Now the TestNG will run the methods in the same order that returned in the list. The IMethodInterceptor listener in TestNG contains only one method:

intercept: This method returns the list of methods that will be executed by TestNG.

Syntax: List<IMethodInstance> intercept (List<IMethodInstance methods, ITestContext context);

Conclusion

Listeners in TestNG build a good testing environment and scenarios. Listeners give us the power to listen to certain events and act accordingly. Since there are so many events in TestNG and so many requirements, so are the listeners. With some listeners gaining popularity among the testing community, others are used less often but certainly important. If there is a situation in which the tester wants to execute only priority = 1 test cases, they need to use the IMethodInterceptor listener in TestNG, which is although not so popular. So, bottom line, go through these listeners two or three times and keep them on the tips to be efficient in TestNG. With this note, we will move on to the next post on TestNG.