Software tests that look at the system as a black box are extremely valuable. These tests are external to the application and exercise the application as a user would. This article examines how Spring Roo supports web testing using the Selenium automated web testing tool.
This article is based on Spring Roo in Action, published in April, 2012. It is being reproduced here by permission from Manning Publications. Manning early access books and ebooks are sold exclusively through Manning. Visit the book’s page for more information.
Authors: Ken Rimple and Srini Penchikala with Gordon Dickens
Testing shouldn’t stop at the unit test or integration test level. These tests all exercise a particular component or set of components in compositions that you define yourself. But the best way to verify that the entire stack functions properly is to use some sort of external, black box test—meaning a test external to the application itself. At a bare minimum, what you need is a mechanism to assert tests against the user interface so that you can start testing at the level of a user interaction. Roo has support for this approach in the form of the Selenium web testing framework.
What is Selenium?
Selenium is a suite of web testing tools. It can exercise browser-based tests against an application and has a Firefox-based IDE (Selenium IDE) for building tests interactively against a live application. Selenium tests can be written in a number of languages, from HTML to the Java JUnit API to other languages such as Ruby or Python.
Selenium tests can be used in a number of ways, including:
* Feature testing—Testing various use cases in your application, using a browser-based approach.
* Monitoring—Checking that a representative controller returns a valid result, which would indicate that the application is online.
* Load testing—Using Selenium’s distributed testing engines, a heavy load can be placed on the application from a number of machines.
Selenium is widely adopted and there are a number of resources, such as JUnit in Action, Second Edition, that document it in detail. We’ll focus on how to get Selenium up and running against a RESTful controller, and then we’ll look at how to add JUnit-based Selenium tests for more sophisticated testing scenarios.
Installing Selenium
As with everything else in Roo, the Selenium framework is installed with a simple Roo shell command. The selenium test command takes several options, including the mandatory controller class to test:
selenium test--controller~.web.TagController
The class must be a Roo-scaffolded controller. In response to this, Roo performs the following actions:
* Installs the Selenium dependencies and the Codehaus Maven Selenium plug-in in the Maven pom.xml file.
* Creates a WEB-INF/selenium directory to hold HTML tests.
* Builds a test-suite.xhtml master test suite file, which Roo will maintain whenever a new test is installed.
* Builds a test case for the entity scaffolded by the controller, with the name of test-entity.xhtml.
You’ll see immediately that Roo’s support for Selenium mostly focuses on scaffolded controllers. This may be a bit limiting, but you can install support for any controller you want by using the JUnit API. To run your tests, you first have to launch your web server. Open a new command prompt, switch to the project root directory, and issue the following command to launch the Jetty web server:
mvn package jetty:run
You’ll need a running instance of your application in order to run Selenium tests. To trigger the tests, issue the following command from another operating system prompt to run your tests:
mvn selenium:selenese
This command launches the Selenium test runner, which should launch an instance of the Firefox browser and run your tests. After the tests are finished, the Selenium Maven plug-in should declare the build a success. A test report will be generated in HTML format and placed in target/surefire-reports/selenium.html. The contents of this test are shown in figure 1.
Figure 1 Successful Selenium report showing test run of the test-tag.xhtml test
Looking at the test report, you’ll see that it contains counts of the number of tests, how many passes and failures, and the details for each test. Any failed test or test command is shown in red, successes are shown in green. You’ll also see a detailed log of each test step underneath the pass or fail data. This is a comprehensive test report.
So, now that you know how to install Selenium and generate and execute your tests, let’s take a look at the test suite and the test that you initially generated on your Tag object.
Autogenerated Selenium tests
The generated tests are controlled by a master test suite file, test-suite.xhtml, located in src/main/webapp/. Let’s review the contents of that file after you’ve generated the Tag test, as shown in the following listing.
Listing 1 test-suite.xhtml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<title>Test suite for coursemanagertestproject</title>
</head>
<body>
<table>
<tr>
<td>
<b>Suite Of Tests</b>
</td>
</tr>
<tr>
<td>
<a href="http://localhost:8080/coursemanagertest➥
/resources/selenium/test-tag.xhtml">
Selenium test for TagController
</a>
</td>
</tr>
</table>
</body>
</html>
The test suite is similar in concept to JUnit test suites—it contains a list of each Selenium test file you’ll execute.
Roo maintains the test suite, adding to it every time a new Selenium test is generated from the Roo shell. You can write your own test suites, place them in the same directory, and add them to this test suite file.
Roo also generated your test case, based on the fields of your entity. Let’s take a look at the test-tag.xhtml test file next.
Listing 2 test-tag.xhtml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head profile="http://selenium-ide.openqa.org/profiles/test-case">
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type"/>
<link href="http://localhost:8080/" rel="selenium.base"/>
<title>Selenium test for TagController</title>
</head>
<body>
<table border="1" cellpadding="1" cellspacing="1">
<thead>
<tr>
<td colspan="3" rowspan="1">Selenium test for TagController</td>
</tr>
</thead>
<tbody>
<tr>
<td>open</td>
<td>http://localhost:8080/coursemanagertest/tags➥
?form&lang=en_US</td> #1
<td> </td>
</tr>
<tr>
<td>type</td>
<td>_tag_id</td>
<td>someTag1</td>
</tr>
<tr>
<td>type</td> #2
<td>_description_id</td>
<td>someDescription1</td>
</tr>
<tr>
<td>clickAndWait</td> #3
<td>//input[@id='proceed']</td>
<td> </td>
</tr>
</tbody>
</table>
</body>
</html>
#1 Browse to page
#2 Type into field
#3 Submit form
The HTML-based Selenium test language was designed so that power users and advanced business experts could read and interpret it. You can see that the test starts by opening the URL to the tag controller (#1). Then, using the HTML ID of the field elements, the type command enters a value into each field (#2). Finally, the clickAndWait command (#3) tells Selenium to press the proceed button and wait for a valid response.
WHAT CAN YOU CHANGE? This generated controller test file shouldn’t be edited, because it’s scaffolded like the controllers and entities themselves. Any change to the entity will also affect a change to the generated test file.
Roo will append only scaffolded tests to the test-suite.xhtml file so you can add additional test files. Any additional entity will add an entry to the test suite. In this way, you can do some basic testing of the create method of the form.
But, what if you’re not scaffolding, or you want to take it a step further? You have two options: either you can add the additional test to the suite and write it in HTML semantics, or you can use the JUnit framework to generate test cases. Let’s look at both approaches.
Writing your own Selenium test
The generated Selenium test submits form data based on legal values from the Bean Validation annotations, clicks the proceed button, and verifies that the next page is displayed. This isn’t an in-depth test. What if the controller fails?
Selenium tests can draw upon the full list of commands, collectively known as Selenese. These commands are documented at http://seleniumhq.org/docs/. Bear in mind, well-written tests attempt to perform a single activity and verify the result.
The test opens an entity creation page, types data on the page, and submits the form. You can go a step farther and assert that the data submitted by the test is reflected in the new page. Here’s a variation on the test that checks that the fields are shown in the show view, which is displayed after you submit. Copy your test-tag.jspx test to a test-tag-with-assertions.tagx, and add the lines in the following example to the end of the table:
These commands verify that the div with the id specified in @id holds the value in the third table element. You can use the assertText command to fail the test and stop running further commands, but the verifyText command you’re using marks the test as failed while it continues to run the rest of the commands in the test file.
If you want to know more about Selenium, we suggest you install the Selenium IDE, a Firefox plug-in that allows you to record, edit, and play back Selenium test cases. Figure 2 shows the IDE in action, reviewing a step in a test case.
Figure 2 The Selenium IDE editor—note the context-sensitive help in the Reference tab
The Selenium IDE has full support for editing the commands generated by your tests. Fire it up and import your HTML test case into the editor. You can run the test interactively, debugging and modifying the commands until you have the test you want. You can also save this test. Note: Don’t save it with a preexisting generated test name or it’ll get overwritten when Roo adds another field to the entity.
Key Selenese commands
When you’re working in the Selenium IDE, you’ll see a drop-down list of commands. These commands are the language of Selenium, known as Selenese. There are a number of key commands that perform activities ranging from typing text into fields, to comparing values on forms, to verifying that text is or is not present, to submitting forms.
Table 1 shows a few common commands.
Table 1 Selenese commands
| Selenium command | Usage | First parameter | Second parameter |
| type | Types the value of an input field. You can also use this command to select from an option field, using the data value, not the visible option. | locator | value |
| click[AndWait] | Clicks on a clickable element such as a button. If used with the suffix AndWait, assumes a server call has been made, and waits up to the configured timeout time for a response from the server. a | locator | value |
| check | Selects a radio button or checkbox field. | locator | none |
| open | Opens a URL in the frame under test. This is the first action in your scaffolded tests. | url | none |
| waitForPageToLoad | Pauses the script until a new page is fully loaded. | timeout (optional) | none |
a. Many commands in the HTML Selenese dialect can be suffixed with AndWait. Consult the reference documentation for details.
Many more Selenese commands are available. Consult the Selenium reference guide, experiment with the Selenium IDE, and write your own tests.
MORE INFORMATION ON SELENIUM COMMANDS If you want to learn more about Selenese, you can refer to the excellent documentation online at http://seleniumhq.org or JUnit in Action, Second Edition.
If you want to run an additional XHTML test, you’ll have to add it to the Selenium testsuite.xhtml file. Assuming you named your new test test-tag-with-verify.xhtml in the same directory, you would add it to the test table, as shown in the following example:
<tr>
<td>
<a href="http://localhost:8080/coursemanagertest/➥
resources/selenium/test-tag-with-verify.xhtml">
Better Selenium test for TagController
</a>
</td>
</tr>
You may look at this and think, “Wow, this is handy.” If so, stop here and start collaborating with your subject matter experts on your tests, using this language as a kind of shared notation. But some of you may also think, “Ewwww. Writing code in HTML?” That’s fine as well. For you, Selenium has an answer. Several, in fact.
Adding JUnit semantics
If it seems wrong to you to write test code in an HTML or XML markup language because you think, as we do, that code is code, and XML is configuration, you can rest easy. Selenium has language bindings for a number of higher-level languages, such as Java (JUnit 3 and JUnit 4, TestNG), Groovy (JUnit), C#, Ruby, RSpec, Perl, Python, and PHP. This means that APIs are available to a wide variety of programmers, and as such makes Selenium a go-to technology for many web testing efforts.
So, let’s get started using Selenium with JUnit. You need to take several steps:
* Install the Selenium Java Client driver in your Maven pom.xml file.
* Write your JUnit tests, or convert them from HTML using the Selenium IDE.
* Optionally, configure Maven to run your tests in the integration test phase.
To install your Java Selenium API, add the following dependency to the pom.xml section:
<dependency>
<groupId>org.seleniumhq.selenium.client-drivers</groupId>
<artifactId>selenium-java-client-driver</artifactId>
<version>1.0.2</version>
<scope>test</scope>
</dependency>
Now, to convert your test into JUnit, you’ll use the helpful language translation feature of the Selenium IDE. The latest Selenium IDE has removed the Format menu option, and recommends cutting/pasting in a given language format. But you can bring the menu item back again. For more information, select Options > Format > Want the Formats Back? Click to read more. After you install the IDE in your Firefox browser, launch it using the Tools > Selenium IDE menu option. The IDE will appear. Clear any test text from the code window and use the Options > Format menu to select the HTML code format, as shown in figure 3.
Figure 3 Selecting the code format
Note that all of the other formats also display in the drop-down menu in the previous figure. Paste your XHTML test code into the code editor on the right, and then switch the code to JUnit 4 by changing the format to JUnit 4 (Remote Control). Pretty nifty. The following listing shows the generated code for your sample test, modified so that you can make it consistent with the rest of your application framework.
Listing 3 The generated JUnit test
package org.rooina.coursemanager.web;
import com.thoughtworks.selenium.*;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.util.regex.Pattern;
public class TagSeleniumTest extends SeleneseTestCase {
@Before
public void setUp() throws Exception {
selenium = new DefaultSelenium("localhost", 4444,
"*chrome", "http://localhost:8080/");
selenium.start();
}
@Test
public void testAddTagAndVerify() throws Exception {
selenium.open("http://localhost:8080/coursemanagertest/➥
tags?form&lang=en_US"); selenium.type("_tag_id", "someTag1");
selenium.type("_description_id", "someDescription1");
selenium.click("//input[@id='proceed']");
selenium.waitForPageToLoad("30000");
assertTrue(selenium.isTextPresent("Show Tag"));
verifyEquals("someTag1", selenium.getText(➥
"//div[@id='_s_org_rooina_coursemanager_model_Tag_tag_tag_id']"));
verifyEquals("someDescription1", selenium.getText(➥
"//div[@id='_s_org_rooina_coursemanager_model➥
_Tag_description_description_id']"));
}
@After
public void tearDown() throws Exception {
selenium.stop();
}
}
This is a preferable test for most Java developers to use—for one thing, you can debug the code. Instead of using the XTML syntax, you can use real, honest-to-goodness compiled Java code to write your tests. You can also use [CTRL-SPACE] for code assistance in your favorite IDE. Now, that feels more like it! Let’s inspect this code a little more.
In the setUp() method, the test creates a Selenium test runner client engine, which looks for a running Selenium server at port 4444. See the sidebar in this chapter on running a Selenium server to configure this. You’ll also need to be running your web application; otherwise, the tests won’t run, because they can’t connect to the server.
Your test method contains calls to the selenium object, which communicates with the Selenium server to execute your tests. Now you can script tests to execute calls to your test web browser, typing data in fields and clicking various buttons. Key methods include the selenium.open() function, which browses to a page; selenium.isText-Present(), which verifies that text is present within the resulting web page; and the combination of selenium.click(), which presses a form button, and selenium .waitForPageToLoad(), which will pause for a period of time to make sure a page is loaded in response to that button click.
Why do you have to fire up the Selenium server?
It may seem strange that the HTML-based Selenium tests don’t require you to fire up your own server process, but the JUnit ones do. There’s a simple reason: the mvn selenium:selenese goal does launch and stop the Selenium server, but when running normal JUnit tests, the Maven surfire test runner plug-in isn’t aware of your Selenium test requirements.
You can configure Selenium, and even Jetty, to run your Selenium JUnit tests during the integration test phase of Maven, rather than the unit test phase. You can even start the Selenium server and Jetty web server when running your integration tests and execute the entire test suite automatically.
The WebDriver API
If you think starting a Selenium server in order to run tests seems complicated, and you’d like to try something more advanced, you can use the WebDriver API to write and execute Java Selenium tests in Selenium 2. This eliminates the need to fire up a Selenium server, and the API is more direct and simplified.
To use the WebDriver API, replace your Maven dependency on the Selenium Java client driver with this:
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.16.1</version>
<scope>test</scope>
</dependency>
With this API, you only need to boot the web server, not the Selenium driver. Web-Driver doesn’t need a Selenium remote control server instance.
Our sample coursemanager project in the chapter-09-testing folder here uses the Web-Driver API for two tests: ITCourseSelenium.java and ItTagSelenium.java, located in the web test directory.
Let’s take a quick look at the ItTagSelenium.java test to compare the API to the previous example:
public class ITTagSelenium {
public class ITTagSelenium {
private WebDriver webDriver;
@Before
public void setUp() throws Exception {
webDriver = new FirefoxDriver();
}
@Test
public void testCreateTag() throws Exception {
webDriver.get(
"http://localhost:8080/coursemanager/tags?form&lang=en_US");
webDriver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
webDriver.findElement(By.id("_name_id")).sendKeys("someTag1");
webDriver.findElement(By.id("_description_id"))
.sendKeys("someDescription1");
webDriver.findElement(By.id("proceed")).click();
Assert.assertEquals(true, 0 < webDriver.getPageSource()
.indexOf("Show Tag"));
}
@After
public void tearDown() throws Exception {
webDriver.close();
}
}
You can see some differences between the Selenium 1 API and the new WebDriver API used in Selenium 2. First, WebDriver has a more fluent, chained API, and second, the web drivers are created and used. No Selenium server needs to be configured to run WebDriver tests. To run this test, execute it in JUnit.
SEE IT IN ACTION The sample coursemanager project in the chapter-09-testing folder has been written using the WebDriver API, and uses the Maven failsafe plug-in to fire off integration tests after the project is packaged. We’ve also configured the project so that the Jetty web server is booted before integration tests fire and shut down after they’ve been run. To execute the tests, issue the mvn verify command.
You can find out more about the WebDriver at the Selenium project website, http://seleniumhq.org.
Final thoughts on web testing
There are a number of considerations we haven’t discussed, and testing via the web is an enormous topic. Keep in mind that the most difficult part of web testing is getting to the right level of detail. You can certainly write web tests that dig deeply into validation rules and test service logic. Many of those tests would be less brittle if built at the appropriate layer—for example, logic tests should be written as unit tests where possible because they won’t change if somebody makes adjustments to the user interface.