For Node, there are two types of automated testing: unit testing and acceptance testing. Unit testing tests code logic directly and is applicable to all types of applications. Acceptance testing, however, is an additional layer of testing most commonly used for web applications. This article discusses the Tobi and Soda frameworks for acceptance testing.
Authors: Mike Cantelon and TJ Holowaychuk
This article is based on “Node.js in Action“, to be published in May 2012. It is being reproduced here by permission from Manning Publications. Manning early access books and ebooks are sold exclusively through Manning. Visit the book’s page for more information.
Acceptance Testing
Acceptance testing, also called functional testing, tests outcome, not logic. So, once you’ve created a suite of unit tests for your project, acceptance testing provides an additional level of protection against bugs that unit testing might not have covered. Acceptance testing is similar, conceptually, to testing by end users following a list of things to test. Being automated, however, it’s fast and doesn’t require human labor. Acceptance testing also deals with complications created by client-side JavaScript behavior. If there’s a serious problem created by client-side JavaScript, server-side unit testing won’t catch it but thorough acceptance testing will. For example, your application may use client-side JavaScript form validation. Acceptance testing will ensure that your validation logic works, rejecting and accepting input appropriately. Or, for another example, you may have AJAX-driven administration functionality—such as the abilities to browse content to selected featured content for a website’s front page—that should only be available to authenticated users. To deal with this, you could write a test to ensure the AJAX request produces expected results when the user is logged in and write another test to make sure that those who aren’t authenticated can’t access this data.
In this article, you’ll learn how to use two acceptance testing frameworks: Tobi and Soda. While Soda provides the benefit of harnessing real browsers for acceptance testing, Tobi, which we’ll look at next, is easier to learn and get up and running.
Tobi
Tobi is an easy-to-use acceptance testing framework that emulates the browser and leverages should.js, offering access to its assertion capabilities. The framework leverages two third-party modules, jsdom and htmlparser, to simulate a web browser, allowing access to a virtual DOM.
In the world of client-side JavaScript development, web developers often use the JQuery library when they need to manipulate the DOM. JQuery can also be used on the server side, and Tobi’s use of JQuery minimizes the learning required to create tests with it. Tobi can test external sites over the network or can interface directly with Express/Connect applications.
Enter the following to install Tobi:
$ npm install tobi
The following test script is an example of using Tobi to test the login functionality of a website. The test attempts to create a todo item and then looks for it on the response page. If you run the script using Node and no exceptions are thrown, the test passes.
Listing 1 Testing a remote site’s login capability using Tobi
var tobi = require(‘tobi’)
, browser = tobi.createBrowser(3000, ‘127.0.0.1’); #1
browser.get(‘/’, function(res, $){ #2
$(‘form’)
.fill({ item: ‘Floss the cat’ }) #3
.submit(function(res, $) { #4
$(‘ul’)
.html()
.indexOf(‘Floss the cat’)
.should.not.equal(-1);
});
});
#1 Creates browser
#2 Gets todo form
#3 Fills in form
#4 Submits data
The script creates a simulated browser, uses it to perform an HTTP GET request for a login form, fills in the form’s name and password fields, and then submits the form. The script then checks the contents of a with the div class messages.
If the div contains the text “Login successful.” then the test passes. If you want to test an Express application you’ve built rather than a remote site, it’s similarly straightforward. Following is an example of a one-page Express site. Note that the last line populates the exports object with the application object. This allows Tobi to use require to access the application for testing.
Listing 2 Sample Express web application
var express = require(‘express’)#1 Sends HTML response
, app = express.createServer();
app.get(‘/about’, function(req, res) {
res.send( #1
+ ‘<html>‘
+ ‘<body>‘
+ ‘<div>‘
+ ‘<h1>About</h1>‘
+ ‘</div>‘
+ ‘</body>‘
+ ‘</html>‘
);
});
module.exports = app;
You can test the above application without even running it. The following Tobi test shows how you’d do this.
var tobi = require(‘tobi’)
, app = require(‘./app’)
, browser = tobi.createBrowser(app);
browser.get(‘/about’, function(res, $){
res.should.have.status(200);
$(‘div’).should.have.one(‘h1’, ‘About’);
app.close();
});
Tobi includes no test runner but can be used with unit testing frameworks such as Mocha or Nodeunit.
Soda
Soda takes a different approach to acceptance testing. While other Node acceptance testing frameworks simulate browsers, Soda remote-controls real browsers. Soda, as shown in figure 1, does this by sending instructions to Selenium Server (also known as Selenium RC) or the Sauce Labs Sauce Cloud on-demand testing service.
Figure 1 Soda is an acceptance testing framework that allows real browsers to be remote-controlled. Whether using Selenium RC or the Sauce Labs service, Soda provides an API that allows a Node to direct testing that takes into account the realities of different browser implementations.
Selenium Server will open browsers on the machine on which it’s installed, whereas Sauce Cloud will open virtual ones on a server somewhere on the Internet. Selenium Server and Sauce Cloud, rather than Soda, do the actual talking to the browsers, but they relay any requested info back to Soda. If you want to do a number of tests in parallel and not tax your own hardware, then using Sauce Cloud may be desirable.
To do testing with Soda, you need to install the Soda npm package and the Selenium Server (if you’re not using Sauce Labs). Enter the following to install Soda:
npm install soda
If your machine has Java installed, Selenium Server is painless to install. All you have to do is download a recent .jar file from the Selenium Downloads page. Once you’ve downloaded the file, you can run the server using a command similar to the following.
java -jar selenium-server-standalone-2.6.0.jar
Once the server is running, you can include the following code in a script to set up for running tests. In the call to createClient, the host and port indicate the host and port used to connect to the Selenium server. By default, these should be 127.0.0.1 and 4444, respectively. The url in the call to createClient specifies the base URL that should be opened in the browser for testing, and the browser specifies the browser to be used for testing.
var soda = require(‘soda’)
, assert = require(‘assert’);
var browser = soda.createClient({
host: ‘localhost’
, port: 4444
, url: ‘http://www.reddit.com’
, browser: ‘firefox’
});
In order to get feedback on what your testing script is doing, you may want to include the following code. This code prints each Selenium command as it’s attempted.
browser.on(‘command’, function(cmd, args){
console.log(‘undefined: undefined’, cmd, args.join(‘, ‘));
});
Next in your test script should be the tests themselves. Following is an sample test that attempts to log a user into Reddit and fails if the text “logout” isn’t present on the resulting page. Operations like clickAndWait are referred to by the Selenium community as Selenese and are documented online.
Listing 3 A Soda test allows you to enter Selenese to control the actions of a browser.
browser
.chain #1
.session() #2
.open(‘/’) #3
.type(‘user’, ‘mcantelon’) #4
.type(‘passwd’, ‘mahsecret’)
.clickAndWait(‘//button[@type=“submit”]’) #5
.assertTextPresent(‘logout’) #6
.testComplete() #7
.end(function(err){
if (err) throw err;
console.log(‘Done!’);
});
#1 Enables method chaining
#2 Starts Selenium session
#3 Opens URL
#4 Enters text into form field
#5 Clicks button and wait
#6 Makes sure text exists
#7 Marks test as complete
If you go the Sauce Cloud route, simply sign up for the service as the Sauce Labs website and change the code in your test script that returns browser to something like the following.
Listing 4 Using Soda to control a Sauce Cloud browser
var browser = soda.createSauceClient({
‘url’: ‘http://www.reddit.com/’
, ‘username’: ‘yourusername’ #1
, ‘access-key’: ‘youraccesskey’ #2
, ‘os’: ‘Windows 2003’ #3
, ‘browser’: ‘firefox’ #4
, ‘browser-version’: ‘3.6’ #5
, ‘name’: ‘This is an example test’
, ‘max-duration’: 300 #6
});
#1 Sauce Labs username
#2 Sauce Labs API key
#3 Desired operating system
#4 Desired browser type
#5 Desired browser version
#6 Makes test fail if it takes too long
And that’s all. You’ve now learned the fundamentals of a powerful testing method that can complement your unit tests and make your applications much more resistant to accidentally added bugs.
Summary
In the realm of acceptance testing, Tobi is a great place to start. Tobi is easy to set up and get started, and developers familiar with JQuery will be up and running easily. For those requiring acceptance testing that takes into account browser discrepancies, Soda may be worth running, but testing is slower and you must learn Selenese.