Acceptance testing at synyx – Part 4

In the last posts we set up our infrastructure to be able to aquire Browsers that run on a remote host and we created a selenium Grid infrastructure that is scalable and reboot safe.
Within this post I want to describe how to extend the grid with windows nodes to be able to test on our beloved Internet Explorer.

Adding windows nodes to the grid

To be able to run tests on Windows browsers I wanted to go a similar way as on the linux-machines: As soon as the machine boots the selenium-server should get started and registers itself to the selenium-hub (set up like mentioned before).
First I installed a current version of Java JRE and created a folder C:\selenium where I put all the needed files to (node-config, selenium-server jar, InternetExplorerDriver and so on.
Then I created a startup-script selenium-server.bat which basically executes the selenium-server process the way the linux-startscripts do and put it to C:\selenium\selenium-server.bat:

"C:\Program Files (x86)\Java\jre7"\bin\java
  -Dwebdriver.ie.driver=C:\selenium\IEDriverServer.exe
  -jar C:\selenium\selenium-server-standalone-2.28.0.jar
  -role node -nodeConfig C:\selenium\nodeconfig.json

The nodeconfig.json looks similar to the linux configurations except that the capabilities-section tells the hub there are Internet Explorers.Click here to see it. Currently we dont add Chromes and Firefoxes to the hub but this would be possible.
In order to run the script as a service I used Non-Sucking Service Manager which registers to Windows’ services and reduces the amount of clicking that has to be done. It also takes care of restarting the service if it hangs and so on.
To get this done in Win7 you have to open an “Adminstrator Commandline” by clicking Start, enter cmd into the search field and execute with CTRL+SHIFT+Return. Then nssm can be used to install the service:

C:\selenium\nssm.exe install selenium-server C:\selenium\selenium-server.bat

Service accessing the desktop

Because selenium-server needs to access the desktop you have to allow this by clicking your way though: Navigate through Controlpanel -> Services -> selenium-server -> Properties.
There go to the “Log On” tab and check [x] allow service to interact with desktop. Start the service or reboot the machine and you’re done. As soon as the service is started it should show up at the console of your selenium-hub node.

Problem 1: Uploading files

Unfortunately we experienced a problem with the described solution: As soon as one of our selenium tests included a file upload a strange popup showed up in Internet Exporer that prevented the tests from succeeding.
Actually selenium does really nice work when it comes to uploading files: If a path to a file is entered into an input-field WebDriver detects this and (in case of RemoteWebDriver) uploads it to a temporary file on the selenium-node. Then it “chooses” this file on the file-chooser in the browser. But somehow the corresponding dialog in Internet Explorer causes problems when the process is not running as a “real” user. This is probably because the service-user has no “home-directory” where the file-upload dialog could be initialized to. And this causes an error-popup which makes the test fail.
We fixed this by taking another approach: The service can be configured not to run as the service-user but as a given windows user. As soon as we used this option the popup was gone and the tests that used file uploads went back to green.

Problem 2: Performance

Now our Windows7 node has some strange performance issues which are probably related to the “accessing the desktop” feature. The tests run significantly slower (like factor 5 to 10 times) when the service is run as non-service user as described in the solution for the upload problem. This also is the case when the service runs with [x] allow service to interact with desktop AND noone is logged in to the VM by rdesktop.
Currently we are investigating this issue and trying to work on a solution. A workaround to this could be keeping the corresponding display session open somehow (e.g. by enabling VNC). As soon as we found a solution for this problem I’ll post an update.

Coming up next…

This post completes (at least for now ;-)) all the technical stuff. Next we will look at what an acceptance test is, how we define acceptance criteria and ultimately how we write and report these tests.