Wait in the Watir

Ten years ago, the choice for open source UI Automation consisted primarily of Watir and Selenium. Many companies who chose Watir did so based on the way it automatically waited for a page to finish loading. As websites have become increasingly dynamic, the very notion of a page “finishing loading” no longer makes sense. As such, Watir has fallen behind in how it synchronizes tests with the page and has failed to keep on top of what was once a major competitive advantage.

(The article’s title is a reference to my favorite song to dance to circa 2002)

The first and most basic step (and by no means an entirely sufficient one) to addressing this issue is to automatically allow some time for an element to become “ready” before any action is taken on it. That would typically mean polling for an element to be present and/or enabled. It is interesting to note that not only has Capybara had this feature from the beginning of the project, it is so confident in the usefulness of their implementation that for their 2.0 release they removed their wait_until method entirely.

The resistance to this approach in Watir is another example of the shift in focus for Watir in its transition to its webdriver implementation. There is a popular belief in the Selenium community that a prerequisite for being a good tester is a thorough understanding of how a website is implemented. This knowledge allows for testers to identify and deal with more issues than just the functionality of the website. Alister Scott gives an example of using intermittent failures to track down configuration and environment issues.

Too many companies in their pursuit of test coverage do not take the time to evaluate the purpose of each test added to a test suite. Consider what information is obtained from the test passing, what information is obtained from the test failing, and the value of that information relative to the costs of investigating a false positive or a false negative. Cost effective tests are focused and repeatable. If one expects to rely on periodically failing tests to find unrelated problems, then the application is missing the proper tests. Automated tests need to verify that the developer’s code does what was intended rather than failing due to things like staging environment limitations. Unless staging and production are exactly the same (which is very unlikely), any issues found aren’t ones that will affect customers or the company’s bottom line.

Automated tests need to focus on verifying the site’s functionality, not the site’s implementation. Understanding the intricacies of the DOM and the CSS transitions and the asynchronous javascript calls are not necessary to validate functionality. It is bad practice for developers to expect that their users to be almost as skilled and experienced as they are in order to use their software. Increasing synchronization lowers the barrier of entry to using Watir and increases a company’s return on investment by empowering less experienced employees to write less brittle and more maintainable tests.

Even for more experienced automators, better synchronization reduces the amount of time necessary to write the tests. There just is not a good reason to expect that the majority of time spent writing a test include things like determining what is the last element to load in the DOM before a modal shows up, or determine how many pixels wide a sidebar should be after a CSS transition before a button becomes enabled before it can be clicked. Both of these can be determined, and explicitly polled for in order to obtain synchronization, or the test automation library can do its best to accommodate the intention of the user in order to test the actual functionality of the site.

Also of note, neither the automatic waits being proposed, nor the implicit waits implemented in the webdriver spec are likely to be sufficient for testing many of today’s dynamic websites. Unlike automatic waits, these implicit waits do not play well with explicit waits (the timeouts can compound in unintended ways). Additionally, implicit waits happen every time an element is located, so querying for the presence of an element that is not there will time out, rather than immediately returning false. Since automatic waits only wait before an action is specified, querying for an element to exist or be present will always happen without waiting.

Automatic waiting is a completely backward compatible feature, such that tests passing without automatic waits will continue to pass with them. The one important consideration is code that is using the antipattern of coding by exception, otherwise known as using exceptions for flow control. Exceptions should indicate a real problem in the code, not merely one of the valid alternatives that determines what code to execute next. If test code uses this pattern and automatic waits are enabled, the tests will still pass, but the exception will only be rescued at the end of the default wait time, which can significantly increase the total test time. The solution would be to use the provided predicate methods for (exists?, present?, enabled?, etc).

Watir needs to follow Capybara’s example and make it easier for the vast majority of Watir users to more easily test the functionality of their web applications.