Sunday, April 11, 2010

What features does a graphical test runner need?

As part of my PyUnit investigation, I wanted to look at a graphical test runner. While unittestgui.py is not available as part of PyDev, it does ship as part of the 1.4.1 version of the PyUnit download. Here's a screenshot:


This tool has the following features:
  • allows input of the test suite name
  • re-reads the test suite from disk every time you click the Start button
  • displays a progressive count of all tests run, failures, errors and remaining tests
  • a progress bar that changes from green to red on the first failure
  • a clickable list of failures providing details
  • the status bar shows the name of the currently running test
Interestingly, while there is no way in the GUI to stop a test in progress, there are commented out lines in the code that try to implement this.

From my reading, it seems like people don't think much of this test runner, and there are other GUI runners out there, but before looking at them, it's interesting to see what other features I'd like to find in one.

In no particular order:
  • abort and pause/resume - sometimes if things are going badly, it's not necessary for all the test to run
  • elapsed time and average time per test so far and total time when done
  • export results - with various format options, including XML and CSV
  • logging
  • graphical display of test suite structure/contents with the ability to select part of test suite to run
  • ability to see which tests have run, which tests will run soon, etc
  • optional stop-on-error

0 comments:

Post a Comment

Note: Only a member of this blog may post a comment.