December 1999
Volume 2 • Number 1

Contents

Choosing a Tool to Automate Software Testing

Choosing which tool to buy is a project in its own right. Start by identifying the current problems in testing, and then evaluate different solutions, including the purchase of test automation tools. The cost justification for the tool should be written up as a business case. Organizational constraints on tool purchase must also be identified. The decision to purchase should be based on a business case for a realistic anticipated cost-benefit ratio. This article is based on Chapter 10 of the book Software Test Automation (Addison Wesley 1999).

Key words: business case, buy vs. build, testing tools, tool selection, tool vendors

by Mark Fewster and Dorothy Graham, Grove Consultants

INTRODUCTION: TOOL SELECTION AND IMPLEMENTATION

The tool selection process evaluates the many test automation tools available and selects one that is appropriate for the organization. The tool implementation process then ensures that the selected tool is used effectively throughout the organization. The tool selection and implementation process is shown in Figure 1. This article covers the left-hand side of the diagram only.

The person in charge of selecting a tool to be used by many people within an organization must approach the selection process in a formal and detailed way. If a tool will be used on an experimental basis with only a few people, then the tool selection process will be on a smaller scale, less detailed, and less formal. If a tool will be used by many people in the organization, then the tool selection process will be more rigorous. In both cases, the process will be the same. If conducting a formal evaluation, all of the ideas in this article will probably be used; if conducting an informal evaluation, a few useful tips may be picked up from this article.

This material is addressed because many of the people charged with selecting testing tools have never done a formal selection. There are books and courses that cover the topic of general tool selection and implementation processes in greater depth, including IEEE standards 1209 (IEEE 1992) and 1348 (IEEE 1995).