December 2001
Volume 4 • Number 1

Contents

Resource Reviews

SOFTWARE TOOL

Cost of Quality
The Harrington Group

QUALITY MANAGEMENT

The eProcess Edge: Creating Customer Value
and Business Wealth in the Internet Era
By Peter Keen and Mark McDonald

Practical Guide to Software Quality Management
By John W. Horch

Handbook of Software Quality Assurance, Third Edition
By G. Gordon Schulmeyer and James I. McManus, eds.

TESTING

Testing Object-Oriented Systems: Models, Patterns, and Tools
By Robert V. Binder

SOFTWARE PROCESSES

Software Engineering: A Practitioner’s Approach, Fifth Edition
By Roger S. Pressman

Managing Software Requirements: A Unified Approach
By Dean Leffingwell and Don Widrig

Quality Information and Knowledge Management
By Kuan-Tse Huang, Yang W. Lee, and Robert Y. Wang

 

SOFTWARE TOOL

Cost of Quality, version 3.2

The Harrington Group, Inc. Copyright© 1995 – 1997.

(CSQE Body of Knowledge: Software Project Management)

Reviewed by Barbara Burkholder

The Harrington Group was founded in 1991 with the objective of filling a need in the quality assurance software market. As a supplier of software tools for quality and enterprise management, Cost of Quality was the group’s first program released to address the lack of available tools for implementing quality systems. The result: an easy-to-use, well-documented utility designed to help users organize and implement a cost-of-quality (COQ) program. Although this application cannot be considered groundbreaking, it certainly has potential as a means for jump-starting the implementation of a COQ system.

Installation of this application is quick and painless. Instructions are clear, and the online overview provides tips for using the application and applying the data within the business enterprise. Included in the extensive online help is an Incredibly Quick Start guide that delivers just what its title implies. Also included with the software package are a user’s manual; a comprehensive, 27-page Microsoft Word document; and sample project data.

The predefined information in the system is geared toward manufacturing enterprises. The application, however, is highly customizable, making adaptation for any type of operation quite easy. Standard reports and graphs are available via the system’s wizard, and may be viewed online or printed. System requirements are Windows 95, 98, ME, or Windows 2000/NT 4.0, and 20-MB of hard-disk space. The system is network ready, and uses a Microsoft Access database. Purchase includes a 30-day buy-back guarantee, and technical support is free to registered users. The Harrington Group Web site is www.harrington-group.com.

One of the highlights of the Cost of Quality is the information defining quality, quality costs, and measurement included in its online help. The sections titled “Cost of Quality Primer,” “The Cost of Quality,” and “Total Quality Management Today” contain a wealth of information that can be used to educate individuals new to the world of quality management, or as a quick reference for seasoned quality management professionals. There is also a 12-point implementation plan suggested for implementing a COQ system.

Barbara Burkholder (bburkholder@elogex.com) earned her MBA from the McColl School of Business at Queens College, Charlotte, NC, and is a QAI certified quality analyst with 18 years of information technology experience in the financial services and transportation logistics industries. Burkholder is active in local software quality organizations, including the Charlotte Information Technology Quality Assurance Association (CITQAA) and the Charlotte Software Process Improvement Network. She serves on the board of the McColl School Graduate Business Alumni Association. Burkholder is currently employed as the quality assurance and testing manager for Elogex, Inc.

 

QUALITY MANAGEMENT

The eProcess Edge: Creating Customer Value
and Business Wealth in the Internet Era.

Peter Keen and Mark McDonald. 2000. Berkeley, Calif.: Osborne/McGraw-Hill. 300 pages. ISBN0-07-212626-4

(CSQE Body of Knowledge: Software Processes)

Reviewed by G. Timothy Surratt

As the title suggests, this is a business book not a technical book. The authors state in the introduction the book “is written for executives who want to understand how to take action and implement the solutions required to deliver eCommerce processes and capabilities to their customers and suppliers.” The book is based on consulting and research done by the authors with a number of companies implementing eCommerce. Their fundamental message is eCommerce is much more than a Web page or an online version of existing processes. Instead, success requires a rethinking of the strategies and processes used to deliver value to the customer.

The book is structured in three parts, each taking the fundamental points to an increasing level of detail. Part 1, “Defining the eProcess Edge,” outlines and supports the authors’ fundamental points: view everything from the customer’s perspective, build a value network to deliver what the customer wants, align the business and the business strategy to handle parts that are truly key to the business, and engage excellent partners to enhance your own capabilities. The authors cite four key strategies:

1. Embed process rules in the software interface. This transfers information and decision making to the customer and serves to both simplify and personalize the interaction.
2. Out-task processes and capabilities electronically. This means get the best provider of a service (UPS for logistics as an example) to perform that function.
3. In-source new capabilities electronically. Provide links to value-added services that augment your offering.
4. Be exceptional at handling exceptions. Basically, if something goes wrong, make sure someone responds to the customer.

The end of Part 1 reprises Keen’s previous book, The Process Edge, and contrasts the approach for eProcesses to standard total quality management (TQM)/reengineering approaches. The authors also introduce some to the analysis tools used in Part 3.

Part 2 is labeled “The Relationship Imperative.” These chapters introduce the idea of a value network and the need to define relationships, not just transactions, between the players in the network. The authors go on to introduce a classification of capabilities needed: identity—brand and differentiation; priority—establish an operational performance advantage; background—the ubiquitous support processes; and mandated—required by law. Evaluating each process as to the capability it provides and whether it is an asset or a liability allows one to formulate the strategy for that process.

Part 3, “Delivering eProcess Results,” takes one to the most detailed level of the book. Here the authors discuss the topics of embedding business rules in software, out-tasking, in-sourcing, and exception handling. They also return to the issue of managing a networked organization, suggesting that service-level agreements are the key rather than some hierarchical or network organization. True to their original purpose, the authors return to an executive agenda for eProcess at the end of the book.

Clearly this book is not going to help readers design a better Web site. It will help them design a better business. To the extent that readers are concerned with the latter, this book is worth reading.

G. Timothy Surratt (tsurratt@xnet.com) is the secretary of ASQ’s Software Division and a Senior member of ASQ. Formerly a quality manager with Lucent Technologies and Bell Labs, Surratt has more than 20 years of software engineering and quality management experience. Surratt is also the associate director of the ASQ Chicago Section Training Institute responsible for quality management. He is currently a consultant with The Sutton Group, LLC. He has a doctorate in chemical physics from Cal Tech, and is an ASQ certified quality manager, engineer, and auditor.

 

Practical Guide to Software Quality Management

John W. Horch. 1996. Boston: Artech House. 259 pages. ISBN 0-89006-865-8

(CSQE Body of Knowledge: Software Process, Software Project Management, Software Quality Assurance)

Reviewed by Gordon W. Skelton

Quality is a crucial concern for all those involved in software production. As has been stated, quality must be built in, not added on. Therefore, one must take a holistic approach to software quality. To do so it is essential that one not only understand what quality means, in respect to software, but also know how one goes about assuring that the end product meets the quality standards set by the development organization.

Horch begins by providing a high-level overview of the elements composing a software quality system (SQS). Because of the variety and number of elements contained in a quality system, Horch limits the discussion to an overview, requiring additional reading on each of the subtopics in order for one to fully implement a quality system. In chapter 1 he outlines the elements he believes should belong in an SQS: 1) standards, 2) reviewing, 3) testing, 4) defect analysis, 5) configuration management, 6) security, 7) education, and 8) vendor management. The first five elements are considered key components of an SQS while security, education, and vendor management are only associated quality issues.

Following a limited discussion on each of these topics, the author addresses the issues surrounding documentation and implementation of an SQS. Here, as with the major components of software quality management, Horch points to the need for integrating quality and the SQS into the entire software development life cycle. He stresses that one must produce documen-tation throughout the development life cycle, including proper documentation of the software quality system itself.

Chapter 9 provides an overview of necessary changes within an organization for implementing an SQS, the roles the organization must play, and how continuous improvement is essential to a successful SQS.

Horch’s book is an introductory text for people who have little or no experience with software quality programs. It focuses on the elements the author sees as being key components of a software quality system. Appendices A through I, drawn from IEEE standards, support the concepts presented in the book.

This book can be recommended as a beginning text for organizations and people who want to establish a software quality system or simply want to expand their basic knowledge of software quality management. It should not, however, be considered a desk reference for people who are already steeped in the creation and management of software quality systems. Horch’s book is suggested as a reference for those studying the IEEE software engineering body of knowledge (SWEBOK) and for ASQ’s software quality engineering exam (CSQE). In both cases the text can be a handy condensed reference.

Gordon Skelton (gwskelton@mvt.com) is vice president for information services for Mississippi Valley Title Insurance Company in Jackson, Miss. In addition, Skelton is on the faculty of the University of Mississippi, Jackson Engineering Graduate Program. He is an ASQ certified software quality engineer and is a member of the IEEE Computer Society, ACM, and AITP. Skelton’s professional areas of interest are software quality assurance, software engineering, process improvement, and software testing.

 

Handbook of Software Quality Assurance, Third Edition

G. Gordon Schulmeyer and James I. McManus, eds. 1999. Upper Saddle River, N. J.: Prentice Hall. 712 pages.

ISBN 0-13-010470-1

(CSQE Body of Knowledge: Software Quality Management)

Reviewed by Greg Jones

This book is a collection of chapters on different subjects written by different authors, and is more of an anthology than a single work. That approach has both benefits and drawbacks. A benefit is that each chapter can stand on its own. A drawback is that the changing authors can produce inconsistent voice and style. This inconsistency is encountered from time to time in this book, and in one case within a single chapter. As a result, although it is overall an excellent resource, the book does not come together as well as it could have. Since each chapter was written separately, this review will treat each one briefly in turn.

Chapter 1: Software Quality Assurance: Coming to Terms
This is a good discussion of terminology, with comparisons between different sources, allowing readers to select or blend a meaning suitable to their own situation.

Chapter 2: How Does Software Quality Assurance Fit In?
Chapter 2 continues with the theme of chapter 1 by providing context for how the term “software quality assurance” fits with various types of software, from operating systems through mission-critical, real-time, interactive, business, and legacy software. The book also discusses the Y2K problem as a type of legacy software and provides the Y2K status of governmental agencies (presumably this section will be removed in future editions). The implications of business process reengineering on software are also included as legacy activities.

Recommendations of activities and responsibilities of the software quality assurance (SQA) function are given for each type of software, along with general recommendations for all situations. For each type, a discussion of other related issues is also included. These recommendations are invaluable to a novice tasked with setting up a new software QA group. Without background in the role of SQA or the SQA process, along with some management experience, this assignment can be very difficult. Following these types of software is a discussion of the role of SQA with respect to other disciplines, such as software configuration management, software maintenance, and IV&V. Finally there is a summary table of all the major points.

Chapter 3: Software Quality Lessons from the Quality Experts
This chapter should be included in all software quality books. It is a discussion of how the messages of major “quality experts” can be applied to software. After a general discussion of the background for the quality movement, the chapter addresses each of these experts in detail. These experts include not only the American trio of J. M. Juran, W. Edwards Deming, and Phillip B. Crosby, but also the Japanese quartet of Ishikawa, Taguchi, Akao, and Shingo. The major benefit of this excellent chapter is the application of these classic concepts to software development, particularly Deming’s 14 points, seven deadly diseases, and obstacles. Also interesting are the quotes from books by these authors.

Chapter 4: Standardization of Software Quality Assurance—Where Is It All Going?
This chapter is not quite as helpful as the others. It is a survey of efforts toward standardization. It begins with a history of software-related standards, primarily military, and goes through commercial standards such as those sponsored by IEEE and ISO/IEC, ending up with the Software Engineering Institute’s Capability Maturity Model (CMM).

Unfortunately, much of this subject is treated in too much detail. The result is a daunting array of acronyms, numbers, and tables. In fact, one table is included that shows which concepts or topics appear in which of more than two dozen different standards. This chapter also has its own appendix, which lists tables of contents for several of the standards.
This chapter would have been more helpful if the emphasis on military standards was reduced and greater coverage was given to commercial standards. For example, a comparison of the Software Engineering Institute’s Capability Maturity Model (CMM) and ISO 12207 would have been helpful, or a discussion of current and planned progress toward reducing the standards quagmire. As it is, this chapter will be skipped over by many practitioners.

Chapter 5: Software Quality Program Organization
As the first of several that address personnel and staffing issues, chapter 5 deals with the setup and organization of a software quality program (SQP). Starting with concepts, the chapter moves through definitions and ends up discussing organizational issues such as responsibilities of recommended groups within the SQP. The SQP structure is related to a project management structure. Additional roles are defined and examples given for small and large organizations.

This chapter is useful for setting up an SQP function, for understanding roles found in other organizations, or as a model for assessing how well an existing function matches the recommendations. Its drawback is that it comes across as very wordy and theoretical. It would have helped to include examples of how real organizations implemented an SQP function, why they did it in a certain way, and how (upon what basis) they made the necessary decisions. This chapter also omits a discussion of how corporate culture can affect the organizational structure and the decisions necessary to implement something like the recommended program. It would also have been interesting and helpful to tie this chapter back to chapter 2, which describes how SQA fits into various environments.

Chapter 6: Personnel Requirements to Make Software Quality Assurance Work
Continuing with the staffing and management aspects of SQA, chapter 6 discusses the desired characteristics of SQA staff. This information would be helpful to management in conjunction with chapter 5 and chapter 2 in setting up a new SQA function in a project or an organization. For the individual professional, this chapter is less useful, except perhaps as a guide to what skills should be cultivated.

The chapter includes another discussion of organization design and structure, at a somewhat higher level than chapter 5. It then provides a recommended process for staffing a new SQA organization, treating the acquisition of staff as an implementation project in and of itself. This approach results in complicating the description of a process that a personnel or human resources department usually does anyway.

Following this are very helpful sections on characteristics of a good SQA engineer, training (including the idea of converting hardware engineers into SQA staff), the technique of rotating staff, and the usefulness of recent college graduates. The section on SQA employment requisitions is helpful when writing the job description to give to personnel. The final two sections on what to expect and on developing career paths are excellent aids to a software manager. Also included is an appendix providing typical job descriptions.

Overall, this chapter seems targeted toward a novice manager in software or SQA, who needs assistance in acquiring and developing staff. If one is not a manager, the chapter is marginally useful only as a guide to self-development.

Chapter 7: American Society for Quality Software Quality Engineer Certification Program
Continuing on the theme of self-development, chapter 7 describes the certified software quality engineer (CSQE) program offered by ASQ’s Software Division. The background for the certification, the reasons for obtaining it, and the overall process to follow are discussed. Also included is a detailed description of how the exam was developed and is maintained.
The most useful parts of this chapter are those on how to prepare for the exam, the topics in the body of knowledge covered by the exam, and the sample questions. Following this are a bibliography of recommended sources for the exam and a brief description of the recertification process.

Although this chapter is an excellent description of the CSQE, most of this information can be found in the CSQE application itself or on the ASQ Web site. It would have been more helpful to condense or eliminate the duplicate information and provide a wider discussion of certification itself. This discussion could have included descriptions of other software quality certifications, of the relative merits, histories, reputations, and applications of each, and of the differences in their bodies of knowledge. Also helpful would have been a discussion of the ongoing progress toward professionalism of the software engineering discipline, such as the efforts toward a professional engineer registration. All these would have made for a more balanced exploration of the certification question.

Chapter 8: The Cost of Software Quality
Completing the focus on management is a chapter on cost of quality. This is a good introduction to the concepts, yet spends a lot of text on a static activity analysis and assessment of an existing software process. Included are sample forms intended to be completed as data are collected regarding the process. This activity analysis is very useful as a baseline.

Unfortunately, many SQA professionals do not have the time to perform a complete analysis of the entire software process. Suggestions are needed for where to begin to collect cost-of-quality measurements, or at least for how to determine the most likely areas to evaluate, then how to proceed from there. Another barrier many face is how to convince management to support a cost-of-quality effort; recommendations based on experience would be helpful here as well.

The sections on potential misuse and productivity are important topics and are very helpful. These are good concepts with which to finish a chapter that is a good introduction, but could have also included some practical pointers on implementation.

Chapter 9: Inspections as an Up-Front Quality Technique
This is a very good chapter, containing a clear and organized description of inspections as a preventive technique. Enough textual detail is provided to allow a novice to understand and implement the beginnings of an inspection process, yet enough quantitative metric techniques are also included to assist more experienced readers. Importantly, examples of results from real companies are also included.

Chapter 10: Software Configuration Management—A Practical Look
Chapter 10 is another good chapter. It expands the idea of software configuration management from the usual scope of code only, to also include specifications and other documentation, storage media, and potentially hardware as well. A generous section on real-world considerations is provided, which offers numerous examples of charts and tables.

Chapter 11: The Pareto Principle Applied to Software Quality Assurance
After an introduction and historical background of the Pareto principle, this chapter is a series of examples of how the Pareto principle has been applied in real-world situations. At the end is a summary containing the steps for applying the principle, as given by Juran. If the reader already understands use of this technique, this chapter can seem repetitive. There is also some information in the summary about software tools that can assist in Pareto analysis; these should be updated or removed.

Chapter 12: Understanding the Capability Maturity Model (CMMsm) and the Role of SQA in Software Development Maturity
The chapter begins with a description of the CMMsm and each of its five levels, then focuses just on the software quality assurance (SQA) key process area (KPA) of level 2. The four goals of this KPA are stated, then translated into practices that would help meet the goals. A typical SQA plan is presented as a practical way to meet the intent of this KPA. Finally, the chapter discusses how the level 2 SQA activities should lay the groundwork for higher maturity levels, particularly software quality management at level 4. This is a good treatment and explanation of the SQA KPA, in terms that are practical and easy to understand.

Chapter 13: SEI CMMsm Level 5: Boeing Space Transportation Systems Software
This entire chapter is devoted to a description of how the Boeing Space Transportation Systems (STS) organization achieved an assessed CMMsm level 5 in 1996. The organization itself is described, then its history with process improvement. Also included is a large, very helpful section on “challenges.” These are barriers and problems that STS had to overcome, including both mistakes made and lessons learned. Most enlightening are the issues involving cultural change and how STS dealt with them. At the end of this chapter, the reader is given the bottom line--the results of the process improvements STS made, using both internal (for example, defect rate, productivity) and external (for example, customer satisfaction) data. After all, this is why process improvement is done, not just to achieve level 5. In addition, implications for future business are explored.

Even though a reader may not work for a huge aerospace defense contractor, the experience described in this chapter is very valuable and should be considered. There are surprisingly few Department of Defense (DoD) acronyms, jargon, and military-speak, making for a very readable and universally applicable chapter.

Chapter 14: Software Quality Assurance CASE Tools
This is a relatively short chapter discussing how CASE tools can be used to facilitate software quality assurance. The CASE concept is also placed within the larger context of a software engineering environment (SEE). General benefits of CASE tool usage are described, then specific applications to SQA. At the end of the chapter many sources and names of specific tools are provided, although some of the listed tools are not traditionally classified as CASE tools (such as statistical software). Also provided are lists of vendors and results of an Internet search. Due to the changing nature of software vendors and other organizations, these listings should be checked for changes in address, Web site, and even existence of the companies. Disclaimers are provided to this effect.

Unless one is looking specifically for these types of tools, this chapter is not very helpful, and readers would probably benefit from performing their own, up-to-date searches. This chapter does, however, offer a good place to start. It might be more informative if the section on applicability of tools were expanded to discuss in more detail the dangers and pitfalls of tool selection and use, including when, why, and how to select a tool.

Chapter 15: Software Quality Assurance Metrics
A discussion of software metrics frequently runs the risk of making the reader’s eyes glaze over due to the mathematics. Happily, this is not true of chapter 15. After a brief level-set of definitions, important concepts in measurement and metrics (including process metrics) are discussed without the need for formulas. Once a general metrics methodology has been presented, a section describes an overall metrics model, based on the work of which provides 11 factors for a quality software product and characteristics of those factors. Also discussed are 11 different quality factors that relate more to the software project and process rather than to the software product itself. Following this are descriptions of several real-world implementations of software metrics. At this point, the charts, graphs, and formulas start. At the conclusion of the chapter is a useful table that shows how different aspects of software metrics are manifested at increasing CMM levels. This table, however, only goes through level 3.

Chapter 16: Practical Applications of Software Quality Assurance to Mission-Critical Software
For readers not involved with mission-critical DoD software, chapter 16 can be skipped entirely. This chapter is very specific to this type of software; it would have been better if the concept of “mission critical” had been expanded to include other potentially life-threatening or safety-related software, such as nuclear plant control systems or medical equipment. In these settings the discipline of SQA does have some special concerns that should be addressed separately from the general-use commercial software discussed in the next chapter, and which are not fully explored in chapter 21.

As written, however, this chapter reads like a DoD software process manual, going through each development phase in great detail. It is also full of DoD acronyms (many of them denoting military standards) and examples written to comply with those standards, even down to the nomenclature format of test-case IDs. This chapter is not recommended for readers who have not spent a significant part of their professional life working on military software.

Chapter 17: Practical Applications of Software Quality Assurance to Commercial Software
This excellent chapter begins with an overview of the growth of commercial SQA out of military projects and how that origin resulted in organizational and cultural issues seen today in commercial software development. A general framework for a commercial SQA program is provided, beginning with its vision, mission, and objectives. Following the identification of example initiatives to start with, the V-Model (although not labeled as such) is conscientiously chosen as an example of a product life cycle. Staffing and roles are discussed, then some helpful tips are given based on experiences.

Following this is an extensive case study of a disguised software development company as it started up and improved upon an SQA program. The case study is extremely helpful, as it goes into just enough detail to allow the reader to compare the theoretical recommendations at the beginning of the chapter to what actually happened when those recommendations were implemented. This section, although excellent, could have benefited from a tie back to the information provided in chapters 2, 5, and 6.

Chapter 18: Effective Methods of Information Services Quality Assurance
This long and complex chapter begins rather disconcertingly by describing quality assurance as part of “data processing” and continues to use the term throughout the introduction. The use of such an outdated term puts a negative connotation in the mind of the reader from the outset. Furthermore, the term information services is used without really defining what that is. The implication seems to be that information services is the name of a department within a larger organization whose business is not software production. Interestingly, the chapter uses the term quality assurance throughout, rather than the more specific term software quality assurance used in the rest of the book. The implication seems to be that this chapter addresses more than just software quality; perhaps hardware quality as well?

After the short introduction, the chapter offers a lengthy and detailed discussion of a 1998 survey on “Information Services Quality Practices and Salary” conducted by the Quality Assurance Institute (QAI) at its conference in April of that year. The point of the detailed analysis seems to be a report on the state of SQA at the time. However, no conclusions are drawn from this material, and it does not seem to relate to the rest of the chapter.

After this lengthy digression, the author begins to discuss the QAI approach to quality management. Three strategic principles are offered: manage toward results, manage by processes, and manage with facts. For managing processes, a progressive, five-level framework is given; however, the first level of this framework is called “manage people.” Only at the second level does one “manage processes.” This framework seems so similar to the CMMsm that it would have been helpful if the author had compared the QAI model to the CMMsm and pointed out the differences and their reasons. In addition to the CMMsm-like model, QAI also identifies six categories of processes to manage, one of which is “people.”

Separate from the entangled process and people management, the next section addresses the “manage with facts” strategy and discusses measurement. Several sample lists of more-or-less measurable quality factors are provided, and analysis techniques outlined. In the last major section, the chapter concludes by discussing how to implement quality assurance in an organization.

This chapter seems to need division into three parts; the part covering the survey seems irrelevant and could be discarded. The middle part is most apropos of the chapter title, and the third part is another viewpoint of a topic covered elsewhere in the book (for example, chapters 6, 17, and parts of 2 and 5).

Chapter 19: Statistical Methods Applied to Software Quality Control
The chapter begins by defining quality control in terms of manufacturing, then applying it to software development. A software development process called “quality programming” (QP) is then described in detail. This process includes a modeling step that is structured in such a way that test designs and acceptance criteria can be determined using statistical methods on quantitative values derived from the model.

After the explanation of the QP approach, several experiences using it are described. These projects range from a DoD Air Defense System through scientific systems and DBMS software, in several languages, on many platforms, and through many generations of computation technology since 1980. Statistical results for some of these projects are provided. The chapter concludes with recommendations for solution of technical problems in project management, based on Deming’s 14 points.

As part of this book, chapter 19 seems almost out of place for most readers. The placement in the back of the book was a good decision. Most readers would probably not be interested in such a topic that might normally be seen in a research journal, but its inclusion here may meet the needs of the more advanced practitioners who pick up the book.

Chapter 20: Software Reliability Management
The chapter opens by providing a background to the measurement of software reliability based on a hardware model, in which hardware functions relatively consistently until it suddenly breaks. The author points out the need for different paradigm of managing software reliability; any faults that software has were there since the beginning. It is only a change in environmental conditions that cause the fault to manifest. The author describes a new way of looking at software through the three characteristics categories of primitives, flexibility, and graphics, and describes the advantages of this approach, specifically in the area of standardization. The author also provides guidance for the application, selection, and optimization of the specific measures used for gauging reliability. Use of the measures recommended in the IEEE 982 documents is encouraged. Standardization is achieved in that if one of these measures is selected, the IEEE 982 documents provide guidance to its consistent use.

In this approach, reliability management is achieved by continuously optimizing reliability throughout the development process, as reflected in the standard measures. Example matrices are provided for recording these standard measures used to determine reliability through typical life-cycle phases.

The chapter also has its own appendix, an excerpt from IEEE Guide 982.2 - 1989, describing the sample measures for reliable software.

Overall, although it starts out promising enough, the chapter ends up seeming to be an endorsement and recommendation for use of the IEEE standard. Also, the term reliability management has a connotation implying primarily the long-term operation and maintenance of software, not the development phases. What is missing from this chapter is information on how to manage the reliability of software when it is in production, at the time its customers need reliability the most; reliability of a system in development is not nearly as important as reliability in operation. Also, this chapter should have reiterated or supported the definition of reliability in chapter 1.

Chapter 21: Software Safety and Its Relation to Software Quality Assurance
This chapter contains some of the material that would have made chapter 16 more helpful. After an introduction and anecdotal descriptions of some accidents involving death or injury caused by faulty software, several terms are defined. Notably the term reliability is restated from chapter 1, even though it was not referenced in the previous chapter. Also in this introductory section, applicable software safety standards are discussed.

The author then discusses organizational issues related to software safety. Three organizational maturity levels and other considerations are proposed for a software safety program. The IEEE 1228-1994 standard for software safety plans is outlined and summarized in practical terms. The description in chapter 5 of organizational boundaries is presented as a way to address software safety issues. Finally, a technique is offered to analyze and mitigate hazards.

This short chapter deals with an emerging issue that deserves more visibility; it should have been expanded to the size of chapters 16 and 17. Alternately, it could have been the springboard for a chapter on software risk mitigation, contingency planning, and business continuity planning.

As it is, though, it provides a basic understanding of software safety and risk mitigation issues.

Greg Jones (greg.b.jones@bankofamerica.com) is a technology project consultant at Bank of America, specializing in software process improvement and the bank’s change management process. He was previously a quality assurance manager at a financial information company, and a programmer/requirements analyst at a large public utility. Jones received a master’s of engineering degree in computer science at North Carolina State University and a bachelor’s degree in nuclear engineering from Texas A&M University. He is an ASQ certified software quality engineer and quality improvement associate and a Quality Assurance Institute quality analyst. Jones is the founder and president of the Charlotte SPIN, past president of the Charlotte IT QA Association, and is the ASQ Division Deputy Regional Councilor for North Carolina.

 

TESTING

Testing Object-Oriented Systems: Models, Patterns and Tools

Robert V. Binder. 2000. Reading, Mass.: Addison-Wesley Longman, Inc. 1152 pages. ISBN 0-201-80938-9

(CSQE Body of Knowledge: Software Inspection, Testing, Verification, and Validation)

Reviewed by Pieter Botman

Exactly who tests object-oriented (OO) systems these days? “eXtreme” programmers? Systems engineers? OO analysts and software archi-tects? Are testers expected to be specialists, locked into performing black-box functional tests on those classes, packages, or systems that are “thrown over the wall” at them?

There may be a new generation of software professionals out there—those not specifically dedicated to testing, and who have spent most of their professional careers using OO techniques and technologies. Is there a foundation of testing theory and guidelines for practice related to the testing of OO systems?

First, the good news: OO systems and techniques are now accepted as part of modern software engineering practice, and practical knowledge and theory relating to this area are now being compiled and published. Now, the bad news: object orientation does not in itself lessen the need for engineering rigor—there is more to know, and more to be aware of in testing OO systems. Inheritance, polymorphism, late binding, and encapsulation all affect strategy and tactics when planning the testing of OO systems.

Robert Binder has credibility in the field. He has already written several books and articles on the testing of OO systems. The book is written for “anyone who wants to improve the dependability of OO systems.” But this is not a book written for newcomers; it assumes readers have experience with object modeling, OO design, and OO programming.

The book begins with a series of interesting chapters entitled “Preliminaries.” Binder immediately challenges readers to consider the test cases needed to adequately test a class representing a triangle, for which the source-code declarations are given. It is a safe bet that not many readers come up with the list of test cases (65 of them in all) supplied by the author. Shocked out of complacency at the beginning, the reader is prepared for something more rigorous than a few flow diagrams.

In Part 2, Binder devotes a substantial portion of the book to establishing (or reestablishing) a sound theoretical basis for testing, including distinctions of scope (unit, integration, system) and intent (fault-directed and conformance). In Part 2 he introduces model-based testing, and compares informal approaches (“cartoons”) to more formal approaches in generating testable models. He covers combinational models and state machines. But he does not segregate the OO considerations from these discussions—in each of these areas, he touches on practical aspects of the topic related to OOA/D. For example, he compares a genuine, usable state model to that offered by UML and some other OOA/OOD definitions:

Although OOA/D cartoons may be useful for preliminary analysis and design, they are inadequate for testing.
…
Most OOA/D definitions of state are untestable: they cannot support an executable determination of what state an object is in.

Part 3 of the book covers results-oriented testing of classes, subsystems, and application systems. This material is even more practical and closer to the everyday artifacts of classes, packages, and components. There are many practical, unambiguous guidelines for planning, selecting, design, and management of tests here. He relates these guidelines as patterns, and incorporates well-known standards and concepts such as traceability matrices, IEEE 829 test plans, adequacy of code coverage, data flow coverage, path sensitization, and domain/partition testing. This section covers testing patterns for classes, components, subsystems, and integration testing in general. This is followed by chapters that address testing at the application and systems level, and regression testing.

Binder makes the point that with OO systems, effective testing involves a view of more than simple units of source code (or classes). Class relationships, responsibilities, and other design aspects all influence test design. It is easy to see why UML is introduced. Binder considers it important to devote an entire chapter to “A Tester’s Guide to UML,” which relates elements of the various UML diagrams to testing. He also cautions that while UML can be useful in describing requirements, object/class relationships, and interactions there are some weaknesses in the representation.

Part 4, entitled “Tools,” includes chapters on test automation, a practical look at assertions, test oracles, and test harness design—all with a level of detail going far beyond mere introduction.

The appendix describes one implementation of a COTS tool suite integrated to form an environment to support automated testing of a large C++ client-server application. Binder was involved in setting up this environment for a customer of his but is careful to present balanced and objective criteria for a fairly sophisticated test automation environment. And at 44 pages, the glossary is larger than glossaries in some books covering the entire field of software engineering.

While this book is very broad in scope, it is deep enough to be considered a reference in some respects. Binder is thorough and does not skim lightly over topics. He strives for clarity and true software engineering rigor in his explanations and his practical testing guidelines. But his style is not overly formal, and the book is peppered with diagrams, charts, and small snippets of code (often Java), making it easy to follow.

Despite the fact that the primary subject of this book is testing and verification, it offers insights about principles of good OO design and construction. Good designers need to think ahead, and “design for test,” while testers must comprehend a great deal of an OO system’s design in order to design worthwhile test suites.

Roger Pressman in Software Engineering: A Practioner’s Approach (McGraw-Hill 2001) admits that “the literature for OO testing is relatively sparse,” and calls this work “the most comprehensive treatment of the subject published to date.” This book is certain to be valuable and useful at many levels to both testing specialists and OO designers.

Pieter Botman (p.botman@ieee.org) is a professional engineer registered in the Province of British Columbia. With more than 20 years of software engineering experience, he is currently an independent consultant, assisting companies in the areas of software process assessment/improvement, project management, quality management, and product management.

 

SOFTWARE PROCESSES

Software Engineering: A Practitioner’s Approach, Fifth Edition

Roger S. Pressman. 2001. New York: McGraw-Hill. 854 pages. ISBN 00736555783

(CSQE Body of Knowledge: All)

Reviewed by Milt Boyd

The 20th anniversary, or fifth edition, of this book has 31 chapters organized into five parts. These parts are: “The Product and the Process,” “Managing Software Projects,” “Conventional Methods for Software Engineering,” “Object Oriented Software Engineering,” and “Advanced Topics in Software Engineering.” Compared to the fourth edition, the look and layout have been improved, with a positive impact on readability. Five chapters are new material, or extensive updates of prior material, and the others are revised.

Chapters have a “quick look” that summarizes these key ideas: What is it? Who does it? Why is it important? What are the steps? What is the work product? And, how do I ensure that I’ve done it right?

Each chapter starts with a list of key concepts with their page numbers, and ends with references, problems, and further readings. The margins of the text contain advice, quotes, questions, key points, Web references, and cross-references.

In addition, McGraw-Hill hosts a Web site for the text (www.mhhe.com/engcs/compsci/pressman), with resources for students, instructors, and profes-sionals. These resources include detailed checklists, outlines of documents, slides and guides, examples, and supplementary content. Although it is impossible to examine all 856 pages in detail, the treatment of all topics in the book is impressive.

Software quality assurance is the topic of the 32 pages in chapter 8, in Part 2 “Managing Software Projects.” Other chapters in Part 2 are Project Management Concepts, Software Process and Project Metrics, Software Project Planning, Risk Analysis and Management, Project Scheduling and Tracking, and Software Configuration Management. This provides firm context for SQA as a management practice, and an integral process in software development projects. Pressman prefers “an umbrella activity that is applied throughout the software process.” The book provides good information on all areas of the CSQE body of knowledge and very good information about software engineering processes, program and project management, software metrics, and configuration management. For practitioners, for those training practitioners, or for those in training to become practitioners, this book provides a well-organized and useful guide to the field, with good treatment of SQA, and is highly recommended.

Milt Boyd (MiltBoyd@aol.com) is a member of ASQ’s Software, Reliability, and Quality Management Divisions. He is an ASQ certified quality manager and is certificated by IRCA as a lead auditor of quality management systems. He is currently employed as a system engineer by Avidyne Co. of Massachusetts.

 

Managing Software Requirements: A Unified Approach

Dean Leffingwell and Don Widrig. 2000. Reading, Mass.: Addison-Wesley Longman. 491 pages. ISBN 0-201-61593-2

(CSQE Body of Knowledge: Software Processes)

Reviewed by Norm Moreau

The authors stated that the objective of their book is to provide a rational approach for building systems that customers want and for preventing software systems projects from being over budget and behind schedule. The book would be beneficial to large and small companies interested in getting requirements right before development is in full swing and before managing requirements throughout the project life cycle.

Given the authors’ background, the book does have a hardware/software slant. To that end, there is a great deal of emphasis on a systems approach to requirements management, which is actually very refreshing. Telecommunications and other organizations driven by hardware needs will find this book very useful. Organizations with strictly business applications will also find the systems approach helpful, but will have to make parallels to their own industries.

Another important aspect of this book is that it is based on the Unified Modeling Language (UML). From this reviewers perspective the model restriction has not taken away from the value of content. The book takes a pragmatic approach built around six requisite team skills for effective requirements management. The premise is that effective requirements management can only be accomplished through an effective software team.

The step-by-step approach, case studies, and appendices presented in this book provide sufficient detail for an organization or software team to implement a systematic and structured approach to eliciting, organizing, and documenting the requirements of a system. This book is an excellent resource for training teams on requirements management principles and techniques. The home lighting automation system case study that is built upon throughout the book makes it easy for everyone to understand how each team skill is applied.

Following is a brief summary of each team skill.

Team skill 1: Analyzing the problem uses a five-step problem analysis process to understand the problem to be solved before application development begins. As the elements of this skill evolves the value of a systems approach becomes very clear.

Team skill 2: Understanding user needs, which invokes three well known “syndromes” to help better understand the challenges of identify requirements and provides a context for the elicitation techniques presented. The syndromes used are the “yes, but” reaction of human nature when shown what software can do, the never-ending search for “undiscovered ruins (or requirements),” and the communication gap often experienced between the “user and the developer.” Seven techniques are presented to help teams overcome the difficulties of these “syndromes.” An example of the detail the authors go to to provide valuable and useful information is demonstrated in this team skill where the use of simple soft skills techniques such as brainstorming and fishbone diagramming really help readers understand how to manage requirements definition meetings.

Team skill 3: Defining the system moves from understanding the needs of the user to starting to define the solution. Movement from the problem domain to the solution domain begins with a description of the documents needed to help organize the requirements of complex hardware and software systems. A great deal of emphasis is placed on the vision document and the champion responsible for it, as a crucial document that every project should have.

Team skill 4: Managing scope describes several tools for setting project priorities. As expected the team size increases and the authors do a nice job in addressing the players and their roles in managing scope. Included are project management basics and negotiation skills for engaging customers to help with the scope management problem.

Team skill 5: Refining the system definition presents a logical construct for documenting requirements in an SRS package that contains use cases, documents, database forms, and other requirements repositories that help constitute clearly defined requirements. Quality measures are presented for assessing the quality of the SRS package.

Team skill 6: Building the right system the authors suggest, is best done by using requirements and use cases to drive the implementation architecture and design. This is then confirmed through verification and validation. Methods for performing hazard and risk analysis are presented to determine what can go wrong and how to avoid or mitigate them. This is considered a plus for organizations that have to deal with systems that perform health and safety concerns. Much emphasis is placed on maintaining requirements traceability throughout the life cycle. Several traceability tools and techniques are presented. Other life-cycle issues, including managing change, are presented and their relationship to requirements management are described. The only aspect of this book that did not add much value was the appendices on CMM and ISO. It would have been more helpful if the authors had looked at the CMMI and ISO 9001-2000 (even if they had used a draft version).

Overall, this book is a great reference for requirements development and managing requirements throughout the project life cycle. In spite of the systems and tool-specific slant, this would be an invaluable handbook for the requirements practitioner or anyone involved in the requirements business.

Norman P. Moreau (nmoreau@erols.com) is the president and a senior management consultant for Theseus Professional Services, LLC in Maryland. He has been a software quality professional for 15 years and supports clients with process improvement, and ISO 9001-2000 and SEI-CMM implementation. Moreau currently serves as a deputy region 5 councilor, is a Senior member of ASQ, a registered professional engineer, a certified software quality engineer and certified quality auditor, and a registered ISO lead auditor.

 

Quality Information and Knowledge Management

Kuan-Tse Huang, Yang W. Lee, and Richard Y. Wang. 2001. Englewood Cliffs, N. J.: Prentice Hall PTR. 209 pages.

ISBN 0-13-010141-9.

(CSQE Body of Knowledge: Software Processes (process and technology change management); Project Management)

Reviewed by J. David Blaine

The nine chapters in Quality Information and Knowledge present a compelling case that organizations should treat information and knowledge as fundamental assets. Not an entirely revolutionary concept, but this is not your father’s quality information book.
The three authors are industry leaders in intellectual capital management. They present valuable guidelines and goals for improving an organization’s treatment of information. A treatment that can make the difference between excelling at meeting your customer expectations or, perhaps, going out of business. The book can add value to a software quality engineer (SQE) who is involved in assessing, improving or using information within an organization.

This book asserts two propositions:

1. Organizations must create a reservoir of high-quality information.
2. Organizations must create a wealth of organizational knowledge.

The topics in this book include:

• Creating an environment for creating and sharing knowledge
• Transforming informal, subjective understanding into explicit knowledge
• Translating organizational experience and insights into quality information
• Creating solutions using intellectual capital, knowledge reuse, and data mining
• Defining responsibilities for a new organizational role: Information product manager
• Getting the most value from one’s intranet and other corporate information systems

Throughout the book examples from real companies are used (the names have been changed).

The authors use an analogy to manufacturing to explain the context and process of the information life cycle. The customers of information in an organization are many and varied, just as in manufacturing.

As manufacturing uses the Deming total quality management (TQM) cycle to improve quality, improving the quality of information uses a total data quality management (TDQM) cycle: define, measure, analyze and improve. The authors note that just as product quality has many dimensions, so does data quality, including: accuracy, completeness, consistency, and timeliness. Although there is no general agreement on information quality dimensions in the literature, the authors’ definitions are thoughtful and can serve the needs of an SQE involved in knowledge engineering efforts.

An important aspect of the TDQM cycle for improving the quality of an organization’s information is a new role: the information product manager (IPM). The authors list the attributes that this IPM should have and the activities that the IPM carries out. Throughout the rest of the book the authors refer to an IPM as the person conducting information quality improvement efforts.

The authors list four principles for managing information as a product, they are:

1. Companies must understand their information needs. This is easy to say but difficult to do. The authors provide some helpful guidelines using the experiences of real companies that were the subject of the research for the book.
2. Companies must manage information as the product of a well-defined information production process. This is, indeed, the central theme of this book. The authors contrast this with managing information as a by-product and show the pitfalls of that approach.
3. Companies must manage the life cycle of their information products.
4. Companies should appoint an information product manager to guide their information processes and resulting products. Any new technology or culture change needs a champion and senior management support.

The chapter concludes with five clear tasks that a company can do to establish an information quality program.

Chapter 3 presents three approaches to getting higher quality information in organizations. The authors present several definitions of deficiencies in the quality of data. The deficiencies are defined as conflicts between the way an information system represents real world data vs. how users perceive the real world and the data in the information system. The authors conclude by offering four dimensions or metrics about information quality: 1) complete; 2) unambiguous; 3) meaningful; and 4) correct.

Given these four dimensions the authors explain how an information system could be deficient in these areas from user perspectives. This chapter gives the reader a basis for the understanding the information product problems and solutions presented in the remainder of the book.

The measurement chapter presents the traditional approach to software metrics as applied to IQ. The motivation for measurement and analysis is to effectively manage. The authors point out that creating a precise definition of information quality, in one’s own organization, is an essential first step. Next, the relevant processes must be identified. This chapter presents a TQM-based approach to metrics development but does not stop there. An interesting methodology, including a tool is presented that IQ engineers can use in their own organizations. The tool and underlying methodology were developed as a result of total data quality management research at MIT.

This reviewer found the method and tool particularly intriguing. However, the tool does not seem to be publicly available, and the authors provide a resource (or link) to the tool. One somewhat irritating feature of this chapter: the authors provide a blank survey form used for assessing an organization’s information quality but copyrighted it and explicitly forbade its use without permission. This reviewer, and perhaps other SQEs, would like to have tools readers can use.

Chapter 5 provides guidelines on how to create and maintain corporate knowledge. In a sidebar the authors tell the story of a customer placing a phone order for boots for his daughter from a store she frequents. The store, somehow, remembers his daughter’s shoe size and recommends the appropriate size for this model, an illustration of the value of organizational knowledge.

Creating and maintaining corporate knowledge is an important factor in today’s economy, especially in high-technology companies. The shoe store in the sidebar is contrasted with a quick turn around eyeglass shop in which a customer must return newly made glasses for rework. The authors offer the phrase “organizational Alzheimer’s” to connote this symptom. Readers of this review may reflect on their own experiences with vendors, or their employers, for instances of this symptom.

This chapter defines organizational (corporate) knowledge and describes the important links between individuals’ knowledge and organizational-level knowledge. The authors define three modes of knowledge that SQEs should be able to relate to and be able to use in their own functions:

Know-what pertains to factual knowledge, at the individual, group, and corporate level

Know-how pertains to work practices and procedures

Know-why pertains the underlying axioms, reasons, and assumptions of work practices

The chapter then provides guidance in assessing organizational knowledge, provides a motivation that SQEs could use in their own organization to help obtain buy-in for initiating an information quality improvement effort and, finally, describes three high-level processes for creating organizational knowledge. This chapter also concludes with a copyrighted blank survey that readers can view but not use.

“Knowledge management fundamentally changes how a firm conducts business.” This direct quote underscores one of the main theses of this book. When a company treats knowledge as a corporate asset (as also suggested in Software as Capitol by Howard Baetjer) then that company is on the road to success. Ten strategies, which have been successfully used by organizations, are given in this chapter. Although they are somewhat high level, SQEs can uses these principles in their efforts to engender a culture change toward IQ. The strategies are:

1. Establish a knowledge management methodology. A bit more detail than the well worn “plan the work, work the plan” adage. The authors present eight components of this strategy, taken from IBM’s intellectual capital management method: a vision, processes, competency community (workers fluent in a particular area), technologies, and incentives. SQEs may recognize these components as derivatives of the SEI’s triad: people, process, and technology framework.
2. Designate a point person. The authors point out, rightly, that adoption of a new technology or cultural change needs a champion.
3. Empower knowledge workers. Before readers suffer gastrointestinal distress from this tiresome phrase, the authors do present some helpful ideas for ensuring that all individuals are part of the IQ program.
4. Manage customer-centric knowledge. All quality engineers will recognize the need to delight the customer. The authors apply this notion to the IQ program by listing five easy steps: 1) define customers; 2) understand customer changing needs; 3) identify knowledge that differentiates customers; 4) integrate information across product lines; and, 5) link all customer-related activities to a common database.
5. Manage core competencies. This refers to corporate level competency, not individuals.
6. Foster collaboration and innovation. This is a tall order. However, the authors provide some pragmatic guidance for organizations to accomplish this.
7. Learn from best practices. This also is a page from well-known quality principles. The authors note that benchmarking and internal projects are sources for best practices.
8. Extend knowledge sourcing. Retrieve information from multiple sources, add value, and deliver it to customers.
9. Interconnect communities of expertise.
10. Report the measured value of knowledge assets. The authors use Skandia as an example of a company that has integrated knowledge value into its bottom line. The company supplements its annual report with the value of its intellectual capital.

Moving targets are hard to hit. What can a company do to keep satisfying its information customers when the environment, technology, and the customers themselves keep changing?

The authors offer a solution borrowed from objected-oriented software engineering: create solutions from reusable solutions and knowledge assets. They advise organizations to establish a knowledge reuse process and an enterprisewide knowledge structure.

Chapter 7 provides interesting definitions of intellectual capital, intellectual asset, and solution. The distinction between these is central to the development of an information quality program. Intellectual capital is both the inventory of knowledge-based assets and the capacity to acquire and quickly assimilate new learning.

An intellectual asset is a group of knowledge items that are reusable and that have, in the past, added value to the organization. A solution is an integration and packaging of assets, products, and services for delivery to solve a specific business problem

The authors explain how to harvest assets for reuse by these steps:

• Ensure the understanding of the principles of intellectual asset reuse
• Decide how to measure the results of this reuse
• Understand how to implement intellectual asset reuse
• Create and implement a vision on how to become a mature intellectual asset reuse company

The authors give several helpful examples. Among the examples is IBM, which implements intellectual asset management in one way by having a “go to” person for the each of the many technologies and expertise areas embraced by IBM. Through personal experience while employed at IBM this reviewer saw the value in this approach. A small IBM office experiencing problems in product design was able to mine the expertise database and gather a set of experts within a few days. Problem solved.

The knowledge asset reuse process presented by the authors is strikingly similar to the Experience Factory paradigm published by the Software Engineering Laboratory. The experience factory, and the authors reuse process, emphasize the notion of gathering lessons learned from a project and “packaging” the lessons for use by the next project
A knowledge asset development process is described in this chapter. An important part of the process is the notion of an “owner” of the asset—an asset without an owner will die on the vine. Other roles of the process are also described.

An enterprise knowledge structure is described by showing an entity relationship between problem solving and knowledge architecture. This diagram can help anyone charged with the responsibility to setup a knowledge asset database.

This chapter concludes with the authors’ definition of data vs. knowledge. Although not quite in sync with the traditional schema of data -> information -> knowledge -> wisdom, the authors’ definitions and underlying concepts are helpful. For this book “wisdom” is replaced by “business success.”

Chapter 8 presents several technologies that can be deployed to create a corporate knowledge infrastructure. Most of the examples are from early adopters of the knowledge asset paradigm promulgated by the authors. The examples are interesting and can be helpful to SQEs involved in an IQ effort.

Particularly interesting is the “knowledge café” for team collaboration. This is a Lotus Notes application developed at IBM. A view of the user interface is given and its simple layout and the navigation functions are very helpful.

The book concludes with a chapter that gives motivations and justifications for adopting the information quality viewpoint that knowledge can and should be managed as an asset by organizations.

J. David Blaine (jblaine@san.rr.com) is a CSQE and a project management professional. He has been in the software engineering field for 24 years and holds a bachelor’s degree and a master’s degree in math/computer science and a master’s degree in electrical and computer engineering. He works for Ensemble Communications. Blaine was a software developer and project manager prior to his most recent role in software quality and process improvement.