October 2002
Volume 9 • Number 4

Contents

Executive Briefs

Quality Management in Education: Building Excellence and Equity in Student Performance

Jacqueline S. Goldberg, North Broward County Hospital District, and Bryan R. Cole, Texas A&M University.

This research is based on a study of Brazosport ISD, an exemplary school district in Texas that used a quality management approach that resulted in greater equity and higher student performance. Faced with deteriorating test scores in several schools, particularly those with a high population of economically disadvantaged students, this district elected to apply the philosophy, tools, and methods of quality management as a means to raise student achievement through system alignment and improvement of instructional processes.

The district focused on a teaching strategy being used by one of its teachers. The strategy, called the eight-step process, is a common-sense approach that is similar to the plan-do-check-act model. The strategy involves:

1. Data disaggregation
2. Developing timelines for the teaching process
3. Instructional focus
4. Assessment
5. Tutorials
6. Enrichment
7. Maintenance of skills and knowledge
8. Monitoring continued improvement

After two years of applying the method, the school being studied received a high rating from the state. The quality improvement framework was then deployed throughout all of Brazosport ISD’s schools. Teachers were held accountable for all of their students. If they acted as if they believed all students could learn, and systems were put in place to support this, then student achievement could be improved.

In addition to the emphasis on using disaggregated data to ensure teaching and learning processes were designed to meet individual learning needs, other processes were redesigned and improved, such as curriculum alignment, staff development, support services, and the instructional schedule. Brazosport ISD also possessed the leadership that is necessary to make such a transformation. Superintendent Gerald Anderson was involved, committed, and able to motivate people.

In validation of the success of its new methodology and the results achieved, Brazosport ISD won the Texas Quality Award in 1998 and received a site visit for the Malcolm Baldrige National Quality Award in 1999, the only school district to date to achieve both honors.

 

Development and Validation of a Perceived Business Quality Measurement Instrument

Juan Carlos Bou Llusar and Cesar Camison Zornoza, Universitat Jaume I.

In this article the authors propose a methodology for the construction and validation of a measurement instrument for perceived business quality (PBQ). This instrument will be applicable to the study of the external effects of quality in any kind of business. First, the authors suggest the development of the PBQ concept, and next, based on the methods of service quality measurement, they develop and validate a measurement instrument through the use of the structural equations models methodology.

A customer’s perceptions of the quality of a company include all elements that are susceptible to being perceived and evaluated by the customer, including price, reputation, image, and so on. Rust and Oliver (1994) believe there are three dimensions or basic components of PBQ: perceived product quality, perceived service quality, and business orientation toward quality (either external or internal).

Using structural equations models methodology, the authors developed a measurement instrument. To evaluate validity and reliability, they performed the following tests:

• Item analysis
• Evaluation of the proposed measurement model adjustment to the data collected by the scale application
• Assessment of reliability and validity of estimated parameters
• Assessment of convergent and discriminant validity

Results of this study show that customers differentiate between product quality, service quality, and business orientation toward quality. This differentiation has significant implications for quality improvement. Additionally, the results obtained in the application of the PBQ measurement instrument show the existence of a high reliability and construct validity. Overall, the results show that the PBQ measurement instrument meets the main requirements for measurement instruments in social sciences, and can be used to evaluate perceived quality in all business types. The PBQ measurement instrument also allows managers to focus on improving dimensions indicated as being very important to their customers.

 

Estimating Interrater Reliability of Examiner Scoring for a State Quality Award

Garry D. Coleman, University of Tennessee, Eileen M. Van Aken, Virginia Tech, and Jianming Shen, Lennox Industries.

Examiner scores for two years of a state quality award were analyzed by sector to estimate interrater reliability, a desirable characteristic that is always present to some degree. For this study, interrater reliability was defined as the similarity of the scores among examiners when scoring the same applicants on a single category. There are several reasons for estimating interrater reliability:

1. To create visibility for the variability that exists among examiners’ scores
2. To add to the public database on examiner score variability
3. To provide information to help those conducting self-assessments to understand why they should expect differences between self-assessment scores and the scores of award examiners
4. To explore the issue of interchangeability of examiners

The intraclass correlation coefficients (ICC), ICC(1,1) and ICC(2,1) were chosen as the statistics to estimate reliability to enable the researchers to generalize results from specific examiner teams to the larger pool of examiners, thus providing an assessment of the overall scoring effectiveness in the state quality award. The ICC was chosen because it offered the precision, comprehensiveness, and flexibility needed to deal with the complexity of reliability assessment.

Participants in the study were 1998 and 1999 examiners for a state quality award. Historically, the state quality award examiners were assigned to one of four sectors: private sector manufacturing, private sector service, public sector service, and public sector local agencies. Recently, however, examiners have sometimes been asked to score applications in other sectors.

In most cases where adequate data were available, the examiners displayed low to moderate interrater reliability. In cases with small data sets, the results consistently showed low interrater reliability. When the small data set was further aggravated by an incomplete data structure, interrater reliability coefficients were especially low. Inconclusive results such as these are a risk of using field data from an actual quality award.

 

Cost-Effectiveness Based Performance Evaluation for Suppliers and Operations

Chee-Cheng Chen and Ching-Chow Yang, Chung-Yuan Christian University.

This research establishes a cost-effectiveness based performance evaluation system for suppliers and operations. The purpose is to provide a methodology for “integrating supplier and manufacturer capabilities through a common goal—profitability improvement—based on lowering the cost of purchased materials.” The relevance of the system to design, purchasing, and other functions is also presented.

Cost of quality (COQ) concepts affect operating costs, profitability, and customer needs. Phil Crosby (1984) encouraged manufacturers to measure COQ. He classified COQ into two broad categories:

1. The price of conformance, which includes the costs of ensuring that goods are produced defect free
2. The price of nonconformance, which includes all costs incurred due to defective quality problems occurring on the first run

The objective of the authors research is to specify the interaction and mutual movement of the three groups in the supply chain—supplier, manufacturer, and customer—based on the quality performance in different stages and its consequential cost. Their goal is to establish a rating system for the supplier’s performance from the orientation of the lowest total involved quality cost (TIQC) using an efficient predetermined cost structure matrix, and prove the suitability of this system using real case simulation and demonstration.

The benefits of measuring supplier quality performance using total involved quality cost (TIQC) are:

1. A common measurement language—the language of money
2. Simple and visible numbers along with direct and indirect loss ratios to help management and employees understand the importance of doing things right the first time.

The authors developed a TIQC and used it in planning and establishing this evaluation system for supplier performance. Very satisfactory and exacting results were obtained in the system simulation and demonstration. The evaluation system using the TIQC can help manufacturers select the best suppliers and drive operational quality improvements.

Return to top