This Article Abstract Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Jacobsen, P. B. Articles by Malafa, M. Search for Related Content PubMed Articles by Jacobsen, P. B. Articles by Malafa, M.

Measuring Quality of Care in the Treatment of Colorectal Cancer: The Moffitt Quality Practice Initiative

H. Lee Moffitt Cancer Center & Research Institute, Tampa, FL; Space Coast Medical Associates, Titusville, FL; Center for Cancer Care and Research, Lakeland, FL; and Sarasota Memorial Hospital, Sarasota, FL

Corresponding author: Paul B. Jacobsen, PhD, Health Outcomes and Behavior Program, Moffitt Cancer Center and Research Institute, 12902 Magnolia Dr, MRC-PSY, Tampa, FL 33612; jacobsen{at}moffitt.usf.edu

	Abstract

Top Abstract Introduction Methods Results Discussion Authors' Disclosures of... References

 
Purpose: The Moffitt Quality Practice Initiative (MQPI) is a practice-based system of quality self-assessment, the ultimate goal of which is to improve the quality of cancer care at a statewide level. The initial phase of this project focused on developing procedures, determining feasibility, and evaluating utility for assessing quality of care for colorectal cancer within an existing affiliate network.

Patients and Methods: Representatives from four oncology groups selected quality measures consistent with evidence-, consensus-, and safety-based guidelines that could be abstracted from medical records. Trained abstractors then reviewed records of all eligible colorectal patients seen by each practice in 2004. Frequencies of responses for each indicator were tabulated for overall and practice-specific level of adherence and were compared among practices.

Results: Adherence was uniformly high for several indicators, including confirmatory pathology report, staging information, and chemotherapy discussion or recommendation. Lower adherence was evident across practices for performance of carcinoembryonic tests and complete colonoscopic evaluations. Significant variation among practices was evident only for consent for chemotherapy.

Conclusion: The initial phase of MQPI demonstrated the feasibility and utility of assessing quality indicators for colorectal cancer among members of an existing affiliate network. Findings identified areas where adherence to care was uniformly high, but also identified areas where both overall and practice-specific adherence were less than optimal. These efforts lay the groundwork for expanding MQPI in several directions that have in common the potential to improve the quality of cancer care on a statewide basis.

	Introduction

Top Abstract Introduction Methods Results Discussion Authors' Disclosures of... References

 
Since 1999, the Moffitt Cancer Center & Research Institute, a National Cancer Institute–designated Comprehensive Cancer Center, has been developing affiliations with strategically located hospitals and practices throughout Florida, the state with the second highest death rate from cancer in the United States.¹ Throughout the last 7 years, the Moffitt Affiliate Network has grown to include 17 community hospitals and more than 280 affiliated oncologists. It is estimated that upwards of 20% of all cancer patients in Florida are currently treated by network members. Affiliation activities include collaborations in the areas of cancer prevention and screening, continuing medical education, clinical service delivery, and clinical research. This infrastructure also provides a unique opportunity for the Moffitt Cancer Center to study and achieve one of its prime objectives—the establishment of higher standards of cancer care statewide.

In 1999, the Institute of Medicine's National Cancer Policy Board published "Ensuring Quality Cancer Care," a landmark report that called attention to problems with the quality of cancer care in the United States.² The board concluded that for many people in the United States, there is a wide gap between the ideal versus the reality of their experience with cancer care. Among the recommendations was a call for the development of systems to measure and monitor quality of care using a core set of indicators.²

Stimulated by the board's report, a number of quality-of-care initiatives have been implemented by national organizations. A major example of such an effort is the National Initiative for Cancer Care Quality (NICCQ), which is sponsored by ASCO.³ Based on literature review and expert reviews, this project developed 36 quality measures for breast cancer and 25 quality measures for colorectal cancer.⁴ In the initial evaluation of these measures, data were collected from five metropolitan areas in the United States for patients newly diagnosed in 1998 with stage I to III breast cancer and stage II to III colorectal cancer.⁴ Findings indicated that among colorectal cancer patients, adherence ranged from 57% to 93% for individual indicators, and was less than 85% for 14 of the 25 indicators.

Although these findings identify important opportunities for improvement, they are based on geographically defined populations and, consequently, it is difficult to determine the care prescribed by individual institutions or practices. Toward this end, the Quality Oncology Practice Initiative (QOPI)⁵ was initiated with support from ASCO to provide oncology practices with a tool for self-examination that can promote excellence in cancer care. The QOPI process, based on retrospective review of medical records, has the capability of yielding measurements of practice quality that allow for comparisons across practices and over time. In the initial phase of this project, 11 quality indicators were evaluated in seven oncology groups in two rounds conducted 6 months apart. Quality indicators included issues related to patient safety (eg, use of chemotherapy flow sheets), application of evidence-based treatment (eg, use of serotonin antagonist with high emetogenic chemotherapy), and patient-centric care (eg, consent for chemotherapy treatment). Findings indicated significant variation in adherence among the practices on eight (73%) of the 11 indicators.

Building on both the NICCQ and QOPI efforts, the Moffitt Quality Practice Initiative (MQPI) was established with the ultimate goal of improving the quality of cancer care among all member institutions and throughout the state of Florida. The current report focuses on the methodology and findings of our initial experience conducting quality-of-care assessments at four of 17 member institutions. Similar to QOPI, the current project relied on retrospective reviews of medical records seen in selected medical oncology practices. In contrast to QOPI, the current project focused on two specific cancers (colon and rectal cancer) and included all cases seen in a calendar year. The project had three aims: demonstrate the feasibility of conducting a multisite evaluation of quality of care for colorectal cancer within an existing affiliate network; obtain practice-specific and aggregate information about adherence to quality care indicators; and provide a specific target or targets for quality improvement efforts.

	Methods

Top Abstract Introduction Methods Results Discussion Authors' Disclosures of... References

 
Selection of Practices
At the time this project was conducted, the Moffitt Affiliate Network comprised 17 affiliate institutions located in the state of Florida. Selection of practices for participation was guided by the following considerations: (1) practice included more than one medical oncologist; (2) ability and agreement to provide experienced medical record abstractor; (3) agreement to participate in the development of the quality standards, data collection form, and data collection process; and (4) ability to contribute at least 40 cases of colon or rectal cancer seen in calendar year 2004. Three Moffitt Affiliate Network members were selected that met these criteria: Space Coast Medical Associates, Titusville, Florida; Center for Cancer Care and Research, Lakeland, Florida; and Sarasota Memorial Hospital, Sarasota, Florida. All three of these community-based oncology practices agreed to participate. Data were also collected at the Moffitt Cancer Center, a National Cancer Institute–designated comprehensive cancer center located at the University of South Florida. The project received approval from institutional review boards at each participating institution. To maintain patient privacy, collection of demographic data was limited to patient sex, and all records were coded with a unique project identifier before transmission to the central data collection site. Based on exempt approval status, written informed consent from patients was not required to access medical records.

Selection of Quality Measures
Selection of quality measures focused on assessing indicators consistent with evidence-, consensus-, and safety-based guidelines. Item selection began by reviewing those used previously by QOPI to develop a practice-based system of quality assessment.⁵ Additional items were constructed based on review of relevant literature and suggestions by project participants. An initial list of quality-of-care indicators was developed and approved by consensus during a meeting attended by representatives from all participating practices. The measures were further reviewed to ensure that they could be easily abstracted from office-based medical oncology records and that responses could be coded in binary terms (yes or no). The final list of indicators appears in Table 1. In addition to these quality indicators, an item was included for exploratory purposes to identify the number of cases diagnosed via routine screening.

Medical Record Review and Data Entry
A training manual for data identification, abstraction, and entry was developed and reviewed at a meeting of participating clinicians and data abstractors to ensure consistency across practices. A research physician from Moffitt Cancer Center was designated and trained as the chief abstractor for the project and trained and monitored all the other data abstractors. The training and monitoring was comprised of three phases. The first phase consisted of detailed on-site training. The chief abstractor reviewed five cases of colorectal cancer from 2003 (the year before the study year) with each abstractor to ensure accuracy and reliability of data collection. During the second phase, each abstractor reviewed five additional cases from 2003. The same medical records were reviewed independently by the chief abstractor and assessed for concordance. Additional training was provided if necessary before practices were approved for project initiation. The third phase occurred after the completion of the initial 15 cases at each practice; the chief abstractor reviewed three randomly selected cases to ensure ongoing quality of data collection and entry. This procedure was repeated following an additional second set of 15 cases from each practice. Abstractors were trained in data entry using a secured Web-based application designed for collection and transmission of data to the Moffitt Cancer Center. This application gave users the capability to skip questions that did not apply, thus minimizing possible data entry errors.

Statistical Analysis
Overall and practice-specific adherence rates were calculated for each indicator. Variation in adherence across practices was evaluated by conducting Fisher's exact tests with a two-tailed significance level of P < .05. An a priori statistical power evaluation was not conducted due to the exploratory nature of the project.

	Results

Top Abstract Introduction Methods Results Discussion Authors' Disclosures of... References

 
Medical records for 260 patients were accrued and abstracted. The number of records per practice ranged from 46 to 102 (mean, 65 records). Fifty-eight percent of patients (n = 150) were male. Seventy-four percent of patients (n = 192) had been diagnosed with colon cancer. Of these patients, 64% had nonmetastatic disease (n = 122) and 36% had metastatic disease (n = 70). The remaining 26% of patients (n = 68) had been diagnosed with rectal cancer. Of these patients, 65% (n = 44) had been diagnosed with nonmetastatic disease, and 35% (n = 24) had been diagnosed with metastatic disease.

The percentage of patients whose cancer had been detected by routine screening ranged from 10% to 20% across practices (mean, 13%). Variation across practices was not statistically significant (P = .42).

Information about overall and practice-specific performance on quality indicators appears in Table 2. The rates for each practice are presented in masked form to preserve anonymity per prior agreement with participating institutions. Adherence rates were uniformly high ( 85%) and did not differ significantly (P values .10) by practice for the following indicators: pathology report confirming malignancy, staging according to American Joint Committee on Cancer or Dukes criteria, chemotherapy discussed or recommended, and chemotherapy flow sheet used. Adherence rates were slightly lower (75% to 89%) for performance of a carcinoembryonic antigen (CEA) test in the 6 months after surgery or chemotherapy, but did not vary significantly by practice (P = .28). Adherence rates were generally moderate (range, 37% to 67%) and did not differ significantly by practice (P values .23) for performance of a complete colon evaluation within 12 months of diagnosis and performance of a CEA test before surgery or chemotherapy. Finally, adherence rates did vary significantly by practice (P < .001) for consent for chemotherapy. While three practices demonstrated high rates of adherence to this indicator ( 97%), one practice demonstrated a much lower rate of adherence (38%).

	Discussion

Top Abstract Introduction Methods Results Discussion Authors' Disclosures of... References

The second aim was to obtain information about adherence to eight quality-care indicators. Through a process that included multiple checks on quality control, overall and practice-specific rates of adherence were calculated for these indicators. With the notable exception of documented consent for chemotherapy, there was limited variation in adherence across practices. Similarities exist for rates of adherence reported here and those reported elsewhere. For example, both QOPI⁵ and MQPI found adherence rates of 85% across practices for the presence of a pathology report in the medical record, and use of chemotherapy flow sheets. In addition, both projects reported significant variation across practices for documentation of consent for chemotherapy. Differences, however, were also evident. While we found uniformly high ( 85%) adherence for staging having been completed, QOPI reported a lower range of adherence (78% to 93%) and significant variation across practices.⁵ Nevertheless, the general consistency in adherence rates is remarkable given the differences across projects in patient characteristics, case selection procedures, and practice characteristics.

The third aim was to provide a specific target or targets for quality improvement efforts. Findings suggested that one practice in particular needed to address their low rate of documented consent for chemotherapy. This problem was apparently due to this institution's lack of policies and procedures for obtaining patient consent for chemotherapy. In addition, all four practices could benefit from examining current practices for performance of baseline and follow-up CEA testing, and performance of a complete colon evaluation within 12 months of diagnosis. Ethical considerations justify improving consent rates, and empirical evidence justifies improving performance of CEA testing and complete colonoscopic evaluations.

Although CEA monitoring is not considered useful for screening purposes,⁶ evidence suggests the utility of baseline measurement at the time of diagnosis. Preoperative CEA levels have been shown to correlate with stage of disease and to be an independent prognostic indicator following curative-intent surgery.^7,8 Elevated levels of CEA are associated with reduced survival⁹ and shorter time to recurrence¹⁰ for stage II and III colorectal cancer. Preoperative levels also seem to be a significant prognostic factor for stage I disease,¹⁰ but not for stage IV metastastic disease.^8,10 Additionally, there is strong evidence to support the usefulness of CEA level in detecting disease recurrence. In patients with resected colorectal cancer, CEA levels should normalize within 4 to 6 weeks,¹¹ and persistent elevation is considered highly suggestive of residual or systemic disease.¹² It should be noted, however, that some patients may have normal preoperative CEA levels despite significant disease burden. In these cases, clinicians may chose not to conduct follow-up CEA testing since it is likely to be uninformative. The possibility that the current set of medical records included patients like this cannot be ruled out. Overall, it has been estimated that the CEA test possesses sensitivity of 77% and specificity of 98% for detection of recurrence.⁷ Given its sensitivity and specificity in detecting recurrent disease, CEA monitoring has been endorsed by both ASCO¹³ and the National Comprehensive Cancer Network.¹⁴

Between 2% and 7% of patients with colorectal cancer may harbor one or more additional synchronous cancers at the time of diagnosis.^15–17 Approximately 25% of patients may also have synchronous benign polyps.¹⁷ Based on such evidence, the American Cancer Society and US Multi-Society Task Force on Colorectal Cancer strongly recommend that the colon and rectum be carefully cleared of synchronous neoplasia¹⁸; in obstructed colons, where possible, double contrast enema or computed tomography colonography should be done preoperatively, and colonoscopy should be performed 3 to 6 months after surgery.¹⁸

With quality improvement as a goal, three types of feedback were provided to professional staff from each practice shortly after the data were tabulated. First, staff received information about the overall rate of adherence to each quality indicator. Second, they were informed of their rate of adherence to each indicator. Third, they were informed of how well the other three practices performed on each indicator; this information was presented in a blinded fashion such that each practice was unable to identify which rates belonged to the other three practices. Feedback has resulted in a quality improvement effort to improve rates of consenting for chemotherapy at the one practice with low adherence and has initiated discussion at all four practices on current patterns of care related to performance of CEA tests and complete colonoscopic examinations.

Two limitations of this initial evaluation of MQPI should be noted. First, the project was limited to patients with colorectal cancer. It remains unknown whether or not practitioners evaluated in this project demonstrated similar rates of adherence to quality care in treating other forms of cancer. However, the clinical significance of this project is underscored by the fact that colorectal cancer represents the second most common cause of cancer deaths in the men living in the United States and the third most common cause in women living in the United States.¹ Moreover, a focus on colorectal cancer permitted examination of disease-specific indicators of quality care, such as performance of CEA tests. A second limitation involved the voluntary nature of participation in the project. As a consequence, no conclusions can be drawn about the rate of adherence to the quality indicators among all members of the Moffitt Affiliate Network or, more generally, among providers of cancer treatment in Florida.

Our initial experience suggests several future directions for MQPI. One direction is to expand participation in the assessment of the quality of colorectal cancer care to all institutions in the Moffitt Affiliate Network. In addition to allowing each institution to compare itself with all other network members, such a project would provide useful information about the quality of care provided by the network as a whole. Another direction is to resurvey the original four participating institutions to see if aspects of care identified as needing improvement (eg, performance of CEA tests and consent for chemotherapy) have improved over time. Yet another direction would be to abstract additional information about the quality of colorectal cancer care from records that have already been reviewed at the four participating institutions. For example, the NICCQ⁴ includes several diagnostic and treatment indicators of the quality of colorectal cancer care, that were not part of the initial survey. By expanding the review to include NICCQ indicators, direct comparisons can be made between adherence rates for Moffitt Affiliate Network members and population-based estimates reported by the NICCQ. Finally, the potential exists to expand the current effort within the Moffitt Affiliate Network to survey the quality of care for other common cancers such as breast or prostate cancer.

Given the potential interest of many oncology practices in conducting similar evaluations, several recommendations can be offered for implementing procedures to monitor quality of care. First, the ability to conduct such evaluations is greatly facilitated by participation in organized multicenter efforts. For example, participation in the current project provided each institution with a standard set of indicators that allowed them to directly compare their performance with that of the other institutions. In addition, each institution received a training manual and instruction for office staff, designed to ensure the standardization of the abstracting procedures and, thus, the quality of the data collected. Second, practices must recognize that the measurement of quality will require an investment of resources. In the current project, each practice provided the services of office staff who reviewed medical records and entered abstracted data into a computerized database. It is estimated that these tasks took approximately 45 minutes per case to complete. Assuming $20 per hour for staff expenses, cost can be estimated at $1,500 per 100 cases reviewed. Participation in the larger effort meant that the individual practices did not have to bear other costs of conducting the quality evaluation (eg, creation of the Web portal and analysis of the data collected). It must be acknowledged, however, that in many settings, office staff may not be available or qualified to conduct record reviews. To address this issue in our affiliate network, we are currently exploring the costs and logistics of providing such practices with the services of an abstractor who would travel to the site. Third, investments in information technology have the potential to greatly facilitate the measurement of quality indicators. Conversion to electronic medical records raises the possibility that information relevant to quality assessment can be abstracted in an automated fashion at minimal cost and at more frequent intervals than is feasible with reviews of paper records.

The initial phase of MQPI demonstrated that feasibility and utility of assessing quality indicators for specific cancers, among members of an existing affiliate network of cancer treatment providers. Findings identified areas in which adherence to quality care was consistently high, but also identified areas in which adherence was less than optimal. To the best of our knowledge to date, this is the first project to accomplish these specific aims. The experience obtained has laid a valuable foundation for future efforts to expand this initiative in several directions, all of which have in common the potential to improve the quality of cancer care in a US state among the largest in population and highest in rates of death due to cancer.

	Authors' Disclosures of Potential Conflicts of Interest

Top Abstract Introduction Methods Results Discussion Authors' Disclosures of... References

 
The authors indicated no potential conflicts of interest.

	Notes

 
Presented in part at the 42nd Annual Meeting of the American Society of Clinical Oncology, June 2-6, 2006, Atlanta, GA.

	References

Top Abstract Introduction Methods Results Discussion Authors' Disclosures of... References

 

1. American Cancer Society: Cancer Facts and Figures 2006. Atlanta, GA, American Cancer Society, 2006
2. Hewitt M, Simone JV: Ensuring quality cancer care. Washington, DC, Institute of Medicine and National Research Council, 1999
3. Schneider EC, Epstein AM, Malin JL, et al: Developing a system to assess the quality of cancer care: ASCO's National Initiative on Cancer Care Quality. J Clin Oncol 15:2985-2991, 2004
4. Malin JL, Schneider EC, Epstein AM, et al: Results of the National Initiative for Cancer Care Quality: How can we improve the quality of cancer care in the United States? J Clin Oncol 24:626-634, 2006[Abstract/Free Full Text]
5. Neuss MN, Desch CE, McNiff KK, et al: A process for measuring the quality of cancer care: The Quality Oncology Practice Initiative. J Clin Oncol 23:6233-6239, 2005[Abstract/Free Full Text]
6. American Society of Clinical Oncology: Clinical practice guidelines for the use of tumor markers in breast and colorectal cancer. J Clin Oncol 14:2843-2877, 1996[Abstract/Free Full Text]
7. Carriquiry LA, Pineyro A: Should carcinoembryonic antigen be used in the management of patients with colorectal cancer? Dis Colon Rectum 42:921-929, 1999[CrossRef][Medline]
8. Wanebo HJ, Rao B, Pinsky C: Preoperative carcinoembryonic antigen level as a prognostic indicator in colorectal cancer. N Engl J Med 299:448-451, 1978[Abstract]
9. Harrison LE, Guillem JG, Paty P: Preoperative carcinoembryonic antigen predicts outcomes in node-negative colon cancer patients: A multvariate analysis of 572 patients. J Am Coll Surg 185:55-59, 1997[CrossRef][Medline]
10. Wichmann MW, Muller C, Hornung HM: Results of long-term follow-up after curative resection of Dukes A colorectal cancer. World J Surg 26:732-736, 2002[CrossRef][Medline]
11. Fletcher RH: Carcinoembryonic antigen. Ann Intern Med 104:66-73, 1986[CrossRef][Medline]
12. Arnaud JP, Koehl C, Adloff M: Carcinoembryonic antigen (CEA) in diagnosis and prognosis of colorectal carcinoma. Dis Colon Rectum 23:141-144, 1980[Medline]
13. Bast RC, Ravdin P, Hayes DF, et al: 2000 update of recommendations for the use of tumor markers in breast and colorectal cancer: Clinical practice guidelines of the American Society of Clinical Oncology. J Clin Oncol 19:1865-1878, 2001[Abstract/Free Full Text]
14. Engstrom PF, Benson AB, Chen Y, et al: The NCCN colon cancer clinical practice guidelines. J Natl Compr Canc Netw 3:468-491, 2005[Medline]
15. Barrier A, Houry S, Hauguier M: The appropriate use of colonoscopy in the curative management of colorectal cancer. Int J Colorectal Dis 13:93-98, 1998[CrossRef][Medline]
16. Langevin JM, Nivatvongs S: The true incidence of synchronous cancer of the large bowel: A prospective study. Am J Surg 147:330-333, 1984[CrossRef][Medline]
17. Pagana TJ, Ledesma EJ, Mittelman A: The use of colonoscopy in the study of synchronous colorectal neoplasms. Cancer 53:356-359, 1984[CrossRef][Medline]
18. Rex DK, Kahi CJ, Levin B: Guidelines for colonscopy surveillance after cancer resection: A consensus update by the American Cancer Society and U.S. Multi-Society Task Force on Colorectal Cancer. CA Cancer J Clin 56:160-167, 2006[Abstract/Free Full Text]
19. Benson AB, Desch CE, Flynn PJ, et al: 2000 update of American Society of Clinical Oncology colorectal cancer surveillance guidelines. J Clin Oncol 18:3586-3599, 2000[Free Full Text]
20. Engstrom PF, Benson AB, Chen Y, et al: The NCCN rectal cancer clinical practice guidelines in oncology. J Natl Compr Canc Netw 3:492-508, 2005[Medline]

This Article Abstract Full Text (PDF) Purchase Article View Shopping Cart Alert me when this article is cited Alert me if a correction is posted Services Email this article to a colleague Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via Google Scholar Google Scholar Articles by Jacobsen, P. B. Articles by Malafa, M. Search for Related Content PubMed Articles by Jacobsen, P. B. Articles by Malafa, M.