Publication
Article
Identifying Controlled Substance Patterns of Utilization Requiring Evaluation Using Administrative Claims Data
Author(s):
Objectives: To develop a systems approach to identify, for furtherevaluation, patients with potential controlled substance misuseor mismanagement using software queries applied toadministrative health claims data.
Study Design: Retrospective validation of the system usinginsurance claims.
Patients and Methods: Data from administrative health claimsdatabases representing nearly 7 million individuals younger than65 years were used by multidisciplinary expert panels to developand validate controlled substance patterns of utilization requiringevaluation (CS-PURE) criteria.
Results: Thirty-four CS-PURE queries were developed in SASand applied to administrative claims records to identify patientswith potential controlled substance misuse or mismanagement.From these, we identified 10 CS-PURE with the highest expertagreement that intervention was warranted. Expert panel agreementthat CS-PURE correctly identified cases ranged from 48% to100%, with at least 50% agreement in 9 of 10 CS-PURE. Theprevalence rates for CS-PURE ranged from 0.001% to 0.252%. Thistranslates to identifying between 5 and 1116 patients for individualCS-PURE in a 500 000-member health plan.
Conclusions: We developed and empirically validated a groupof queries using CS-PURE to identify patients with potential controlledsubstance misuse or mismanagement that would warrantfurther evaluation by the treating physician, a quality assurancefunction, or the medical director. Claims-based CS-PURE identificationis generalizable to most health insurers with access to medicaland pharmaceutical claims records. Although CS-PURE are notdirect measures of misuse, they can direct attention to potentialproblems to determine if intervention is needed.
(Am J Manag Care. 2004;10:783-790)
The diversion, abuse, and inappropriate use ofcontrolled substances are subjects of continuingconcern among the medical community, insurers,and policy makers. However, a balance must beachieved between preventing diversion and abuse andencouraging the use of controlled substances for legitimatemedical need, particularly for pain management.1-3 Several clinical practice guidelines, consensusstatements from professional associations, and statelaws and policies emphasize that it is essential for opioidanalgesics to be available for the treatment of moderate-to-severe pain and that prescribing should beindividualized to the patient.2,4-14 Although someprogress has been made in treating pain, undertreatmentof pain is still prevalent.15-17
Media coverage of diversion and abuse of controlledsubstances and uncertainty regarding potential disciplinaryaction may cause physicians to hesitate when consideringtreatment for a patient who may requirelong-term or high doses of opioids.1,18 This is exacerbatedwhen physicians have difficulty discerning between apatient with a legitimate pain problem and one who isfeigning pain to obtain drugs for abuse or diversion.19 Because pain is subjective and cannot be measured orruled out by laboratory tests or physical examination,physicians rely largely on their interpretation of patientinterviews and histories to determine a patient's needfor analgesics. However, they often find themselves inthe predicament of wanting to treat seemingly legitimatepatients without having information about theirpatients' prescription drug and medical histories thatwould help them identify and address any problems.Indeed, a 1999 report from the Institute of Medicinestressed that most medical errors do not result fromindividual practitioners' recklessness; rather, they areattributable to faulty processes and systems that leadpeople to make mistakes or fail to prevent them throughlack of information and support in a complex workingenvironment.20 Solving problems within healthcarerequires the design of systems and processes to helpavoid errors, to minimize the damage caused by errorsthat occur, and to analyze the patterns of errors and discoverways to prevent them.
Despite technological advances and the wealth ofstrategic knowledge within administrative health claimsdatabases, only 17 states operate electronic prescriptionmonitoring programs, which vary in their goals,structure, and oversight by the health profession.21-26 Presently, few health plans analyze the data to identifypotential misuse of controlled substances. Access tothis aggregate information on patients is not readilyprovided to physicians, restricting their ability to providequality care. In response to this need, we developeda software program that identifies patients withpotential prescription mismanagement or abuse anddiversion issues.
This article lays out a road map for a system to complementstate programs, where they exist, and providea stand-alone tool for physicians in other states. Thesystem detects controlled substance patterns of utilizationrequiring evaluation (CS-PURE) that suggest needfor further evaluation. The CS-PURE criteria, 10 ofwhich are presented herein (Table 1), were developedand validated by experts primarily from the medicalprofession but also from the pharmaceutical diversioninvestigation community, based on a combination ofevidence and professional consensus.
METHODS
Data
The data for this study were drawn from healthinsurance claims databases collected during 2000.These databases were used to test the development andapplication of CS-PURE criteria. The database used todevelop the original CS-PURE prototype consisted of acompilation of claims from different self-insuredemployers and third-party administrators from nearlyall of the 50 states. It contains a mix of health plantypes, including indemnity fee-for-service plans, preferredprovider organizations, independent practiceassociations, and health maintenance organizations.Nearly 7 million covered lives are included in the singleyear of data, drawn from the workforce younger than 65years and their covered dependents. Database 1 is fromone of the nation's largest managed care plans. Datawere used from 5 distinct markets in which independentpractice association and preferred provider organization products are offered, 3 in southern states and 2in midwestern states. Together, these markets containedalmost 3 million covered lives from the populationyounger than 65 not covered by Medicaid. Database2, generated from a health plan that offered preferredprovider organization and health maintenance organizationproducts in the eastern United States, containedrecords for almost 800 000 covered lives younger than65. This health plan is representative of most moderate-to-large insurers in the United States in terms of demographicsand structure.
Development of Patterns ofControlled Substance Misuse
An 11-member multidisciplinary expert panel, consistingof 2 addictionists, 3 pain physicians, 2 psychologists,1 psychiatrist, and 3 pain management nurses,convened in December 2001. Their goal was to defineCS-PURE criteria that could be applied to claims databasesto identify possible abuse or diversion of controlledsubstances by patients or mismanagement by prescribers.The CS-PURE are not conclusive of inappropriateuse; rather, they aim at improving patient safety andoutcomes by alerting prescribers and insurers of potentialproblems so that further evaluation can be conductedby them. The expert panel reached consensus on 38prototypical CS-PURE for evaluation and did not excludeparticular patient groups in general, with 2 exceptions(≥3 prescriptions for injectable opioid for patients withouta cancer diagnosis during a year, and any patientwith a prescription for benzodiazepine or opioid with aprior substance abuse diagnosis). Some CS-PURE criteriawere based on similar patterns, but reflected variationsin specific medications used and changes in theduration of consecutive or overlapping days of medicationuse, for example, continuous overlap of 2 or morebenzodiazepines for at least 30, 60, or 90 days.
Operationalization of CS-PURE
Computer programs based on the expert panel's original38 CS-PURE were developed using SAS, version 8.2,to operationalize and apply CS-PURE to the initiallydeveloped database. Detailed use profiles were producedfor the patients identified by each of the prototypicalCS-PURE. These profiles were reviewed andassessed for the accuracy of the computer coding by aproject team comprising pharmacists, computer programmers,and health services researchers. At the conclusionof this process, the original 38 CS-PURE werereduced to 34 CS-PURE (detailed in a list available fromthe author). This change reflected the deletion of 4 ofthe original CS-PURE criteria because they identified alow number of patients.
Assessing the Generalizability andValidity of CS-PURE
The CS-PURE specification was assessed in terms ofgeneralizability and validity. The CS-PURE were consideredto be generalizable if they could be implementedusing data from different insurers and managed careorganizations and if they identified approximately thesame percentage of patients in each plan, indicating thepresence of general or similar trends. Alternatively,there could be a plausible explanation for dramaticvariation in prevalence rates across plans, such as differencesin state laws regarding prescribing practices orthe presence of prescription monitoring programs. TheCS-PURE criteria were considered to have a high degreeof validity if they identified the target population with alow rate of false-positive results. The methods forassessing these dimensions are described herein.
Generalizability Assessment.
Estimation of whetherthe data requirements of CS-PURE were too stringent tobe widely applicable across plans occurred naturallyduring the refinement of the profiles, as alreadydescribed. Once CS-PURE were developed in such away that the required data elements would be readilyavailable from most insurers, they were applied to datafrom the 5 database 1 markets and database 2 to computeperiod prevalence rates.Several multivariate tests of generalizability acrossthe health plans were also completed. Logistic and linearprobability regression analyses were used to determinewhether the health plans were systematicallydifferent in their probability to predict patients' CSPUREafter controlling for general health status usingthe ambulatory diagnostic group system developed byJohns Hopkins University.27
Validation Assessment.
To validate the specificationof CS-PURE, a second expert panel meeting was convenedin December 2002. The 10-member panel, 7 ofwhom were members of the original panel in December2001, consisted of 6 physicians specializing in painmedicine or addiction, a pain management nurse, and 1current and 2 former diversion investigators. In preparationfor this meeting, 180 patient profiles of medicaland pharmacy claims history were generated by applyingthe initial CS-PURE to database 2. This database waschosen because it provided multiregion variation and itwas likely to be most similar to an average managedcare plan database. A given patient could be flagged by1 or multiple CS-PURE. Using these results, a randomselection of proportionately sampled patients was generatedbased on the period prevalence of the 34 CSPURE.Each profile represented the medical andpharmacy claims history for a given insured person, displayedin chronological order of service date or prescription fill date. In addition, minimal informationabout the patient (eg, age and sex), as well as eachpatient's total medical care expenditure, was presented.The profiles flagged all CS-PURE found for a givenpatient for 2000.
Panelists were divided into 3 teams of 3 or 4 memberseach, with each team containing at least 1 painphysician and a diversion investigator. As each profilewas reviewed, panelists discussed the available information.Each expert was then asked to individually scoreeach profile along 3 dimensions: (1) whether the case ofprescription medication use appeared to be outside thescope of accepted practice, (2) whether further evaluationby the health plan would be recommended, and (3)when an intervention was judged to be needed, whetherthe intervention should be with the patient, the prescriber,or both.
Following the meeting, scores of individual expertswere evaluated, and agreement scores were computedfor the profiles within CS-PURE. The scores were tabulatedto construct the share of agreement in theresponses of the 3 questions and provide an assessmentof validity among the expert panel.
CS-PURE Refinement
Using the expert panel results, a logistic regressionanalysis was undertaken for 2 reasons: (1) to examinethe validity of the ratings for each of the CS-PURE criteriaand (2) to order CS-PURE by their ability to identifycases of interest. The dependent variable in theregression was the rating assigned by each expertreviewer for each profile that he or she reviewed. Thesewere regressed on 34 dichotomous variables, 1 for eachof the CS-PURE criteria. No omitted variable was necessarybecause profiles could, and often did, trigger multipleCS-PURE. This generated odds ratios and tests ofstatistical significance for each of the 34 CS-PURE. Wethen culled the list of 34 CS-PURE, concentrating ononly the top 10 CS-PURE. The choice of 10 as the numberof CS-PURE to examine was somewhat arbitrary,relying on natural breaks in the analysis and on our estimateof the resources that would be necessary to implementthem. The selection of the 10 CS-PURE was basedon the union of statistically significant positive oddsratios from the logistic regression analysis results andthe highest expert agreement scores that interventionwas warranted.
RESULTS
CS-PURE Prevalence
Six health plans were used to generate the periodprevalence estimates of the 10 CS-PURE criteria mostpositively associated with the probability of identifyinga misuse case. The prevalence results are displayed inTable 1. Five of the 6 health plans' prevalence data werecombined to represent database 1 plans. The other planwas database 2. Prevalence estimates between the 2insurers were similar. Notable exceptions were CSPURE06,in which meperidine hydrochloride prescribingwas nearly 2 times greater in database 2, andCS-PURE05, in which the number of prescriptions for 4g/d or more of acetaminophen was roughly 2 timesgreater in database 1. To appreciate the size of the populationsidentified as an estimate of potential workloadfor the prescribers and administrators of the plans, thenumber of patients who would be identified in a500 000-member health plan was calculated. In almostall cases within the top 10 population, the number ofpatients identified by each of the CS-PURE criteria wasfewer than 1000. In some cases, the estimated targetpopulation was fewer than 20 patients per 500 000 coveredlives. Although these estimates are small, they areconsistent across the different health plans and arefound in more than 1 type of health insurer or region ofthe country.
CS-PURE Validity
Table 2 presents the results of the expert panel'sassessment of the validity of CS-PURE. The overallpanel agreement that a case of potential misuse was correctlyidentified is summarized, as well as the clinicaland law enforcement representatives' assessments ofspecificity. The overall agreement of correct identificationof a case was at least 50% for 9 of the top 10 CSPURE.In CS-PURE01, in which patients had 6 or moredifferent prescribers for the same controlled substance,the overall percentage of patients the experts agreed onas being valid identifications was 55%. In almost allcases, law enforcement and medical professionals werein agreement on the share of patients to be identified.
The expert panel also largely agreed that a patientcorrectly identified as a potential misuse candidateshould receive further evaluation and that some form ofintervention should be directed at the prescribingphysician or the patient (in some cases, both). For CSPURE05and CS-PURE06, the expert panel had 100%agreement that all of the patients sampled were correctlyspecified and should be evaluated, with furtheraction possibly warranted. Both of these CS-PURE areclear indicators of inappropriate prescribing, based oncurrent guidelines. The maximum safe daily dose ofacetaminophen has long been established at 4 g/d, andguidelines from professional associations warn thatmeperidine should not be used for more than 48 hoursfor acute pain and should not be prescribed for persistent pain. This is because of the formation of an activemetabolite called normeperidine, which is a centralnervous system excitotoxin that produces anxiety,tremors, myoclonus, and generalized seizures when itaccumulates with repetitive dosing.4
CS-PURE Generalizability
r
To assess generalizability, we first computed periodprevalence for each of the CS-PURE criteria. Next, usinga logistic regression model, we regressed a binaryresponse variable, which corresponded to whetherpatients did or did not have CS-PURE, on dummy variablesfor each plan from database 1, with the single planfrom database 2 serving as the reference category andan ambulatory care group score to control for casemix.27 The overall sample size was 221 836 patientshaving at least 1 of the CS-PURE for services received in2000. Pseudo 2 estimates ranged from 0.01 to 0.45.
Statistically significant health plan—specific effectson prevalence were found. All of the models had at least1 health plan dummy variable that was significant, suggestinggeographic variation in the distribution of CSPUREresults. No single health plan consistently had thelargest positive or negative effect on the probability ofCS-PURE prevalence, indicating that, while the healthplan populations were different, they were not systematicallybiased in a particular direction or degree in themagnitude of their effect.
DISCUSSION
We developed a claims-based, computerized methodto identify (1) patients who may be misusing controlledsubstances and (2) prescribers who may be providingpharmacologic management that warrants evaluation.Using claims data to improve quality of care is a specializedart requiring collaboration between clinical experts,programmers, systems analysts, and health servicesresearchers to correctly interpret and identify CS-PUREfrom items found in a typical database. The major componentsof claims data are medical and pharmacyclaims, member eligibility, and provider data.28,29 Thisusually includes the patient's diagnoses, tests and proceduresperformed, prescriptions filled (including the specificdrug, quantity, and number of days supplied),duration and level of hospitalization with dates andcosts, and a unique identifier of the provider who renderedthe service. The CS-PURE can be applied usingthese data elements, which are commonly available tomost health plans, and are written in SAS programminglanguage, which is also common to most plans (SAS programmingis available on CD-ROM from the author).
Practicing and specialty-specific physicians from differentparts of the country, together with pharmaceuticaldiversion investigators, were involved in all aspectsof the development of our claims-based CS-PURE. Thistype of tool can be implemented with little additionalexpenditure by insurers and is unobtrusive, with lowadministrative burden to physicians because the datahave already been collected.
The top 10 CS-PURE criteria identified patients witha high probability of controlled substance misuse ormismanagement so that resources can be targeted andpractitioners are not overwhelmed with data, as the projectednumber of patients identified in a 500 000-memberhealth plan ranged from 5 to 1116. These numbersare manageable for possible intervention, but also showthat the prevalence of potential problems with controlledsubstances cannot be ignored. The lowest prevalenceswere associated with CS-PURE09 and 10, whichwere most restrictive in terms of overlap for 90 or moreconsecutive days. None of the 10 CS-PURE included athreshold for the number of pills or dosage of a controlledsubstance, because the clinical experts wereemphatic that there is no such thing as underprescribingor overprescribing; rather, prescribing should becategorized as appropriate or inappropriate based on ananalysis of individual patient characteristics. The onlydosage threshold occurred in CS-PURE05 for 4 g/d ormore of acetaminophen, which has long been establishedas the maximum safe dosage of this analgesic.4 Acetaminophen is included in CS-PURE because it iscommercially available in fixed combination with opioidanalgesics, which should, but often in practice doesnot, limit the maximum dosage of these combinationproducts. Carisoprodol was included in CS-PUREbecause it is frequently abused, although it is not a controlledsubstance by federal regulations; an activemetabolite of carisoprodol is meprobamate, a scheduleIV anxiolytic.30
Application of Results
The CS-PURE are not direct measures of quality, butare surrogates to direct physicians' attention to potentialproblems requiring evaluation. Therefore, analysisof claims data to identify CS-PURE should be followedby a process of communication with physicians todetermine if intervention is needed. For example, themedical director of a health plan could generate a timelyletter or an e-mail alert to the primary or all prescribersof a patient flagged by CS-PURE so that anyproblem could be promptly addressed.
The cases would then be stratified; some complexpatients may require a case manager, while other individualsmay be triaged into appropriate treatment, forexample, an interdisciplinary pain clinic or substanceabuse treatment center. It is important that physiciansare supported with mechanisms, such as clinical pathwaystied to differential diagnoses, to assist in respondingto the data sent to them, for example, todifferentiate between people with inadequately treatedpain and those seeking drugs for abuse or diversion.Current information on proper pain managementshould be available to physicians, with targeted educationfor physicians who have been "scammed" or whomay be outdated in their prescribing practices toimprove their management of patients. Although physicianshave been known to resist notices about how toprescribe or treat based on claims data,31 in this case,physicians are provided with valuable information,some of which they would otherwise have no way ofknowing, that can help them not only provide bettercare but also protect their prescriptive authority.
Ongoing Refinement
As the prototypical CS-PURE criteria are applied byhealth plans, analysis of the results with subsequentrefinement will increase the usefulness of CS-PURElogic to physicians and medical directors. The medicalprofession should be involved in evaluating the qualityand accuracy of data sources, methods, and results.This also requires the development of reliable outcomeindicators to assess the effectiveness of applying CSPUREto claims data in reducing prescription drugabuse and in improving quality of care.
Limitations
Our study has several limitations. Claims data weredesigned to support a financial transaction rather thanto convey clinical information. Pharmacy claims datarepresent only filled prescriptions and do not generallyinclude information on prescriptions paid for in cash. Inaddition, the billing diagnosis code includes many termsthat may be associated with pain. Commonly used codingand billing logic allow physicians to arbitrarily citeonly 1 of multiple diagnoses.32 Other limitations ofclaims databases include the omission of some nonreimbursable services, inconsistent or inaccurate coding,the possibility of corrupted data, and the absence ofcodes that capture disease or symptom severity.28 Underreporting due to unmet deductibles can also leadto incomplete data.29 The data requirements to applyCS-PURE are stringent. Clear identification of uniqueprescribing physician and pharmacy identifiers isrequired. Many times, a group practice may be representedby a single prescribing physician identifier, andthis may damage the credibility of CS-PURE. New standardson health insurance data, imposed by the HealthInsurance Portability and Accountability Act, will guaranteethe future specification of a unique prescribingidentifier. An additional limitation is that the presenceof a data field does not guarantee its quality. Many CSPURErequire specific information on dosage frompharmacy claims, but health insurers often do not collectthis information. Without a dosage listed on theclaim record, CS-PURE must rely on the National DrugCode dosage differences. To explore new CS-PUREspecification, accurate dosage information will beessential. Despite these limitations, claims data arereadily accessible to plans and constitute the mostdetailed and uniform body of information available.
CONCLUSION
We have demonstrated the development of anempirically validated group of queries that draws onpattern recognition within existing claims databasesfrom different health plans to detect prescribing andmedical utilization patterns of controlled substancesthat may represent improper use. This has potentialuse as a valuable tool to assist physicians in managingtheir patients who require treatment with controlledsubstances. The strategies outlined in this report cannotbe implemented by an individual physician or themedical profession as a whole, but are an initiative thatrequires resources that are only available to insurers,that is, claims data. The development of such a systemsapproach is an opportunity for insurers to assist physiciansto address the possibility of inappropriate use ofcontrolled substances. A partnership between plansand prescribers is needed to systematically providephysicians with timely access to information that isvital to achieve their ultimate goal of improving clinicalcare and outcomes. The medical profession has astrong incentive to take a leadership role in developingsuch programs. Physicians are, first and foremost,responsible for the quality of care provided to theirpatients. In addition, by federal and state law, they areresponsible for their prescribing of controlled substances.They should therefore desire that insurers providethem with useful analyses of available informationthat will assist them in their practice. However, technologycan ignite fears about accuracy, liability, privacy,and security. Physicians have reason to be cautious,which is why physician leadership is critical in developingand implementing such systems and ensuringthat safeguards are in place.
Acknowledgments
We gratefully acknowledge the members of the 2 expert panels: SusanDerby, NP; Daniel Doleys, PhD; Douglas Gourlay, MD; Steven Hahn, MD;Howard Heit, MD; Madelyn Kitaj, MD; Eliot Krames, MD; BridgetMcDonough, RN; Fred Sheftel, MD; Richard Stieg, MD; Julie Waight, FNP; J.David Haddox, DDS, MD; Dale Ferranto, MS; Landon S. Gibbs, BA; andScott Gunn.
From the HSI Network LLC (STP, LAS) and Department of Healthcare Management,Carlson School of Management, University of Minnesota (STP), Minneapolis, and Center forHealth Care Policy and Evaluation, United Health Group, Eden Prarie (MDF, TSR), Minn;and Purdue Pharma LP, Stamford, Conn (SSK, RS, JDH), and Department of Anesthesiology,School of Medicine, University of Connecticut, Storrs (JDH).
This study was supported by Purdue Pharma LP.
Address correspondence to: Stephen T. Parente, PhD, Department of HealthcareManagement, Carlson School of Management, University of Minnesota, 321 19th AvenueSouth, Room 3-149, Minneapolis, MN 55455. E-mail: sparente@csom.umn.edu.
J Law Med Ethics.
1. Hoffmann DE, Tarzian AJ. Achieving the right balance in oversight of physicianopioid prescribing for pain: the role of state medical boards. 2003;31:21-40.
2. American Pain Society Web site. Promoting pain relief and preventing abuse ofpain medications: a critical balancing act: a joint statement from 21 health organizationsand the Drug Enforcement Administration. 2001. Available at:http://www.ampainsoc.org/advocacy/promoting.htm. Accessed Febuary 20, 2002.
Drug Alcohol Depend.
3. Zacny J, Bigelow G, Compton P, Foley K, Iguchi M, Sannerud C. College onProblems of Drug Dependence taskforce on prescription opioid non-medical useand abuse: position statement. 2003;69:215-232.
Principles of Analgesic Use in the Treatment of AcutePain and Cancer Pain.
4. American Pain Society. 4th ed. Glenview, Ill: American Pain Society; 1999.
Management of Cancer Pain: ClinicalPractice Guideline No. 9.
5. Jacox A, Carr DN, Payne R, et al. Rockville, Md: Agency for Health Care Policy andResearch, US Dept of Health and Human Services, Public Health Service; March1994. AHCPR publication 94-0592.
6. American Pain Society Web site. The use of opioids for the treatment of chronicpain: a consensus statement from American Academy of Pain Medicine andAmerican Pain Society. 1997. Available at:http://www.ampainsoc.org/advocacy/opioids.htm. Accessed February 20, 2002.
Guideline for the Management of Pain inOsteoarthritis, Rheumatoid Arthritis and Juvenile Chronic Arthritis: ClinicalPractice Guideline No. 2.
7. American Pain Society. Glenview, Ill: American Pain Society; 2002.
J Am Geriatr Soc.
8. American Geriatrics Society Panel on Persistent Pain in Older Persons. Themanagement of persistent pain in older persons. 2002;50(suppl6):S205-S224.
9. American Medical Association Web site. Report 11 of the Council on ScientificAffairs (A-99): use of opioids in chronic noncancer pain. 1999. Available at:http://www.ama-assn.org/ama/pub/article/2036-2341.html. Accessed March 1,2002.
West J Med.
10. Dickinson BD, Altman RD, Nielsen NH, Williams MA. Use of opioids to treatchronic, noncancer pain. 2000;172:107-115.
Guideline for the Management of Acute and ChronicPain in Sickle-Cell Disease: Clinical Practice Guideline No. 1.
11. American Pain Society. Glenview, Ill:American Pain Society; 1999.
J Law Med Ethics.
12. Gilson AM, Joranson DE, Maurer MA. Improving state medical board policies:influence of a model. 2003;31:119-129.
13. Federation of State Medical Boards Web of the United States, Inc. ModelPolicy for the Use of Controlled Substances for the Treatment of Pain. May 2004.Available at: http://www.fsmb.org. Accessed October 1, 2004.
Cancer Pract.
14. Dahl JL, Bennett ME, Bromley MD, Joranson DE. Success of the state pain initiatives:moving pain management forward. 2002;10(suppl 1):S9-S13.
PainManag Nurs.
15. Jacob E, American Pain Society. Pain management in sickle cell disease. 2001;2:121-131.
ClinRheumatol.
16. Katz WA. Musculoskeletal pain and its socioeconomic implications. 2002;21(suppl 1):S2-S4.
Cancer.
17. Anderson KO, Richman SP, Hurley J, et al. Cancer pain management amongunderserved minority outpatients: perceived needs and barriers to optimal control. 2002;94:2295-2304.
18. Robeznieks A. Pain specialists fear chilling effect after arrest. October 27,2003. Available at: http://www.ama-assn.org/amednews/2003/10/27/prsc1027.htm. Accessed February 20, 2004.
Clin J Pain.
19. Savage SR. Assessment for addiction in pain-treatment settings. 2002;18(suppl 4):S28-S38.
To Err Is Human: Building a Safer HealthSystem.
20. Kohn LT, Corrigan JM, Donaldson MS, Committee on Quality of Health Carein America, Institute of Medicine, eds. Washington, DC: National Academy Press; 1999.
J Law Med Ethics.
21. Ziegler SJ, Lovrich NP Jr. Pain relief, prescription drugs, and prosecution: afour-state survey of chief prosecutors. 2003;31:75-100.
J Pain Symptom Manage.
22. Joranson DE, Carrow GM, Ryan KM, et al. Pain management and prescriptionmonitoring. 2002;23:231-238.
J Health CareFinance.
23. Forgione DA, Neuenschwander P, Vermeer TE. Diversion of prescription drugsto the black market: what the states are doing to curb the tide. 2001;27:65-78.
Prescription Drugs: State Monitoring Programs Provide Useful Tool to ReduceDiversion.
24. US General Accounting Office Report to the Subcommittee on Oversight andInvestigations, Committee on Energy and Commerce, House of Representatives. Washington, DC: US General Accounting Office; May 2002.Publication GAO-02-634.
Diversion and Abuse of Prescription Drugs: A Closer Look atState Prescription Monitoring Programs.
25. US Department of Justice, Drug Enforcement Administration, DiversionControl Program. Alexandria, Va: Drug EnforcementAdministration; April 2000.
J Law Med Ethics.
26. Brushwood DB. Maximizing the value of electronic prescription monitoringprograms. 2003;31:41-54.
MedCare.
27. Weiner JP, Starfield BH, Steinwachs DM, Mumford LM. Development andapplication of a population-oriented measure of ambulatory care case-mix. 1991;29:452-472.
Am J Health Syst Pharm.
28. Armstrong EP, Manuchehri F. Ambulatory care databases for managed careorganizations. 1997;54:1973-1983.
QRB Qual RevBull.
29. Weiner JP, Powe NR, Steinwachs DM, Dent G. Applying insurance claimsdata to assess quality of care: a comparison of potential indicators. 1990;16:424-438.
30. Drug Abuse Warning Network Web site. Emergency department trends fromDAWN: final estimates 1994 to 2001. August 2002. DAWN Series, No. D-21.DHHS publication SMA-02-3635. Available at:http://dawninfo.samhsa.gov/pubs_94_02/edpubs/2001final/default.asp. AccessedOctober 10, 2003.
JAMA.
31. Hofer TP, Hayward RA, Greenfield S, Wagner EH, Kaplan SH, Manning WG.The unreliability of individual physician "report cards" for assessing the costs andquality of care of a chronic disease. 1999;281:2098-2105.
Spine.
32. Deyo RA, Taylor VM, Diehr P, et al. Analysis of automated administrative andsurvey databases to study patterns and outcomes of care. 1994;19(suppl):2083S-2091S.
