A Quality Indicator Framework for High-Risk Areas in Obstetrical Care

Published: October 2019

Authors: Dr. Lisa A. Calder, Cara L. Bowman, Qian Yang, Tunde Gondocz, Christina Young, Cathy Zhang, Anna MacIntyre, Renee Darling, Dr. J. Peter O’Neill, Dr. Charmaine Roye, Dr. Guylaine Lefebvre

Acknowledgments: The authors wish to thank Dr. Sharon Caughey for her contributions to the framework, and Ria De Gorter for her assistance in preparing this manuscript. We also thank Joanna Noble from the Healthcare Insurance Reciprocal of Canada for reviewing and commenting on the manuscript.

How to cite this document >>


We developed a comprehensive set of pragmatic quality improvement indicators for areas of obstetrical practice associated with increased medical-legal risk. An analysis of two national databases identified medical-legal high-risk areas, and a systematic review of quality improvement initiatives located relevant published studies. We selected metrics that mapped to the high-risk areas, and developed new indicators and measures to address gaps. Five areas of increased medical-legal risk were identified among 691 closed cases (2010-2014), and five published quality indicator frameworks were located. We developed 35 process of care, 21 clinical care, and 4 balancing indicators and measures for assisted vaginal delivery, labour induction and augmentation, shoulder dystocia, collaborative care, and decision to perform Caesarean section. Quality improvement teams and researchers can use this framework to facilitate quality of care improvements in obstetrics.


There has recently been an emphasis on the necessity of improving obstetrical care worldwide.1-3 Obstetrical care is also an important focus for medical-legal risk reduction because, while the frequency of medical-legal cases is relatively rare, the severity of harm for patients and their families and subsequent costs to healthcare systems are high. It has long been purported that engaging in quality improvement efforts may also lead to a potential reduction in medical-legal risk.4-6 We have recently published an analysis that demonstrates that reduction in patient safety indicators is associated with reduced medical-legal risk when examined regionally across Canada.7

Successful quality improvement efforts rely on appropriate measurement. While this is an intuitive principle, there has been a recent call to measure what matters.8 Not only is there currently an excess of measurements that do not matter, there is also a tension between what is measured for epidemiological research and what is necessary to measure the impact of quality improvement efforts. Too often, quality improvements are impeded by measurement of only clinical outcomes, for example maternal mortality, which, while important, is so rare in developed countries that detecting a signal of improvement at a local level is significantly challenging.

To advance the quality of care in obstetrics and, hence, decrease medical-legal risk, it is essential that appropriate quality measures are selected. In a joint report with several other Canadian organizations including Accreditation Canada, Salus Global and the Healthcare Insurance Reciprocal of Canada (HIROC), we identified that there are five high-risk areas of practice in obstetrics.9 These include: 1) Assisted vaginal delivery; 2) Induction and augmentation of labour; 3) Shoulder dystocia; 4) Collaborative care; and 5) Decision to delivery time for Caesarean section. We created a quality indicator framework to facilitate quality improvement efforts in obstetrical care for the five identified high-risk areas of obstetrics based on medical-legal data.


We used two data sources to set a foundation for this framework: a medical-legal case review and a systematic literature review.

The medical-legal case review was a retrospective analysis of 691 cases from 2010-14 involving obstetrical care but excluding midwifery care.10 The cases were sourced from two medical-legal organizations: the Canadian Medical Protective Association, a not-for-profit organization currently representing over 100,000 physicians and the Healthcare Insurance Reciprocal of Canada, the largest provider of medical liability insurance for Canadian healthcare organizations and their employees. The cases were coded using international, national, and internal frameworks, and a descriptive analysis was performed.11 A summary of the results is presented in Table 2.

We performed a systematic literature review to identify quality improvement studies in obstetrics for the five high-risk areas. We searched Medline, CINAHL, Database of Health Technology Assessment, Cochrane databases from 2005-2016, as well as grey literature from websites and repositories of relevant organizations and conferences. Two reviewers independently screened titles, abstracts and full text articles. We found 6,193 citations and included 73 articles. The detailed methods are published elsewhere.12 A summary of results is presented in Table 3. This review identified that there was a paucity of robust quality indicators for the areas of interest. Subsequently, we performed expert consultation with local obstetricians and nurses to determine challenges in measurement in obstetrics quality improvement.13

Based on quality improvement measurement principles, we developed criteria for robust quality metrics.14 (See Table 2.) This allowed us to evaluate pre-existing quality measures from the literature. We were interested in overall measures to inform quality improvement efforts. We used the Institute for Health Improvement’s 3 types of measures: 1) process, 2) outcome and 3) balancing.15 We chose not to focus on structural or patient experience measures in this framework.16 We refined our framework to distinguish measures from indicators. We defined measures according to the National Quality Forum as reference points against which other things can be measured, to bring into comparison against a standard. We defined indicators as a type of measure which gave an indication of a quality of care issue rather than a specific measurement.16 Thus, the quality indicator framework includes process measures and indicators, clinical outcome measures and indicators, and balancing measures and indicators. We defined a clinical outcome measure as a defined result of care provided.17

We selected relevant quality indicators and measures from the literature that mapped to the identified high-risk medical-legal areas, and developed new potential measures and indicators as necessary.18-22 We also examined the national hospital discharge database and a provincial birth registry to which the hospitals were already submitting data and prioritized those measures that could be constructed with existing data. Subsequently, we created the quality indicator framework with expert consultation and iterative refinement. We evaluated the use of select indicators in a pilot project conducted with obstetrical teams at a local community hospital. The purpose of this focused evaluation was to determine the face validity, or a subjective assessment of whether the indicators measure what they were intended to measure. The pilot project was designed to determine the effectiveness of an educational intervention focused on implementing quality improvement projects in obstetrics. The choice of project was based on the identified high-risk areas of practice in the medical-legal case review. The team of 157 frontline healthcare providers were coached on measurement using the quality indicator framework described in this paper.


We report 35 metrics in the quality indicator framework, 39 quality indicators, and 21 measures in Table 1. During the pilot project, we worked with a team that used measures for the decision to perform a Caesarean section. Our definition included the time leading up to the decision to perform the procedure as well as the time from the decision to the procedure occurring. Participants reported that the measures were pragmatic and easy to record. This was despite the fact that system issues contributing to delayed decisions are frequently not recorded in health records and, as such, can be harder to detect.


We note several limitations with our quality indicator framework. It is possible we missed key articles with our systematic literature review. We also relied on expert consensus to determine our final list of measures, and that may have resulted in exclusion of potentially useful measures. The measures we propose are restricted to five areas of increased medical-legal risk and do not reflect all aspects of obstetrics care. Finally, not all measures have been validated, and some process of care and clinical measures may not be achievable at some sites depending on data collection resources and the robustness of health record documentation. These metrics are intended to be a starting point and will require local contextualization and, in many cases, further definition to allow for robust application.


We created a framework that is pragmatic and measures what matters to improve obstetrical care. The framework focuses on five data-driven high-risk areas in obstetrics that we believe deserve quality improvement focus. We also believe that efforts in these areas will show improvements in the measures provided, and subsequently be associated with reductions in medical-legal risk. As such, this framework may be used as a tool to guide future quality improvement efforts and contribute to such improvements and reductions in medical-legal risk.

Previous studies have described quality indicators for specific areas of obstetrics, with the majority focused on maternal mortality secondary to postpartum hemorrhage, overall Caesarean section rate, and elective inductions.23,24 We were unable to find in our systematic literature review any universally accepted standards or guidelines for measurement of quality improvement in obstetrics. The UK Royal College of Obstetricians and Gynaecologists developed 18 quality indicators for 7 main categories, of which there is overlap for induction of labour, use of instruments and somewhat for the decision to delivery time for urgent Caesarean section.1 Likewise, Boulkedid et al conducted a French Delphi panel to determine quality indicators for antenatal and inpatient obstetrical care.23 They identified 28 indicators with some relevance to induction of labour and assisted delivery.23 We note that in this document, the vast majority of indicators are clinical, with few process of care measures, and no balancing measures. 

The Joint Commission, Centers for Medicare and Medicaid Services along with the National Quality Forum in the United States released a set of quality measures that for the hospital setting were all clinical measures. These included incidence of episiotomy, elective deliveries >= 37 weeks and < 39 weeks, Caesarean section rates, antenatal steroid prescription, and exclusive breast milk feeding. While these measures may be useful as a broad illustration of obstetrical quality of care at an institutional level, they are insufficient to improve any of the supporting clinical processes. Process of care measures and balancing measures are essential to appreciate the impact of local quality improvement efforts because they are often more timely, easier to measure, occur with sufficient frequency to determine local impact of interventions, and can uncover unintended consequences.

We also found one regional obstetric quality improvement evaluation study by Salus Global, a company that has created a specific quality improvement program to improve institutional obstetrical care in Canada.25 In this article, the outcomes used were all clinical with the exception of length of stay at maternal hospitals. The levels of improvements were not clinically significant (with the exception of length of stay) but may have been more demonstrable if process of care outcomes were utilized. Overall, we noted in the literature a lack of consistency in quality measure use across studies for the five high-risk areas we identified, which limits the comparability of interventions.


Currently, there is a broad variety of published quality improvement efforts in obstetrics, but a lack of focus on key clinical areas, with the exception of post-partum hemorrhage. After analysis of the medical-legal cases in two national databases, we believe there is a pressing need to improve the quality of care in the five identified areas of risk. This quality indicator framework is intended to help address this need.


This framework can help advance efforts so that those working in quality improvement can determine the impact of their interventions. We encourage researchers and quality improvement professionals to consider carefully what measures they are using and the measures we have put forth in this framework.

High-risk Area Assisted vaginal delivery Unless otherwise stated, all values would be calculated as a proportion of assisted vaginal deliveries

Process-of-care Indicators

  • Existence of protocol* (y/n)
  • Adherence of protocol to SOGC** guidelines (low/mod/high)
  • Usability of protocol (low/mod/high)
  • Adherence to protocol where in place
  • Documentation of training/simulation to maintain assisted vaginal delivery skills for obstetrics (y/n)
  • Documentation of handovers between providers

Clinical Outcome Indicators

  • Number of assisted delivery pulls per vacuum delivery
  • Failed forceps delivery leading to Caesarean section delivery
  • Neonatal Intensive Care Unit admission

Balancing Indicators

  • Caesarean section delivery

Process-of-care Measures

  • Delay-to-delivery time (if delivered by Caesarean section)
  • Both forceps and vacuum used for a single delivery

Clinical Outcome Measures

  • Poor maternal outcome related to assisted vaginal delivery (e.g. 3rd/4th degree, cervical/high vaginal tears, episiotomy dehiscence)
  • Poor neonatal outcome potentially related to delivery (e.g. hypoxic encephalopathy)
  • Neonatal scalp abrasion, scalp hematoma, facial abrasion, facial hematoma, neonatal jaundice
  • Post-partum hemorrhage requiring transfusion

Balancing Measures

  • Poor maternal outcome related to Caesarean section (e.g. sepsis, PPH, wound dehiscence/infection)
High-risk Area Induction and augmentation of labour Unless otherwise stated, all values would be calculated as a proportion of deliveries involving induction or augmentation

Process-of-care Indicators

  • Existence of protocol* (y/n)
  • Adherence of protocol to SOGC guidelines (low/mod/high)
  • Usability of protocol (low/mod/high)
  • Adherence to protocol where in place (e.g. patients induced as per time specified on registration form)

Clinical Outcome Indicators

  • Elective inductions <39 weeks (as a proportion of all inductions)
  • Cervical ripening agents used
  • Uterine tachysystole resulting in abnormal fetal heart rate
  • Number of assisted delivery pulls per vacuum delivery

Balancing Indicators

  • Failed or prolonged inductions

Process-of-care Measures

  • Oxytocin threshold exceeded (as a proportion of deliveries using oxytocin)

Clinical Outcome Measures

  • Caesarean section delivery
  • Uterine rupture
  • Post-partum hemorrhage requiring transfusion
  • Assisted delivery
  • Poor neonatal outcome (e.g. hypoxic encephalopathy) potentially related to delivery –APGARs (5'), pH
High-risk Area Shoulder dystocia Unless otherwise stated, all values would be calculated as a proportion of deliveries involving shoulder dystocia

Process-of-care Indicators

  • Existence of protocol* (y/n)
  • Adherence of protocol to SOGC guidelines (low/mod/high)
  • Usability of protocol (low/mod/high)
  • Adherence to protocol where in place
  • Evidence that neonatal size was estimated prior to labour
  • Presence of risk assessment for shoulder dystocia
  • Documentation of shoulder dystocia management

Clinical Outcome Indicators

  • Number of assisted delivery pulls per vacuum delivery

Process-of-care Measures

  • Delay-to-delivery time (if delivered by Caesarean section)

Clinical Outcome Measures

  • Assisted delivery
  • Poor neonatal outcome potentially related to delivery (e.g. hypoxic encephalopathy)
  • Neonatal brachial plexus injury identified while in hospital
  • Neonates requiring resuscitation

Balancing Measures

  • Caesarean section delivery
High-risk Area Collaborative care Unless otherwise stated, all values would be calculated as a proportion of all deliveries

Process-of-care Indicators

  • Number of interprofessional huddles
  • Documentation of structured handover (proportion of handovers overall)
  • Existence of protocol for shared care*** (y/n)
  • Documented patient complaints (from all hospital sources) due to critical information not being shared
  • Intentionally elicited feedback from team by manager (patient centeredness)
  • Delay in care resulting from a lack of clarity in making healthcare decision
  • Transfer of care between family physician and obstetrician or midwife and obstetrician while on unit (rate and timing)
  • Missing antenatal record and ultrasound records
High-risk Area Decision to delivery time for Caesarean section Unless otherwise stated, all values would be calculated as a proportion of deliveries involving a Caesarean section

Process-of-care Indicators

  • Both forceps and vacuum used for a single delivery
  • Evidence in chart that discussions took place about decision (rationale for decision documented)
  • Evidence of delay to delivery, where there is evidence of miscommunication (any team members) contributing to delay (as a proportion of cases with a delay to delivery)
  • Evidence in chart of delays due to lack of availability of operating room or equipment issues

Process-of-care Measures

  • Excessive delay in time to decision (needs expert consensus based on local operating room timings and clinical conditions)
  • Excessive delay in time from decision to incision (needs expert consensus based on local operating room timings and clinical conditions)

  • *Consider evaluation of quality of protocol.
  • **SOGC = Society of Obstetricians and Gynaecologists of Canada
  • ***This assumes sound policy with process in place, accountabilities outlined, and mechanism(s) to monitor team functioning. Can include protocol for transfer of care between providers.

Collapse section

Areas of practice at high risk:

  • Induction and augmentation of labour with oxytocin
  • Intrapartum fetal surveillance
  • Assisted vaginal delivery
  • Decision to delivery time for Caesarean section
  • Management of shoulder dystocia

Phases of care involving a patient safety incident were most frequently intrapartum delivery but often involved more than one phase of care.

Provider factors were the most frequent contributing factor, and included:

  • Provider decision-making
  • Lack of situational awareness
  • Team communication

System factors were also identified, and included:

  • Inadequate processes and protocols
  • Second-on-call contingency plan issues
  • Resourcing issues

Collapse section

Areas of practice at high risk Number of articles
Induction and augmentation of labour with oxytocin 30
Collaborative care 24
Management of shoulder dystocia 17
Assisted vaginal delivery 7
Decision to delivery time for Caesarean section 1
Other areas of focus for quality improvement in obstetrics Number of articles
Post-partum hemorrhage 13
Intrapartum fetal surveillance 12
Non-urgent Caesarean section 7

Collapse section


  1. Carroll F, Knight H, Cromwell D, Gurol-Urganci I, van der Meulen J. Patterns of maternity care in English NHS trusts. London, United Kingdom: The Royal College of Obstetricians and Gynaecologists; March 2016.
  2. Curtin SC, Gregory KD, Korst LM, Uddin SFG. Maternal morbidity for vaginal and cesarean deliveries, according to previous cesarean history: New data from the birth certificate, 2013. U.S. Department of Health and Human Services, Centers of Disease Control and Prevention; May 20 2015.
  3. Rushwan H. The FIGO leadership in Obstetrics and Gynecology for Impact and Change (LOGIC) Initiative in Maternal Newborn Health. International Journal of Gynecology and Obstetrics. 2014;127(Supplement 1):S1-S2.
  4. Sakala C, Yang YT, Corry MP. Maternity care and liability: most promising policy strategies for improvement. Womens Health Issues. 2013;23(1):e25-37.
  5. Dalton GD, Samaropoulos XF, Dalton AC. Improvements in the safety of patient care can help end the medical malpractice crisis in the United States. Health Policy. 2008;86(2-3):153-162.
  6. Pettker CM, Thung SF, Lipkind HS, et al. A comprehensive obstetric patient safety program reduces liability claims and payments. American Journal of Obstetrics & Gynecology. 2014;211(4):319-325.
  7. Yang Q, Zhang C, Hines K, Calder LA. Improved hospital safety performance and reduced medical-legal risk: an ecological study using 2 Canadian databases. CMAJ Open. 2018;6(4):E561-566.
  8. Berwick DM. Era 3 for Medicine and Health Care. JAMA. 2016;315(13):1329-1330.
  9. Obstetrics Services in Canada: Advancing Quality and Strengthening Safety. Ottawa, Ontario: Accreditation Canada, the Healthcare Insurance Reciprocal of Canada, the Canadian Medical Protective Association, Salus Global Corporation;2016.
  10. Delivery in focus: Strengthening obstetrical care in Canada. Ottawa, Ontario: Healthcare Insurance Reciprocal of Canada and the Canadian Medical Protective Association;2018.
  11. McCleery A, Devenny K, Ogilby C, et al. Using medicallegal data to support safe medical care: A contributing factor coding framework. J Healthc Risk Manag. 2018.
  12. A systematic review of quality improvement interventions in labour and delivery. PROSPERO 2016 CRD42016052118 2016. Available from: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42016052118.
  13. Lefebvre G, Calder L, Hines K, et al. Methods and lessons from a focused quality improvement program in obstetrics. Society of Obstetricians and Gynaecologists of Canada Annual Clinical and Scientific Conference 2017; June 20-23, 2017; Ottawa, Canada.
  14. Science of Improvement: How to Improve. Institute for Healthcare Improvement. Available from: http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementHowtoImprove.aspx.
  15. Science of improvement: Establishing Measures. Institute for Healthcare Improvement.  Available from: http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementEstablishingMeasures.aspx.
  16. Campbell SM, Braspenning J, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002;11:358-364.
  17. The ABCs of Measurement. Washington, DC: National Quality Forum;2015.
  18. Arora KS, Shields LE, Grobman WA, D’Alton ME, Lappen JR, Mercer BM. Triggers, Bundles, Protocols, and Checklists – What Every Maternal Care Provider Needs to Know. American Journal of Obstetrics & Gynecology. 2015.
  19. Bonfill X, Roque M, Aller MB, et al. Development of quality of care indicators from systematic reviews: the case of hospital delivery. Implementation Science. 2013;8:42.
  20. Measuring Patient Harm in Canadian Hospitals: Technical Report. Ottawa, ON: Canadian Institute for Health Information;2016.
  21. How-to Guide: Prevent Obstetrical Adverse Events. Cambridge, MA: Institute for Healthcare Improvement;2012.
  22. Vincent C, Burnett S, Carthey J. The measurement and monitoring of safety. London, United Kingdom: The Health Foundation; April 2013.
  23. Boulkedid R, Sibony O, Goffinet F, Fauconnier A, Branger B, Alberti C. Quality indicators for continuous monitoring to improve maternal and infant health in maternity departments: a modified Delphi survey of an international multidisciplinary panel. PLoS One. 2013;8(4):e60663.
  24. Melman S, Schoorel EC, de Boer K, et al. Development and Measurement of Guidelines-Based Quality Indicators of Caesarean Section Care in the Netherlands: A RAND-Modified Delphi Procedure and Retrospective Medical Chart Review. PLoS One. 2016;11(1):e0145771.
  25. Thanh NX, Jacobs P, Wanke MI, Hense A, Sauve R. Outcomes of the introduction of the MOREOB continuing education program in Alberta. Journal of Obstetrics and Gynaecology Canada. 2010;32(8):749-755.
  26. Calder L, Bowman C, De Gorter R, Wright E, Yang Q, Lefebvre G. A systematic literature review of quality improvement initiatives in obstetrics. Institute for Healthcare Improvement’s Scientific Symposium on Quality Improvement in Health Care , December 11 2017; Orlando, Florida.

Copyright information: The Canadian Medical Protective Association retain all intellectual property rights for the information presented here, unless otherwise specified.

How to cite this document: Calder LA, Bowman CL, Yang Q, Gondocz T, Young C, Zhang C, MacIntyre A, Darling R, O’Neill JP, Roye C, Lefebvre G. 2019. A quality indicator framework for high-risk areas in obstetrical care. Unpublished manuscript, Canadian Medical Protective Association.