Skip to main content

Table 4 Indicators for the final evaluation of a health systems intervention to improve care during and after a pregnancy complicated by diabetes

From: Improving systems of care during and after a pregnancy complicated by hyperglycaemia: A protocol for a complex health systems intervention

Objective

Final evaluation question addressed*

Indicator

Data source

Data collection

REACH

 Increase (FNQ) and sustain (NT) engagement of clinicians with the project

3

Health practitioner awareness of Partnership and activities

Health professionals

Interviews

3

Health practitioner attendance at Partnership education events

Project activity log

Surveys

3

Use of project online health professional educational resources

Website

Activity log

Metrics from website

 Improve health practitioner awareness of DIP Clinical Register

3

Health practitioner awareness of DIP Clinical Register

Health professionals

DIP Clinical Register

Interviews

Surveys

 Increase (FNQ) and sustain (NT) coverage of the DIP Clinical Register

1

DIP Clinical Register coverage; trajectory of coverage over time

Comparison of DIP Clinical Register with health service data

Health service reports

 Determine the number and characteristics of women accessing and not accessing antenatal care

 

Number of women accessing antenatal care, including number and timing of visits

Health service electronic health records

DIP Clinical Register

Health service reports1

Audit

EFFECTIVENESS

 Enhance support for health practitioners

1

Health practitioner perception of support

Health professionals

Interviews

4

Health practitioner and champion reports of which activities have been useful in enhancing support

Champions

Surveys

 Increase health practitioner awareness of and confidence in managing hyperglycaemia in pregnancy

1

Health practitioners perceived knowledge and confidence, and changes from Partnership formative work

Health professionals

Health service electronic health records

Interviews

Surveys

1

Rates of completion of recommended glucose screening in early pregnancy for high risk women

DIP Clinical Register

Formative DIP Models of Care work

Audit

 Earlier hyperglycaemia in pregnancy screening women at high risk

1

Rates of completion of recommended early pregnancy screening for high risk women

DIP Clinical Register

Audit

 Improved blood glucose levels for women with diabetes in pregnancy

2

Mean first- and third-trimester HbA1c and changes over time

DIP Clinical Register

Audit

 Improved birth and neonatal outcomes

2

Gestational age at delivery

Mode of delivery

Birth weight

Large for gestational age

Small for gestational age

Macrosomia

Neonatal obstetric trauma

Neonatal hypoglycaemia requiring treatment

Neonatal special care admission

5-minute APGAR score less than 5

Neonatal jaundice requiring treatment

Neonatal respiratory distress

DIP Clinical Register

Audit

 Improve health practitioners’ awareness of postpartum guidelines

1

Health practitioner awareness of guidelines and changes over time

Health professionals

Champions

Interviews

Surveys

 Improve postpartum management, according to guidelines, following diabetes in pregnancy

2

Proportion of women completing postpartum glucose testing

Postpartum weight, body mass index, waist circumference

Proportion of women breastfeeding

Proportion of women smoking

Proportion of women prescribed contraception, or who have discussed contraception with a health practitioner

Changes over time in all indicators

Electronic health records

DIP Clinical Register

Audit

 Enhance communication between primary healthcare and hospital services

1

Health practitioner perception of communication between primary healthcare and hospital services

Health professionals

Interviews

3

Health practitioner and champion reports of which activities have contributed to changes

Champions

Surveys

 Improve referral pathways and care coordination for services caring for women with hyperglycaemia in pregnancy

1

Health practitioner knowledge of referral pathways

Health professionals

Interviews

1

Health practitioner perception of improvements in care coordination

Champions

Surveys

3

Health practitioner and champion reports of which activities have contributed to changes

 Improve discharge processes

1

Health practitioner perception of usefulness of discharge summaries

Health professionals

Interviews

1

Health practitioner and champion perception of impact of discharge processes on postpartum care

Champions

Surveys

ADOPTION

 Enhance referrals to DIP Clinical Register

1

DIP Clinical Register coverage

Comparison of DIP Clinical Register with health service data

Health professionals

Health service reports1

Interviews

Surveys

3

Health service perceptions of referral process

 Improve practitioner use of DIP Clinical Register

1

Health practitioner use of DIP Clinical Register and reports

Health professionals

Interviews

3

Health practitioner reports of which aspects of reports are useful in practice

DIP Clinical Register

Surveys

External use of DIP Clinical Register website, e.g. website metrics

 Identify enablers and barriers impacting on adoption of project activities

3

Health practitioner, implementer and champion reports of enablers and barriers

Health professionals

Implementers

Champions

Interviews

 Determine acceptability and value of project activities

  •Are project activities socially appropriate/ acceptable?

  •What is the social importance of project outcomes?

  •Which project activities are perceived as valuable?

4

Health practitioner, implementer and champion perceptions of acceptability and value of project activities

Health professionals

Implementers

Champions

Women

Interviews

IMPLEMENTATION

 Determine if project activities have been delivered as intended

3

Proportion of planned activities delivered

Project activity log

Implementers

Interviews

Audit of activity log

 Determine if project activities have been adapted, e.g. to fit local needs

3

Adaptations of planned activities and rationale

Implementers

Interviews

 Identify enablers and barriers impacting on implementation of project activities

3

Enablers and barriers as identified by implementation team

Implementers

Health professionals

Champions

Interviews

MAINTENANCE

 Sustain DIP Clinical Register through integration with other structures

5

Health practitioners and services perceptions of sustainability of the DIP Clinical Register

Resources required and cost of maintaining DIP Clinical Register

Health professionals

Champions

Implementers

Activity log

Interviews

Surveys

Cost-consequences analysis

 Identify project activities sustainable beyond project completion, and method for funding or integration into existing services

5

Health professional, champion and implementer perception of sustainability of project activities

Resources required for project activity sustainability

Health professionals

Champions

Implementers

Activity log

Interviews

Cost-consequences analysis

  1. *Evaluation questions: 1. To what degree has the health systems intervention improved systems of care during and after a pregnancy complicated by hyperglycaemia?; 2. To what degree has the health systems intervention improved maternal and neonatal outcomes during and after a pregnancy complicated by hyperglycaemia?; 3. If improvements are demonstrated, which activities of the health systems intervention have contributed to this improvement, and what enablers and barriers have impacted on the success of these activities?; 4. Are activities considered socially valid by healthcare providers, champions and stakeholders?; 5. How do the resources required for project activities balance against the benefits?; 6. How can the Partnership support the continuation of successful project activities after completion of the health systems intervention?
  2. Abbreviations: NT Northern Territory, FNQ Far North Queensland
  3. 1NT Midwives’ Data Collection, FNQ Queensland Health Case Mix reports