Skip to main content

Table 5 Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development (PROCEED Structure)

From: Pragmatic applications of implementation science frameworks to regulatory science: an assessment of FDA Risk Evaluation and Mitigation Strategies (REMS) (2014–2018)

PROCEED Phase

General Description

Description as applied to

REMS Assessments

Assessment Guidance Category

Definitions of Assessment Guidance Categories

5. Implementation

Design intervention;

Assess availability of resources;

Implement program

A. REMS Design (PRECEDE assumptions):

• Alignment with patient preferences

• Alignment with healthcare system and clinical workflow norms

• Alignment with health insurance requirements and medical practice guidelines

Not included

Not applicable

B. Availability of Resources:

• Capacity and capability of the healthcare personnel to fulfill REMS requirements (human resources)

• Availability (costs) of medical supplies and facilities necessary to fulfill REMS requirements (physical resources)

Not included

Not applicable

C. Program Outreach and Communication:

• Date of product or REMS launch

• Date communication materials distributed (by mode and frequency, including website access)

• Dates of training programs (by mode and frequency)

Program Outreach and Communication

Measures of the extent to which the REMS materials reached the intended stakeholders

6. Process Evaluation

Determine whether planned program put into action has been successfully implemented in target audience;

Ongoing evaluation and modification of program components

A. Program Implementation

At the Health Care Setting-, Health Care Provider- and Patient-Level

• Number successfully certified, enrolled or receiving the medication; proportion of target audience certified or enrolled.

• Characteristics of those certified, enrolled or receiving the medication compared with non-adopters in the same target audience – representativeness

• Unenrollment (voluntary) of health care providers (and reasons)

Program Implementation and Operations

Measures of the extent to which the intended stakeholders are participating in the program, how effectively the REMS program is being implemented and any unintended consequences such as patient access or burden to the healthcare system

B. Quality Process Improvement/System-Level Evaluation (by risk mitigation strategy)

• Unintended interruptions and/or delays

• Training curriculum fidelity and adaptation (by training modality)

• Audit findings and corrective actions, including analysis of Call Center complaints and dispensing systems

• Number (type) of REMS modifications

Program Implementation and Operations

Measures of the extent to which the intended stakeholders are participating in the program, how effectively the REMS program is being implemented and any unintended consequences such as patient access or burden to the healthcare system

7. Impact Evaluation

Determine effectiveness and efficiency of program in terms of intermediate objectives and change in predisposing, enabling, and reinforcing factors (immediate observable effects of the program); examines impact of program intervention on environmental and behavioral factors identified in PRECEDE phase of intervention design

A. Effectiveness: Knowledge and Skill

At the Health Care Provider- and Patient-Level

• Effectiveness of certification training

• Knowledge and awareness of key risk messages

Knowledge

Measures of the extent of stakeholders’ knowledge about the REMS-related risk or knowledge of any safe use conditions that are needed in order to mitigate the risk

B. Effectiveness: Compliance

At the Health Care Provider- and Patient-Level

• Behavioral intention (self-reported) to take appropriate risk mitigation or safe use behaviors

• Risk mitigation and safe use behaviors (observed)

• Decertification (required) of health care provider (and reasons)

Evaluation of Safe Use Conditions

Measures of the extent to which safe use conditions are being adopted or followed

C. Efficiency

• Evidence of integration into clinical workflows and perceived burden given health care professional preferences

• Patient access and perceived burden given patient preferences

• Unenrollment (voluntary) of patients (due to burden)

• Clinical efficiency (e.g., Number Needed to Screen to mitigate risk)

Program Implementation and Operations

Measures of the extent to which the intended stakeholders are participating in the program, how effectively the REMS program is being implemented and any unintended consequences such as patient access or burden to the healthcare system

D. Effectiveness: Environmental Factors

• Change in predisposing, reinforcing, and enabling behavioral factors

• Administrative and policy factors (e.g., treatment guidelines and insurance prescribing requirements)

Not included

Not applicable

8. Outcome Evaluation

Evaluate effectiveness of program implementation on outcomes identified in PRECEDE phase of intervention design and how health status has changed overall

o Mitigating the health risks (or surrogate outcomes) as stated in the REMS Goals

o Rates of expected and unexpected adverse events related to REMS implementation

o Unenrollment of patients (due to safety outcomes being targeted by the REMS program)

Health Outcomes and/or Surrogates of Health Outcomes

Adverse Event Surveillance

Measures of the safety-related health outcome of interest or a surrogate of a health outcome

  1. Not included: no appropriate category from the Assessment Guidance. Not applicable: not currently assessed in the REMS program to have application