Table 3 Item by Item Test-Retest & Inter-rater Reliability Showing % Agreement and Kappa Statistics (N = 24 Files)

History

Demographic Data Complete

subjective

S

87.50

0.3739

84.72

0.1410

History

Privacy Form present and complete

subjective

E

83.33

0.7467

59.72

0.3737

History

New Patient Form present and complete

subjective

S

66.67

0.3725

48.61

0.0111

History

Health Status Survey/Pain Diagram reviewed by clinician

objective

S

75.00

0.4000

66.67

0.2334

History

Outcome Measures appropriate

objective

S

A

79.17

0.5699

72.22

0.4198

History

Past Health History

subjective

S

A

62.50

−0.0385

66.67

0.1111

History

Family Health History

subjective

S

A

62.50

0.4114

47.22

0.1582

History

Social History

subjective

S

A

87.50

0.6066

66.67

0.0759

History

Flags

subjective

A

54.17

0.2000

37.50

−0.0679

History

Primary Complaint Complete

subjective

S

A

83.33

0.5556

50.00

−0.2101

History

Secondary Complaint (if applicable)

subjective

S

62.50

0.3684

45.83

0.1063

History

Consent to Physical Examination obtained and documented

subjective

S

A

100.00

1.0000

97.22

0.9434

History

Differential Diagnoses rendered

subjective

S

A

83.34

0.6129

75.00

0.4311

History

Complexity Completed

objective

A

87.50

0.6538

75.00

0.3069

Physical Examination

Observation/posture

subjective

S

95.84

0.8710

90.28

0.7483

Physical Examination

Vitals

subjective

S

A

75.00

0.4947

58.33

0.3411

Physical Examination

Range of Motion

subjective

E

79.17

0.5219

68.06

0.3196

Physical Examination

Palpation

subjective

E

100.00

1.0000

100.00

1.0000

Physical Examination

Orthopaedic Procedures

subjective

S

A

95.84

0.7857

88.89

0.5590

Physical Examination

Correlates with history

subjective

A

95.84

0.7857

91.67

0.6364

Physical Examination

Neurologic Procedures

subjective

S

A

95.84

0.8909

94.44

0.8633

Physical Examination

Procedures sufficient to: Exclude differential diagnoses

subjective

A

87.51

0.5714

86.11

0.4865

Physical Examination

Procedures sufficient to: Render clinical diagnosis

subjective

S

A

100.00

1.0000

91.67

0.5917

Physical Examination

Complexity Completed

objective

A

91.67

0.7612

86.11

0.5740

Diagnosis

Diagnoses rendered

subjective

S

A

95.84

0.8182

97.22

0.8786

Diagnosis

Appropriate and supported by findings

subjective

A

95.84

0.7857

80.56

0.3429

Diagnosis

Complexity Completed

objective

A

95.84

0.8806

91.67

0.7690

Plan of Management

Further Evaluation

subjective

S

87.51

0.5294

80.56

0.4661

Plan of Management

Frequency, duration

subjective

S

A

95.84

0.8182

90.28

0.5758

Plan of Management

Therapy details

subjective

S

A

83.34

0.4353

83.33

0.4368

Plan of Management

Active program - planned/ detailed

subjective

A

79.17

0.5604

62.50

0.1914

Plan of Management

Evidence-based

objective

A

95.84

0.73333

86.11

0.3182

Plan of Management

Complexity Completed

objective

A

87.50

0.6418

88.89

0.6883

Plan of Management

Dx & POM - Verified by clinician

objective

E

91.67

0.6364

83.33

0.4525

Goals & Outcomes

Appropriate Specific & measurable goals & outcomes present

objective

A

75.00

0.4217

63.89

0.0517

Goals & Outcomes

Repeat OM measures administered

subjective

A

66.67

0.3786

51.39

0.1286

Prognosis

Prognosis appropriately defined

subjective

S

A

91.67

0.7209

80.56

0.4540

Prognosis

Short term & long term prognosis, support by positive and negative prognostic factors

subjective

S

A

75.00

0.5826

58.33

0.3463

Consent / ROF

ROF complete

subjective

S

A

87.50

0.6539

80.56

0.5512

Consent / ROF

Risks – major/minor & benefits

subjective

S

A

91.67

0.7176

91.67

0.7512

Consent / ROF

Recording of patient Questions in ROF

objective

S

87.51

0.7714

97.22

0.9428

Consent / ROF

ROF signed by patient

objective

S

95.83

0.8571

94.44

0.8093

Consent / ROF

ROF signed by clinician

objective

S

95.83

0.8571

94.44

0.8093

Consent / ROF

ROF/IC current

objective

S

A

95.83

0.8571

94.44

0.8093

Consent / ROF

IC signed by patient:

objective

S

95.83

0.8571

91.67

0.7323

Consent / ROF

IC signed by clinician

objective

S

95.83

0.8571

94.44

0.8093

Consent / ROF

IC signed by intern

objective

E

95.83

0.8571

94.44

0.8093

Case Report / Physicians letter

Clear concise narrative

subjective

E

70.83

0.4167

63.89

0.1628

Case Report / Physicians letter

Professional format

subjective

E

75.00

0.3077

76.39

0.2842

Documentation

Collaborative care/correspondence (if applicable)

subjective

A

75.00

0.4667

70.83

0.2738

Documentation

Exchange of Medical Info form present & complete

objective

E

87.50

0.8059

34.72

−0.0349

Documentation

Dashboard (brown) boxes current & complete (Social Hx, Med.Hx, Ongoing Concerns, Reminder Section)

subjective

S

A

70.84

0.4894

66.67

0.3410

Documentation

Time to conditional case sign-off < 7 days

subjective

E

50.00

0.3043

30.56

−0.0164

Documentation

Case Signed off

objective

E

87.50

0.5714

73.61

0.2392

Documentation

SOAP notes: - Complete

subjective

S

87.50

−0.0588

77.78

−0.0159

Documentation

Clinician verification of SOAP

objective

S

95.83

0.0000

94.44

−0.0286

Documentation

Response to care documented in SOAP

subjective

S

79.16

0.5200

66.67

0.1787

Documentation

Timely uploading of documents

objective

S

A

62.50

0.2703

63.89

0.2034

Documentation

Compliance documented in file

subjective

S

A

66.66

0.4037

48.61

−0.0238

Re-evaluation

Completed as scheduled in POM

subjective

S

A

87.50

0.8059

29.17

−0.0588

Re-evaluation

Includes relevant details - Thorough & Complete

subjective

S

A

74.99

0.6308

29.17

−0.0217

N = 38

N = 37

N = 10

N = 61

N = 61

N = 61

N = 61

32/38 (84%)

31/37 (84%)

9/10 (90%)

52 > 70%

k > 0.6 32/61

37 > 70%

k > 0.6 16/61

23/38 (60%)

23/37 (60%)

5/10 (50%)