Table 1 Notation

 IP

Inpatient

 OP

Outpatient

 EP

Emergency patient

i − th regular service period, i = 1, …, N

k − th overtime service period, k = 1, …, K

Patient type,  j = IP, OP, EP

h − th outpatient scheduled for regular service period i, \( h=1,\dots, {Ag}_i^{\mathrm{OP}} \)

 t_i, t_(N + k)

Start of i − th regular period and of k − th overtime period

 t_i + 1, t_{(N + k + 1)}

End of i − th regular period and of k − th extra period

 \( {w}_i^j \)

Number of type j patients waiting for service at the start of regular period i, for j = IP, OP, EP

 \( {w}_{\left(N+k\right)}^j \)

Number of type j patients waiting for service at the start of overtime period N + k, for j = IP, OP

 Z

Space containing all possible states of N regular periods

 Z_i

Set of all feasible states at the start of regular period i, immediately before waiting patients are selected for service, such that Z_i ∈ Z

 z_i

State at the start of regular period i, such that \( {z}_i=\left({w}_i^{\mathrm{IP}},{w}_i^{\mathrm{OP}},{w}_i^{\mathrm{EP}}\right)\in {Z}_i \), gives the number of IPs, OPs and EPs waiting to be served

 |z_i|

Sum of elements in state z_i, \( \left|{z}_i\right|={w}_i^{\mathrm{IP}}+{w}_i^{\mathrm{OP}}+{w}_i^{\mathrm{EP}} \)

 S

Space containing all possible states of K overtime periods

 S_k

 s_k

State at the start of overtime period k, such that \( {s}_k=\left({w}_{\left(N+k\right)}^{\mathrm{IP}},{w}_{\left(N+k\right)}^{\mathrm{OP}}\right)\in {S}_k \), gives the number of IPs and OPs waiting to be served

 |s_k|

Sum of elements in state s_k, \( \left|{s}_k\right|={w}_{\left(N+k\right)}^{\mathrm{IP}}+{w}_{\left(N+k\right)}^{\mathrm{OP}} \)

 A

Set of all possible actions in N regular periods

 \( {A}_{z_i} \)

Set of all feasible actions in state z_i, such that \( {A}_{z_i}\in A \)

 a_i

Action taken in regular period i, such that \( {a}_i=\left({a}_i^{\mathrm{IP}},{a}_i^{\mathrm{OP}},{a}_i^{\mathrm{EP}}\right)\in {A}_{z_i} \), represent all IPs, OPs and EPs selected for service

 B

Set of all possible actions in K overtime periods

 \( {B}_{s_k} \)

Set of all feasible actions in state s_k, such that \( {B}_{s_k}\in B \)

 b_k

Action taken in overtime period k, such that \( {b}_k=\left({b}_{\left(N+k\right)}^{\mathrm{IP}},{b}_{\left(N+k\right)}^{\mathrm{OP}}\right)\in {B}_{s_k} \)_, represent all IPs and OPs selected for service

 N

Total number of regular periods

 K

Total number of overtime periods

 \( {p}_i^j \)

Arrival probability of patient type j during regular period i, for j = IP, OP, EP

 Ag^OP

Appointment schedule of OPs

 \( {Ag}_i^{\mathrm{OP}} \)

Number of OPs scheduled in regular period i

 C_i, C_(N + k)

Service capacity in each service period, given by the number of equipment available in regular periods i and in overtime periods N + k

 wc^j

Individual waiting cost for patient type j during regular period, for j = IP, OP

 oc^j

Individual overtime cost of serving patient type j, for j = IP, OP

 pc^j

Individual penalty cost for not serving patient type j, for j = IP, OP

 P_i

Transition probability between states i and i + 1

 \( {P}_i^j \)

Transition probability between states i and i + 1 for patients type j, such that j = IP, OP, EP

 wc_i

Total waiting cost during regular period i

 oc_k

Total overtime cost in period k

 pc_{N + K + 1}

Total penalty cost for not providing service to patients

 TC_N + K

Total overall cost during a finite horizon decision period (one business day), with N + K decision points

 V_i(z_i)

Minimum expected cost for each regular period i

 V_k(s_k)

Minimum expected cost for each overtime period k

 V_{N + K + 1}(s_{N + K + 1})

Minimum expected penalty cost