Content uploaded by Ali Gharbi
Author content
All content in this area was uploaded by Ali Gharbi on Aug 28, 2016
Content may be subject to copyright.
11th International Conference on Modeling, Optimization and SIMulation - MOSIM’16
August 22-24
Montr´eal, Qu´ebec, Canada
“Innovation in Technology for performant Systems”
ADAPTIVE CONTROL OF HYBRID FAILURE-PRONE
MANUFACTURING SYSTEMS UNDER DEMAND VARIATION
AND UNCERTAINTY
Vladimir POLOTSKI, Jean-Pierre KENNE and Ali GHARBI
Ecole de Technologie Superieure Ecole de Technologie Superieure
Mechanical Engineering Automated Production Engineering
1100 Notre-Dame Ouest 1100 Notre-Dame Ouest
vladimir.polotski, jean-pierre.kenne@etsmtl.ca ali.gharbi@etsmtl.ca
ABSTRACT: Hybrid manufacturing systems that utilize in production process both raw materials and used
products collected from the market at the end of their life are frequently met in practice. Effective production
management in such systems requires coordination between monitoring, reverse logistics, planning, and
production control. An important problem encountered along the implementation of these activities is related
to the uncertainty and variation of exogenous parameters of the system, such as market demand for the final
products and the flow level of returned products. These parameters are used in the decision procedures and
often supposed to be constant and known. In order to adapt such assumptions to the industrial practice, the
procedures for online estimation of the demand and return levels have to be developed, and existing methodology
has to be extended to the case of demand and return variations. In our work we present some results in this
direction: (1) we show how the state observers can be used for estimating constant or periodically varying
demand and return levels, and (2) we show how the optimality conditions can be modified and numerically
implemented in order to address the case of variable market demand and return product flow.
KEYWORDS: manufacturing, remanufacturing, demand uncertainty, observers, adaptive
control
1 INTRODUCTION
In recent years the hybrid systems using both man-
ufacturing (direct line) and remanufacturing (reverse
logistics) received significant attention due to the re-
quirements of sustainable development and environ-
mental concerns. The problems inherent to reman-
ufacturing were studied since the nineties (Fleish-
man et al. 1997), optimization of returned product
recovery as well as production planning and inven-
tory management was considered in (Kiesmuller and
Scherer, 2003) An approach based on the stochastic
control and dynamic programming was proposed in
(Kenne et al. 2012), where the failure prone system
composed of two machines working in manufacturing
and remanufacturing mode respectively was consid-
ered. Using the same approach the systems under
diffusion type demand were analyzed in (Ouaret et
al. 2013). Important issues in reverse logistics, and
in particular the coordination of manufacturing and
remanufacturing are addressed in the recent book by
(Gupta, 2014). A comprehensive review of the meth-
ods relevant to the systems that use reverse logistics
can be found in (Govindan et al. 2015).
When the production planning in hybrid systems is
considered, the variation and uncertainty in the de-
mand and return levels become important issues be-
cause they adversely affect the coordination between
manufacturing and remanufacturing. In order to re-
solve these issues (1) the reliable estimates of the de-
mand and return levels have to be constructed, and
(2) the optimal solutions adaptable to the variations
of the demand and return levels have to be found.
Although dealing with the demand and return vari-
ation and uncertainty is of particular importance for
hybrid manufacturing-remanufacturing systems, it is
also relevant to conventional manufacturing systems
since the market demand is often unknown and/or
varies in time. The underlying problems have been
addressed in the literature but, the relevant publica-
tions are relatively sparse.
Generic approaches to the analysis of the systems
with only partially observed inventory characteristics
are discussed in (Sethi and Shi, 2013). Practically im-
portant class of such systems (so called ”zero balance
walk” - model) is investigated in (Bensoussan et al.
2007), where the rigorous analysis of optimality con-
MOSIM’16 - August 22-24 Montr´eal, Qu´ebec, Canada
ditions (Bellman equation) is performed and a feed-
back control policy is described. For the systems that
use remanufacturing in their production process the
effect of uncertainty in the demand and (especially) in
the return level is more pronounced (Bulmus et al.,
2014, Govindan et al., 2015) and currently attracts
the increasing interest of the researchers. For exam-
ple, in (Mukhopadhyay and Ma, 2009) authors inves-
tigate the the production and supply strategy under
uncertainty; in (Fung and Kongfa, 2014) authors de-
scribe the decisions about the acquisition price and
production rates that the company has to make while
facing uncertainty in the demand and return levels.
The systems with variable (but known) demand and
return were investigated in (Miner and Kleber, 2001,
and Kleber et al., 2002). In the first of theses pa-
pers, the optimal manufacturing and remanufactur-
ing policies for fully reliable system were determined
using optimal control methodology. In the second pa-
per the results were extended to the case of multiple
recovery options. Recently, in (Feng et al., 2013) this
methodology was applied to the system with perish-
able products. Only fully reliable systems were con-
sidered in these papers.
This short overview of the literature the liter-
ature shows that the variation and uncertainty
of the demand and return levels is and impor-
tant subject rarely addressed in the context of
failure-prone and especially hybrid manufacturing-
remanufacturing systems.
The paper is organized as follows: in section 2 we pro-
vide some motivations for our research, then formu-
late the problem and describe the proposed method-
ology. In the rest of the paper we apply this method-
ology to the simplified one-machine-one-product sys-
tem. In section 3 we develop the observer-based es-
timation procedure for the unknown demand level.
In section 4 we consider the varying demand. First,
we extend our observer-based estimation procedure
to this case. Second, we propose a novel approach
for approximating the solutions of Hamilton-Jacoby-
Bellman equations in non-stationary case, which ap-
plicable to the case of varying demand and return
levels. In section 5 we provide some conclusions and
outline the future works.
2 Problem formulation and proposed
methodology
2.1 Motivation
Addressing the optimal control problem for the man-
ufacturing systems the construction of feedback pol-
icy is most often considered. The demand level is
supposed to be known. In case of manufacturing-
remanufacturing systems that use products returned
from the market as a source for production ( in paral-
lel to row materials) - both demand and return levels
are often supposed to be known and therefore avail-
able for determining the decision making procedure.
Such assumptions may not hold in practice since both
demand and return vary in time and do not become
immediately known. Demand level is often not avail-
able at all and different forecast models are considered
in order to construct the estimate of the demand and
provide the necessary information do decision maker.
In the context of reverse logistics, the same is true for
the return level which is often more affected by var-
ious sources of uncertainty (Govindan et al., 2015).
Thus modeling the return level as being (1) variable
and (2) uncertain is important from both practical
and theoretic points of view.
2.2 Systems under study and problem de-
scription
Our main target in this study is the hybrid system
composed of two facilities (machines) : manufactur-
ing and remanufacturing ones. The machines are
subject to (random) failures followed by (random)
repairs. The times between successive failures and
the repair times are exponentially distributed with
rates piand rirespectively (i= 1,2 is an index of
the machine). Let us denote by ξi, i = 1,2 the
binary variables corresponding to the random state
of the machine Mi:ξi= 1 when the machine Mi
is up and ξi= 0, when it is down. To describe
the state of the whole system we define the variable
η=ξ1×ξ2∈ {(1,1),(1,0),(0,1),(0,0)}∼{1,2,3,4}.
The system structure is illustrated in figure 1.
Transitions between the state can be conventionally
described by a state transition matrix
G={qij }= (1)
−(p1+p2)p2p10
r2−(r2+p1) 0 p1
r10−(r1+p2)p2
0r1r2−(r1+r2)
where qαβ is the transition rate from state αto β,
(α, β ∈ {0,1}).
Let x1be the serviceable inventory, x2- the return
inventory, u1, U1- the production and maximal pro-
duction rates of the manufacturing machine M1,
u2, U2- the production and maximal production rate
of the remanufacturing machine M2,u3, U3the dis-
posal and maximal disposal rates, d(t), R(t) - the cus-
tomer demand and return rates, that can be variable
and/or unknown. The evolution of the system can be
described by the following equations:
˙x1(t) = u1(t) + u2−d(t)
˙x2(t) = R(t)−u2−u3(2)
With an additional state constraint:
MOSIM’16 - August 22-24 Montr´eal, Qu´ebec, Canada
x2(t)≥0 (3)
The inequality (3) asserts that the system evolutions
occur in the half plane x2≥0 because the return
inventory can not be negative.
Since no production is possible when the machine is
down, production rates satisfy the constraints:
0≤u1≤U1ξ1,0≤u2≤ˆ
U2ξ2,(4)
here ˆ
U2(x) = U2if x > 0
Rif x= 0 because an additional
bound has to be imposed on u2due to (3).
The disposal option is always available, therefore:
0≤u3≤U3(5)
We can now define Γ - the set of admissible control
policies u1(.), u2(.), u3(.) :
Γ(x, α) =
u1(.), u2(.), u3(.) :
0≤u1(.)≤U1Ind12
0≤u2(.)≤ˆ
U2(x)Ind1,3
0≤u3(.)≤U3
here the notation Indij =1 if α=ior α=j
0 otherwise
has been introduced.
The instantaneous cost h(·) is defined as follows:
h(x, u)= c+
1x+
1+c−
1x−
1+c2x2+cmu1+cru2+cdu3
where c−
1, c+
1, c2are unit costs of backlog, serviceable
and return inventory holding, cm, cr, cdare unit costs
of manufacturing, remanufacturing and disposal re-
spectively; x+=max(0, x), x−= max(0,−x).
2.3 Optimality conditions
The objective is to determine manufacturing, reman-
ufacturing and disposal policies u1, u2,and u3in or-
der to minimize the expected discounted cost (ρis the
discount rate):
J(y, u, α, t) = Et
0e−ρth(x(s), u(s))ds |
x(0) = y, η(0) = α, u(.)∈Γ(x, α)}(6)
here x={x1, x2}, u ={u1, u2, u3}, the integration
interval [0, t] is finite in general, and notation notation
J(x, u, α) is used for limt→∞ J(x, u, α, t)
Value functions are conventionally defined as follows
(Gershwin, 2011; Kenne et al., 2012):
V(x, t, α) = inf
u∈Γ{J(x, u, t, α)}, α = 1, . . . , 4 (7)
Optimality conditions can be written in the form
of Hamilton-Jacobi-Bellman (HJB) equations (Gersh-
win, 2011):
0 = ∂V (x,t,α
∂t −ρ V (x, t, α) + h(x, u)+
minu∈Γ(u1Ind12 +u2Ind13 −d(t)) ∂V (x,t,α)
∂x1
(R(t)−u2Ind13 −u3)∂V (x,t,α)
∂x2+
βqαβ (V(x, tβ)−V(x, t, α))
(8)
It is important to emphasize that because our objec-
tive is to investigate the optimal behavior under vari-
able and/or uncertain we keep non-stationary terms
∂V (·
∂t which are conventionally omitted.
To analyze the the situations that arise in case of
unknown and variable demand and return levels, we
propose below some new approaches. However the
analysis of the hybrid system described above with
these approaches goes beyond the scope of this paper.
We apply the proposed methodology to the simpler
case: namely to the one-machine-one-product system,
for which the analytical solution has been obtained
in (Akella and Kumar, 1986) for the case of known
constant demand.
2.4 Proposed methodology
In contrast to the standard assumption that the de-
mand level is a known constant we suppose that it is
unknown and/or varying.
We consider the following cases:
(1) demand is constant and unknown
(2) the demand is variable and known
(3) the demand is variable and unknown
2.4.1 Constant unknown demand
For the case of unknown constant demand, we pro-
pose the estimation procedure based on the use of
state observers.
The demand estimate converges asymptotically to the
exact (unknown) demand) and this estimate can be
used for computing optimal policy (instead of the ex-
act unknown demand level).
2.4.2 Variable known demand
For the case of variable demand two approaches can
be considered. First is a simplified approach that
consists of using instantaneous demand level in the
analytical formulation in order to compute the (vari-
able) hedging level. Second is a novel approach we
propose here. It consists of keeping (conventionally
omitted) non-stationary terms in HJB equations (as
shown in equations (8)) to characterize the partic-
ularity of variable demand. With these additional
terms we may compute numerically the value func-
MOSIM’16 - August 22-24 Montr´eal, Qu´ebec, Canada
tions, hedging levels and finally the optimal policies
for the variable demand case.
2.4.3 Variable unknown demand
For the case of unknown varying demand an approach
we propose is inspired by the so-called separation
principle from the feedback control theory. We first
compute the estimates of the demand and return lev-
els using state observers, and next, we replace the
unknown demand and return levels by the obtained
estimates and use them in the modified HJB equa-
tions taking into account non-stationary terms as it
is described in the previous subsection.
3 State observers for demand estimation
We suppose that inventory level can be directly
measured, and the production capacity is precisely
known. That means that when the machine switches
to failure mode (and therefore its capacity falls to zero
level) - it becomes immediately known as well. A dy-
namical system, called state observer is constructed,
that takes as inputs the current production level, the
measured inventory level; its output is the estimated
demand level. The estimates provided by the sate ob-
server are known to converge under some conditions
to the actual (unknown, variable) demand level.
In the rest of the paper we consider a one-machine-
one-product system (Akella and Kumar, 1986) with
failure/repair rates, cost parameters policy con-
straints and objective function reduced in appropriate
way from the description given in section 2.2.
˙x(t) = u(t)−d(9)
We suppose that the inventory measurement provides
the precise value of x(t)
y=x(t) (10)
To define the observer dynamics, we first define the
observer gains that allow to place the observer poles
into the desired location. For our case we propose
the double pole located at the point λ1=λ2=−2
on the complex plane. To do that we set the degree
of stability µ(equal to µ= 2 in our case), and then
proceed with defining the gains as follows
g1=−2µ;g2=µ2(11)
This definition leads to the observer with estimation
process of second order.
˙
˜x=u−˜
d+g1(˜x−y)
˙
˜
d=g2(˜x−y)(12)
Corresponding error dynamics is of second order, thus
the demand estimation error converges exponentially
with the rate 2 (bounded by exp −2t). The described
Figure 1: Structure of the hybrid system
Figure 2: Estimation of constant demand
procedure works well for a constant demand and pro-
vides the demand estimate converging exponentially
fast to the actual unknown demand.
The results of the estimation process as well as corre-
sponding inventory dynamics are illustrated in figure
2. It is important to emphasize that the the discrete
stochastic jumps in the production u(t) due to failure
repair random perturbations do not affect the estima-
tion process. That is because the production (even
affected by the failures) is known, and as it is being
integrated into the estimation procedure, the result-
ing error dynamics is invariant to such perturbations.
In a particular case illustrated in figure 2 we have used
the system with the following parameters: MT T F =
0.1 (p= 10), M T T R = 0.6 (r= 1.66), U =
1.5, d = 1. To compute the hedging point we used
c+= 1, c−= 50, ρ = 0.05, this results in z= 1.35.
Second order observer dynamics is synthesized with
µ= 2, g1=−4, g2= 4.
MOSIM’16 - August 22-24 Montr´eal, Qu´ebec, Canada
Figure 3: Estimation of variable demand
For the case of variable demand this procedure also
works, but there will be a systematic error as can
be observed in figure 3. Higher order error dynamics
can be designed in order to improve the convergence
quality. This is addressed in the next section.
4 Dealing with variable demand
We describe below a model of slow-varying-periodic
demand and address ( using this model) two impor-
tant aspects characterizing the case of variable de-
mand: (1) - estimation procedure adapted to demand
variations and (2) - policy optimization procedure
adapted to demand variations.
Let us consider the demand varying periodically ac-
cording to the model:
d(t) = d0+d1cos(ωt −ϕ0) (13)
Choosing for our example a trigonometric function,
we target the practically important case of seasonal
variations in the demand (see e.g. Kleber et al.,
2002). Also by choosing d0> d1, we limit ourselves
with positive and bounded demand.
Here d0naturally corresponds to the average demand
level, d1- to the amplitude of periodical variations,
ωto the frequency of variation and ϕ0- the initial
phase.
4.1 State observer for periodically varying
demand
Following the ”classical” technique (see e.g. Kwak-
ernaak and Sivan, 1972), the model (13) can be ob-
tained as a solution of the following system of differ-
ential equations:
Figure 4: Dynamics of inventory and demand estima-
tion: time-varying case
˙
d0= 0
˙
dv=ω vd(14)
˙vd=−ω dv
The overall demand consists of the constant (average)
a and varying portion:
d=dm+dv(15)
Integrating the demand dynamics (14,15) into the
inventory dynamics (9) with the measurement (10)
we get a complete model. An observer for the peri-
odic demand estimation can be constructed as follows
(Kwakernaak and Sivan, 1972):
˙
˜x=g1(˜x−y) + u−(˜
dm+˜
dv) + g1(˜x−y)
˙
˜
dm=g2(˜x−y) (16)
˙
˜
dv=g3(˜x−y) + ω˜vd
˙
˜vd=g4(˜x−y)−ω˜vd
Observer gains are determined in a way similar to
(11) using the degree of stability µ(set to µ=1.5 in
this case)
g1= 4µ;g2=µ4/ω2g3=ω2−6µ+µ4/ω2
g4= 4(µ3/ω −µω) (17)
The dynamics of the demand, its observer-based
estimate, corresponding inventory dynamics as well
as the behavior of hedging point (original and esti-
mated) are illustrated in figure 4. The results corre-
spond to the following parameters of the underlying
models: M T T F = 10, M T T R = 0.6, ρ = 0.05, ω =
MOSIM’16 - August 22-24 Montr´eal, Qu´ebec, Canada
Figure 5: Dynamics of the inventory and demand es-
timation: lower frequency case
.3, d0= 1, d1= 0.3, ϕ = 0 (last 3 parameters re
not ”known” by the observer model!). One can see
that the demand estimate (shown in green) is now
converging to the original demand (blue line) - com-
paring to the second-order estimator shown in fig-
ure 3 were an offset is clearly visible. As before the
failure/repair events (marked by arrows) resulting in
the abrupt drops in the inventory (solid red line) do
not affect estimation process. One can also see the
convergence of the hedging point estimate ( magenta
solid line) to the original hedging level (solid black
line), which itself varies due to demand variations.
This ”original” hedging point curve is obtained using
the analytical expression from (Akella and Kumar,
1986 ). On-demand production characterized by the
inventory curve closely following the hedging curve is
clearly visible in the center of the plot. It starts from
t≃50 to t≃75 - then is shortly interrupted by the
failure-repair, resumed and continued until t≃140,
were interrupted again for a longer period and re-
sumed at t≃200.
In figure 5 we illustrate the results obtained for the
slower varying demand with ω= 0.25. One can see
that the estimates converge to the original demand
and hedging points much faster and without smaller
overshoot as compared with figure 4 (ω= 0.3).
In figure 6 we illustrate the behavior for ϕ=
1.57, ω = 0.25. The demand estimate converges very
fast in this case, although the convergence of the
hedging point estimate takes longer (magenta line
converges to black line). It worth noting the long pe-
riod of the maximal production (until t≃100), with
the slope of the inventory curve decreasing as the de-
mand gets higher; after reaching the hedging level the
Figure 6: Dynamics of the inventory and demand es-
timation: fast estimation error decay
inventory follows it with very short interruptions due
to failures (at t≃110 and t≃220).
4.2 Optimality conditions and numerical ap-
proach for slow varying demand
An approach we describe below consists of approx-
imated evaluation of non-stationary terms in HJB
equations and their subsequent integration into nu-
merical solution of these equations.
In what follows we apply this approach to the M1P1-
system with variable demand determined by the
model (6) We suppose that d1< d0/2 and also con-
sider ωbeing small, usually ω < ρ.
We consider the proposed solution as a first step for
addressing the hybrid systems with both demand and
return levels varying in time d=d(t), R =R(t),
and in particular - varying periodically according to
(14,15).
In general parameter varying case, the HJB equations
contain the terms ∂Vi(...)
∂t additionally to the conven-
tionally terms studied ∂Vi(...)
∂xj) as it is shown in. This
non-stationary terms are usually dropped along the
analysis of stationary case (see e.g. Gershwin, 2011).
We propose the following approach.
1. Consider the range of the demand variation. For
the demand model (5) it is D= [d0−d1, d0+d1].
2. Divide this range into Nseveral (small) intervals
Ij= [dj, dj+1], j = 1, . . . , N ;d1=d0−d1, dN=
d0+d1.
MOSIM’16 - August 22-24 Montr´eal, Qu´ebec, Canada
Figure 7: Value function (operational mode) for var-
ious demand levels
3. Compute the solutions of conventional HJB equa-
tions (without non-stationary terms) for discrete de-
mand levels d=dj(taken at times tj), and obtain
numerically the value functions Vj(x, α) ( α= 1,2 for
this case with α= 1 standing for operational mode
and α= 0 - for failure mode).
4. Compute the numerical estimates for the time-
derivative using value functions for consecutive j:
∂V j(x, α)
∂t ≃Vj+1 (x, α)−Vj(x, α)
tj+1 −tj
5. Recompute the numerical solutions of the HJB
equations with the time derivative terms integrated
into the grid data.
The described approach is implemented, and the re-
sults obtained for the following system parameters
U1= 0.27, M T T F = 25(p= 0.04), M T T R =
6.67(q= 0.15), ρ = 0.01, c+= 1, c−= 20, ω =
0.005, d0= 0.19, d1= 0.02 are shown below. In
figure 7 the value functions for operational mode
(V(x, 1) are shown for different ”frozen” demand lev-
els (from 0.17 to 0.21). Corresponding hedging point
policies are illustrated in figure 8, and value functions
for failure mode (V(x, 0)) - in figure 9.
One can observe the following evolutions: opera-
tional mode value functions are moving consistently
up when the demand level increases, also the point
of minimum of value functions (hedging point) con-
sistently increases (clearly visible in figure 8, but also
in figure 7). Failure mode value functions are located
above the operational mode value functions for each
particular level of the demand. However, globally,
this feature does not hold - the failure mode value
function for the low demand level (e.g d= 0.17) is
Figure 8: Policy switching for various demand levels
Figure 9: Value function (failure mode) for various
demand levels
MOSIM’16 - August 22-24 Montr´eal, Qu´ebec, Canada
Figure 10: Value function for varying (decreas-
ing/stable/increasing demand level
below the operational mode value function for the
high demand level (e.g d= 0.21).
Figure 10 illustrates the results obtained after taking
into account the demand variation for a particular
demand level (d= 0.19 in this case). One can ob-
serve 3 operational-mode-value-function curves corre-
sponding to the constant, increasing and decreasing
demand respectively. Hedging levels obtained are 6,
5.75 and 5.5 and respectively.
When the demand varies, the hedging point also
varies, namely it increases (decreases) when the de-
mand increases (decreases), but there is also antici-
patory effect: the hedging level increases more and
earlier then it would be if we compare two corre-
sponding demand levels. The evolution of the hedging
point along the whole period of the demand evolution
is illustrated in figure 11. Blue dashed curve corre-
sponds zs(t) to the different (but stationary) levels
of the demand. Green solid curve zns(t) corresponds
to the ”non-stationary” hedging level obtained when
local variations of the demand (increase or decrease)
are taken into account, (solid blue line on the bot-
tom just show the corresponding demand evolution).
One can clearly see the anticipatory effect in zns(t)vs
zs(t): the increases and decreases of the zns(t) (green
curve) are advanced with respect to those of zs(t)
(blue curve). But close to the extremal points of the
demand, when it varies slower - both curves get close
zns(t)≃zs(t).
Figure 11: Hedging points computed along the peri-
odically varying demand
5 CONCLUSIONS and FUTURE WORKS
Stochastic optimization problem for failure prone sys-
tems under uncertain and variable demand is inves-
tigated. The problem is initially formulated for the
hybrid manufacturing- remanufacturing system that
uses for production both row materials and returned
products. For such systems both demand and return
level may become uncertain and varying. Considering
the investigation of such systems as an ultimate goal,
we describe in this work the methodology for dealing
with demand variation and uncertainty, but apply it
to the simpler one-machine-one-product system.
For the case of unknown demand, the observer-based
technique is described, and it is shown how the de-
mand estimates converging to the original (unknown)
demand can be constructed. The constant unknown
demand is considered first. Next, the technique is
adapted to the case of variable unknown demand and
convergence properties of the constructed estimates
are investigated. As the obtained estimates converges
exponentially to the original demand, they can be
used in the optimization procedures instead those un-
known demand levels.
However the variation of the demand (and return in
case of hybrid system) has and additional effect on
the optimization procedure. When the demand varies
in time, the conventional optimality condition are
not applicable. We have described a novel approach
that takes into account the non-stationary terms in
HJB equations, that is conventionally omitted be-
cause the limiting stationary solution is of interest.
Our proposed approach is implemented as an add-on
to the conventional (so called) Kushner numerical al-
MOSIM’16 - August 22-24 Montr´eal, Qu´ebec, Canada
gorithm: we first compute the optimal hedging point
policy for various demand levels, next, we estimate
numerically the time-derivative of the value functions,
and finally, recompute the optimal policy taking into
account the estimated time-derivative.
Observer-based technique is fully applicable to the
case of two-machine hybrid system when both ser-
viceable and return inventory level are observable.
Full implementation of the estimation procedure
and its detailed investigation is a part of ongo-
ing work. Application of the technique based on
the non-stationary HJB equations to the hybrid
manufacturing-remanufacturing system will require
numerical implementation of the developed algo-
rithms for the multidimensional case and is a subject
of our future work.
REFERENCES
Akella R. and P. R. Kumar, 1986. Optimal Con-
trol if Production rate in a Failure Prone Manu-
facturing System. IEEE Transactions on Auto-
matic Control, 31, 2, 116–126.
S. Bulmus S., S. Zhu and R. Teunter, 2013. Ca-
pacity and production decisions under a reman-
ufacturing strategy. International Journal of
Production Economics, 145, 359–370, 2013.
Bensoussan A., M. Cakanyildurim and S. Sethi,
2007. Partially Observed Inventory Systems:
The Case of Zero Balance Walk. SIAM Journal
on Control & Optimization, 46, 1, 176–209.
Feng L., J. Zhang and W. Tang, 2013. Optimal
control of production and remanufacturing for a
recovery system with perishable items. Interna-
tional Journal of Production Research, 51, 13,
3997–3994.
Fleischmann M., J.M. Bloemhof-Ruwaard, R.
Dekker, E. Van der Laan, J.A.E.E. Van Nunen
and L.N. Van Wassenhove. 1997. Quantitative
models for reverse logistics : A review. European
Journal of Operational Research, 103, 1, 1–17.
Zhang Fuan and Hu Kongfa, 2014. Production de-
cisions in remanufacturing with uncertain return
and demand. Advanced Materials Research, 988,
751-754.
Gershwin S. B., 2011. Manufacturing Systems Engi-
neering. Massachusetts Institute of Technology.
Govindan, K., H. Soleimani and D. Kannan, 2015.
Reverse logistics and closed-loop supply chain:
A comprehensive review to explore the future.
European Journal of Operational Research, 240
(3), pp. 603626.
S. M. Gupta, editor. 2014. Reverse Supply chains:
Issues and analysis. Boca Raton, FL, CRC
Press, Taylor & Francis Group.
Kenne J.-P., P. Dejax and A. Gharbi, 2012. Pro-
duction planning of a hybrid manufacturing - re-
manufacturing system under uncertainty within
a closed-loop supply chain. International Jour-
nal of Production Economics, 135, 81–93.
Kiesmuller G.P. and C.W. Scherer, 2003. Compu-
tational issues in a stochastic finite horizon one
product recovery inventory model. European
Journal of Operational Research, 146, 553–579.
Kleber R., S. Minner and G. Kiesmuller, 2002. A
continuous time inventory model for a product
recovery system with multiple options. Inter-
national Journal of Production Economics, 79,
121–141.
Kushner H. J. and P. Dupuis,1992. Numerical meth-
ods for stochastic control problems in continuous
time. New York: Springer Verlag.
Kwakernaak H. and R. Sivan, 1972. Linear Optimal
control Systems. Wiley- Interscience.
Minner S. and R. Kleber, 2001. Optimal control
of production and remanufacturing in a simple
recovery model with linear cost functions. OR
Spectrum, 23, 3, 3–24.
Mukhopadhyay S. and H. Ma, 2009. Joint pro-
curement and production decisions in remanu-
facturing under quality and demand uncertainty.
International Journal of Production Economics,
120, 5–17.
Ouaret S., V. Polotski, J.P. Kenne and A. Gharbi,
2013. Optimal Production Control of Hybrid
Manufacturing/Remanufacturing Failure-Prone
Systems under Diffusion-Type Demand. Scien-
tific Research: Applied Mathematics, 4, 3 550–
559.
Sethi S. and R. Shi, 2013. Managing with Incom-
plete Inventory Information (i3). IFAC Confer-
ence MIM 2013, Moscow, Russia.
