STOCHASTIC APPROXIMATION WITH COMPOUND DELAYED OBSERVATIONS

- The Robbins-Monro stochastic approximation procedure is modified so as to be applicable in the presence of compound delayed observations. The efficiency of the modified procedure is investigated. The asymptotic values of the efficiency are compared with those obtained by an approximation based on geometric distribution.


INTRODUCTION
Stochastic approximation is a procedure for finding the root of an equation, or the solution of a system of equations when the values of the respective function can only be observed with experimental errors at recursively determined points. There are a large number of papers and literatures (e.g. [2], [3] and [4]) dealing with stochastic approximation procedure.
The procedure for finding the root is called Robbins-Monro procedure (see [5]). The Robbins & Monro [5] stochastic approximation procedure is applied in situations when the results of experiments become known only after a compound random time delay as well as the results of sub-experiments becomes known after a random time delay. The results obtained show that the replacement of the procedure in the presence of delayed observations by the procedure with compound delayed observations gives, as a rule, a satisfactory approximation to the efficiency of the procedure. In principle, however, our proposal can be applied to other stochastic approximation or recursive estimation procedures.

ROBBINS-MONRO PROCEDURE WITH COMPOUND DELAYED
OBSERVATIONS The Robbins-Monro stochastic approximation procedure with delayed observations has been investigated by Dupač and Herkenrath [1] for a geometrical delay distribution. In order to eliminate (or at least diminish) time-losses due to delays of observations, it has been proposed in this paper to allocate the experiments into K parallel series in the following way: The

Here
) (x m is a function whose zero point θ is to be found, is the observational error corresponding to an observation of m made at point x and becoming known during the interval x is the current approximation to θ in the i th series at time 0 − n ; , , N n a n ∈ is a zero sequence of positive constants, typically 0 0 ), /( n n n a a n + = is non-negative; ) (n t is [the integer part of] the delay of the result of an experiment made at time . σ is the asymptotic variance of the same normed approximation in a procedure with no delays (see [4]). Hence e is also the relative asymptotic efficiency of n θ as a statistical estimator.
Allocating experiments into K series is not the only possible approach to the problem of stochastic approximation with delays. An alternative approach, not discussed in the quoted paper, will be sketched here: There is only a single approximation sequence. Each experiment is partitioned into N parts where each part is treated as a sub-experiment and parts or experiments are loosed if no result of the preceding one has become known during the time interval between any two consecutive sub-experiments or experiments respectively. This approach is represented throughout two investigated loss systems 0 derived in the service system theory, identifying the inter-arrival and the compound inter-arrival times with n / 1 time and unit time interval between subexperiments and between experiments, Bernoulli and Binomial service times with the delay and the compound delay of the result of a sub-experiment and an experiment, and K inspectors with K series respectively .
Assume that n parts will be inspected with or without needing repairs, where n N − parts are loosed, i.e. they will leave without receiving their inspections, if the inspector is occupied. If a part is inspected where the result shows that it needs no repairs, then the time of inspection is assumed to be , 0 1 − n and if the result shows it needs repairs, then the time of inspection and repairs is assumed to be 0 At the beginning, assume that an experiment starts its inspection at time t . If a part is inspected without repairs, then it will leave immediately before time n t 1 + , i.e. at , where the next one arrives at time n t 1 + and starts its inspection without delay. If a part is inspected with repairs, then it will leave before time the inspector is delayed one time unit such that the time delay starts from the arrival of the next part. If all the parts of an experiment need no repairs, then the experiment will leave immediately before the compound time , where the next one arrives at time 1 + t and starts its inspection without delay. If all the parts need no repairs except, for example, the last one needs repairs, then the experiment will leave before the compound time , where the inspector is delayed one time unit such that the time delay starts from the arrival of the next experiment. Observe that the result of an experiment becomes known only after a compound random time delay, which is produced by taking the compound of the delay of its parts. Our proposal can be applied in the presence of many biological or lifetime experiments. Let . Consider the situation that the 0 n parts are inspected first, followed by the 1 n parts. The process of inspections and repairs can be described as follows: The i th part, The loosed parts are not allowed to enter any other free inspector for the delay of one of them will loose the next experiment and loss in the system will be increase.
According to the described process, it is seen that the compound random time delay D of the experiment can be produced by the compound of the delays 0 ,..., 2 , 1 , n i D i = and the delays 1 ,..., of 0 n parts that are inspected first without delays and the next 1 n parts that are inspected with a time unit delay for each respectively. The event that represents the compound delay D is given by: Assume that the K inspectors are in a parallel position, where an experiment consisting of N parts is arrived each time unit. At the beginning all the K inspectors are free and an inspector receives an experiment each time unit. If all inspectors are occupied, then experiments will be loosed.
The process of inspections and repairs of an experiment is described as follows: Assume that an experiment enters a free inspector at time instant t . If it finishes its inspections of its parts without repairs before time 1 + t , i.e. at time 0 1 − + t , then its compound random time delay D equals zero with probability . 1 n p A study of a service system in the special case 1 = N was given by Dupač and Herkenrath [1], where the service time has geometric distribution.
By the state of the system we usually understand various numerical characteristics connected with experiments located in the system at a given moment of time (i.e. enters the system but its parts are not yet inspected). The study of the stochastic processes describing the behavior in time of these characteristics is the main object of service theory. In our study, it is seen that the states of the system are: where K j i j ,..., 2 , 1 ; = is an integer represents the compound of n sub experiments that are inspected with or without repairs. A standard argument of the queuing theory shows that the system with states (1) represents a Markov chain where each of its states consists of K components and each component is produced by the compound of n sub experiments. A chain with this property is called a compound Markov chain and its states are called compound states. Note that ) 0 ,..., 0 , 0 ( is the initial compound state and ) ,..., 2 , is the last possible compound state, where each component is greater than the preceding one, that can be reached from the initial compound state. If , K n p then there always is at least an inspector free, i.e. there is no loss of experiments and the efficiency of the procedure equals 1.
It is shown that the stationary transition probabilities ij p of the compound Markov chain in the case 2 = K are given by: , where all the inspectors are occupied, can be eliminated from the system as successive transitions from these compound states to compound states contains at least one free inspector and occur deterministically, with probability 1. The resulting matrix of transition probabilities is called the reduced transition matrix.
From the reduced transition matrix we find the following: i) All states are irreducible closed sets; therefore they contain persistent non-null states. ii) All states have period 1 because 0 for all i . Then from the previous conditions all states are ergodic (see [2]).In this case, there a unique stationary distribution π that can be calculated by solving the system of equations: , π π = T P (2) together with the added requirement: , where T denotes the transpose of the matrix, π is the stationary distribution matrix of the compound Makov chain; and P is the matrix of transition probabilities.

METHODOLOGY
To solve the system (2) we use the following steps: (1) First, assume the values K n, to form the transition matrix P . (2) Form the system (2) using the transition matrix P .
(3) One of the remaining equations can always be deleted, another one is to be added, namely the requirement ∑ α α π =1.
(4) The unknowns α π withα containing no 0′ s can be easily eliminated from the previous system, as successive transitions from these states to states containing 0′ s occur deterministically, with probability 1, where the resulting system is called reduced system. (5) The reduced system is solved for α π , α contains at least one 0, using the Matlab program (ver.5.3).
(6) Each assumed value of ) (d E determines a value 1 p that corresponds the solution α π withα contains at least one 0. (7) Finally, the solution α π is used to compute efficiency e of the procedure where e = ∑ α α π with α contains at least one 0.

RESULTS AND DISCUSSION
The asymptotic efficiencies are shown in Table 1 . The results obtained in Table 1, show that the asymptotic efficiencies vary directly as the number of inspectors for fixed delay and vary inversely as the time delay for fixed number of inspectors. This gives as a rule that: The loss in parts of an experiment depends on the number of inspectors and the time of delay. Therefore, to minimize the number of loosed parts, increase the number of inspectors or decrease delay.
In Table 1, the comparison of asymptotic values of ( e ) with those obtained by an approximation based on geometric distribution ( g e ), seems to be satisfactory. The consequent recommendation for the choice of K seems to be reasonable even for a more general delay distribution not drastically different from the binomial one. The headings ) (d E in Table 1 are then made use of, instead of 1 p .