A Limiting Description in a Gaussian One-Armed Bandit Problem with Both Unknown Parameters

We consider the limiting description of control in a Gaussian one-armed bandit problem, which is a mathematical model for optimizing batch processing of big data in the presence of two alternative methods with known efficiency of the first method. We establish that this description is given by a second-order partial differential equation in which the variance of one-step income is known. This means that in the case of big data, the variance can be arbitrarily accurate estimated at a short initial stage of processing, and then the obtained estimate is used by the control strategy.

Keywords

one-armed bandit; Bayesian and minimax approaches; invariant description; batch processing.

References

1. Kolnogorov A.V. Invariant Description of Control in a Gaussian One-Armed Bandit Problem. Bulletin of the South Ural State University. Series: Mathematical Modelling, Programming and Computer Software, 2024, vol. 17, no. 1, pp. 27-36. DOI: 10.14529/mmp240103
2. Sragovich V.G. Mathematical Theory of Adaptive Control. Singapore, World Scientific, 2006. DOI: 10.1142/5857
3. Lattimore T., Szepesvari C. Bandit Algorithms. Cambridge, Cambridge University Press, 2020. DOI: 10.1017/9781108571401
4. Kolnogorov A.V. One-Armed Bandit Problem for Parallel Data Processing Systems. Problems of Information Transmission, 2015, vol. 51, no. 2, pp. 177-191. DOI: 10.1134/S0032946015020088
5. Bather J.A. The Minimax Risk for the Two-Armed Bandit Problem. Mathematical Learning Models - Theory and Algorithms, 1983, vol. 20, pp. 1-11. DOI: 10.1007/978-1-4612-5612-0_1
6. Chernoff H., Ray S.N. A Bayes Sequential Sampling Inspection Plan. The Annals of Mathematical Statistics, 1965, vol. 36, pp. 1387-1407. DOI: 10.1214/aoms/1177699898
7. Bradt R.N., Johnson S.M., Karlin S. On Sequential Designs for Maximizing the Sum of n Observations. The Annals of Mathematical Statistics, 1956, vol. 27, pp. 1060-1074. DOI: 10.1214/aoms/1177728073