|
|
Digital Library of the
European Council for Modelling and Simulation |
|
Title: |
Simulation-Based Optimization Of Markov Controlled Processes With
Unknown Parameters |
|
Authors: |
Enrique Campos-Náñez |
|
Published in:
|
(2009).ECMS
2009 Proceedings edited by J. Otamendi, A. Bargiela, J. L. Montes, L. M. Doncel
Pedrera. European Council for Modeling and
Simulation. doi:10.7148/2009 ISBN: 978-0-9553018-8-9 23rd
European Conference on Modelling and Simulation, Madrid, June
9-12, 2009 |
|
Citation
format: |
Campos-Nanez,
E. (2009). Simulation-Based Optimization Of Markov Controlled Processes With
Unknown Parameters. ECMS 2009 Proceedings edited by J. Otamendi,
A. Bargiela, J. L. Montes, L. M. Doncel Pedrera (pp.
537-543). European Council for Modeling and Simulation. doi:10.7148/2009-0537-0543 |
|
DOI: |
http://dx.doi.org/10.7148/2009-0537-0543 |
|
Abstract: |
We
consider simulation-based gradient-estimation and its use in Markov
controlled processes with unknown pa- rameters. We
consider a Markov reward process con- trolled by both a set of tunable
parameters, and a set of fixed but unknown. We analyze the use a recursive
identification procedure, and their application to exist- ing
gradient-based algorithms based on simulation. We show that simple
modifications of available gradient es- timation algorithms, namely assuming parameter cer- tainty, can accommodate
system parameter identifica- tion,
without sacrificing the convergence of these to local optima by following a
two-time-scale recursive identi- fication/optimization procedure. This approach is illus- trated through an application to the algorithm proposed
in (Marbach and Tsitsiklis,
2001). We illustrate our re- sults with a small
numerical example, which we further use to test the ability of the proposed
scheme to track slow changing system parameters. |
|
Full
text: |
|
