Approximating Optimal Policies for Agents with Limited Execution Resources

Dmitri A. Dolgov, and Edmund H. Durfee

In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03). Pages 1107--1112. 2003.

Copyright © 2003 IJCAI. Online proceedings are available at http://dli.iiit.ac.in/ijcai/IJCAI-2003/content.htm.

Abstract
An agent with limited consumable execution resources needs policies that attempt to achieve good performance while respecting these limitations. Otherwise, an agent (such as a plane) might fail catastrophically (crash) when it runs out of resources (fuel) at the wrong time (in midair). We present a new approach to constructing policies for agents with limited execution resources that builds on principles of real-time AI, as well as research in constrained Markov decision processes. Specifically, we formulate, solve, and analyze the policy optimization problem where constraints are imposed on the probability of exceeding the resource limits. We describe and empirically evaluate our solution technique to show that it is computationally reasonable, and that it generates policies that sacrifice some potential reward in order to make the kinds of precise guarantees about the probability of resource overutilization that are crucial for mission-critical applications.


BibTex
@inproceedings{ dolgov03approximating,
   paperID   = "IJCAI-03",
   author    = "Dmitri A. Dolgov and Edmund H. Durfee",
   booktitle = "Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03)",
   address   = "Acapulco, Mexico",
   title     = "Approximating Optimal Policies for Agents with Limited Execution Resources",
   year      = "2003",
   pages     = "1107--1112"
}


Download:
pdf [pdf]        ps [ps.gz]