Please use this identifier to cite or link to this item:
Title: Using a spreadsheet to find the steady state of a Markov process
Authors: Cottrell, Wayne D.
Issue Date: 2000
Publisher: EUT Edizioni Università di Trieste
ISTIEE Istituto per lo studio dei trasporti nell’integrazione economica europea
Source: Wayne D. Cottrell, "Using a spreadsheet to find the steady state of a Markov process", in: European Transport / Trasporti Europei, VI (2000) 14, pp. 5-1
Series/Report no.: European Transport / Trasporti Europei
VI (2000) 14
A macro, which automatically executes a series of spreadsheet software commands, is developed to find the steady-state condition of a Markov decision process (MDP). The typical infrastructure management approach is to assess system conditions, then select and implement mitigating strategies with the joint objectives of maximizing benefits and minimizing costs. Infrastructure management decisions must often be made under conditions of uncertainty, making a probabilistic model of the system useful. An MDP is a relatively simple model that exhibits stochastic dependence — stochastic in that future conditions are based on probabilities, and "memoryless" dependence in that the next condition is based only on the existing state. One of the features of an MDP is the achievement of a steady state following repeated applications of a management policy. Toward facilitating the analytical requirements of an MDP, a computerized procedure to determine the steady-state condition of an MDP is developed. The procedure might serve as one component of a complete MDP-based infrastructure management system. A comprehensive system would also compute the rewards and costs associated with a given policy and determine the optimal policy, possibly through a linear programming formulation.
Type: Article
ISSN: 1129-5627
Appears in Collections:European Transport / Trasporti Europei (2000) 14/VI

Files in This Item:
File Description SizeFormat
Cottrell_ET14.pdf2.47 MBAdobe PDFThumbnail
Show full item record

CORE Recommender

Page view(s) 10

checked on Jan 25, 2020

Download(s) 50

checked on Jan 25, 2020

Google ScholarTM


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.