Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
Zahra Rajabi
pymdptoolbox
Commits
4a305db3
Commit
4a305db3
authored
Feb 04, 2013
by
Steven Cordwell
Browse files
start a module docstring
parent
d1ab747a
Changes
1
Hide whitespace changes
Inline
Side-by-side
mdp.py
View file @
4a305db3
# -*- coding: utf-8 -*-
"""Markov Decision Process (MDP) Toolbox v4.0 for Python
Copyright (c) 2011, 2012, 2013 Steven Cordwell
Copyright (c) 2009, Iadine Chadès
Copyright (c) 2009, Marie-Josée Cros
Copyright (c) 2009, Frédérick Garcia
Copyright (c) 2009, Régis Sabbadin
The MDP toolbox provides classes and functions for the resolution of
descrete-time Markov Decision Processes.
All rights reserved.
Available classes
-----------------
MDP
Base Markov decision process class
FiniteHorizon
Finite horizon MDP
LP
Linear programming MDP
PolicyIteration
Policy iteration MDP
PolicyIterationModified
Modified policy iteration MDP
QLearning
Q-learning MDP
RelativeValueIteration
Relative value iteration MDP
ValueIteration
Value iteration MDP
ValueIterationGS
Gauss-Seidel value iteration MDP
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
Available functions
-------------------
check
Check that an MDP is properly defined
checkSquareStochastic
Check that a matrix is square and stochastic
exampleForest
A simple forest management example
exampleRand
A random example
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name of the <ORGANIZATION> nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
How to use the documentation
----------------------------
Documentation is available both as docstrings provided with the code and
in html or pdf format from
`The MDP toolbox homepage <http://www.somewhere.com>`_. The docstring
examples assume that the `mdp` module has been imported::
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>>> import mdp
Code snippets are indicated by three greater-than signs::
>>> x = 17
>>> x = x + 1
The documentation can be displayed with
`IPython <http://ipython.scipy.org>`_. For example, to view the docstring of
the ValueIteration class use ``mdp.ValueIteration?<ENTER>``, and to view its
source code use ``mdp.ValueIteration??<ENTER>``.
"""
# Copyright (c) 2011, 2012, 2013 Steven Cordwell
# Copyright (c) 2009, Iadine Chadès
# Copyright (c) 2009, Marie-Josée Cros
# Copyright (c) 2009, Frédérick Garcia
# Copyright (c) 2009, Régis Sabbadin
#
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
# * Neither the name of the <ORGANIZATION> nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
from
math
import
ceil
,
log
,
sqrt
from
random
import
randint
,
random
from
time
import
time
...
...
@@ -260,17 +315,32 @@ def checkSquareStochastic(Z):
Returns None if no error has been detected
"""
s1
,
s2
=
Z
.
shape
# try to get the shape of the matrix
try
:
s1
,
s2
=
Z
.
shape
except
AttributeError
:
raise
TypeError
(
"Matrix should be a numpy type."
)
except
ValueError
:
raise
ValueError
(
mdperr
[
"mat_square"
])
# check that the matrix is square, and that each row sums to one
if
s1
!=
s2
:
raise
ValueError
(
mdperr
[
"mat_square"
])
elif
(
absolute
(
Z
.
sum
(
axis
=
1
)
-
ones
(
s2
))).
max
()
>
10e-12
:
raise
ValueError
(
mdperr
[
"mat_stoch"
])
elif
((
type
(
Z
)
==
ndarray
)
or
(
type
(
Z
)
==
matrix
))
and
(
Z
<
0
).
any
():
raise
ValueError
(
mdperr
[
"mat_nonneg"
])
elif
(
type
(
Z
)
==
sparse
)
and
(
Z
.
data
<
0
).
any
():
raise
ValueError
(
mdperr
[
"mat_nonneg"
])
else
:
return
(
None
)
# make sure that there are no values less than zero
try
:
if
(
Z
<
0
).
any
():
raise
ValueError
(
mdperr
[
"mat_nonneg"
])
except
AttributeError
:
try
:
if
(
Z
.
data
<
0
).
any
():
raise
ValueError
(
mdperr
[
"mat_nonneg"
])
except
AttributeError
:
raise
TypeError
(
"Matrix should be a numpy type."
)
except
(
ValueError
,
TypeError
):
raise
return
(
None
)
def
exampleForest
(
S
=
3
,
r1
=
4
,
r2
=
2
,
p
=
0.1
):
"""Generate a MDP example based on a simple forest management scenario.
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment