...
 
Commits (24)
build/
dist/
docs/_build/
.coverage
*.db
*.py[co]
*~
*.egg-info/
......@@ -3,8 +3,19 @@ Markov Decision Process (MDP) Toolbox for Python
.. image:: https://travis-ci.org/sawcordwell/pymdptoolbox.svg?branch=master
:target: https://travis-ci.org/sawcordwell/pymdptoolbox
:alt: Build Status
.. image:: https://coveralls.io/repos/sawcordwell/pymdptoolbox/badge.png
:target: https://coveralls.io/r/sawcordwell/pymdptoolbox
:target: https://coveralls.io/r/sawcordwell/pymdptoolbox
:alt: Code Coverage
.. image:: https://pypip.in/py_versions/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi/pymdptoolbox/
:alt: Supported Python versions
.. image:: https://pypip.in/implementation/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi/pymdptoolbox/
:alt: Supported Python implementations
.. image:: https://pypip.in/license/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi/pymdptoolbox/
:alt: License
The MDP toolbox provides classes and functions for the resolution of
descrete-time Markov Decision Processes. The list of algorithms that have been
......@@ -17,6 +28,10 @@ The classes and functions were developped based on the
`Biometry and Artificial Intelligence Unit <http://mia.toulouse.inra.fr/>`_ of
`INRA Toulouse <http://www.toulouse.inra.fr/>`_ (France). There are editions
available for MATLAB, GNU Octave, Scilab and R.
The suite of MDP toolboxes are described in Chades I, Chapron G, Cros M-J,
Garcia F & Sabbadin R (2014) 'MDPtoolbox: a multi-platform toolbox to solve
stochastic dynamic programming problems', *Ecography*, vol. 37, no. 9, pp.
916–920, doi `10.1111/ecog.00888 <http://dx.doi.org/10.1111/ecog.00888>`_.
Features
--------
......@@ -27,15 +42,12 @@ Features
- Optional linear programming support using
`cvxopt <http://abel.ee.ucla.edu/cvxopt/>`_
Documentation
-------------
Documentation is available as docstrings in the module code.
.. TODO and as html in the doc folder or from `the MDPtoolbox homepage <>`_.
PLEASE NOTE: the linear programming algorithm is currently unavailable except
for testing purposes due to incorrect behaviour.
Installation
------------
NumPy and SciPy must be on your system to use of this toolbox. Please have a
NumPy and SciPy must be on your system to use this toolbox. Please have a
look at their documentation to get them installed. If you are installing
onto Ubuntu or Debian and using Python 2 then this will pull in all the
dependencies:
......@@ -48,14 +60,40 @@ dependencies to have a fully featured cvxopt then run:
``sudo apt-get install python3-numpy python3-scipy liblapack-dev libatlas-base-dev libgsl0-dev fftw-dev libglpk-dev libdsdp-dev``
I recommend using `pip <https://pip.pypa.io/en/latest/>`_ to install
The two main ways of downloading the package is either from the Python Package
Index or from GitHub. Both of these are explained below.
Python Package Index (PyPI)
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. image:: https://pypip.in/download/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi//pymdptoolbox/
:alt: Downloads
.. image:: https://pypip.in/version/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi/pymdptoolbox/
:alt: Latest Version
.. image:: https://pypip.in/status/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi/pymdptoolbox/
:alt: Development Status
.. image:: https://pypip.in/wheel/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi/pymdptoolbox/
:alt: Wheel Status
.. image:: https://pypip.in/egg/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi/pymdptoolbox/
:alt: Egg Status
.. image:: https://pypip.in/format/pymdptoolbox/badge.svg
:target: https://pypi.python.org/pypi/pymdptoolbox/
:alt: Download format
The toolbox's PyPI page is https://pypi.python.org/pypi/pymdptoolbox/ and there
are both zip and tar.gz archive options available that can be downloaded.
However, I recommend using `pip <https://pip.pypa.io/en/latest/>`_ to install
the toolbox if you have it available. Just type
``pip install pymdptoolbox``
at the console and it should take care of downloading and installing everything
for you. If you also want cvxopt to be automatically downloaded and installed
so that you can solve MDPs using linear programming then type:
so that you can help test the linear programming algorithm then type
``pip install "pymdptoolbox[LP]"``
......@@ -63,34 +101,43 @@ If you want it to be installed just for you rather than system wide then do
``pip install --user pymdptoolbox``
Otherwise, you can download the package manually from the web
1. Download the latest stable release from
https://pypi.python.org/pypi/pymdptoolbox or clone the Git repository
``git clone https://github.com/sawcordwell/pymdptoolbox.git``
If you downloaded the package manually from PyPI
2. If you downloaded the `*.zip` or `*.tar.gz` archive, then extract it
1. Extract the `*.zip` or `*.tar.gz` archive
``tar -xzvf pymdptoolbox-<VERSION>.tar.gz``
``unzip pymdptoolbox-<VERSION>``
3. Change to the PyMDPtoolbox directory
2. Change to the PyMDPtoolbox directory
``cd pymdptoolbox``
4. Install via Setuptools, either to the root filesystem or to your home
3. Install via Setuptools, either to the root filesystem or to your home
directory if you don't have administrative access.
``python setup.py install``
``python setup.py install --user``
Read the
`Setuptools documentation <https://pythonhosted.org/setuptools/>`_ for
more advanced information.
Of course you can also use virtualenv or simply just unpack it to your working
directory.
GitHub
~~~~~~
Clone the Git repository
``git clone https://github.com/sawcordwell/pymdptoolbox.git``
and then follow from step two above. To learn how to use Git then I reccomend
reading the freely available `Pro Git book <http://git-scm.com/book>`_ written
by Scott Chacon and Ben Straub and published by Apress.
Quick Use
---------
Start Python in your favourite way. The following example shows you how to
......@@ -98,12 +145,29 @@ import the module, set up an example Markov decision problem using a discount
value of 0.9, solve it using the value iteration algorithm, and then check the
optimal policy.
>>> import mdptoolbox.example
>>> P, R = mdptoolbox.example.forest()
>>> vi = mdptoolbox.mdp.ValueIteration(P, R, 0.9)
>>> vi.run()
>>> vi.policy
(0, 0, 0)
.. code:: python
import mdptoolbox.example
P, R = mdptoolbox.example.forest()
vi = mdptoolbox.mdp.ValueIteration(P, R, 0.9)
vi.run()
vi.policy # result is (0, 0, 0)
Documentation
-------------
Documentation is available at http://pymdptoolbox.readthedocs.org/
and also as docstrings in the module code.
If you use `IPython <http://ipython.scipy.org>`_ to work with the toolbox,
then you can view the docstrings by using a question mark ``?``. For example:
.. code:: python
import mdptoolbox
mdptoolbox?<ENTER>
mdptoolbox.mdp?<ENTER>
mdptoolbox.mdp.ValueIteration?<ENTER>
will display the relevant documentation.
Contribute
----------
......@@ -117,5 +181,5 @@ Use the issue tracker.
License
-------
The project is licensed under the BSD license. See LICENSE.txt for details.
The project is licensed under the BSD license. See `<LICENSE.txt>`_ for details.
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest coverage gettext
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " applehelp to make an Apple Help Book"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
@echo " coverage to run coverage check of the documentation (if enabled)"
clean:
rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/PythonMarkovDecisionProcessToolbox.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/PythonMarkovDecisionProcessToolbox.qhc"
applehelp:
$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
@echo
@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
@echo "N.B. You won't be able to view it unless you put it in" \
"~/Library/Documentation/Help or install it in your application" \
"bundle."
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/PythonMarkovDecisionProcessToolbox"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/PythonMarkovDecisionProcessToolbox"
@echo "# devhelp"
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
coverage:
$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
@echo "Testing of coverage in the sources finished, look at the " \
"results in $(BUILDDIR)/coverage/python.txt."
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
.. automodule:: mdptoolbox.example
:members:
:undoc-members:
:inherited-members:
:show-inheritance:
.. automodule:: mdptoolbox.mdp
:members:
:undoc-members:
:inherited-members:
:show-inheritance:
.. automodule:: mdptoolbox
:members:
:undoc-members:
:inherited-members:
:show-inheritance:
.. automodule:: mdptoolbox.util
:members:
:undoc-members:
:inherited-members:
:show-inheritance:
This diff is collapsed.
.. include:: ../README.rst
Contents
========
.. toctree::
:maxdepth: 2
api/mdptoolbox
api/mdp
api/util
api/example
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
@ECHO OFF
REM Command file for Sphinx documentation
if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set BUILDDIR=_build
set ALLSPHINXOPTS=-d %BUILDDIR%/doctrees %SPHINXOPTS% .
set I18NSPHINXOPTS=%SPHINXOPTS% .
if NOT "%PAPER%" == "" (
set ALLSPHINXOPTS=-D latex_paper_size=%PAPER% %ALLSPHINXOPTS%
set I18NSPHINXOPTS=-D latex_paper_size=%PAPER% %I18NSPHINXOPTS%
)
if "%1" == "" goto help
if "%1" == "help" (
:help
echo.Please use `make ^<target^>` where ^<target^> is one of
echo. html to make standalone HTML files
echo. dirhtml to make HTML files named index.html in directories
echo. singlehtml to make a single large HTML file
echo. pickle to make pickle files
echo. json to make JSON files
echo. htmlhelp to make HTML files and a HTML help project
echo. qthelp to make HTML files and a qthelp project
echo. devhelp to make HTML files and a Devhelp project
echo. epub to make an epub
echo. latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter
echo. text to make text files
echo. man to make manual pages
echo. texinfo to make Texinfo files
echo. gettext to make PO message catalogs
echo. changes to make an overview over all changed/added/deprecated items
echo. xml to make Docutils-native XML files
echo. pseudoxml to make pseudoxml-XML files for display purposes
echo. linkcheck to check all external links for integrity
echo. doctest to run all doctests embedded in the documentation if enabled
echo. coverage to run coverage check of the documentation if enabled
goto end
)
if "%1" == "clean" (
for /d %%i in (%BUILDDIR%\*) do rmdir /q /s %%i
del /q /s %BUILDDIR%\*
goto end
)
REM Check if sphinx-build is available and fallback to Python version if any
%SPHINXBUILD% 2> nul
if errorlevel 9009 goto sphinx_python
goto sphinx_ok
:sphinx_python
set SPHINXBUILD=python -m sphinx.__init__
%SPHINXBUILD% 2> nul
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)
:sphinx_ok
if "%1" == "html" (
%SPHINXBUILD% -b html %ALLSPHINXOPTS% %BUILDDIR%/html
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The HTML pages are in %BUILDDIR%/html.
goto end
)
if "%1" == "dirhtml" (
%SPHINXBUILD% -b dirhtml %ALLSPHINXOPTS% %BUILDDIR%/dirhtml
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The HTML pages are in %BUILDDIR%/dirhtml.
goto end
)
if "%1" == "singlehtml" (
%SPHINXBUILD% -b singlehtml %ALLSPHINXOPTS% %BUILDDIR%/singlehtml
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The HTML pages are in %BUILDDIR%/singlehtml.
goto end
)
if "%1" == "pickle" (
%SPHINXBUILD% -b pickle %ALLSPHINXOPTS% %BUILDDIR%/pickle
if errorlevel 1 exit /b 1
echo.
echo.Build finished; now you can process the pickle files.
goto end
)
if "%1" == "json" (
%SPHINXBUILD% -b json %ALLSPHINXOPTS% %BUILDDIR%/json
if errorlevel 1 exit /b 1
echo.
echo.Build finished; now you can process the JSON files.
goto end
)
if "%1" == "htmlhelp" (
%SPHINXBUILD% -b htmlhelp %ALLSPHINXOPTS% %BUILDDIR%/htmlhelp
if errorlevel 1 exit /b 1
echo.
echo.Build finished; now you can run HTML Help Workshop with the ^
.hhp project file in %BUILDDIR%/htmlhelp.
goto end
)
if "%1" == "qthelp" (
%SPHINXBUILD% -b qthelp %ALLSPHINXOPTS% %BUILDDIR%/qthelp
if errorlevel 1 exit /b 1
echo.
echo.Build finished; now you can run "qcollectiongenerator" with the ^
.qhcp project file in %BUILDDIR%/qthelp, like this:
echo.^> qcollectiongenerator %BUILDDIR%\qthelp\PythonMarkovDecisionProcessToolbox.qhcp
echo.To view the help file:
echo.^> assistant -collectionFile %BUILDDIR%\qthelp\PythonMarkovDecisionProcessToolbox.ghc
goto end
)
if "%1" == "devhelp" (
%SPHINXBUILD% -b devhelp %ALLSPHINXOPTS% %BUILDDIR%/devhelp
if errorlevel 1 exit /b 1
echo.
echo.Build finished.
goto end
)
if "%1" == "epub" (
%SPHINXBUILD% -b epub %ALLSPHINXOPTS% %BUILDDIR%/epub
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The epub file is in %BUILDDIR%/epub.
goto end
)
if "%1" == "latex" (
%SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
if errorlevel 1 exit /b 1
echo.
echo.Build finished; the LaTeX files are in %BUILDDIR%/latex.
goto end
)
if "%1" == "latexpdf" (
%SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
cd %BUILDDIR%/latex
make all-pdf
cd %~dp0
echo.
echo.Build finished; the PDF files are in %BUILDDIR%/latex.
goto end
)
if "%1" == "latexpdfja" (
%SPHINXBUILD% -b latex %ALLSPHINXOPTS% %BUILDDIR%/latex
cd %BUILDDIR%/latex
make all-pdf-ja
cd %~dp0
echo.
echo.Build finished; the PDF files are in %BUILDDIR%/latex.
goto end
)
if "%1" == "text" (
%SPHINXBUILD% -b text %ALLSPHINXOPTS% %BUILDDIR%/text
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The text files are in %BUILDDIR%/text.
goto end
)
if "%1" == "man" (
%SPHINXBUILD% -b man %ALLSPHINXOPTS% %BUILDDIR%/man
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The manual pages are in %BUILDDIR%/man.
goto end
)
if "%1" == "texinfo" (
%SPHINXBUILD% -b texinfo %ALLSPHINXOPTS% %BUILDDIR%/texinfo
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The Texinfo files are in %BUILDDIR%/texinfo.
goto end
)
if "%1" == "gettext" (
%SPHINXBUILD% -b gettext %I18NSPHINXOPTS% %BUILDDIR%/locale
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The message catalogs are in %BUILDDIR%/locale.
goto end
)
if "%1" == "changes" (
%SPHINXBUILD% -b changes %ALLSPHINXOPTS% %BUILDDIR%/changes
if errorlevel 1 exit /b 1
echo.
echo.The overview file is in %BUILDDIR%/changes.
goto end
)
if "%1" == "linkcheck" (
%SPHINXBUILD% -b linkcheck %ALLSPHINXOPTS% %BUILDDIR%/linkcheck
if errorlevel 1 exit /b 1
echo.
echo.Link check complete; look for any errors in the above output ^
or in %BUILDDIR%/linkcheck/output.txt.
goto end
)
if "%1" == "doctest" (
%SPHINXBUILD% -b doctest %ALLSPHINXOPTS% %BUILDDIR%/doctest
if errorlevel 1 exit /b 1
echo.
echo.Testing of doctests in the sources finished, look at the ^
results in %BUILDDIR%/doctest/output.txt.
goto end
)
if "%1" == "coverage" (
%SPHINXBUILD% -b coverage %ALLSPHINXOPTS% %BUILDDIR%/coverage
if errorlevel 1 exit /b 1
echo.
echo.Testing of coverage in the sources finished, look at the ^
results in %BUILDDIR%/coverage/python.txt.
goto end
)
if "%1" == "xml" (
%SPHINXBUILD% -b xml %ALLSPHINXOPTS% %BUILDDIR%/xml
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The XML files are in %BUILDDIR%/xml.
goto end
)
if "%1" == "pseudoxml" (
%SPHINXBUILD% -b pseudoxml %ALLSPHINXOPTS% %BUILDDIR%/pseudoxml
if errorlevel 1 exit /b 1
echo.
echo.Build finished. The pseudo-XML files are in %BUILDDIR%/pseudoxml.
goto end
)
:end
sphinxcontrib-napoleon==0.3.1
......@@ -2,7 +2,7 @@
from setuptools import setup, find_packages
setup(name="pymdptoolbox",
version="0",
version="4.0-b4-dev",
author="Steven A. W. Cordwell",
author_email="steven.cordwell@uqconnect.edu.au",
url="https://github.com/sawcordwell/pymdptoolbox",
......@@ -14,7 +14,7 @@ setup(name="pymdptoolbox",
"along with several variations.",
download_url="https://pypi.python.org/pypi/pymdptoolbox",
classifiers=[
"Development Status :: 2 - Pre-Alpha",
"Development Status :: 4 - Beta",
"Environment :: Console",
"Intended Audience :: Education",
"Intended Audience :: Science/Research",
......@@ -25,6 +25,7 @@ setup(name="pymdptoolbox",
"Programming Language :: Python :: 2",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.2",
"Programming Language :: Python :: 3.3",
"Programming Language :: Python :: 3.4",
"Topic :: Scientific/Engineering",
......
......@@ -262,7 +262,7 @@ def getTransitionAndRewardArrays(s):
# Loop over all actions
for a in range(ACTIONS):
# Assign the transition probabilities for this state, action pair
P[a][idx] = getTransitionProbabilities(x, F, s, a)
P[a][idx] = getTransitionProbabilities(s, x, F, a)
return(P, R)
def solveMDP():
......
......@@ -7,17 +7,18 @@ descrete-time Markov Decision Processes.
Available modules
-----------------
example
:mod:`~mdptoolbox.example`
Examples of transition and reward matrices that form valid MDPs
mdp
:mod:`~mdptoolbox.mdp`
Makov decision process algorithms
util
:mod:`~mdptoolbox.util`
Functions for validating and working with an MDP
How to use the documentation
----------------------------
Documentation is available both as docstrings provided with the code and
in html or pdf format from
in html or pdf format from
`The MDP toolbox homepage <http://www.somewhere.com>`_. The docstring
examples assume that the ``mdptoolbox`` package is imported like so::
......@@ -44,19 +45,19 @@ source code use ``mdp.ValueIteration??<ENTER>``.
Acknowledgments
---------------
This module is modified from the MDPtoolbox (c) 2009 INRA available at
This module is modified from the MDPtoolbox (c) 2009 INRA available at
http://www.inra.fr/mia/T/MDPtoolbox/.
"""
# Copyright (c) 2011-2013 Steven A. W. Cordwell
# Copyright (c) 2009 INRA
#
#
# All rights reserved.
#
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
#
# * Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
......@@ -65,7 +66,7 @@ http://www.inra.fr/mia/T/MDPtoolbox/.
# * Neither the name of the <ORGANIZATION> nor the names of its contributors
# may be used to endorse or promote products derived from this software
# without specific prior written permission.
#
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
......
......@@ -7,10 +7,13 @@ reward matrices.
Available functions
-------------------
forest
:func:`~mdptoolbox.example.forest`
A simple forest management example
rand
:func:`~mdptoolbox.example.rand`
A random example
:func:`~mdptoolbox.example.small`
A very small example
"""
......@@ -46,6 +49,7 @@ rand
import numpy as _np
import scipy.sparse as _sp
def forest(S=3, r1=4, r2=2, p=0.1, is_sparse=False):
"""Generate a MDP example based on a simple forest management scenario.
......@@ -184,6 +188,7 @@ def forest(S=3, r1=4, r2=2, p=0.1, is_sparse=False):
R[S - 1, 1] = r2
return(P, R)
def _randDense(states, actions, mask):
"""Generate random dense ``P`` and ``R``. See ``rand`` for details.
......@@ -201,7 +206,7 @@ def _randDense(states, actions, mask):
m[m <= r] = 0
m[m > r] = 1
elif mask.shape == (actions, states, states):
m = mask[action][state] # mask[action, state, :]
m = mask[action][state] # mask[action, state, :]
else:
m = mask[state]
# Make sure that there is atleast one transition in each state
......@@ -213,6 +218,7 @@ def _randDense(states, actions, mask):
_np.ones(states, dtype=int)))
return(P, R)
def _randSparse(states, actions, mask):
"""Generate random sparse ``P`` and ``R``. See ``rand`` for details.
......@@ -233,10 +239,10 @@ def _randSparse(states, actions, mask):
m[m <= 2/3.0] = 0
m[m > 2/3.0] = 1
elif mask.shape == (actions, states, states):
m = mask[action][state] # mask[action, state, :]
m = mask[action][state] # mask[action, state, :]
else:
m = mask[state]
n = int(m.sum()) # m[state, :]
n = int(m.sum()) # m[state, :]
if n == 0:
m[_np.random.randint(0, states)] = 1
n = 1
......@@ -258,6 +264,7 @@ def _randSparse(states, actions, mask):
R[action] = RR.tocsr()
return(P, R)
def rand(S, A, is_sparse=False, mask=None):
"""Generate a random Markov Decision Process.
......@@ -342,8 +349,9 @@ def rand(S, A, is_sparse=False, mask=None):
if mask is not None:
# the mask needs to be SxS or AxSxS
try:
assert mask.shape in ((S, S), (A, S, S)), "'mask' must have " \
"dimensions S×S or A×S×S."
assert mask.shape in ((S, S), (A, S, S)), (
"'mask' must have dimensions S×S or A×S×S."
)
except AttributeError:
raise TypeError("'mask' must be a numpy array or matrix.")
# generate the transition and reward matrices based on S, A and mask
......@@ -353,6 +361,7 @@ def rand(S, A, is_sparse=False, mask=None):
P, R = _randDense(S, A, mask)
return(P, R)
def small():
"""A very small Markov decision process.
......
This diff is collapsed.
......@@ -8,17 +8,18 @@ solved.
Available functions
-------------------
check
:func:`~mdptoolbox.util.check`
Check that an MDP is properly defined
checkSquareStochastic
:func:`~mdptoolbox.util.checkSquareStochastic`
Check that a matrix is square and stochastic
getSpan
:func:`~mdptoolbox.util.getSpan`
Calculate the span of an array
isNonNegative
:func:`~mdptoolbox.util.isNonNegative`
Check if a matrix has only non-negative elements
isSquare
:func:`~mdptoolbox.util.isSquare`
Check if a matrix is square
isStochastic
:func:`~mdptoolbox.util.isStochastic`
Check if a matrix is row stochastic
"""
......@@ -89,6 +90,7 @@ _MDPERR = {
"actions greater than 0. i.e. R.shape = (S, A) or (A, S, S)."
}
def _checkDimensionsListLike(arrays):
"""Check that each array in a list of arrays has the same size.
......@@ -101,6 +103,7 @@ def _checkDimensionsListLike(arrays):
raise _error.InvalidError(_MDPERR["obj_square"])
return dim1, dim2, dim3
def _checkRewardsListLike(reward, n_actions, n_states):
"""Check that a list-like reward input is valid.
......@@ -118,6 +121,7 @@ def _checkRewardsListLike(reward, n_actions, n_states):
raise _error.InvalidError(_MDPERR["R_shape"])
return dim1, dim2, dim3
def isSquare(matrix):
"""Check that ``matrix`` is square.
......@@ -138,6 +142,7 @@ def isSquare(matrix):
return True
return False
def isStochastic(matrix):
"""Check that ``matrix`` is row stochastic.
......@@ -154,6 +159,7 @@ def isStochastic(matrix):
absdiff = (_np.abs(matrix.sum(axis=1) - _np.ones(matrix.shape[0])))
return (absdiff.max() <= 10*_np.spacing(_np.float64(1)))
def isNonNegative(matrix):
"""Check that ``matrix`` is row non-negative.
......@@ -176,6 +182,7 @@ def isNonNegative(matrix):
return True
return False
def checkSquareStochastic(matrix):
"""Check if ``matrix`` is a square and row-stochastic.
......@@ -205,6 +212,7 @@ def checkSquareStochastic(matrix):
if not isNonNegative(matrix):
raise _error.NonNegativeError
def check(P, R):
"""Check if ``P`` and ``R`` define a valid Markov Decision Process (MDP).
......@@ -293,10 +301,11 @@ def check(P, R):
for aa in range(aP):
checkSquareStochastic(P[aa])
def getSpan(W):
"""Return the span of W
sp(W) = max W(s) - min W(s)
def getSpan(array):
"""Return the span of `array`
span(array) = max array(s) - min array(s)
"""
return W.max() - W.min()
return array.max() - array.min()
......@@ -6,11 +6,111 @@ Created on Sat Aug 24 14:18:51 2013
"""
import numpy as np
import scipy.sparse as sp
import mdptoolbox, mdptoolbox.example
import mdptoolbox.example
from .utils import SMALLNUM, P_small, R_small
class TestUnitsMDP(object):
def test_MDP_has_startRun_method(self):
P, R = mdptoolbox.example.small()
sdp = mdptoolbox.mdp.MDP(P, R, None, None, None)
sdp._startRun()
assert sdp.time is not None
def test_MDP_has_endRun_method(self):
P, R = mdptoolbox.example.small()
sdp = mdptoolbox.mdp.MDP(P, R, None, None, None)
sdp._startRun()
sdp.V = np.zeros(1) # to prevent AttributeError in sdp._endRun()
sdp.policy = np.zeros(1) # to prevent AttributeError in sdp._endRun()
time = sdp.time
sdp._endRun()
assert time != sdp.time
class TestMDP(object):
P = (((0.0, 0.0, 0.6, 0.4, 0.0),
(0.0, 0.0, 0.0, 0.0, 1.0),
(0.0, 0.0, 1.0, 0.0, 0.0),
(0.0, 0.0, 0.0, 1.0, 0.0),
(0.0, 0.0, 0.0, 0.0, 1.0)),
((0.0, 0.4, 0.0, 0.0, 0.6),
(0.0, 1.0, 0.0, 0.0, 0.0),
(0.0, 0.0, 0.0, 0.0, 1.0),
(0.0, 0.0, 0.0, 0.0, 1.0),
(0.0, 0.0, 0.0, 0.0, 1.0)))
R = (((0, 0, 0, 0, 0),
(0, 0, 0, 0, 0),
(0, 0, 0, 0, 0),
(0, 0, 0, 0, 0),
(0, 0, 0, 0, 0)),
((0, 0, 0, 0, 1),
(0, 0, 0, 0, 0),
(0, 0, 0, 0, 0),
(0, 0, 0, 0, 0),
(0, 0, 0, 0, 0)))
computed_R = ((0.0, 0.0, 0.0, 0.0, 0.0),
(0.6, 0.0, 0.0, 0.0, 0.0))
def test_a(self):
P = np.array(self.P)
R = np.array(self.R)
sdp = mdptoolbox.mdp.MDP(P, R, None, None, None)
assert sdp.S == 5
assert sdp.A == 2
assert (sdp.R[0] == np.array(self.computed_R[0])).all()
assert (sdp.R[1] == np.array(self.computed_R[1])).all()
def test_b(self):
P = np.array(self.P)
R = np.array(self.computed_R).T
sdp = mdptoolbox.mdp.MDP(P, R, None, None, None)
assert sdp.S == 5
assert sdp.A == 2
assert (sdp.R[0] == R[:, 0]).all()
assert (sdp.R[1] == R[:, 1]).all()
def test_c(self):
P = np.array(self.P)
R = (0, 1, 0, 1, 0)
sdp = mdptoolbox.mdp.MDP(P, R, None, None, None)
assert sdp.S == 5
assert sdp.A == 2
assert (sdp.R[0] == np.array(R)).all()
assert (sdp.R[1] == np.array(R)).all()
assert id(sdp.R[0]) == id(sdp.R[1])
def test_d(self):
P = [None] * 2
P[0] = sp.csr_matrix(np.array(self.P[0]))
P[1] = sp.csr_matrix(np.array(self.P[1]))
R = [None] * 2
R[0] = sp.csr_matrix(np.array(self.R[0]))
R[1] = sp.csr_matrix(np.array(self.R[1]))
sdp = mdptoolbox.mdp.MDP(P, R, None, None, None)
assert sdp.S == 5
assert sdp.A == 2
assert (sdp.R[0] == np.array(self.computed_R[0])).all()
assert (sdp.R[1] == np.array(self.computed_R[1])).all()
def test_e(self):
P = np.empty(shape=2, dtype=object)
P[0] = np.array(self.P[0])
P[1] = np.array(self.P[1])
R = np.empty(shape=2, dtype=object)
R[0] = np.array(self.R[0])
R[1] = np.array(self.R[1])
sdp = mdptoolbox.mdp.MDP(P, R, None, None, None)
assert sdp.S == 5
assert sdp.A == 2
assert (sdp.R[0] == np.array(self.computed_R[0])).all()
assert (sdp.R[1] == np.array(self.computed_R[1])).all()
def test_MDP_P_R_1():
P1 = []
P1.append(np.array(np.matrix('0.5 0.5; 0.8 0.2')))
......@@ -20,13 +120,14 @@ def test_MDP_P_R_1():
R1.append(np.array(np.matrix('5, -1')))
R1.append(np.array(np.matrix('10, 2')))
R1 = tuple(R1)
a = mdptoolbox.mdp.MDP(P_small, R_small, 0.9, 0.01, 1)
a = mdptoolbox.mdp.MDP(P_small, R_small, None, None, None)
assert type(a.P) == type(P1)
assert type(a.R) == type(R1)
for kk in range(2):
assert (a.P[kk] == P1[kk]).all()
assert (a.R[kk] == R1[kk]).all()
def test_MDP_P_R_2():
R = np.array([[[5, 10], [-1, 2]], [[1, 2], [3, 4]]])
P1 = []
......@@ -37,20 +138,21 @@ def test_MDP_P_R_2():
R1.append(np.array(np.matrix('7.5, -0.4')))
R1.append(np.array(np.matrix('2, 3.9')))
R1 = tuple(R1)
a = mdptoolbox.mdp.MDP(P_small, R, 0.9, 0.01, 1)
a = mdptoolbox.mdp.MDP(P_small, R, None, None, None)
assert type(a.P) == type(P1)
assert type(a.R) == type(R1)
for kk in range(2):
assert (a.P[kk] == P1[kk]).all()
assert (np.absolute(a.R[kk] - R1[kk]) < SMALLNUM).all()
def test_MDP_P_R_3():
P = np.array([[[0.6116, 0.3884],[0, 1]],[[0.6674, 0.3326],[0, 1]]])
P = np.array([[[0.6116, 0.3884], [0, 1]], [[0.6674, 0.3326], [0, 1]]])
R = np.array([[[-0.2433, 0.7073],[0, 0.1871]],[[-0.0069, 0.6433],[0, 0.2898]]])
PR = []
PR.append(np.array(np.matrix('0.12591304, 0.1871')))
PR.append(np.array(np.matrix('0.20935652,0.2898')))
PR = tuple(PR)
a = mdptoolbox.mdp.MDP(P, R, 0.9, 0.01, 1)
a = mdptoolbox.mdp.MDP(P, R, None, None, None)
for kk in range(2):
assert (np.absolute(a.R[kk] - PR[kk]) < SMALLNUM).all()
......@@ -6,11 +6,13 @@ Created on Sat Aug 24 15:00:29 2013
"""
import numpy as np
import scipy.sparse as sp
import mdptoolbox
from .utils import SMALLNUM, P_forest, R_forest, P_small, R_small, P_sparse
from .utils import P_forest_sparse, R_forest_sparse
from .utils import SMALLNUM, P_forest, R_forest, P_small, R_small, P_sparse, \
P_forest_sparse, R_forest_sparse, \
assert_sequence_almost_equal
def test_PolicyIteration_init_policy0():
sdp = mdptoolbox.mdp.PolicyIteration(P_small, R_small, 0.9)
......@@ -120,3 +122,13 @@ def test_PolicyIterative_forest_sparse():
assert sdp.policy == p
assert (np.absolute(np.array(sdp.V) - v) < SMALLNUM).all()
assert sdp.iter == itr
def test_goggle_code_issue_5():
P = [sp.csr_matrix([[0.5, 0.5], [0.8, 0.2]]),
sp.csr_matrix([[0.0, 1.0], [0.1, 0.9]])]
P = np.array(P)
R = np.array([[5, 10], [-1, 2]])
pi = mdptoolbox.mdp.PolicyIteration(P, R, 0.96)
pi.run()
expected = (100.67873303167413, 94.45701357466055)
assert_sequence_almost_equal(pi.V, expected)
# -*- coding: utf-8 -*-
import numpy as np
import scipy.sparse as sp
import mdptoolbox
class BaseTestIssue7(object):
discount = 0.9
P = [None] * 2
P[0] = np.array([
[ 0. , 0. , 0. , 0.64, 0. , 0. , 0.36, 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0.93, 0. , 0. , 0.07, 0. ],
[ 0. , 0. , 0. , 0. , 0. , 0.2 , 0. , 0. , 0.8 ],
[ 0. , 0. , 0. , 1. , 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 1. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. , 1. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. , 0. , 1. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. , 0. , 0. , 1. , 0. ],
[ 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 1. ]
])
P[1] = np.array([
[ 0. , 0. , 0.4 , 0. , 0.6 , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 1. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 1. , 0. , 0. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0.87, 0.13, 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 1. , 0. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. , 1. , 0. , 0. , 0. ],
[ 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0.11, 0.89],
[ 0. , 0. , 0. , 0. , 0. , 0. , 0. , 1. , 0. ],
[ 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 1. ]
])
R = [None] * 2
R[0] = np.zeros((9, 9))
R[1] = np.array([
[ 0., 0., 0., 0., 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 1., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0.]
])
computed_R = (np.array((0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0)),
np.array((0.6, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0)))
policy = (1, 1, 0, 0, 0, 0, 0, 0, 0)
def dense_P_dense_R(self, algorithm):
sdp = algorithm(self.P, self.R, self.discount)
if algorithm != mdptoolbox.mdp.QLearning:
assert (sdp.R[0] == self.computed_R[0]).all()
assert (sdp.R[1] == self.computed_R[1]).all()
assert not sp.issparse(sdp.P[0])
assert not sp.issparse(sdp.P[1])
assert not sp.issparse(sdp.R[0])
assert not sp.issparse(sdp.R[1])
sdp.run()
if algorithm != mdptoolbox.mdp.QLearning:
assert sdp.policy == self.policy, sdp.policy
def sparse_P_dense_R(self, algorithm):
P = list(map(sp.csr_matrix, self.P))
sdp = algorithm(P, self.R, self.discount)
if algorithm != mdptoolbox.mdp.QLearning:
assert (sdp.R[0] == self.computed_R[0]).all()
assert (sdp.R[1] == self.computed_R[1]).all()
assert sp.issparse(sdp.P[0])
assert sp.issparse(sdp.P[1])
assert not sp.issparse(sdp.R[0])
assert not sp.issparse(sdp.R[1])
sdp.run()
if algorithm != mdptoolbox.mdp.QLearning:
assert sdp.policy == self.policy, sdp.policy
def dense_P_sparse_R(self, algorithm):
R = list(map(sp.csr_matrix, self.R))
sdp = algorithm(self.P, R, self.discount)
if algorithm != mdptoolbox.mdp.QLearning:
assert (sdp.R[0] == self.computed_R[0]).all()
assert (sdp.R[1] == self.computed_R[1]).all()
assert not sp.issparse(sdp.P[0])
assert not sp.issparse(sdp.P[1])
#assert sp.issparse(sdp.R[0])
#assert sp.issparse(sdp.R[1])
sdp.run()
if algorithm != mdptoolbox.mdp.QLearning:
assert sdp.policy == self.policy, sdp.policy
def sparse_P_sparse_R(self, algorithm):
P = list(map(sp.csr_matrix, self.P))
R = list(map(sp.csr_matrix, self.R))
sdp = algorithm(P, R, self.discount)
if algorithm != mdptoolbox.mdp.QLearning:
assert (sdp.R[0] == self.computed_R[0]).all()
assert (sdp.R[1] == self.computed_R[1]).all()
assert sp.issparse(sdp.P[0])
assert sp.issparse(sdp.P[1])
#assert sp.issparse(sdp.R[0])
#assert sp.issparse(sdp.R[1])
sdp.run()
if algorithm != mdptoolbox.mdp.QLearning:
assert sdp.policy == self.policy, sdp.policy
# Needs some work before can use, need to pass horizon
#class TestFiniteHorizon(BaseTestIssue7):
#
# def test_dense_P_dense_R(self):
# self.dense_P_dense_R(mdptoolbox.mdp.FiniteHorizon)
#
# def test_sparse_P_dense_R(self):
# self.sparse_P_dense_R(mdptoolbox.mdp.FiniteHorizon)
#
# def test_dense_P_sparse_R(self):
# self.dense_P_sparse_R(mdptoolbox.mdp.FiniteHorizon)
#
# def test_sparse_P_sparse_R(self):
# self.sparse_P_sparse_R(mdptoolbox.mdp.FiniteHorizon)
#class TestLP(BaseTestIssue7):
#
# def test_dense_P_dense_R(self):
# self.dense_P_dense_R(mdptoolbox.mdp.LP)
#
# def test_sparse_P_dense_R(self):
# self.sparse_P_dense_R(mdptoolbox.mdp.LP)
#
# def test_dense_P_sparse_R(self):
# self.dense_P_sparse_R(mdptoolbox.mdp.LP)
#
# def test_sparse_P_sparse_R(self):
# self.sparse_P_sparse_R(mdptoolbox.mdp.LP)
class TestPolicyIteration(BaseTestIssue7):
def test_dense_P_dense_R(self):
self.dense_P_dense_R(mdptoolbox.mdp.PolicyIteration)
def test_sparse_P_dense_R(self):
self.sparse_P_dense_R(mdptoolbox.mdp.PolicyIteration)
def test_dense_P_sparse_R(self):
self.dense_P_sparse_R(mdptoolbox.mdp.PolicyIteration)
def test_sparse_P_sparse_R(self):
self.sparse_P_sparse_R(mdptoolbox.mdp.PolicyIteration)
class TestPolicyIterationModified(BaseTestIssue7):
def test_dense_P_dense_R(self):
self.dense_P_dense_R(mdptoolbox.mdp.PolicyIterationModified)
def test_sparse_P_dense_R(self):
self.sparse_P_dense_R(mdptoolbox.mdp.PolicyIterationModified)
def test_dense_P_sparse_R(self):
self.dense_P_sparse_R(mdptoolbox.mdp.PolicyIterationModified)
def test_sparse_P_sparse_R(self):
self.sparse_P_sparse_R(mdptoolbox.mdp.PolicyIterationModified)
class TestQLearning(BaseTestIssue7):
def test_dense_P_dense_R(self):
self.dense_P_dense_R(mdptoolbox.mdp.QLearning)
def test_sparse_P_dense_R(self):
self.sparse_P_dense_R(mdptoolbox.mdp.QLearning)
def test_dense_P_sparse_R(self):
self.dense_P_sparse_R(mdptoolbox.mdp.QLearning)
def test_sparse_P_sparse_R(self):
self.sparse_P_sparse_R(mdptoolbox.mdp.QLearning)
class TestValueIteration(BaseTestIssue7):
def test_dense_P_dense_R(self):
self.dense_P_dense_R(mdptoolbox.mdp.ValueIteration)
def test_sparse_P_dense_R(self):
self.sparse_P_dense_R(mdptoolbox.mdp.ValueIteration)
def test_dense_P_sparse_R(self):
self.dense_P_sparse_R(mdptoolbox.mdp.ValueIteration)
def test_sparse_P_sparse_R(self):
self.sparse_P_sparse_R(mdptoolbox.mdp.ValueIteration)
class TestRelativeValueIteration(BaseTestIssue7):
def test_dense_P_dense_R(self):
self.dense_P_dense_R(mdptoolbox.mdp.RelativeValueIteration)
def test_sparse_P_dense_R(self):
self.sparse_P_dense_R(mdptoolbox.mdp.RelativeValueIteration)
def test_dense_P_sparse_R(self):
self.dense_P_sparse_R(mdptoolbox.mdp.RelativeValueIteration)
def test_sparse_P_sparse_R(self):
self.sparse_P_sparse_R(mdptoolbox.mdp.RelativeValueIteration)
class TestValueIterationGS(BaseTestIssue7):
def test_dense_P_dense_R(self):
self.dense_P_dense_R(mdptoolbox.mdp.ValueIterationGS)
def test_sparse_P_dense_R(self):
self.sparse_P_dense_R(mdptoolbox.mdp.ValueIterationGS)
def test_dense_P_sparse_R(self):
self.dense_P_sparse_R(mdptoolbox.mdp.ValueIterationGS)
def test_sparse_P_sparse_R(self):
self.sparse_P_sparse_R(mdptoolbox.mdp.ValueIterationGS)