Commit f1d63882 authored by Steven Cordwell's avatar Steven Cordwell
Browse files

Merge branch 'docs'

Conflicts:
	setup.py
parents 0b5e6f02 9347d6a9
2014-03-11
v4.0b1 - fist beta release created on the testing branch
2013-09-11 2013-09-11
v4.0a4 - Package is useable with Python 3.3, used 2tp3 script to convert. Still v4.0a4 - Package is useable with Python 3.3, used 2tp3 script to convert. Still
needs a bit of cleaning up. needs a bit of cleaning up.
......
2013-02-05
The first alpha release of v4.0 will be tagged today, which indicates that the
documentation and testing is starting to approach a level suitable for a final
release. A 4.0 release date has not been set, but once there are unittests for
all classes and functions, and the documentation is in a stable state, then it
can be tagged.
2013-01-26
The release of v0.14 will be made today. With this all the functionality should
be available to use. The initial version release of 4.0 will be made once the
documentation has been finalised and unit tests have been written to cover all
of the classes. The first release will be 4.0 to match the MATLAB, Octave,
Scilab and R MDP Toolboxes that are available at
http://www.inra.fr/mia/T/MDPtoolbox/
...@@ -6,6 +6,22 @@ descrete-time Markov Decision Processes. The list of algorithms that have been ...@@ -6,6 +6,22 @@ descrete-time Markov Decision Processes. The list of algorithms that have been
implemented includes backwards induction, linear programming, policy iteration, implemented includes backwards induction, linear programming, policy iteration,
q-learning and value iteration along with several variations. q-learning and value iteration along with several variations.
The classes and functions were developped based on the
`MATLAB <http://www.mathworks.com/products/matlab/>`_
`MDP toolbox http://www.inra.fr/mia/T/MDPtoolbox/>`_ by the
`Biometry and Artificial Intelligence Unit <http://mia.toulouse.inra.fr/>`_ of
`INRA Toulouse <http://www.toulouse.inra.fr/>`_ (France). There are editions
available for MATLAB, GNU Octave, Scilab and R.
Features
--------
- Eight MDP algorithms implemented
- Fast array manipulation using `NumPy <http://www.numpy.org>`_
- Full sparse matrix support using
`SciPy's sparse package <http://www.scipy.org/SciPyPackages/Sparse>`_
- Optional linear programming support using
`cvxopt <http://abel.ee.ucla.edu/cvxopt/>`_
Documentation Documentation
------------- -------------
Documentation is available as docstrings in the module code and as html in the Documentation is available as docstrings in the module code and as html in the
...@@ -22,7 +38,7 @@ Installation ...@@ -22,7 +38,7 @@ Installation
``tar -xzvf pymdptoolbox-<VERSION>.tar.gz`` ``tar -xzvf pymdptoolbox-<VERSION>.tar.gz``
``unzip pymdptoolbox-<VERSION>`` ``unzip pymdptoolbox-<VERSION>``
3. Change to the MDP toolbox directory 3. Change to the PyMDPtoolbox directory
``cd pymdptoolbox`` ``cd pymdptoolbox``
4. Install via Distutils, either to the filesystem or to a home directory 4. Install via Distutils, either to the filesystem or to a home directory
...@@ -33,11 +49,24 @@ Quick Use ...@@ -33,11 +49,24 @@ Quick Use
--------- ---------
Start Python in your favourite way. Then follow the example below to import the Start Python in your favourite way. Then follow the example below to import the
module, set up an example Markov decision problem using a discount value of 0.9, module, set up an example Markov decision problem using a discount value of 0.9,
and solve it using the value iteration algorithm. solve it using the value iteration algorithm, and then check the optimal policy.
>>> import mdptoolbox, mdptoolbox.example >>> import mdptoolbox.example
>>> P, R = mdptoolbox.example.forest() >>> P, R = mdptoolbox.example.forest()
>>> vi = mdptoolbox.mdp.ValueIteration(P, R, 0.9) >>> vi = mdptoolbox.mdp.ValueIteration(P, R, 0.9)
>>> vi.run()
>>> vi.policy >>> vi.policy
(0, 0, 0) (0, 0, 0)
Contribute
----------
Issue Tracker: https://code.google.com/p/pymdptoolbox/issues/list
Source Code: https://code.google.com/p/pymdptoolbox/source/checkout
Support
-------
License
-------
The project is licensed under the BSD license. See LICENSE.txt for details.
1. Improve the documentation and rewrite it as neccessary. 1. Improve the documentation and rewrite it as neccessary.
2. Work on converting the storage of the transitions and rewards as tuples of 2. Implement a nicer linnear programming interface to cvxopt, or write own
arrays.
3. Implement a nicer linnear programming interface to cvxopt, or write own
linear programming code. linear programming code.
4. Write unittests for all the classes. 3. Write unittests for all the classes.
5. Implement own exception class.
6. Move evalPolicy* functions to be module functions, as these are useful for
checking policies
7. Try to use rows for the Bellman computations rather than columns (perhaps 4. Implement own exception class.
this storage is more Pythonic? or more NumPyic?)
8. Create the mask as a sparse matrix in exampleRand 5. Move evalPolicy* functions to be functions of the util module, as these are
useful for checking policies in cases other than policy iteration algorithms.
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
from distutils.core import setup from distutils.core import setup
setup(name="pymdptoolbox", setup(name="pymdptoolbox",
version="4", version="0",
author="Steven A. W. Cordwell", author="Steven A. W. Cordwell",
author_email="steven.cordwell@uqconnect.edu.au", author_email="steven.cordwell@uqconnect.edu.au",
url="http://code.google.com/p/pymdptoolbox/", url="http://code.google.com/p/pymdptoolbox/",
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment