Incremental Policy Iteration for POMDPs

This web-site contains the source code for the following paper:

Marek Grzes
and Pascal Poupart: Incremental Policy Iteration with Guaranteed Escape from Local Optima in POMDP Planning. Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS). Istanbul, Turkey, 2015. [pdf] [slides] [poster] [bibtex]


Note that all jar files will be copied to the ~/lib directory. The following steps could be followed to compile the main project and all dependencies.

tar xzf mgjpomdp-IPI-branch.tar.gz
# CPLEX paths have to be updated in mgjpomdp-IPI-branch/build.xml
# CPLEX paths have to be updated at the end of the mgjpomdp-IPI-branch/ script
bash mgjpomdp-IPI-branch/

The mgjpomdp-IPI-branch/ bash script will download all dependencies, compile the entire project, put it in one jar file, and then run the planner on the tiger proglem. The full content of the bash scipt is here:


# This bash script downloads and compiles all the code that is required to build my mgjpomdp package. This script
# assumes that cplex is installed. The paths to cplex should be set in the mgjpomdp/build.xml file. The execution of the
# main program requires -Djava.library.path=/opt/ibm/ILOG/CPLEX_Studio1261/cplex/bin/x86-64_linux/ and
# /opt/ibm/ILOG/CPLEX_Studio1261/cplex/lib/cplex.jar as well.
# Marek Grzes

echo "about to build all dependencies"
echo "note that all jar files will be copied to $HOME/lib"

# this should be ../mgjpomdp-IPI-branch

mkdir -p $HOME/lib
cd $HOME/lib/


echo "build mgjcommon"
hg clone
cd mgjcommon/
ant dist dist-src
echo "at this point you should see mgjcommon.jar and mgjcommon-src.jar in $HOME/lib"


echo "build matrix toolkits for java"
hg clone
cd mtj-mdp/
ant package
echo "at this point you should see Matrix_Toolkits_for_Java.jar and Matrix_Toolkits_for_Java-src.jar in $HOME/lib"

cd $HOME/lib


echo "build the POMDP parser; note that this package has to be compiled after mtj-mdp"
echo "this package is required by mglibpomdp"
#sudo apt-get install libglpk-java
git clone
cd libpomdp
ln -s $HOME/lib external


echo "build mgjpomdp"
cd mgjpomdp-IPI-branch
ant dist-one-jar

echo "check if the final program can be executed"
java -cp dist/mgjpomdp-ALL.jar mgjpomdp.apps.IPI --help

echo "run the planner on the tiger problem; the logs will be saved in /tmp"
java -Djava.library.path=/opt/ibm/ILOG/CPLEX_Studio1261/cplex/bin/x86-64_linux/ \
-cp dist/mgjpomdp-ALL.jar:/opt/ibm/ILOG/CPLEX_Studio1261/cplex/lib/cplex.jar mgjpomdp.apps.IPI --show-fsc \
--init-num-nodes=1 --node-rewiring DETocf --max-num-nodes 5

After the project has been successfully compiled, the mgjpomdp-ALL.jar will contain all Java classes (except CPLEX). mgjpomdp-ALL.jar is located in dist/mgjpomdp-ALL.jar and it will also be copied to ~/lib/mgjpomdp-ALL.jar.

[home page]