Incremental Policy Iteration for POMDPs

This web-site contains the source code for the following paper:

Marek Grzes and Pascal Poupart: Incremental Policy Iteration with Guaranteed Escape from Local Optima in POMDP Planning. Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS). Istanbul, Turkey, 2015. [pdf] [slides] [poster] [bibtex]

Installation

Note that all jar files will be copied to the ~/lib directory. The following steps could be followed to compile the main project and all dependencies.

wget https://www.cs.kent.ac.uk/people/staff/mg483/code/IPI/mgjpomdp-IPI-branch.tar.gz
tar xzf mgjpomdp-IPI-branch.tar.gz
# CPLEX paths have to be updated in mgjpomdp-IPI-branch/build.xml
# CPLEX paths have to be updated at the end of the mgjpomdp-IPI-branch/build-deps.sh script
bash mgjpomdp-IPI-branch/build-deps.sh
The mgjpomdp-IPI-branch/build-deps.sh bash script will download all dependencies, compile the entire project, put it in one jar file, and then run the planner on the tiger proglem. The full content of the bash scipt is here:

#!/bin/bash

# This bash script downloads and compiles all the code that is required to build my mgjpomdp package. This script
# assumes that cplex is installed. The paths to cplex should be set in the mgjpomdp/build.xml file. The execution of the
# main program requires -Djava.library.path=/opt/ibm/ILOG/CPLEX_Studio1261/cplex/bin/x86-64_linux/ and
# /opt/ibm/ILOG/CPLEX_Studio1261/cplex/lib/cplex.jar as well.
#
# Marek Grzes

echo "about to build all dependencies"
echo "note that all jar files will be copied to $HOME/lib"

# this should be ../mgjpomdp-IPI-branch
CURRDIR=$(pwd)

mkdir -p $HOME/lib
cd $HOME/lib/
wget https://www.cs.kent.ac.uk/people/staff/mg483/code/IPI/trove-3.0.3.jar
wget https://www.cs.kent.ac.uk/people/staff/mg483/code/IPI/trove-3.0.3-src.jar
wget https://www.cs.kent.ac.uk/people/staff/mg483/code/IPI/junit-4.11.jar

cd $CURRDIR

echo "build mgjcommon"
hg clone https://mgrzes@bitbucket.org/mgrzes/mgjcommon
cd mgjcommon/
ant dist dist-src
echo "at this point you should see mgjcommon.jar and mgjcommon-src.jar in $HOME/lib"

cd $CURRDIR

echo "build matrix toolkits for java"
hg clone https://bitbucket.org/mgrzes/mtj-mdp
cd mtj-mdp/
ant package
echo "at this point you should see Matrix_Toolkits_for_Java.jar and Matrix_Toolkits_for_Java-src.jar in $HOME/lib"

cd $HOME/lib
wget https://www.cs.kent.ac.uk/people/staff/mg483/code/IPI/antlrworks-1.4.3.jar
wget https://www.cs.kent.ac.uk/people/staff/mg483/code/IPI/glpk-java.jar
wget https://www.cs.kent.ac.uk/people/staff/mg483/code/IPI/jmatharray.jar

cd $CURRDIR

echo "build the POMDP parser; note that this package has to be compiled after mtj-mdp"
echo "this package is required by mglibpomdp"
#sudo apt-get install libglpk-java
git clone https://github.com/mgrzes/libpomdp.git
cd libpomdp
ln -s $HOME/lib external
ant

cd $CURRDIR

echo "build mgjpomdp"
cd mgjpomdp-IPI-branch
ant dist-one-jar

echo "check if the final program can be executed"
java -cp dist/mgjpomdp-ALL.jar mgjpomdp.apps.IPI --help

echo "run the planner on the tiger problem; the logs will be saved in /tmp"
java -Djava.library.path=/opt/ibm/ILOG/CPLEX_Studio1261/cplex/bin/x86-64_linux/ \
-cp dist/mgjpomdp-ALL.jar:/opt/ibm/ILOG/CPLEX_Studio1261/cplex/lib/cplex.jar mgjpomdp.apps.IPI --show-fsc \
--init-num-nodes=1 --node-rewiring DETocf --max-num-nodes 5

After the project has been successfully compiled, the mgjpomdp-ALL.jar will contain all Java classes (except CPLEX). mgjpomdp-ALL.jar is located in dist/mgjpomdp-ALL.jar and it will also be copied to ~/lib/mgjpomdp-ALL.jar.

[home page]