WEB-SNAP: POMDP-based Knowledge Engineering for Assistive Systems https://cs.uwaterloo.ca/~mgrzes/code/web-snap/ While this code is made publicly available, all work that uses this code, extends this code or makes a comparison to this code is required to cite my paper: Marek Grzes, Jesse Hoey, Shehroz Khan, Alex Mihailidis, Stephen Czarnuch, Dan Jackson and Andrew Monk: Relational Approach to Knowledge Engineering for POMDP-based Assistance Systems as a Translation of a Psychological Model. International Journal of Approximate Reasoning (IJAR). Special Issue of Applications of Bayesian Networks. Elsevier, 2013. INFO Software implemented by Marek Grzes. A tool for designing POMDP-based assistive systems by people who are not necessarily experts in artificial intelligence. No POMDP related knowledge is required. The designer works with an easy to use web interface were domain details can be specified. After that the full POMDP model (in the SPUDD/SymbolicPerseus format) is generated automatically and it can be used for decision making in prompting/assistive systems. Written in Java and PHP and requires a web-server (e.g., Apache) and a PostgreSQL database. 1. PACKAGE CONTENTS web-main - contains the main web application that should be installed on the web server. In order to import this project into Eclipse PHP, you can use "File->Import->General->Existing Projects into workspace" and select the web-main directory. This way, the entire web-snap directory should be loaded into the current Eclipse workspace. I've just checked that it works on Zend Eclipse for PHP Developers Version: 3.2.0. SnapGenerator - contains Java files that generate the actual POMDP file in the SPUDD/SymbolicPerseus format using the data found in the database. This application reads in the data from the database, and constructs a fully specified POMDP model which can be solved using the SymbolicPerseus planner. The planner also contains a simulator that can be used to simulate the policy after it has been solved. It can be further used to build arbitrary tools that execute the POMDP policy and update the belief state. lib - contains a copy of symbolicPerseus written in Java examples - example models that can be loaded to the database, as well as POMDP files in the symbolicPerseus format that were generated using this software screenshots - several screenshots that show how the web interface looks like 2. INSTALLATION web-main requires a web server with PHP support and a PostgreSQL database. The following shows how to configure the Apache web server and the PostgreSQL database on Ubuntu 12.04 so that web-main can be executed. 2.1 Install the Apache server with PHP plugins sudo apt-get install apache2 libapache2-mod-php5 php5-pgsql Go to http://localhost to see if Apache is running. You should be able to see the default Apache web-sited that says "It works!". 2.1.1 set up htaccess in Apache Go to /etc/apache2/sites-available/default and replace "AllowOverride None" with "AllowOverride AuthConfig". Restart Apache using: sudo /etc/init.d/apache2 restart 2.1.2 Set up PHP for Apache PHP modules for Apache may require the following command to be executed: a2enmod php5 2.2 Install PostgreSQL sudo apt-get install postgresql Change password for the postgres user. Type sudo -u postgres psql and then you can change password for user postgres using: ALTER ROLE postgres WITH ENCRYPTED PASSWORD 'mypass-here'; the same password as above should be set for the unix accout of user postgres: sudo passwd -d postgres sudo su postgres -c passwd If only the above is done, the server should be accessible from the localhost only, the problem is that with this the custom user has to exist also in the system, to avoid this limitation, the following has to be done: Go to /etc/postgresql/9.1/main/pg_hba.conf and replace "ident" with "trust" in the following line: local all all trust Now, you can create your actual user account, "x721demouser", in the database engine: sudo -u postgres createuser -D -A -P x721demouser And then create the default database which is required by web-snap: sudo -u postgres createdb -O x721demouser snap-config At some point I also had to uncomment this line: listen_addresses = 'localhost' and set password_encryption = on in /etc/postgresql/9.1/main/postgresql.conf After performing all steps up to here, I could connect to snap-config as x721demouser using: psql -d snap-config -U x721demouser -W At this point, we have PostgreSQL running with user x721demouser which is going to be our entry point to the database from Java and PHP codes. Example snap databases will be created (loaded) below. snap-config is used for configuration only and does not store any data related to particular snap models. 2.3 Install web-main web-main does not require any compilation. After all the required software has been installed, the full web-snap package should be checked out as follows: cd /var/www hg clone https://bitbucket.org/mgrzes/web-snap After that, the following steps are required: 2.3.1 Check if php is properly configured. You do this using the link http://localhost/web-snap/web-main/index.php/?test and http://localhost/web-snap/web-main/index.php/?phpinfo. If it does now work, you need to verify your installation of Apache and PHP. 2.3.2 Add your password to the /var/www/web-snap/web-main/.htpasswd file. You can encrypt your password using: htpasswd -n snapuser and then add the encrypted version of your password to that file. Ideally, for security reasons, you should move .htpasswd to another location and update the .htaccess file. Right now they are in the same directory to make installation easier. Note that you need to change the path in .htaccess when you are installing web-snap in a different directory. The current password is 'MtNjxQHP4Y' in case you won't skip this step initially. If this step is done correctly, you should be prompted to type a user name and password when you go to: http://localhost/web-snap/web-main/ 2.3.3 In the above web-site, there were no instances to work with. Now, we are going to create several empty databases. Each database will be named a snap-instance and the idea is that one POMDP will be generated from one instance. Hence, one logical database stores one snap-instance, and one instance generates one POMDP. New databases can be created using: sudo -u postgres createdb -O x721demouser snap-demo sudo -u postgres createdb -O x721demouser snap-test 2.3.3 Having two empty logical databased created above, we can load schema (empty tables) to those database using the following command: cd /var/www/web-snap psql -d snap-demo -U x721demouser -W < SnapGenerator/sql-files/create-snapdb.sql psql -d snap-test -U x721demouser -W < SnapGenerator/sql-files/create-snapdb.sql At this point, if we go to http://localhost/web-snap/web-main/, we can select one of the two snap-instances to work with. 2.3.4 Load example models to the database: The two databases created in the previous step are empty.They are empty initially. There is one small example than we can load into an existing database. Let's say that we want to load that example to snap-demo. psql -d snap-demo -U x721demouser -W < SnapGenerator/sql-files/snap-door4-example.sql This is a very simple prompting model where the person is supposed to open the door. In order to do that, the person needs an ability of recognising a door handle. If the person is not able to do the task, the system should prompt her to recognise the door handle. We will show below how to simulate this model, but before that, we need to compile the snap generator. Overall, at this point, the web interface should work if you go to http://localhost/web-snap/web-main. You can work on your model, but in order to be able to generate the POMDP file, you need to compile the SnapGenerator. 2.4 Compile and Install the Snap Generator If the repository was checked out from bitbucket as I mentioned before, SnapGenerator should be found here: /var/www/web-snap/SnapGenerator. 2.4.1 Required Libraries PostgreSQL JDBC which can be installed as follows in Ubuntu sudo apt-get install libpostgresql-jdbc-java 2.4.2 Compilation and Installation cd /var/www/web-snap/SnapGenerator bash create_jar.sh snap-ubuntu.jar should appear as a result in the current directory. Once compiled, there is no need to move the jar file; the web-main application is going to use that jar file from its default location. At this point entire application should be installed. 3. HOW TO USE IT? In this system, every model is stored in a separate logical database which we call a snap-instance; every such instance can be used to generate one POMDP file. Normally, one POMDP should be sufficient to model one assistive task. The problem is that sometimes large POMDPs cannot be solved by the planner and the model has to be split into a number of POMDPs and some form of hierarchical control is required. In this manual, I won't go into details of hierarchical control, but it is supported by this software as well because extra variables required by the hierarchical controller can be generated on request (see the check boxes located above the generate button). For those who are interested in hierarchical control, I would recommend looking into this paper: Jesse Hoey and Marek Grzes: Distributed control of situated assistance in large domains with many tasks. In Proceedings of International Conference on Automated Planning and Scheduling (ICAPS), Freiburg, Germany, 2011. When the new prompting model is designed using web-snap, the designer performs task analysis which breaks down the task into smaller elements. In this work, syndetic modelling form psychology is used because it nicely fits into dynamic Bayesian networks. The following paper describes the underlying methodology in more detail: Jesse Hoey, Thomas Ploetz, Dan Jackson, Patrick Olivier, Andrew Monk and Cuong Pham Rapid Specification and Automated Generation of Prompting Systems to Assist People with Dementia. Pervasive and Mobile Computing, 7, 3, June, 2011. A practical and detailed case study that shows how to use this methodology through this software package can be found in Section 4 of Marek Grzes, Jesse Hoey, Shehroz Khan, Alex Mihailidis, Stephen Czarnuch, Dan Jackson and Andrew Monk: Relational Approach to Knowledge Engineering for POMDP-based Assistance Systems as a Translation of a Psychological Model. International Journal of Approximate Reasoning (IJAR). Special Issue of Applications of Bayesian Networks. Elsevier, 2013. I would encourage anybody who wants to use this software to read that paper and refer to section 4 when designing own models. Additionally, here, I summarise this methodology at the very high level with hope that it will be useful for you to start using the software. We want to use this software to design a new prompting system. The example challenge could be to design a prompting system for the hand washing task where the person with a cognitive disability should be able to wash hands with the help of the promoting system. The entire prompting systems would include hardware, software to control the hardware and collect readings from sensors, and software that implements decision making engine. Here, we are talking about designing such an intelligent decision making engine based on POMDPs. This software adheres to the rules of the psychological interactive unit analysis, and the overall resulting methodology follows the way psychologists would do their analysis. When the interactive unit (IU) analysis is performed by a psychologist or a trained health professional, the outcome of such an analysis is usually the spreadsheet-like document that describes the IU table (see Table 1 in our IJAR paper for an example). An important observation is that the professional, before creating the IU table, writes down its components, where the task states, abilities and behaviours are distinguished. The health professional will also determine how every behaviour of the user changes the state of the world, and what kinds of preconditions are required for those changes to happen. This naturally corresponds to the use of our tool that, in this case, would require accomplishing a number steps that are explained in detail in Section 4 of our paper. The designer normally starts the process by watching the video showing the user washing her hands with the help of a human care giver who would prompt the user when required. The designer watches the video several times an identifies elements required in order to complete the IU analysis. 3.1 Design Your Model Assuming that you want to design a new model, you would start with an empty instance and you would need to perform a number of steps explained in the case study in Section 4 of our IJAR paper. This includes identifying task and environment features, dynamics of user behaviours, the IU table, rewards and costs of prompting, and finally identify sensors and their precision (information about sensors has to be provided by engineers who are familiar with sensors that the POMDP is going to work with). Each of these elements can be specified in corresponding tables through the web interface. 3.2 Generate POMDPs Once the initial model has been defined, the user can generate its formal POMDP specification. For that, the user clicks the "SNAP generator" option in the top menu. In there, there is a detailed description of the process and the "Generate" button can be found. Once clicked, the POMDP model will be generated on the server and it can be saved on a local computer. Having that POMDP file, the user can go to the next step which solves the model using the POMDP planner. 3.3 Solve POMDPs This software comes with the POMDP file for the snap-demo example used before in this manual. In order to solve that example (or any other POMDP model), one can type: cd /var/www/web-snap java -cp lib/symbolicPerseus.jar pomdp.Solver examples/pomdp/snap-demo-door4.txt -g -b 1500 -r 2 where symbolicPerseus.jar is a jar file that contains symbolicPerseus. Other planners could be used as long as they can read in the symbolicPerseus format. 3.4 Simulate POMDPs To simulate the policy using command line cd /var/www/web-snap java -cp lib/symbolicPerseus.jar pomdp.Solver examples/pomdp/snap-demo-door4.txt -i examples/pomdp/snap-demo-door4.pomdp -s 10 The simulator starts and the user can type observations using command line and overwrite (if necessary) actions suggested by the policy. The belief state and Q-values are also printed in order to provide more insight into the quality of the POMDP policy and the behaviour of the overall model. Note, that poor prompting does not mean that the policy is not good enough (symbolicPerseus is an approximated planner); it usually happens when the model is not correct and needs to be adjusted. Normally, the user would generate the POMDP, simulate it, and then go back to the model to refine it. The process requires several iterations of these steps, and usually leads to a model that does what the designer wants to achieve. 3.5 Use POMDP Policies in Your Software After you've successfully simulated your model, you can use it in your own program. For that you would need to look into the source code of the SymbolicPerseus and you would need to extract parts that load the POMDP model and the policy, and parts that do belief update, query the policy, and read in observations. Having this functionality, you will need to write your own code that provides observations (sensors), and takes and executes actions returned by the policy. 4. EXAMPLES Folders examples/sql and examples/pomdp contain several models in the database format and the symbolicPerseus format respectively. As for autumn 2013, I do not remember very well which exact model was used in our IJAR paper, but it is very likely that snap-hw_one_pomdp_version10 is the one that was used for the hand washing experiment with the human actor. In order to load the model examples/sql/snap-hw_one_pomdp_version10_full.sql to the database, the following steps would be required. First, we need a new database for that model: sudo -u postgres createdb -O x721demouser snap-hw_one_pomdp_version10 Second, we need to load the data and the schema: psql -d snap-hw_one_pomdp_version10 -U x721demouser -W < examples/sql/snap-hw_one_pomdp_version10_full.sql After that, the POMDP file for that model can be generated using the web interface. An example output for that model can be found in: examples/pomdp/snap-hw_one_pomdp_version10.txt It is a valid POMDP model in the symbolicPerseus format, and it can be solved and simulated according to our description in Section 3. There are a few other models in examples. For example, examples/sql/snap-hw_one_pomdp_version11_full.sql is another version of the hand washing problem that was used for testing. There are a few other models in examples/sql from other projects. You can load them using the above steps. DISCLAIMER This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ----------------- Have fun! Marek Grzes, Waterloo, 2013