Skip to content

Latest commit

 

History

History
161 lines (110 loc) · 9.31 KB

README.md

File metadata and controls

161 lines (110 loc) · 9.31 KB
Raw

FOND Benchmarks

Flat collection of all FOND domains in circulation. Notes are our (Christian Muise & Sebastian Sardina) best guesses 🤭

These are planning domains that include the oneof effect to model non-deterministic actions (without probabilities). The oneof construct was proposed as part of NPDDL (which starts from level 2 of PDDL 2.1) in the following 2003 workshop paper:

Planning under non-deterministic oneof actions was then first used in the 2006 IPC-5 as an addition ot the probabilistic/uncertainty track (now probabilistic and non-deterministic track):

Notable Changes

  • added empty :parameters block to some actions:
    • finish action in faults, faults-new, and st_faults
    • changetire action in tireworld and st_tireworld.
  • added missing :requirements in domains:
    • st_blocksworld, st_first_responders, st_mapfdu, st_tires, st_faults, puffbot_dialogue_pddl, tidyup-mdp, and zenotravel.

Probabilistic Interesting - (Little and Thiebaux 2007)

(Little and Thiebaux 2007) proposed the following four "probabilistically interesting" domains:

  • climber: requires a strong conservative, less efficient, plan (to avoid falling and dying).
  • river
  • bus-fare
  • triangle-tireworld

Domains taken from FOND replanner safe-planner.

IPC-6 2008

The following four domains from the 2008 IPC-6 FOND track in the uncertainty part:

  • blocksworld (all solvable)
  • faults (all solvable)
  • first-responders
  • forest (at least 48/66 proven unsolvable)
    • type problem renamed to sub-problem to avoid a conflict with the problem keyword.

These domains are already FOND ones, that is, they use the non-deterministic oneof construct. Not every problem has a strong-cyclic plan, some were proved unsolvable (see Table 1 in 2012 PRP paper).

In addition, the following domains were crafted but not used in the final competition:

  • rectangle-tireworld: a variant of triangle-tireworld where the network is rectangular; nondeterministic by removing probabilities from the FOP version.
    • rectangle-tireworld-noghost: same but without the ghostTeleport action.
  • tireworld: adapted the IPC-5 (2006) probabilistic version by replacing probabilities with oneof non-determinism. Provided as an example before competition.

A description of the above domains can be found in 2008 Bryce & Buffet's International Planning Competition Uncertainty Part: Benchmarks and Results report.

PRP (2012)

The 2012 PRP paper used all the four final 2008 IPC-6 FOND track domains and probabilistic interesting ones, plus the following ones.

First, the blocksworld and forest domains were extended as follows:

  • blocksworld-new: scaled the number of blocks from 1 to 50 (the highest number of blocks in the original benchmark set is 20).
  • forest-new: generator slightly to guarantee that every problem has a strong cyclic plan.

Second, the PRP paper modified three domains from 2006 IPC5 Probabilistic Track by stripping the probabilities and replacing them with non-deterministic oneof constructs:

  • blocksworld-2: same domain as the IPC-6 blocksworld but with 15 instances coming from the probabilistic track.
  • elevators
  • zenotravel

Many additional domains were added later to its repo (but not reported in the 2012 paper itself) and became default benchmarks for later FOND planners: 👍

  • blocksworld-ex: FOND version of the probabilistic ex-blocksworld in 2008 IPC-6. In this domain, blocks can explode.
  • first-responders-new: as with forest-new and blocksworld-new, this is first-responders-new but with expanded sets of the benchmark to scale much larger.

FOND-SAT (2018)

These five come from the FOND-SAT repo/paper, crafted to have many "misleading plans" (i.e., weak plans leading to dead-ends):

  • tireworld-spiky: modification of triangle-tireworld.
  • tireworld-truck: modification of tireworld-spiky itself.
  • islands
  • doors: a variation of chain-of-rooms (used by QBFPLAN and MBP).
  • miner

Misc Domains

  • acrobatics: similar to beam-walk; by Miquel Ramirez 2014.
  • beam-walk: (Cimatti, Roveri, & Traverso, 1998)
  • chain-of-rooms: from Constructing conditional plans by a theorem-prover, JAIR 10 (1999), 323–352, as part of the QBFPLAN planner.
  • earth-observation: (J Aldinger & J Löhr, 2013)
  • puffbot-dialog: domain generating the dialogue policy for building dialogue agents in the health-care setting. Reported in Teixeira et. al SAC'21 paper and PhD thesis.
    • have problem-specific objects moved to domain constants, since they are used in actions.
  • tidyup-mdp: from MyND (as far as we can trace it back; see here); with significant clean-up by Christian Muise.
    • modified to fix some PDDL bugs: the cup* objects were moved from the problem files to constants in the domain, since they are mentioned explicitly in action preconditions.

From strong domains: st_xyz

These are domains that were designed to admit strong solutions. Strong cyclic plans exist, but it's the acyclic ones that are desired. 😉

  • st_blocksworld, st_faults, st_first_responders, st_tiresworld: domains from the 2008 IPC-6 FOND track that were modified to have strong solutions by Jicheng Fu et. al ICTAI'13.
  • st_mapfdu: from Thorsten Engesser on planning for implicit coordination; AAAI'20.

Catalogue Tool

Script tool catalogue.py gives you access to all of the benchmarks in the form of dom/prob pairs (absolute paths). Example usage:

from catalogue import confirm_files, COLLECTIONS, DOMAINS

# Prints out every benchmark and the number of instances in them
confirm_files()

# Access the problems for a specific domain
for dom,prob in DOMAINS['acrobatics']:
    print(dom)
    print(prob)

# Access all the benchmarks for a specific collection
for domain_name in COLLECTIONS['all-fond-papers']:
    print(domain_name)

Script can also be run as follows:

$ python catalogue.py
(  8) acrobatics
( 11) beam-walk
( 30) blocksworld
( 15) blocksworld-2
( 50) blocksworld-new
(  1) bus-fare
...
...
...
All domains in collection 'interesting':
         climber
         bus-fare
         river

FOND Planners

Using PDDL with oneof construct:

Not using oneof construct:

  • FIP (2011). Domains need to be (manually) determinized first (with specific naming on outcomes; fewer than 10).
  • GAMER (2009; pdf)
  • NDP (2008)
  • MBP (2003)
  • QBFPLAN (1999)
  • UMOP (2000): universal OBDD-based planning framework for non-deterministic multi-agent domains (described in their NADL domain description language).