Starting v1.9.7 onwards, the contributors list is no longer maintained. Please check the Github contributors list instead.
+
FireWorks was developed primarily by Anubhav Jain at Lawrence Berkeley National Lab, using research funding from Kristin Persson for the Materials Project.
stored_data (dict) – data to store from the run. Does not affect the operation of FireWorks.
-
exit (bool) – if set to True, any remaining Firetasks within the same Firework are skipped.
-
update_spec (dict) – specifies how to update the child FW’s spec
-
mod_spec ([dict]) – update the child FW’s spec using the DictMod language (more flexible
+
stored_data (dict) – data to store from the run. Does not affect the operation of FireWorks.
+
exit (bool) – if set to True, any remaining Firetasks within the same Firework are skipped.
+
update_spec (dict) – specifies how to update the child FW’s spec
+
mod_spec ([dict]) – update the child FW’s spec using the DictMod language (more flexible
than update_spec)
additions ([Workflow]) – a list of WFs/FWs to add as children
detours ([Workflow]) – a list of WFs/FWs to add as children (they will inherit the
current FW’s children)
-
defuse_children (bool) – defuse all the original children of this Firework
-
defuse_workflow (bool) – defuse all incomplete steps of this workflow
-
propagate (bool) – apply any update_spec and mod_spec modifications
+
defuse_children (bool) – defuse all the original children of this Firework
+
defuse_workflow (bool) – defuse all incomplete steps of this workflow
+
propagate (bool) – apply any update_spec and mod_spec modifications
not only to direct children, but to all dependent FireWorks
down to the Workflow’s leaves.
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -199,12 +199,12 @@
name (str) – the name of the resource, should be unique
-
category (str or [str]) – a String describing a specific category of
+
name (str) – the name of the resource, should be unique
+
category (str or [str]) – a String describing a specific category of
job to pull, does not need to be unique. If the FWorker should
pull jobs of multiple categories, use a list of str.
-
query (dict) – a dict query that restricts the type of Firework this resource will run
-
env (dict) – a dict of special environment variables for the resource.
+
query (dict) – a dict query that restricts the type of Firework this resource will run
+
env (dict) – a dict of special environment variables for the resource.
This env is passed to running Firetasks as a _fw_env in the
fw_spec, which provides for abstraction of resource-specific
commands or settings. See fireworks.core.firework.FiretaskBase
@@ -760,27 +760,27 @@
Delete a set of fireworks identified by their fw_ids.
+
ATTENTION: This function serves maintenance purposes and will leave
+workflows untouched. Its use will thus result in a corrupted database.
+Use ‘delete_wf’ instead for consistently deleting workflows together
+with theit fireworks.
rerun_duplicates (bool) – flag for whether duplicates should be rerun
+
recover_launch ('last' or int) – launch_id for last recovery, if set to
‘last’ (default), recovery will find the last available launch.
If it is an int, will recover that specific launch
recover_mode ('prev_dir' or 'cp') – flag to indicate whether to copy
@@ -1433,7 +1477,7 @@
password (str) – A String representing today’s date, e.g. ‘2012-12-31’
-
require_password (bool) – Whether a password is required to reset the DB. Setting to
+
password (str) – A String representing today’s date, e.g. ‘2012-12-31’
+
require_password (bool) – Whether a password is required to reset the DB. Setting to
false is dangerous because running code unintentionally could clear your DB - use
max_reset_wo_password to minimize risk.
-
max_reset_wo_password (int) – A failsafe; when require_password is set to False,
+
max_reset_wo_password (int) – A failsafe; when require_password is set to False,
FWS will not clear DBs that contain more workflows than this parameter
spec_document (dict) – The spec document. Note that only modifications to
the spec key are allowed. So if you supply {“_tasks.1.parameter”: “hello”},
you are effectively modifying spec._tasks.1.parameter in the actual fireworks
collection.
-
mongo (bool) – spec_document uses mongo syntax to directly update the spec
+
mongo (bool) – spec_document uses mongo syntax to directly update the spec
A LazyFirework only has the fw_id, and retrieves other data just-in-time.
This representation can speed up Workflow loading as only “important” FWs need to be
fully loaded.
Error raised if the context manager WFLock can’t acquire the lock on the WF within the selected
time interval (WFLOCK_EXPIRATION_SECS), if the killing of the lock is disabled (WFLOCK_EXPIRATION_KILL)
Lock a Workflow, i.e. for performing update operations
Raises a LockedWorkflowError if the lock couldn’t be acquired withing expire_secs and kill==False.
Calling functions are responsible for handling the error in order to avoid database inconsistencies.
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -93,7 +93,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -129,7 +129,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -165,7 +165,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -196,7 +196,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -218,7 +218,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -275,7 +275,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -311,7 +311,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -347,7 +347,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -419,7 +419,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -450,7 +450,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -481,7 +481,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -1155,14 +1155,14 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -133,14 +133,14 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -88,7 +88,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -119,7 +119,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -191,14 +191,14 @@
query_start (str) – The start time (inclusive) to query in isoformat (YYYY-MM-DDTHH:MM:SS.mmmmmm).
+
query_start (str) – The start time (inclusive) to query in isoformat (YYYY-MM-DDTHH:MM:SS.mmmmmm).
Default is 30 days before current time.
-
query_end (str) – The end time (exclusive) to query in isoformat (YYYY-MM-DDTHH:MM:SS.mmmmmm).
+
query_end (str) – The end time (exclusive) to query in isoformat (YYYY-MM-DDTHH:MM:SS.mmmmmm).
Default is current time.
-
query (dict) – Additional Pymongo queries to filter entries for process.
-
time_field (str) – The field to query time range. Default is “updated_on”.
-
args (dict) – Time difference to calculate query_start from query_end.
+
query (dict) – Additional Pymongo queries to filter entries for process.
+
time_field (str) – The field to query time range. Default is “updated_on”.
+
args (dict) – Time difference to calculate query_start from query_end.
Accepts arguments in python datetime.timedelta function. args and query_start can
not be given at the same time. Default is 30 days.
query_start (str) – The start time (inclusive) to query in isoformat (YYYY-MM-DDTHH:MM:SS.mmmmmm).
+
query_start (str) – The start time (inclusive) to query in isoformat (YYYY-MM-DDTHH:MM:SS.mmmmmm).
Default is 30 days before current time.
-
query_end (str) – The end time (exclusive) to query in isoformat (YYYY-MM-DDTHH:MM:SS.mmmmmm).
+
query_end (str) – The end time (exclusive) to query in isoformat (YYYY-MM-DDTHH:MM:SS.mmmmmm).
Default is current time.
-
time_field (str) – The field to query time range. Default is “time_end”.
-
query (dict) – Additional Pymongo queries to filter entries for process.
-
runtime_stats (bool) – If return runtime stats. Default is False.
-
include_ids (bool) – If return fw_ids. Default is False.
-
args (dict) – Time difference to calculate query_start from query_end.
+
time_field (str) – The field to query time range. Default is “time_end”.
+
query (dict) – Additional Pymongo queries to filter entries for process.
+
runtime_stats (bool) – If return runtime stats. Default is False.
+
include_ids (bool) – If return fw_ids. Default is False.
+
args (dict) – Time difference to calculate query_start from query_end.
Accepts arguments in python datetime.timedelta function. args and query_start can
not be given at the same time. Default is 30 days.
Helper class - run subprocess commands in a different thread with TIMEOUT option.
From https://gist.github.com/kirpit/1306188
Based on jcollado’s solution:
@@ -91,8 +91,8 @@
The QueueAdapter is responsible for all interactions with a specific queue management system.
This includes handling all details of queue script format as well as queue submission and
launch_dir (str) – directory where we want to write the blocks
-
nlaunches (int) – total number of launches desired; “infinite” for loop, 0 for one round
-
njobs_queue (int) – stops submitting jobs when njobs_queue jobs are in the queue, 0 for no limit.
+
launch_dir (str) – directory where we want to write the blocks
+
nlaunches (int) – total number of launches desired; “infinite” for loop, 0 for one round
+
njobs_queue (int) – stops submitting jobs when njobs_queue jobs are in the queue, 0 for no limit.
If 0 skips the check on the number of jobs in the queue.
-
njobs_block (int) – automatically write a new block when njobs_block jobs are in a single block
-
sleep_time (int) – secs to sleep between rapidfire loop iterations
-
reserve (bool) – Whether to queue in reservation mode
-
strm_lvl (str) – level at which to stream log messages
-
timeout (int) – # of seconds after which to stop the rapidfire process
-
fill_mode (bool) – whether to submit jobs even when there is nothing to run (only in
+
njobs_block (int) – automatically write a new block when njobs_block jobs are in a single block
+
sleep_time (int) – secs to sleep between rapidfire loop iterations
+
reserve (bool) – Whether to queue in reservation mode
+
strm_lvl (str) – level at which to stream log messages
+
timeout (int) – # of seconds after which to stop the rapidfire process
+
fill_mode (bool) – whether to submit jobs even when there is nothing to run (only in
non-reservation mode)
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -316,7 +316,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -347,7 +347,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -485,7 +485,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -516,7 +516,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -547,7 +547,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -717,14 +717,14 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -255,7 +255,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -299,7 +299,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -341,7 +341,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -378,7 +378,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -408,9 +408,9 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -459,9 +459,9 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -506,8 +506,8 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -577,7 +577,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -609,15 +609,16 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -700,7 +701,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -759,7 +760,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -811,7 +812,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -895,7 +896,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -950,7 +951,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -1033,7 +1034,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -1075,7 +1076,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -1125,7 +1126,7 @@
fw_spec (dict) – A Firework spec. This comes from the master spec.
+
fw_spec (dict) – A Firework spec. This comes from the master spec.
In addition, this spec contains a special “_fw_env” key that
contains the env settings of the FWorker calling this method.
This provides for abstracting out certain commands or
@@ -1250,14 +1251,14 @@
@keyword n: the number of vertices. Can be omitted, the default is
zero. Note that if the edge list contains vertices with indexes
-larger than or equal to M{m}, then the number of vertices will
-be adjusted accordingly.
-
-
@keyword edges: the edge list where every list item is a pair of
integers. If any of the integers is larger than M{n-1}, the number
-of vertices is adjusted accordingly. C{None} means no edges.
-
-
-
@keyword directed: whether the graph should be directed
-@keyword graph_attrs: the attributes of the graph as a dictionary.
-@keyword vertex_attrs: the attributes of the vertices as a dictionary.
-
-
Every dictionary value must be an iterable with exactly M{n} items.
-
-
-
@keyword edge_attrs: the attributes of the edges as a dictionary. Every
dictionary value must be an iterable with exactly M{m} items where
-M{m} is the number of edges.
text_mode (bool) – whether to use text_mode for file read/write (instead of binary). Might be useful if
working only with text files between Windows and Unix systems
Provide a server that can host shared objects between multiprocessing
Processes (that normally can’t share data). For example, a common LaunchPad is
shared between processes and pinging launches is coordinated to limit DB hits.
Used to help pickle inner classes, e.g. see Workflow.Links
When called with the containing class as the first argument,
and the name of the nested class as the second argument,
@@ -725,7 +640,7 @@
The input for FireWorks is often provided in JSON and YAML and generated by
+third-party software that is unaware of the valid data types in FireWorks. Latent
+mismatches of data types may produce run-time errors, such as missing keywords
+or wrong data types, that are more difficult to handle than a validation of the
+initial input.
+
JSON schema provides a formal human- and machine-readable description of
+the data types used in classes in FireWorks. Additionally, a function is provided
+that checks the validity of JSON and YAML inputs immediately before deserialization.
+
To use the schema the fireworks_schema package must be installed.
+
There are three ways to activate JSON schema validation:
+
+
Call the schema validator explicitly
+
Activate automatic schema validation
+
Modify the list of classes for automatic validation
This is the case when you use Python but read JSON/YAML serialized objects
+provided externally. In the following example, a serialized workflow object is
+loaded from a YAML file and validated against the Workflow schema:
To activate automatic schema validation you must specify:
+
JSON_SCHEMA_VALIDATE:true
+
+
+
in your FWConfig file. For more details about managing your FWConfig file see the
+FW Config tutorial.
+
The default value of JSON_SCHEMA_VALIDATE is false.
+
If automatic validation is turned on, i.e. JSON_SCHEMA_VALIDATE is true,
+then validation is performed only for built-in classes specified in the list
+JSON_SCHEMA_VALIDATE_LIST, whenever an object of these
+classes is loaded from file. You can find the default
+JSON_SCHEMA_VALIDATE_LIST in fw_config.py file in the FireWorks source.
+
+
+
Modify the list of classes for automatic validation¶
+
You can modify the default JSON_SCHEMA_VALIDATE_LIST in your FWConfig file.
+For example, to turn on automatic validation for serialized Firework and
+Workflow objects only: