v1.1.3

• Reverts v1.1.2 change to paths in run outputs JSON, due to unintended interference with caching of successive task calls in workflows
• Fix a rare error arising from concurrent tasks writing into the call cache at the same moment

v1.1.2

miniwdl run ops refinements

• The run outputs JSON for call cache hits now refer to output files under the newly created out/ run directory, rather than out/ from the cached run
  • These are usually just symbolic links to the same files; with config [file_io] output_hardlinks, they're hardlinks to the same inodes
• Insourced the docker image used for HTTP URI downloads using aria2c
• Suppress the "running as root" warning when miniwdl is running inside a container

v1.1.1

miniwdl run ops refinements

• Add option [scheduler] fail_fast = false to stop a workflow after a task failure only after letting other ongoing tasks finish normally
  • In contrast to default behavior of immediately terminating all ongoing tasks
  • Can be used with call caching to avoid necessarily discarding work in progress
• Improve compatibility with filesystems' varying support for "flock"
• With --log-json, produce both JSON and default log files in each run directory

v1.0.0 🚢 🎁 🎓 🍾

With thanks to CZI Science and all these cheery faces!

v0.11.3

The v0.11.x series will serve as the "release candidate" for miniwdl v1.0.0.

miniwdl run

• [WDL 1.1] unzip() function
• Under out/ directory, add dotfiles helping a crawling program to distinguish File and Directory outputs

miniwdl check

• restore --no-shellcheck option (hidden) for compatibility

v0.11.2

WDL

• Restore interpolation placeholder options (sep, default, true, false) as they're deprecated, not removed from WDL 1.1

miniwdl check

• Command-line option to suppress certain warning(s) globally: --suppress Warning1,Warning2
  • --suppress CommandShellCheck replaces --no-shellcheck
• WDL 1.1 warnings for newly-deprecated coercions and placeholder options

v0.11.1

miniwdl run

• [WDL version 1.1] accept input overrides for task runtime settings, e.g. mytask.runtime.cpu=4
• work around module import race condition in python3.9
• add retry logic for some idempotent dockerd requests (complements task-level maxRetries)

miniwdl configure

• create ~/.config if necessary

import WDL

• reminder: Python scripts now must explicitly import the WDL.runtime and WDL.Lint subpackages in order to use their functionality

v0.11.0

miniwdl configure

This new subcommand walks through generating a runner configuration file with common opt-in features, like the call and download caches. It eases first-time setup, allowing further customization.

miniwdl run

• Improved macOS compatibility, though setup still has a few roadbumps (#145)
• The runner "flocks" the top-level run directory, in addition to the log file therein, while the run is in progress.
• Recognize runtime.container as an alias for runtime.docker
• Adds plugin extensibility for call cache (yet to be documented)

miniwdl check

• Startup latency significantly improved (~50%)

WDL

• WDL version 1.1 is now recognized, anticipating its ratification in the OpenWDL community
  • Roughly, this includes all of miniwdl's existing version development features except Directory support
• Python scripts now must explicitly import the WDL.runtime and WDL.Lint subpackages in order to use their functionality, in addition to import WDL (for CLI startup latency improvement)
• Float coerces to String with six decimal places, per spec

v0.10.0

• Adds read_object() and read_objects()
  • only for initializing Map[String,String] and structs
  • Object remains generally unsupported
• Initialize structs from Map[String,_] returned by read_json() or read_object()
  • liable to fail at runtime if types don't match
  • new lint warning UnverifiedStruct
• String literals may contain both escape sequences and embedded, non-ASCII UTF-8 characters
• Call cache: store & reuse prior [sub]workflow outputs, as well as previously-supported task outputs
• Resolve docker image tag (e.g. ubuntu:20.04) to RepoDigest (ubuntu@sha256:c95a8e48...) before scheduling
  • Ensures use of identical image across retries or swarm workers
• (version development) Abbreviated syntax for passthrough call inputs
  • call t { input: x, y=z } is shorthand for call t { input: x=x, y=z }

v0.9.0

"WDL 2.0" preview

This release adds new features defined in the pending "WDL 2.0" language version. For WDL documents beginning with a version development declaration, the parser and runner support:

• Directory inputs and outputs for tasks and workflows
• Directory downloads and caching for s3: URIs (other services pending)
• None literal keyword, to create an empty value explicitly
• call task1 after prevCall { ... } to declare a dependency between workflow calls without an output-input linkage
• Revised struct literals declaring the struct type name instead of object, e.g. Car { make: "Ford", model: "Model T" }
• New standard library functions:
  • min() max()
  • sep() quote() squote()
  • keys() as_map() as_pairs() collect_by_key()

⚠️ version development is a moving target, as "WDL 2.0" is still to be finalized in the OpenWDL community.

miniwdl run

• CLI:
  • key=value inputs accept a space after the = and before the value, allowing shell path autocompletion for File/Directory inputs
  • --empty can specify empty strings (as well as empty arrays)
  • --none to nullify optional inputs with defaults
• The legacy output_links/ (linking to out/) is no longer created in each run directory
• Allow task call caching with URI inputs
• Default docker image (used if runtime.docker isn't specified) updated to ubuntu:20.04
• Call and download cache entries created with previous releases are invalidated, and may be deleted

WDL

• Pair values serialize to & from JSON {"left": x, "right": y} instead of [x,y]
• Allow Map literals with compound key types