Wasmer now supports journals for the state of a WASM process. This gives the ability to persist changes made to the temporary file system and to save and store snapshots of the running process.
The journal file is a linear history of events that occurred when the process was running that if replayed will bring the process made to a discrete and deterministic state.
Journal files can be concatenated, compacted and filtered to change the discrete state.
These journals are maintained in a consistent and durable way thus ensuring that failures of the system while the process is running does not corrupt the journal.
The journal will record state changes to the sandbox built around the WASM process as it runs however it may be important to certain use-cases to take explicit snapshot restoration points in the journal at key points that make sense.
When a snapshot is triggered all the running threads of the process are paused and the state of the WASM memory and thread stacks are recorded into the journal so that they can be restored.
In order to use the snapshot functionality the WASM process must be compiled with
the asyncify modifications, this can be done using the wasm-opt tool.
Note: If a process does not have the asyncify modifications you can still use
the journal functionality for recording the file system and WASM memory state
however the stacks of the threads will be omitted meaning a restoration will
restart the main thread.
Various triggers are possible that will cause a snapshot to be taken at a specific point in time, these are as follows:
Triggered when all the threads in the process go into an idle state. This trigger is useful to take snapshots at convenient moments without causing unnecessary overhead.
For processes that have TTY/STDIN input this is particularly useful.
Triggered when a listen syscall is invoked on a socket. This can be an important milestone to take a snapshot when one wants to speed up the boot time of a WASM process up to the moment where it is ready to accept requests.
Triggered when the process reads stdin for the first time. This can be useful to speed up the boot time of a WASM process.
Triggered when the process reads an environment variable for the first time. This can be useful to speed up the boot time of a CGI WASM process which reads the environment variables to parse the request that it must execute.
Triggered periodically based on a timer (default 10 seconds) which can be specified
using the journal-interval option. This can be useful for asynchronous replication
of a WASM process from one machine to another with a particular lag latency.
Issued if the user sends an interrupt signal (Ctrl + C).
Alarm clock signal (used for timers)
(see man alarm)
The SIGTSTP signal is sent to a process by its controlling terminal to request it to stop temporarily. It is commonly initiated by the user pressing Ctrl-Z.
The SIGSTOP signal instructs the operating system to stop a process for later resumption
When a non-deterministic call is made from WASM process to the outside world (i.e. it reaches out of the sandbox)
- The WASM process that wish to record the state of the threads must have had the
asyncifypost processing step applied to the binary (seewasm-opt). - Taking a snapshot can consume large amounts of memory while its processing.
- Snapshots are not instant and have overhead when generating.
- The layout of the memory must be known by the runtime in order to take snapshots.
On startup if the restore journal file is specified then the runtime will restore the state of the WASM process by reading and processing the log entries in the snapshot journal. This restoration will bring the memory and the thread stacks back to a previous point in time and then resume all the threads.
When a trigger occurs a new journal will be taken of the WASM process which will take the following steps:
- Pause all threads
- Capture the stack of each thread
- Write the thread state to the journal
- Write the memory (excluding stacks) to the journal
- Resume execution.
The implementation is currently able to save and restore the following:
- WASM Memory
- Stack memory
- Call stack
- Open sockets
- Open files
- Terminal text
Writes the log events to a linear log file on the local file system as they are received by the trait. Log files can be concatenated together to make larger log files.
The default implementation does not support snapshots and will error out if an attempt is made to send it events. Using the unsupported capturer as a restoration point will restore nothing but will not error out.
Deduplicates memory and stacks to reduce the number of volume of log events sent to its inner capturer. Compacting the events occurs in line as the events are generated
Filters out a specific set of log events and drops the rest, this capturer can be useful for restoring to a previous call point but retaining the memory changes (e.g. WCGI runner).