atomvm_packbeam

An Erlang Escript and library used to generate an AtomVM AVM file from a set of files (beam files, previously built AVM files, or even arbitrary data files).

This tool roughly approximates the functionality of the AtomVM PackBEAM utility, except:

• Support for multiple data types, include beam files, text files, etc
• "Pruned" extraction of beams from AVM files, so that only the beams that are needed are packed
• Support for embedded OTP applications in your PackBEAM files.

The packbeam tool may be used on its own as a stand-alone command-line utility. More typically, it is used internally as part of the atomvm_rebar3_plugin rebar3 plugin.

Prerequisites

Building packbeam requires a version of Erlang/OTP compatible with AtomVM, as well as a local installation of rebar3. Optionally, any recent version of make may be used to simplify builds. Consult the AtomVM Documentation for information about supported OTP versions.

Build

To build a release, run the following commands:

shell$ rebar3 release
shell$ rebar3 as prod tar

These commands will create an Erlang tar archive containing a versioned release of the atomvm_packbeam tool, e.g.,

...
===> Tarball successfully created: _build/prod/rel/atomvm_packbeam/atomvm_packbeam-0.6.2.tar.gz

in your local working directory.

IMPORTANT! The files in this tar archive do not contain the atomvm_packbeam prefix, so extracting these files without care will create a bin and lib directory in the location into which files from the archive is extracted. See the example below before proceeding!

You can use the install.sh script to install the atomvm_packbeam utility into a location on your local machine. You will need to specify the prefix location into which you want to install the utility, together with it's current version.

shell$ ./install.sh /opt/atomvm_packbeam 0.6.2
atomvm_packbeam version 0.6.2 installed in /opt/atomvm_packbeam.

Note. Some prefix locations may require root permissions to write files to.

Set your PATH environment variable to include the bin directory of the installation prefix (if not already set), and you should then be able to run the packbeam command included therein.

For example:

shell$ export PATH=/opt/atomvm_packbeam/bin:$PATH
shell$ packbeam help
Syntax:
    packbeam <sub-command> <options> <args>
    ...

packbeam command

The packbeam command is used to create an AVM file from a list of beam and other file types, to list the contents of an AVM file, or to delete elements from an AVM file.

The general syntax of the packbeam command takes the form:

packbeam <sub-command> <args>

On-line help is available via the help sub-command:

shell$ packbeam help

packbeam version 0.7.2

Syntax:
    packbeam <sub-command> <options> <args>

The following sub-commands are supported:

    create <options> <output-avm-file> [<input-file>]+
        where:
        <output-avm-file> is the output AVM file,
        [<input-file>]+ is a list of one or more input files,
        and <options> are among the following:
            [--prune|-p]           Prune dependencies
            [--start|-s <module>]  Start module
            [--remove_lines|-r]    Remove line number information from AVM files

    list <options> <avm-file>
        where:
        <avm-file> is an AVM file,
        and <options> are among the following:
            [--format|-f csv|bare|default]  Format output

    extract <options> <avm-file> [<element>]*
        where:
        <avm-file> is an AVM file,
        [<element>]+ is a list of one or more elements to extract
            (if empty, then extract all elements)
        and <options> are among the following:
            [--out|-o <output-directory>]   Output directory into which to write elements
            (if unspecified, use the current working directory)

    delete <options> <avm-file> [<element>]+
        where:
        <avm-file> is an AVM file,
        [<element>]+ is a list of one or more elements to delete,
        and <options> are among the following:
            [--out|-o <output-avm-file>]    Output AVM file

    version
        Print version and exit

    help
        Print this help

The packbeam command will return an exit status of 0 on successful completion of a command. An unspecified non-zero value is returned in the event of an error.

The packbeam sub-commands are described in more detail below.

create sub-command

To create an AVM file from a list of beam files, use the create sub-command to create an AVM file. The first argument is take to be the output AVM file, following by the files you would like to add, e.g.,

shell$ packbeam create mylib.avm mylib/ebin/mylib.beam mylib/ebin/foo.beam mylib/ebin/bar.beam

This command will create an AtomVM AVM file suitable for use with AtomVM.

The input files specified in the create subcommand may be among the following types:

• compiled BEAM files (typically ending in .beam)
• Previously created AVM files
• "Normal" files, e.g., text files, binary files, etc.

Note that beam files specified are stripped of their path information, inside of the generated AVM file. Any files that have the same name will be added in the order they are listed on the command line. However, AtomVM will only resolve the first such file when loading modules at run-time.

Start Entrypoint

If you are building an application that provides a start entrypoint (as opposed to a library, suitable for inclusion in another AVM file), then at least one beam module in an AVM file must contain a start/0 entry-point, i.e., a function called start with arity 0. AtomVM will use this entry-point as the first function to execute, when starting.

Note. It is conventional, but not required, that the first beam file in an AVM file contains the start/0 entry-point. AtomVM will use the first BEAM file that contains an exported start/0 function as the entry-point for the application.

If your application has multiple modules with exported start/0 functions, you may use the --start <module> (alternatively, -s <module>) option to specify the module you would like placed first in your AVM file. The <module> parameter should be the module name (without the .beam suffix, e.g., main).

A previously created AVM file file may be supplied as input (including the same file specified as output, for example). The contents of any input AVM files will be included in the output AVM file. For example, if you are building a library of BEAM files (for example, none of which contain a start/0 entry-point), you may want to archive these into an AVM file, which can be used for downstream applications.

In addition, you may specify a "normal" (i.e., non-beam or non-AVM) file. Normal files are labeled with the path specified on the command line.

shell$ packbeam create mylib.avm mylib.avm mylib/priv/sample.txt

Note. It is conventional in AtomVM for normal files to have the path <module-name>/priv/<file-name>.

Pruning

If you specify the --prune (alternatively, -p) flag, then packbeam will only include beam files that are transitively dependent on the entry-point beam. Transitive dependencies are determined by imports, as well as use of an atom in a module (e.g, as the result of a dynamic function call, based on a module name).

If there is no beam file with a start/0 entry-point defined in the list of input modules and the --prune flag is used, the command will fail. You should not use the --prune flag if you are trying to build libraries suitable for inclusion on other AtomVM applications.

Line number information

By default, the packbeam tool will generate line number information for embedded BEAM files. Line number information is included in Erlang stacktraces, giving developers more clues into bugs in their programs. However, line number information does increase the size of AVM files, and in some cases can have an impact on memory in running applications.

For production applications that have no need for line number information, we recommend using the -r (or --remove_lines) flags, which will strip line number information from embedded BEAM files.

list sub-command

The list sub-command will print the contents of an AVM file to the standard output stream.

To list the elements of an AVM file, specify the location of the AVM file to input as the first argument:

shell$ packbeam list mylib.avm
mylib.beam * [284]
foo.beam [276]
bar.beam [252]
mylib/priv/sample.txt [29]

The elements in the AVM file are printed to the standard output stream and are listed on each line. If a beam file contain an exported start/0 function, it will be marked with an asterisk (*). The size in bytes of each module is also printed in square brackets ([]).

You may use the --format (alternatively, -f) option to specify an output format. The supported formats are:

• csv Output elements in comma-separated value format. Fields include the module name, whether the element is a BEAM file, whether the element provides a start/0 entrypoint, and the size (in bytes) of the element.
• bare Output just the module name, with no annotations.
• default Output the module name, size (in brackets), and whether the file provides a start/0 entrypoint, indicated by an asterisk (*). The default output is used if the --format option is not specified.

extract sub-command

The extract sub-command can be used to extract elements from an AVM file.

To extract one or more elements from an AVM file, specify the location of the AVM file from which to extract elements, followed by the list of elements (as displayed via the list sub-command) to extract. If no elements are listed, then all elements from the AVM file will be extracted.

Non-BEAM ("normal") files that contain paths in their names will be extracted into a directory tree that reflects the path used in the element name. For example, if the element name is mylib/priv/sample.txt, then the sample.txt file will be extracted into the mylib/priv directory (relative to the output directory, detailed below).

You may optionally specify an output directory using the --out option, which will contain the extracted contents of the input AVM file. This directory must exist beforehand, or a runtime error will occur. If no output directory is specified, elements will be extracted into the current working directory.

For example:

shell$ mkdir mydir
shell$ packbeam extract -out mydir mylib.avm foo.beam mylib/priv/sample.txt
Writing to mydir ...
x foo.beam
x mylib/priv/sample.txt

delete sub-command

The delete sub-command can be used to remove elements from an AVM file.

To delete one or more elements from an AVM file, specify the location of the AVM file from which to remove elements, followed by the list of elements (as displayed via the list sub-command) to remove. You may optionally specify an output AVM file using the --out option, which will contain the contents of the input AVM file, minus the specified elements. If no output AVM is specified, the input AVM file will be overwritten.

For example:

shell$ packbeam delete -out mylib2.avm mylib.avm foo.beam bar.beam
shell$ packbeam list mylib2.avm
mylib.beam * [284]
mylib/priv/sample.txt [29]

packbeam_api API

In addition to being an escript command-line utility, this project provides an Erlang API and library for manipulating AVM files. Simply include atomvm_packbeam as a dependency in your rebar.config, and you will have access to this API.

For more detailed information about this API, see the packbeam_api Reference.

Creating PackBEAM files

To create a PackBEAM file, use the packbeam_api:create/2 function. Specify the output path of the AVM you would like to create, followed by a list of paths to the files that will go into the AVM file. Typically, these paths are a list of BEAM files, though you can also include plain data files, in addition to previously created AVM files. Previously-created AVM files will be copied into the output AVM file.

Note. Specify the file system paths to all files. BEAM file path information will be stripped from the AVM element path data. Any plain data files (non-BEAM files) will retain their path information. See the AtomVM Documentation about how to create plain data files in AVM files that users can retrieved via the atomvm:read_priv/2 function.

%% erlang
ok = packbeam_api:create(
    "/path/to/output.avm", [
        "/path/to/foo.beam",
        "/path/to/bar.beam",
        "/path/to/myapp/priv/sample.txt",
        "/path/to/some_lib.avm"
    ]
).

Alternatively, you may specify a set of options with the packbeam_api:create/3 function, which takes a map as the third parameter.

Key	Type	Deafult	Description
prune	boolean()	false	Specify whether to prune the output AVM file. Pruned AVM files can take considerably less space and hence may lead to faster development times.
start	module()	n/a	Specify the start module, if it can't be determined automatically from the application.
application	module()	n/a	Specify the application module. The <application>.app file will be encoded and included as an element in the AVM file with the path <module>/priv/application.bin
include_lines	boolean()	true	Specify whether to include line number information in generated AVM files.

Listing the contents of PackBEAM files

You can list the contents of PackBEAM files using the packbeam_api:list/1 function. Specify the file system path to the PackBEAM file you would like to list:

%% erlang
AVMElements = packbeam_api:list("/path/to/input.avm").

The returned AVMElements is list of an opaque data structures and should not be interpreted by user applications. However, several functions are exposed to retrieve information about elements in this list.

To get the element name, use the packbeam_api:get_element_name/1 function, passing in an AVM element. The return type is a string() and represents the path in the AVM file for the AVM element.

%% erlang
AVMElementName = packbeam_api:get_element_name(AVMElement).

To get the element data (as a binary) use the packbeam_api:get_element_data/1 function, passing in an AVM element. The return type is a binary() containing the actual data in the AVM element.

%% erlang
AVMElementData = packbeam_api:get_element_data(AVMElement).

To get the element module (as an atom) use the packbeam_api:get_element_module/1 function, passing in an AVM element. The return type is a module() and the module name of the AVM element.

Note that if the AVM element is not a BEAM file, this function returns undefined.

%% erlang
AVMElementModule = packbeam_api:get_element_module(AVMElement).

To determine if the element is a BEAM file, use the packbeam_api:is_beam/1 function, passing in an AVM element. The return value is a boolean().

%% erlang
IsBEAM = packbeam_api:is_beam(AVMElement).

To determine if the element is an entrypoint BEAM (i.e., it exports a start/0 function), use the packbeam_api:is_entrypoint/1 function, passing in an AVM element. The return value is a boolean().

%% erlang
IsEntrypoint = packbeam_api:is_entrypoint(AVMElement).

Deleting entries from PackBEAM files

You can delete entries from an AVM file using the packbeam_api:delete/3 function. Specify the file system path to the PackBEAM file you would like to delete from, the output path you would like to write the new AVM file to, and a list of AVM elements you would like to delete:

%% erlang
ok = packbeam_api:delete(
    "/path/to/input.avm",
    "/path/to/ouput.avm",
    ["foo.beam", "myapp/priv/sample.txt"]
).

Note. You may specify the same values for the input and output paths. In this case, the input AVM file will be over-written by the new AVM file.

Extracting entries from PackBEAM files

You can extract elements from an AVM file using the packbeam_api:extract/3 function. Specify the file system path to the PackBEAM file you would like to extract from, a list of AVM elements you would like to extract, and the output directory into which would like to extract the files:

%% erlang
ok = packbeam_api:extract(
    "/path/to/input.avm",
    ["foo.beam", "myapp/priv/sample.txt"],
    "/tmp"
).