This document presents the current best practices and conventions for using ASDF 3, as of 2017. It is not a tutorial, though it starts like one, because it assumes for each category of ASDF user (beginner, simple user, more elaborate user) that he already knows what seems to be common knowledge among such users, and tries to complete this knowledge with less obvious points that are often wrong in systems seen in the wild.
- Trivial Examples
- Simple Examples
- More Elaborate Examples
- Other
Let's start with some trivial examples. We'll see below how these examples evolve as systems grow more complex.
The most trivial use of ASDF is by calling (asdf:load-system :foobar)
to load your library.
Then you can use it.
For instance, if it exports a function some-fun in its package foobar,
then you will be able to call it with (foobar:some-fun ...) or with:
(in-package :foobar)
(some-fun ...)
To run the tests for a system, you may use:
(asdf:test-system :foobar)
The convention is that an error SHOULD be signalled if tests are unsuccessful.
Using keywords to name systems is good and well at the REPL.
However, when writing a program, a bootstrap script, or a system definition,
you SHOULD follow the good style of writing names in the canonical way;
and the canonical name of a system is a string, which by convention is in lower-case
(indeed any symbol used as system designator is downcased by asdf:coerce-name).
Thus, the proper way to designate a system in a program is with lower-case strings, as in:
(asdf:load-system "foobar")
(asdf:test-system "foobar")
(Historical note: MK-DEFSYSTEM and before it Genera's SCT
used string-upcase to normalize system names,
and use that as filenames, which made sense on Genera's filesystem
or using logical pathnames as common with MK-DEFSYSTEM.
Dan Barlow's ASDF, to "Play Nice With Unix", preferred lower-case file names,
and used string-downcase to normalize symbols into strings;
at the same time, he allowed you to specify mixed case or upper-case names by using a string.
The latter was probably a mistake, and I strongly recommend against using anything but lower-case.
If that weren't allowed, there would be a case for making lower-case symbols the default syntax,
and let the various case conversions do their job without quotes.
But that is not where we are.
In any case, system names designate .asd files, not Lisp bindings,
and this determines their syntax.)
A trivial system would have a single Lisp file called foobar.lisp.
That file would depend on some existing libraries,
say alexandria for general purpose utilities,
and trivia for pattern-matching.
To make this system buildable using ASDF,
you create a system definition file called foobar.asd,
with the following contents:
(defsystem "foobar"
:depends-on ("alexandria" "trivia")
:components ((:file "foobar")))
Note how the type lisp of foobar.lisp
is implicit in the name of the file above.
As for contents of that file, they would look like this:
(defpackage :foobar
(:use :common-lisp :alexandria :trivia)
(:export
#:some-function
#:another-function
#:call-with-foobar
#:with-foobar))
(in-package :foobar)
(defun some-function (...)
...)
...
Assuming your system is installed under the ~/common-lisp/ hierarchy
or some other filesystem hierarchy already configured for ASDF,
you can load it with: (asdf:load-system "foobar"),
or shorter (asdf:make :foobar)
If your Lisp was already started when you created that file,
you may have to (asdf:clear-configuration) to re-process the configuration.
Even the most trivial of systems needs some tests, if only because it will have to be modified eventually, and you want to make sure those modifications don't break client code. Tests are also a good way to document expected behavior.
The simplest way to write tests is to have a file foobar-tests.lisp
and modify the above foobar.asd as follows:
(defsystem "foobar"
:depends-on ("alexandria" "trivia")
:components ((:file "foobar"))
:in-order-to ((test-op (test-op "foobar/tests"))))
(defsystem "foobar/tests"
:depends-on ("foobar" "fiveam")
:components ((:file "foobar-tests"))
:perform (test-op (o c) (symbol-call :fiveam '#:run! :foobar)))
The :in-order-to clause in the first system
allows you to use (asdf:test-system :foobar)
which will chain into foobar/tests.
The :perform clause in the second system does the testing itself.
In the test system, fiveam is the name of a popular test library,
and the content of the perform method is how to invoke this library
to run the test suite :foobar.
Obviously your mileage may vary if you use a different library.
The :in-order-to ((test-op (test-op ...))) idiom will not work with ASDF 2:
attempts to test-system will result in failure due to circular dependencies;
however, load-system will still work.
As for the :perform method, you need to specify :after right after test-op
for ASDF 2 to accept the defsystem form at all (but still not make the :in-order-to work);
so it's probably OK if your test system is in a separate .asd file,
but if you want backward compatibility and it's a secondary system, you'll use :after.
Making things work completely on ASDF 2 as well as ASDF 3 would take a lot of pain,
and would be colloquial for neither.
But friends don't let friends use ASDF 2. ASDF 3 has been available since 2013, and all implementations have now been providing it for a long time. Many systems, and growing, assume ASDF 3, possibly some you depend on, and it's pointless to maintain compatibility with a moribund system (especially so if you don't also check that all your dependencies do, too). Upgrade your implementations and/or at least your ASDF, and tell your friends to do as much. The very founding principle of ASDF 2 (and 3) was to provide users with the ability to install a newer ASDF on top of an old one to fix its bugs (an ability that did not exist with ASDF 1); thus, users do not have to maintain compatibility with antique bugs.
These days, I even recommended that you should freely rely
on features of ASDF 3.1 (2014) not present in ASDF 3.0 (2013), such as:
~/common-lisp/ being included in the source-registry by default;
run-program being full-fledged with input and error-output support;
or the package-inferred-system feature.
Conversely, if you want indefinite backward compatibility, why stop at ASDF 2?
Why not also support ASDF 1, and MK-DEFSYSTEM, and
the original Lisp Machine DEFSYSTEM?
A good rule of thumb for me is that it is not worth writing workarounds
for bugs that were fixed or features that were changed two years ago or more.
Two years seems to be the time it takes for a release of ASDF
to become ubiquitously available by default on all implementations;
and, remember, if a user must use an older implementation,
he can always trivially install a newer ASDF on top of it.
Thus, by October 2019, people should not be shy about
dropping support for ASDF versions older than 3.3.
And even before then, if you need a recent ASDF, just document it,
and tell your users to upgrade their implementation and/or
install a recent ASDF on top of their implementation's.
In the previous testing code, symbol-call is a function defined in package uiop.
It helps deal with the fact that the package :fiveam isn't defined yet
at the time the defsystem form is read.
Thus you can't simply write (fiveam:run! :foobar), as
this might cause a failure at read-time when file foobar.asd is first read
and the system fiveam isn't loaded yet.
This of course also holds for packages defined by system foobar itself.
To get a symbol in it that is not a function being called,
you can also use (find-symbol* :some-symbol :foobar),
and you can use symbol-value to get the value bound to such a symbol as denoting a variable.
For complex expressions, you can use eval and read-from-string,
as in (eval (read-from-string "(foobar:some-fun foobar:*some-var*)")),
or equivalently, also from uiop: (eval-input "(foobar:some-fun foobar:*some-var*)").
System definition files are loaded with the current *package* bound to the package :asdf-user,
that uses the packages :cl, :asdf and :uiop.
Therefore, you don't need to and you SHOULD NOT specify cl:, asdf: or uiop: prefixes
when referring to symbols in these respective packages.
It is considered bad form to do so, unless you changed the package,
and even then, your package should probably use these packages, too.
In particular you SHOULD NOT to write (asdf:defsystem "foobar" ...)
instead of (defsystem "foobar" ...).
Also, you SHOULD NOT use (in-package ...) or (defpackage ...)
if all you do can legitimately be done in package ASDF-USER.
System definition files are loaded by a special function asdf::load-asd;
this function locally binds the current *package* to the :asdf-user package,
but it also does many other things.
You MUST NOT ever load a .asd file using cl:load,
as this will not work in general, and may fail in subtle or spectacular ways.
You MUST always use asdf::load-asd for that purpose
(the function is exported in ASDF 3.2 and above, but not in earlier versions).
You SHOULD NOT encourage the illusion that a .asd file can be loaded with cl:load
by using (in-package :asdf-user) or anything,
or by using the package prefix asdf: for defsystem and other such symbols.
You SHOULD be using the slime-asdf extension to SLIME
if you are going to edit .asd file and then load them from SLIME,
as it will automatically use load-asd to load the file contents.
A crucial notion in Common Lisp is that of symbols, to which are associated functions, variables, macros, properties, and other meanings. These symbols are organized in a two-level namespace that is global to a given Lisp image. Symbols are the second level, and the first level are packages. Symbols and packages are traditionally uppercase internally, but you traditionally write them lower-case in source code; the Common Lisp reader is case-converting. These internal representation details matter, because they are exposed when you either write macros at compile-time or introspect the system at runtime.
The conventional way to refer to a package is with its name designated by a lower-case keyword,
starting with the colon character :.
For instance, the standard package COMMON-LISP is commonly designated as :common-lisp
(or (find-package :common-lisp) if you need an actual package object).
Keywords are themselves symbols in the magic package KEYWORD,
where each interned symbol is also a constant bound to its own name as a value.
Using a keyword as designator rather than a string ensures that we can maintain
the same convention of using lower-case in source code while the runtime will use uppercase.
As systems grow, the above pattern quickly becomes insufficient, but systems can still remain simple.
Some systems offer operations that are neither loading in the current image, nor testing. Whichever operation a system is meant to be used with, you may use it with:
(asdf:make :foobar)
This will invoke build-op, which in turn will depend on
the build-operation for the system, if defined, or load-op if not.
Therefore, for usual Lisp systems that want you to load them,
the above will be equivalent to (asdf:load-system :foobar),
but for other Lisp systems, e.g. one that creates a shell command-line executable,
(asdf:make ...) will do the Right Thing™, whatever that Right Thing™ is.
To look at what ASDF knows of your system, try:
(describe (asdf:find-system "foobar"))
If you're looking for the .asd file, try (asdf:system-source-file "foobar").
For the directory in which the .asd file resides, try (asdf:system-source-directory "foobar").
For a specific file under that directory try
(asdf:system-relative-pathname "foobar" "path/to/the/file.extension").
ASDF has the concept of an operation that can act upon a system (or a smaller component thereof). Typical operations that matter to end-users include:
load-opfor loading a system in the current Lisp image, as used byasdf:load-system,test-opfor running tests associated to the system, as used byasdf:test-system,build-opfor doing whatever build operation is associated to the system, (which by default isload-opif the system didn't override it), as used byasdf:make.
Further operations of interest to users include:
compile-op(ensure the system is compiled, without necessarily loading all of it, or any bit of it),load-source-op(load the system from source without compiling),compile-bundle-op(create a single fasl for the entire system, for delivery),monolithic-compile-bundle-op(create a single fasl for the entire system and all its transitive dependencies, for delivery),image-op(create a development image with this system already loaded, for fast startup),program-op(create a standalone application, which we will see below), etc.
Whichever operation you want, (asdf:operate operation system)
will ensure this operation is performed on that system
(after all the necessary dependencies of such an action).
A short-hand asdf:oos is available for asdf:operate
(the name oos is an old acronym for operate-on-system,
and both functions existed as synonyms in the old MK-DEFSYSTEM;
the latter function doesn't exist in ASDF, which instead has just operate).
The operation is typically specified as a symbol that names the operation class.
Since ASDF 3, you can also use a keyword to specify an action in the ASDF package.
Thus, (asdf:oos :load-op :foobar) is equivalent to (asdf:load-system :foobar).
ASDF has a notion of primary system,
that it can find in configured parts of the filesystem (the source-registry),
in a file that has the same name and the type asd.
Thus, primary system foobar is defined in a file foobar.asd.
While arbitrary strings are accepted in system names, it is strongly discouraged
to use anything but lower-case ASCII letters and digits for primary system names,
plus the separators - (hyphen) and . (dot).
The . itself is only recommended in primary system names that are part of an informal hierarchy;
for instance the popular library iolib contains many related systems:
iolib.asdf, iolib.base, iolib.common-lisp, iolib.conf, iolib.examples,
iolib.grovel, iolib.tests.
The main one is ostensibly iolib, but it contains many systems,
and for some reasons (notably proper phase separation)
they cannot all be secondary systems in the same file.
A secondary system is a system defined in the same file as a primary system.
By convention, its name starts the same as the file's primary system,
followed by a slash / and by a suffix made of some arbitrary characters,
preferably ASCII letters and digits
plus the separators - (hyphen), . (dot) and / (slash).
We already saw an example of it with system foobar/tests
in the trivial testing definition above.
The convention above allows ASDF 3 and later to find a secondary system by name,
by first looking for the associated primary system.
ASDF 3.2 or later will issue a warning when you violate this naming convention.
(Historically, ASDF 1 (and 2) allowed arbitrary names for secondary systems,
and in practice many people used foo-test or such for secondary system names in foo.asd;
however, ASDF 1 (and 2) couldn't find those systems by name, and horrific bugs could happen
if a system was simultaneously defined in multiple files.)
A simple system may be made of many files.
Typically, in package.lisp (or packages.lisp)
you'll define the package (or packages)
used by all the files in your system
in one or more forms such as (defpackage :foobar ...).
Then, in a package utils.lisp you'll define utility macros and functions
that you're using throughout your system;
maybe some of them deserve to be moved to alexandria and other utility systems;
and maybe they already exist out there and you just haven't looked hard enough.
Then a file foobar.lisp defines the meat of your system.
Your system definition will look like:
(defsystem "foobar"
:depends-on ("alexandria" "trivia")
:serial t
:components
((:file "package")
(:file "utils")
(:file "foobar")))
The :serial t indicates that each of these files depends on (all) the previous.
As your system grows some more, soon enough instead of a single file foobar.lisp,
you will have several files, one for each aspect of your system, such as
foo.lisp, bar.lisp as well as foobar.lisp.
Moreover, :serial t soon becomes inappropriate:
it will make your code slower to compile, but also to read.
Indeed, those who read the code won't be readily able to tell
which parts of the code they need to keep active in their brains
to understand the code at hand.
Instead, you may prefer to explicitly represent the dependencies
between the components of your system using :depends-on clauses
as follows:
(defsystem "foobar"
:depends-on ("alexandria" "trivia" "trivia.ppcre")
:components
((:file "package")
(:file "utils" :depends-on ("package"))
(:file "foo" :depends-on ("utils"))
(:file "bar" :depends-on ("utils"))
(:file "foobar" :depends-on ("foo" "bar"))))
Out of good style, you SHOULD still list the components in an order that makes sense,
such that the readers can read the files and mentally rebuild the system.
However, note that this order, if coherent, will be respected
only in ASDF 3.3 or later, due to a bug in earlier versions of ASDF.
But that precise order shouldn't matter, or it should be reflected
in the :depends-on declarations (or in a :serial t declaration).
You MAY assume that the current package uses CL at the beginning of a file,
but you MUST NOT assume that it is any particular package at this point:
right now, it is guaranteed to be the CL-USER package of the underlying implementation,
but it is conceivable that in some indeterminate future,
some extension to CL may provide a well-defined portable alternative that ASDF would use.
Therefore, the sane way to write a Lisp file is that it SHOULD start
with an in-package form, optionally preceded by a defpackage form
(or a uiop:define-package form).
You SHOULD NOT write cl:in-package or precede your defpackage with an (in-package :cl-user)
(which is stupid, because to be pedantic you'd have to (cl:in-package :cl-user),
at which point you may as well (cl:defpackage ...) and (cl:in-package ...)).
If it's a regular cl-source-file, it can assume the language is CL indeed,
and that the readtable something reasonable, etc.
You MAY use any of the symbols documented and exported by ASDF or UIOP. Actually, it is warmly recommended to use them everywhere that it matters, instead of less documented or less portable alternatives.
You MUST NOT use asdf:run-shell-command, asdf:system-definition-pathname,
or other deprecated functions that were once recommended in the time of ASDF 1.
They will be removed in the near future (one to two year horizon).
ASDF 3.2 or 3.3 will issue a style-warning when you do, and
some future version of ASDF will issue a full warning,
which will then break the SBCL build.
See backward-interface.lisp for a list of deprecated function — or just heed the damn warnings.
Test systems can also be divided in multiple files.
If possible (which is not always the case), the file names for test files
should match file names for regular code files,
so that the file that tests bar.lisp will be called bar-test.lisp or bar-tests.lisp
(pick singular or plural, but have a story for it and be consistent about it).
To keep things tidy as the test system grows,
you may even put all test files in a subdirectory t/, test/ or tests/.
Your test system definition may then look like:
(defsystem "foobar/tests"
:depends-on ("fiveam" "foobar")
:pathname "t/" ;; specify the subdirectory
:components
((:file "test-suite")
(:file "utils-test" :depends-on ("test-suite"))
(:file "foo-test" :depends-on ("test-suite"))
(:file "bar-test" :depends-on ("test-suite"))
(:file "foobar-test" :depends-on ("test-suite")))
:perform (test-op (o c) (symbol-call :foobar/tests :run-test-suite)))
As the system and its test system both grow, the test system may be moved to its own file
foobar-tests.asd or foobar.tests.asd where it is its own primary system:
(defsystem "foobar-tests" ...)
Other secondary systems may be created beyond test systems: for instance systems that provide independent aspects of the system, or optional add-ons to it. One case is a command that makes the Lisp functionality accessible from a Unix shell.
To build an executable, define a system as follows
(in this case, it's a secondary system, but it could also be a primary system).
You will be able to create an executable file foobar-command
by evaluating (asdf:make :foobar/executable):
(defsystem "foobar/executable"
:build-operation program-op
:build-pathname "foobar-command" ;; shell name
:entry-point "foobar::start-foobar" ;; thunk
:depends-on ("foobar")
:components ((:file "main")))
The build-pathname gives the name of the executable;
a .exe type will be automatically added on Windows.
As a horrible kluge that may go away in the future,
the output will be created in the system's directory (where the foobar.asd file resides)
when the build-operation matches the requested operation.
A future version of ASDF will probably instead have some :output argument to operate,
or some such thing, and drop this ugly special case.
You have been warned. Contact me if you want that sooner, or not at all.
There, file main.lisp defines a function start-foobar in package foobar,
that takes no argument, and initializes and starts the executable ---
after uiop calls its own *image-restore-hook* and evaluates any provided *image-prelude*.
Importantly, libraries may register functions to call in the *image-restore-hook*
using register-image-restore-hook (see also register-image-dump-hook);
UIOP and ASDF themselves make use of this facility.
Typically, start-foobar will be defined as something like:
(defun start-foobar () (main (uiop:command-line-arguments)))
Where function main parses the arguments
(a list of strings, excluding the magic C argv[0], which can be computed as (uiop:argv0))
and does whatever its magic.
You may want to use the full-featured net.didierverna.clon or my small command-line-arguments
or one of a slew of other libraries to parse the command-line arguments.
You may want to use cl-scripting to nicely wrap Lisp code into execution contexts
that handle errors in a nice(r) user-visible way for the shell user.
You may want to use inferior-shell if your program in turn invokes other shell programs.
Instead may also use cl-launch to build executables, or buildapp.
cl-launch is largely compatible with ASDF
(indeed, a lot of code formerly written as part of cl-launch
was later made part of ASDF 3's UIOP).
buildapp, that came before ASDF 3 but after cl-launch,
has a slightly incompatible convention where a main function is called with
a list of arguments that includes the argv[0].
(cl-launch and after it UIOP had many good enough reasons
to start the argument list at the "user arguments", if only because
when invoking a Lisp implementation via cl-launch,
the process' argv[0] may not be available or meaningful,
whereas the user arguments may be only a subset of the actual process' arguments.)
Both cl-launch and buildapp have similar functionality
to handle multicall binaries à la Busybox,
with the same incompatibility as above.
Sometimes, a system can provide an extension to another system.
For instance, if you use both metacopy and contextl,
you may be interested in some system metacopy/with-contextl
that creates synergies between these two systems.
There exists a system asdf-system-connections that will allow you
to define such system connections that are automatically loaded
when both the connected systems are loaded.
While I debugged that mechanism and made sure it works with ASDF 3,
I recommend against using it, because it introduces side-effects within the build.
Instead I recommend explicitly loading the system connections
as part of the larger system that will use them.
Sometimes, you want to force ASDF to re-build some system.
At those times, you can pass the :force argument to asdf:operate
(or its wrappers asdf:load-system, asdf:test-system, asdf:oos, etc.).
Passing an argument t will force rebuild of just the system, and none of its dependencies
(and that also means none of the dependencies that happen to be secondary systems
with the same primary system name).
Passing a list of system designators (preferably lower-case strings)
will force the specific systems to be rebuilt (if they appear in the build plan at all, that is).
Finally, passing :all as argument will force a rebuild of everything, including all dependencies.
Thus, if you ran tests that use the prove test framework,
in which loading the files is itself the test,
and you want to force a re-run, even though ASDF might be satisfied
with already having loaded the files, then you can use:
(asdf:load-system :clack-test :force t)
Note that it is strongly recommended to not have any non-determinism or side-effects
that are not declared to ASDF, that would cause the forcing to be meaningful.
Forcing is thus a debugging feature for ASDF systems that fail this good practice.
In the case of prove, we will have to work with its author so that the correct way
to use it doesn't violate ASDF invariants, but instead properly declare that
ASDF should not consider tests already run.
The converse of :force is :force-not, and you can specify a list of systems to not rebuild.
In this context t means "everything but this system" rather than " this system".
force-not takes precedence over force, and by default includes a list of "immutable" systems
that may be used when delivering extensible applications to customers
(See function asdf::register-immutable-system).
Note that these flags are only for use by the user at the toplevel.
You MUST NOT call asdf:operate with :force or :force-not from within a build.
Actually, you should probably not explicitly use asdf:operate at all,
except maybe inside a .asd file in cases where defsystem-depends-on isn't sufficient.
You SHOULD NOT use cl:require as a substitute for asdf:load-system.
You SHOULD NOT use asdf:require-system except at the toplevel.
Back in the days of ASDF 1, a convenient hook was added to ASDF
so that when you call cl:require, it would try to load the named system,
and if not fall back on the builtin require mechanism.
This was a cool hack, and when you merely wanted a dependency, it was easier to type
(require :foo) than (asdf:operate 'asdf:load-op :foo) as you then had to.
Moreover, on SBCL where ASDF was developed,
ASDF itself was used to compile and load SBCL modules at runtime,
so this hook came naturally.
As ASDF maintainer, I now consider this in bad taste:
- First, this hook is not 100% portable, so it is bad taste to recommend relying on it.
- Second,
cl:requirehas a mechanism for loading things only once by checkingcl:*modules*which may subtly interfere with ASDF's mechanism for keeping things up to date. - Third, it interferes with ASDF's capacity to detect legitimate vs illegitimate
recursive uses of
operateat places that defeat tracking of dependencies. - Fourth, it adds a lot of complexity for dubious gain: at a time you had to type
(asdf:operate 'asdf:load-op :foo),(require :foo)may have been a nice short-hand, but it isn't such a great gain over(asdf:make :foo). - Fifth, SBCL now uses ASDF 3's
compile-bundle-opto create a fasl during the build of SBCL itself, and that fasl can latter be loaded at runtime without ASDF. Therefore the hook has no natural use anymore.
Similarly, in ASDF 2.21 I added a function require-system
that used to called load-system with :force-not (already-loaded-systems),
which was a nice hack at the time, that I latter used as part of the cl:require hook.
That was all a big mistake, as :force-not interferes with the ability to keep a coherent plan
across recursive uses of asdf:operate as required by builds that involve :defsystem-depends-on
and other ASDF extensions.
These days, this function only checks whether the requested component is already loaded,
and if not calls asdf:load-system on it.
This function MUST only be used at the toplevel, never in a script or build.
It may be deprecated in a future version of ASDF.
When you start writing large enough systems, putting everything in one big package leads to a big mess: it's hard to find what function is defined where, or should be defined where; you invent your own symbol prefixing system to avoid name clashes; totally unrelated things end up in the same mother-of-all package; you divide your mother-of-all package into a few subpackages, but as the software keeps growing each of these packages in turn becomes too big.
Meanwhile, as you grow large enough libraries, you find that you loading a big library just to use a small part of it becomes a big hassle, leading to code bloat, too much recompilation, too much re-testing, and not enough understanding of what's going on.
A solution to both these problems is the "one file, one package, one system" style,
once spearheaded by faslpath and quick-build, and now available as part of ASDF
(since ASDF 3.1, 2014) using the package-inferred-system class.
Following this style, your top .asd file defines uses this class for its primary system
(you can still define secondary systems with different classes).
Then, any secondary system, if not explicitly defined,
will be searched for in a file as named by the secondary system suffix,
under the directory that contains the system definition file.
Thus, if ~/common-lisp/foobar-1.0/foobar.asd defines
a primary system of class package-inferred-system,
but no secondary class foobar/x/y, then ASDF will look for a file
~/common-lisp/foobar-1.0/x/y.lisp to contain this system.
That file will be a regular Lisp file,
that will begin with a defpackage or uiop:define-package form.
(The latter form is more friendly to live upgrades,
but also allows to use a mix of packages with a priority on symbol conflicts,
or to reexport imported symbols.)
Dependencies for this system will be deduced from the :use,
:import-form and :shadowing-import-from clauses of that defpackage,
where each package name is downcased and interpreted as a system name,
unless registered otherwise via asdf:register-system-packages.
This allows for large modular libraries, wherein you may use one file, and only that file and its transitive dependencies will be loaded, rather than the entire humongous library.
This also helps you enforce a discipline wherein it is always clear in which file each symbol is defined, which files have symbols used by any other file, etc.
If you need an ASDF extension, the recommended way is to use :defsystem-depends-on.
The extension will define new classes of operations or components, new functions, etc.
If it defines them in the ASDF package, you can refer to them using a keyword of the same name,
e.g. :cffi-wrapper-file for one of the component classes defined by system cffi-grovel,
:static-program-op for one of the operation classes defined by cffi-toolchain, or
:f2cl-system for the system class defined by system f2cl.
Otherwise, you can refer to them using a string that when read yields the symbol, e.g.
the system asdf-finalizers defines a function that you can use as an around-compile hook
for your system using the clause :around-compile "asdf-finalizers:check-finalizers-around-compile".
You cannot usually specify the qualified symbol directy, because it lives in a package
that hasn't been defined yet at the time the defsystem form is read.
Sometimes, you may want to use several build extensions at once,
but a given system, smaller component or operation, can have only one class.
You may split that system in two (say, a primary system and a secondary system)
so each has its own extension (say, make the primary system a package-inferred-system,
but have a secondary system be a program-system).
For more advanced cases, you may have to define a class that inherits from multiple other classes
in your own extension.
(In a prototype OO system, you could just mix and match extensions without defining a new class,
but CLOS is not a prototype OO system.)
ASDF currently allows arbitrary Lisp code in a .asd file.
I would like to deprecate that in the future to make ASDF more declarative.
In the meantime, here are some guidelines for how to write such code properly.
First, you SHOULD minimize the amount of such code and
you SHOULD strongly consider defining an extension
in a separate .asd file that you :defsystem-depends-on.
Then, .asd files are read with the current package being asdf-user.
You MAY use any symbols defined in packages cl, asdf and uiop,
and you SHOULD stay in package asdf-user if it suffices;
you SHOULD NOT needlessly create a package for your system definition file
if you're not going to define any such function nor variable nor macro.
But you MUST NOT pollute that package with bindings that could clash with uses by other systems.
It is poor taste to define functions, variables or macros in package asdf-user,
unless strictly necessary or widely useful, and even then with a long name
that distinctively includes the name of your system.
However it is perfectly acceptable to define methods on existing functions,
preferably using the :perform syntax used above, as part of a defsystem form.
If you must define new functions, variables or macros,
you MUST define a new package in which to define them.
That system would typically be named foobar-system
and would export the relevant symbols, as in:
(defpackage :foobar-system
(:use :cl :asdf :uiop)
(:export #:my-foobar-class ...))
(in-package :foobar-system)
That said, if you need more than trivial definitions, and if these definitions ever need to be used by others, it is appropriate to move them to their own system.
Also, it is poor taste to define symbols used in the system itself (e.g. *version*)
since the system then depends on ASDF and cannot be directly built using Bazel,
or some other future build system.
For versions, consider having ASDF extract the version from a file, as in
:version (:read-file-form "variables.lisp" :at (2 2 2).
See the ASDF manual for details, and asdf.asd itself for an example.
The recommended way to conditionally use code is to rely on the CL features;
yet, you SHOULD NOT use read-time conditionals if you can avoid it,
and instead use the ASDF 3 :feature and :if-feature features:
- To conditionally depend on another system or module,
specify as a
:depends-onargument something like(:feature :sbcl (:require :sb-introspect))or(:feature (:or :ccl :sbcl) "bordeaux-threads"). - To conditionally enable a component,
specify a
:if-featureargument to it like(:file "sbcl-support" :if-feature :sbcl)
Note that the feature expressions MUST use keywords, not symbols in the current package
(unless you're pervert enough to make :keyword your current package).
Also note that ASDF at this time wants all the component classes to be defined, so if your component classes are conditionally defined, you may have to resort back to the otherwise deprecated use of read-time conditionals, as in:
(defsystem "foo"
:defsystem-depends-on ((:feature :sbcl "some-sb-magic"))
:components (#+sbcl (:sb-magic-file "some-magic")))
If the condition you want to express doesn't have a corresponding *features* keyword,
you MAY use a (when ... (pushnew :my-keyword *features*)) form to add a feature keyword for it.
However, you SHOULD NOT needlessly push new features;
in particular, the mere fact of having loaded your system does not warrant a new feature.
A feature is only warranted if your system is some deep infrastructure
with mostly compatible rivals that it needs to be distinguished from.
You MUST NOT call asdf:operate or any of its derivatives,
such as asdf:load-system or asdf:test-system from within a perform method.
Instead, you SHOULD declare proper dependencies between actions using methods on
component-depends-on, or more simply using an :in-order-to clause in your defsystem form.
You MUST NOT call asdf:clear-system or in any way interfere with the build
while in a perform method.
You SHOULD NOT define methods on asdf:operate --- most of the time it's totally the wrong thing
because users would not be "operating" on your system, but on their system that depends on it.
Instead you SHOULD define methods on asdf:perform, asdf:component-depends-on, etc.
You MUST NOT define methods on source-file-type.
This bad ASDF 1 interface must die and will be removed in a future version of ASDF.
Instead, override the slot type for your file class and provide a proper :initform, as in:
(defclass cl-source-file.l (cl-source-file) ((type :initform "l")))
You can then provide the name of that class as class name for individual components,
or as :default-component-class to relevant systems or modules.
An operation MUST actually create all the outputs declared by the output-files method.
If some of these outputs are conditional, the output-files method MUST check the condition
and remove uncreated outputs from the list.
If some output is sometimes non meaningful but that is only known after calling the perform method,
it is sometimes appropriate to create an empty file instead of not creating the file.
If you conditionally avoid performing an action altogether using an :around method,
you MUST still call mark-operation-done on the action in the branch of the method
where the action is not performed.
In that case, both the output-files and input-files methods
should probably return nil when that condition is met.
Alternatively, if avoiding some actions is uniform for all
operations on a given component, you MAY want to override or wrap around
the component-if-feature method for that component instead of defining
all those perform, output-files and input-files methods.
Do not side-effect the current *readtable*,
which is never guaranteed to be writable, and
may not be what is current when you need the modification.
If you need modifications to the readtable, use named-readtables,
reader-interception, or some other adequate mechanism.
To bind the readtable, use named-readtables:in-readtable or cl-syntax:in-syntax
in each file that needs a non-standard readtable, and/or use an :around-compile hook
to automatically bind it around every file in a module or system.
See the syntax-control document (in the syntax-control branch if it wasn't merged yet).
At times, you will have to use some ASDF internals to get your software to work. When this happens, follow these rules to minimize the breakage associated to bitrot:
-
You MUST use the
asdf::package prefix rather thanasdf/foo:to name the symbols internal to ASDF. Indeed, from one version of ASDF to the next, which internal package a symbol will reside in may change; for instanceasdf/find-system:primary-system-namerecently becameasdf/system:primary-system-name(it is still present, not exported, in the former system); but it is always present (though not currently exported) in packageasdf. -
When you use internals, you MUST notify the ASDF maintainers, and request that they either export the symbol and support the API, or offer an alternative API more to their liking for the same functionality. Otherwise, they will be justified in modifying internals that no one is supposed to use, and they will blame you when they later change or remove these internals and your software breaks.
Note that the latter point is true of all software, not just ASDF.