Skip to content

Sources

Jump to bottom Edit New page
Bert Frees edited this page Feb 20, 2018 · 1 revision

Source Code

All source code lives in Github repositories and is made available under the free software licence (LGPL).

Main repositories

A first group of repositories are the principal repositories that make up the DAISY Pipeline 2 software suite.

  • pipeline-assembly: The build script that assembles all components into various packages (for different platforms).

The main component is a Java application. It is heavily modularized. Each of the following repositories produces one or more JAR artefacts (hereafter called "bundles"). Together all these bundles form the Java application.

The main Java component, which provides the web server and the GUI, is complemented by a number of secondary components. They either act as as clients of the web server (the graphical client and the command line client), or work independently (the updater).

A second group of repositories are some auxiliary repositories:

  • pipeline-samples: Sample documents in various formats for demonstrating and testing Pipeline scripts
  • pipeline-build-utils: Various build and test utilities specific to the Pipeline build system

Finally, there are some repositories that have been deprecated:

Dependencies

Next to the main Pipeline repositories there are some noteworthy dependencies (third-party libraries and build tools).

Some libraries were developed as part of DAISY Pipeline but possibly have a larger scope and are therefore treated specially here:

Some libraries are projects developed by DAISY members:

Then there are some third-party projects that we have contributed to (either forked or upstream):

Finally there are some auxiliary projects such as build and test utilities:

Aggregator project

For convenience there is an "aggregator project" (or "super-project") that aggregates all the main repositories and the most important dependencies into a single repository. The various "sub-repositories" are arranged in a tree structure like this:

If you want to build the Pipeline or contribute, it is recommended that you check out this super-repository. You can actually treat it as a single repository, i.e. you can make commits that touch multiple sub-repositories, and you can make pull-requests to daisy/pipeline. See also Contributing below.

Moreover, the super-projects provides a "super-build", meaning that you can build the whole system with one command. See Build System for more info.

Languages

The DAISY Pipeline is written in various programming languages. Languages used in the main code are:

  • Java
  • Scala
  • XProc
  • XSLT
  • Go

Additional languages used in third-party libraries are:

Modules

Pipeline modules are the JAR files (bundles) that contain all the conversion logic and that are "run" by the Pipeline framework. They are the most obvious extension point of the Pipeline.

The core Pipeline modules live in these repositories:

Some examples of third-party modules are:

In general a module is either a "script module" or a "utility module", or it can also be something in between. Script modules contain a Pipeline script, i.e. a top-level converter exposed in the user interface. Utility modules contain common functionality that be used in scripts or other utility modules.

Modules use each other through their APIs and provided (OSGi) services. The API of a module is comprised of its public Java packages and its public resources defined in its "catalog.xml" file. Services are normally implemented in private packages.

The catalog.xml file associates a unique URI to every public resource in the module. Resources can be

  • XSLT files,
  • XProc files (step declarations, libraries, or Pipeline scripts which are regular step declarations annotated with a px:script attribute in the catalog),
  • CSS files,
  • datatype grammar files (annotated with px:data-type in the catalog),
  • or other files.

The public URIs can then be used from other modules to "import" that particular functionality similar to the way Java packages are imported. The import mechanism is of course specific to the language in question: xsl:include or xsl:import for XSLT, p:import for XProc, @import for CSS.

An overview of all modules with documentation of their APIs and provided services can be found here.

Contributing

We accept contributions of any kind to any part of the DAISY Pipeline 2. It can be code changes, tests, documentation, or anything else. Contributions should preferably happen through git/GitHub. The general workflow is the following:

  • Fork the super-repository or one of the individual sub-repositories to your own Github account.
  • Clone the repository and make changes to it.
  • Push the changes back to GitHub and make a pull request.
  • The branch is automatically tested with Travis.
  • We review the PR and possibly request further changes before we merge.

Certain smaller contributions like small changes to documentation can also be made directly within the GitHub web interface.

Contributions should meet a certain standard:

  • Code changes should be accompanied by tests and documentation.
  • Every commit should perform a single well-defined change. Unrelated changes should be in separate commits.
  • Every commit should ideally pass the tests.
  • Commit messages should clearly describe what the commit does and should start with a short subject (separated from the body with an empty line).

In addition, when you make a PR against the super-repository, for technical reasons the branch may not contain any merge commits.

Add a custom footer
Clone this wiki locally