REP: 111 Title: Multiple Package Manager Support for Rosdep Author: Tully Foote Status: Final Type: Standards Track Content-Type: text/x-rst Created: 28-June-2011 ROS-Version: 1.6 Post-History: 30-Jun-2011
This REP add support to the rosdep YAML format [1] to support multiple package managers on an individual platform. This includes a a new 'source' package manager that replacees the previous mechanisms for performing source-based installs.
Previously, the rosdep format has enabled two modes of installation of system dependencies: the platform's preferred package manager, or a custom bash script. Both of these installation mechanisms were limiting.
Using only a single package manager limits integration options on platforms like OS X, where there are multiple mechanisms available for installing system libraries (e.g. MacPorts, distribute/easy_install). For example, for Python libraries, it can be beneficial to use distribute, which works with the default Python installation on OS X 10.6, instead of using the MacPorts Python libraries, which require additional setup by the user.
Similarly, the bash script specification was brittle and provided limited information to rosdep itself. The scripts were required to be idempotent because rosdep was unable to detect whether or not the dependency was already installed.
The goal of this REP is to make rosdep more robust and flexible, as well as enable better support of platforms other than Ubuntu.
The current implementation of rosdep in ROS Diamondback [2] supports the following structures:
Simple
ROSDEP_NAME: OS_NAME1: PACKAGE_ARGUMENTS_A OS_NAME2: PACKAGE_ARGUMENTS_AVersioned
ROSDEP_NAME: OS_NAME1: OS_VERSION1: PACKAGE_ARGUMENTS_A OS_VERSION2: PACKAGE_ARGUMENTS_A2
The names above resolve as follows:
ROSDEP_NAMEis the name referred to by manifest files. Examples:log4cxx,gtest.OS_NAMEis the name of an OS. Examples:ubuntu,fedora,debian,windows,macports.OS_VERSION(''optional'') is the name of specific versions in the OS. Examples:'10.04',squeeze. If noOS_VERSIONis specified, the rule is assumed to apply to all versions.PACKAGE_ARGUMENThas two interpretations. If it is a single line, it is expected to be a list of packages to pass to the package manager for the OS. If this field is a multi-line script, it is an idempotent bash script that is executed to install the dependency.
Example:
python:
ubuntu: python-dev
fedora: python-devel
macports: py26-scipy
boost:
ubuntu:
'11.04': libboost.42-all-dev
'10.10': libboost.42-all-dev
For many Linux-based platforms, there are dominant package managers that are the main mechanism for deploying software, such as deb/apt for Debian-based systems and rpm/yum for RedHat-based systems. However, for non-Linux platforms, as well as for cross-platform language environments (e.g. Python, Java, Ruby), there exist other software deployment mechanisms that are commonly used. There are also platforms like OS X and Windows, where no package manager has a dominant role.
On OS X, rosdep currently uses MacPorts as the default package manager. While MacPorts does provide most of the libraries necessary for integrating ROS on an OS X system, the current implementation limits the flexibility of integration options. One area of integration we wish to improve is Python. The MacPorts Python integration requires a separate Python tree from the default OS X Python interpreter. This has proven to be very brittle for users and a common are of misconfiguration. Now that OS X now deploys with a Python version that is compatible with ROS, we wish to explore using distribute/easy_install for Python libraries instead.
The current support for bash scripts requires these scripts to be idempotent: they must detect the presence of the required dependency and install it if it is not present. There is no mechanism for rosdep to determine for itself whether the required dependency is installed, nor is there any mechanism for removing dependencies installed via this mechanism. This creates confusion on the part of the user as there is no mechanism for analyzing whether or not these dependencies are installed correctly. Creating more structure for source-based installations will enable simpler installation scripts, better error reporting, and more advanced features (e.g. uninstall) in the future.
OS X dependencies are specified generally as macports and are not
versioned to a particular OS X release. OS X releases frequently have
significant differences as Apple does not maintain much backwards
compatibility.
The current ROS OS X integration suffers from 32-bit/64-bit compatibility problems due to the OS X 10.6 release. The current rosdep specification also does not allow integrators to take advantage of the builtin Python 2.5+ interpeter that was introduced in the OS X 10.5 release.
Another limitation of the OS X integration is that the rosdep syntax did not allow for specification of MacPorts "variants" for individual packages. The rosdep syntax interpreted the package argument value as a list of packages, which was not flexible enough for specifying variants on specific packages.
The current OS version identifiers use the release number (e.g. 11.04)
instead of the release codename (e.g. lucid). Using the release
number has been a frequent source of bugs as YAML interprets 11.04
as a floating point number instead of a string. This requires
explicit string escaping when specifying rules (e.g. '11.04').
The new specification for rosdep introduces a PACKAGE_MANAGER
key in the YAML specification, which is a child key of the OS_NAME.
rosdep no longer interprets the PACKAGE_MANAGER_ARGUMENTS and instead
passes them to the specified PACKAGE_MANAGER handler.
- Simple
ROSDEP_NAME: OS_NAME1: PACKAGE_MANAGER1: PACKAGE_ARGUMENTS_A OS_NAME2: PACKAGE_MANAGER2: PACKAGE_ARGUMENTS_A
- Versioned
ROSDEP_NAME: OS_NAME1: OS_VERSION1: PACKAGE_MANAGER1: PACKAGE_ARGUMENTS_A OS_VERSION2: PACKAGE_MANAGER2: PACKAGE_ARGUMENTS_A2
The names above resolve as follows:
ROSDEP_NAMEis the name referred to by manifest files. Examples:log4cxxorgtest.OS_NAMEis the name of an OS. Examples:ubuntu,osx,fedora,debian, orwindows.OS_VERSION(optional) is the name of specific versions in the OS. Examples:lucidorsqueeze. If noOS_VERSIONis specified, the rule is assumed to apply to all versions.PACKAGE_MANAGER(optional in ROS Electric, required in ROS Fuerte) is a key to select which package manager to use for this rosdep. Examples:apt,easy_install,macports.PACKAGE_ARGUMENTis free-form YAML that is be passed to the handler for the specifiedPACKAGE_MANAGER.
For Ubuntu the default package manager is apt. The new syntax is:
rosdep_name:
ubuntu:
apt:
packages: [ debian-package-name, other-debian-package-name]
or versioned as follows:
rosdep_name:
ubuntu:
lucid:
apt:
packages: [debian-package-name, other-debian-package-name]
OS version identifiers will be standardized onto one-word codenames for each distro.
For example:
- debian: squeeze
- ubuntu: lucid, maverick, natty, oneiric
- osx: snow, lion
The OS_NAME key for OS X will be changed to osx and will start
including OS_VERSION keys that represent the version number of the
OS X releases (e.g. lion). The default package is macports,
which can be used as a PACKAGE_MANAGER field.
osx:
lion:
macports: bzip2
For backwards compatibility, the PACKAGE_MANAGER is allowed to be
optional in the ROS Electric case. As both PACKAGE_MANAGER and
OS_VERSION are optional, this creates an ambiguious case where
either OS_VERSION or PACKAGE_MANAGER is specified, but not
both.
In this ambiguous case, rosdep first interprets the key as a
PACKAGE_MANAGER. If this test fails, it will be interpreted as an
OS_VERSION. Developers should exercise caution in keeping
OS_VERSION and PACKAGE_MANAGER keys globally distinct.
Although not specified in this REP, developers of new package managers
for rosdep are encouraged to include a dependency specification in
their PACKAGE_ARGUMENTS values. This dependency specification
should enable rosdep rules to specify dependencies on other rosdep
rules. This, for example, would enable an easy_install package to
also depend on a apt-based install. One use case of this is
rosinstall, which can be installed via easy_install, but also
implicitly requires non-Python libraries for Git, Subversion,
Mercurial, and other non-easy_install packages.
This advisory does not apply to package managers that are builtin to a particular platform, such as apt and yum.
Internally, the rosdep library will provide APIs for developers to trigger installation of these dependencies.
A new source-based package manager is discussed in the motivation for this REP. This specification of this new source-based package manager will be described in a separate forthcoming REP.
rosdep rules are only allowed to specify a single package manager to fulfill them.
All current rosdep.yaml files are compatible with the new proposed syntax, though some features will be deprecated and the files should be upgraded during the ROS Electric cycle to maintain compatibility.
If no package manager is specified, the previous default package manager will be used.
Defaults:
- ubuntu: apt
- debian: apt
- mint: apt
- osx: macports
- freebsd: port
Support for the default package manger will be maintained upto and including ROS Fuerte.
Multi-line values will still be treated as bash scripts in ROS Electric. Support for this will be removed in ROS Fuerte.
The macports rules will be supported during ROS Electric.
For the following codenames backwards compatabilty will be built into the tool to automatically translate rules:
- '10.04' -> lucid
- '10.10' -> maverick
- '11.04' -> natty
It is expected that all future releases will refer to the correct codename, e.g. 'oneiric'.
This will be supported through ROS Fuerte.
| [1] | rosdep documentation (http://www.ros.org/wiki/rosdep) |
| [2] | rosdep.yaml format in ROS Diamondback (http://www.ros.org/wiki/rosdep/rosdep.yaml/diamondback) |
This document has been placed in the public domain.