New Mapper Test Driver Architecture Draft

Why a new architecture?

The current test driver/mapper architecture is inflexible in that it combines the aspect of having symbols that are themselves executable, and the principle of a mapper which translates from concrete to abstract and from abstract to concrete. The fact that concrete symbols must be executable ignores the fact that "abstract" and "concrete" are no absolute concepts, but instead very much depend on the perspective. This requirement of "absolutely concrete" symbols also prevents mappers from being composable.

Proposed new architecture

The core of the proposed new architecture is to separate the above two aspects. Therefore, we have an ExecutableInputSUL, which takes care of executing symbols that are implementations of ExecutableInput -- and nothing more. On the other hand, we have a Mapper interface, which takes care of mapping input symbols from (some form of) abstract level to the (resp. some form of) concrete level, and vice versa for output symbols.

We therefore introduce the following new important classes/interfaces:

• ExecutableInputSUL/ContextExecutableInputSUL: used to execute ExecutableInput symbols, with either a nullary execute() method, or a context-specific execute(C context) method
• Mapper: this is the (more general) replacement for DataMapper; explanation below

The Mapper interface

Mapper is the replacement for the DataMapper interface. It also has four type parameters: AI (abstract input), AO (abstract output), CI (concrete input), CO (concrete output). Note that CI now no longer has a type bound.

A mapper implementation is responsible for lowering/lifting input/output symbols to/from the abstract/concrete level. To sum it up in one sentence: The purpose of a Mapper<AI,AO,CI,CO> is to make a concrete SUL<CI,CO> appear as an abstract SUL<AI,AO>.

The lowering/lifting is done by the mapInput and mapOutput methods, respectively, that are direct equivalents of the old DataMapper.input and DataMapper.output methods.

Mapping Exceptions

Furthermore, a mapper is responsible for handling SULExceptions that occur during invocations of SUL.step(). The API for this has been changed to a more convenient form, which is best explained along an example:

Suppose we have a mapper that uses Strings as abstract outputs. We treat SULExceptions (or rather their causes) according to the following scheme:

• IllegalArgumentExceptions usually result from locally erroneous parameters of a single invocation, but do not affect subsequent invocations. We therefore want to output "error", but otherwise continue the regular execution.
• NullPointerExceptions play a special role. We therefore want to produce a distinct output "npe". Furthermore, we do not continue execution, as it might result from the internal state of our target object being corrupted.
• Exceptions signal (regular) error conditions. We also do not want to continue execution, because we do not now what's going wrong. To signal this abnormal termination, the output "fatal" should be used.
• everything else might be anything, from a user-defined subclass of Throwable to an OutOfMemoryError. It usually is not a good idea to handle these, we therefore just pass them on to the next higher level.

In case of NullPointerExceptions and IllegalArgumentExceptions, we abort the execution on the SUL even though there are still inputs to process. To make clear that the outputs do not correspond to any actual execution, we output "unobserved" for each input that was processed after a NullPointerException or Exception was thrown.

The corresponding exception mapper is pretty concise:

@Override
public MappedException<String> mapException(SULException ex) {
  Throwable cause = ex.getCause();
  if(cause instanceof IllegalArgumentException) {
    return MappedException.ignoreAndContinue("error");
  }
  if(cause instanceof NullPointerException) {
    return MappedException.repeatOutput("npe", "unobserved");
  }
  if(cause instanceof Exception) {
    return MappedException.repeatOutput("fatal", "unobserved");
  }
  // else
  return MappedException.pass(cause);
}

Basic mapper operations

The Mappers class provides some utility operations for Mappers.

• Mappers.apply takes a Mapper<AI,AO,CI,CO> and a SUL<CI,CO> and returns a SUL<AI,AO> (as we remember, this is the prime purpose of a mapper)
• Mappers.compose returns a mapper that is the composition of two specified mappers. In simplified notation, for obtaining a Mapper<AI,AO,CI,CO>, the provided mappers must be of types Mapper<AI,AO,CAI,ACO> for the "outer" and Mapper<CAI,ACO,CI,CO> for the "inner" mapper. (CAI/ACO stands for "concrete/abstract input" and "abstract/concrete output", respectively)

Global API Changes

The SULException is now part of the API package. This exception is used exclusively to wrap other exceptions that occur during an execution of a SUL.step() implementation. This exception is unchecked, so existing code depending on the SUL interface should not break.

Furthermore, I propose moving the ExecutableInput and ContextExecutableInput as well as the respective SUL implementations to the learnlib-core module.

Changes for users of the existing driver API

Hopefully, not much will change. Existing implementations of DataMapper should be easily able to migrate their code to the new Mapper interface. The only changes are of method names to more descriptive names, and the exception handling facility, which is more convenient now.

Instantiations of TestDrivers of form new TestDriver<>(myMapper) will most likely simply have to be replaced by Mappers.apply(myMapper, new ExecutableInputSUL<>()).

The adaption to the new mapper architecture for the reflection use case can be seen here. The usage only differs marginally.