Update activity #12: help defining an activity that executes a program

[ggreg]

Please review. Fix #20.

These changes adds the ability to define an activity that executes an external program and a helper to execute a function with another Python interpreter in a sub-process.

[darkjh]

Should we just forbid the mix of positional arguments and named arguments? What I mean is that if the task has chosen with_named_arguments, then all its arguments should be passed as named arguments.

[ggreg]

I agree that the name is ambiguous. We could only keep the behavior of with_named_arguments but change the name of the function.

[darkjh]

Maybe a workflow test with mixture of different kinds of tasks?

[ggreg]

Yes, I'm improving it by adding some validation based on the signature of the decorated function:

• def cmd(*args, **kwargs) accepts variable arguments and keyword arguments.
• def cmd(*args) does not accept any keyword argument.
• def cmd(**kwargs) only accepts keyword arguments.
• def cmd(a='', *args) accepts the keyword argument (with default value '') a and variable arguments.
• def cmd(a, b) only accepts two positional arguments.
• def cmd(a='', b='blah') only accepts two keyword arguments.

Each kind will have at least a test as an example.

[darkjh]

Ok that will be cool.

[ggreg]

Most tasks have arguments of builtin types (string or int). When the values come from the command line they will be strings. If the task is an actual program, it expects strings. When the task is a program that wraps a callable, the program should know how to convert the command line arguments into the types expected by the callable. Hence we need a way to declare the type of each argument of a task.

[darkjh]

@ggreg would protocols like protobuf or thrift useful here?
The tasks can be declared with a language-independent way.

[ggreg]

I find protocol buffers and thrift too heavy in this context. The way to serialize and define arguments should be flexible and allow such methods. However, I prefer that we start with a simple method such as telling the type in the decorator. The main limitation here would come from nested types for example a list of integers. In this case telling xs=list is not enough.

We could use xs=json.loads to load dynamic types but it does not document the expected type. Should it be a concern? We use Python and can't ensure the type of a function's arguments. So, it's fine. As JSON is still ubiquitous in simpleflow, let parse arguments from the command line as JSON values.

[ggreg]

The Travis build with pypy is failing. Python2.7 build is fine.

I'm figuring out how to fix the pypy build.

@ampelmann @jbbarth please review.

[ampelmann]

LGTM ! 😎

[jbbarth]

So if we have 4 arguments on the command line, we continue and will get obscure errors later, no?

[ggreg]

Good catch! Even with such simple interface I should have use argparse to document how the command behave. Note that it's not excuse for this lack of rigor 😉. At first I only thought of this as a private interface to run functions as programs but it could also be used directly from the command line.

[ggreg]

Parsing the history with the pypy and CPython versions led to different event types and states for child workflows. It comes from the way simple-workflow lookups the name of event types. It iterates of the keys of the EVENTS dict. There is no guaranty in the order of keys in a Python dict, that's why there is a collections.OrderedDict class. See botify-labs/python-simple-workflow#45.

[ggreg]

Green, at last!

[ggreg]

Added a very short document in docs/. Last review before merging.