I/O adapter framework.

Started building an infrastructure to ingest data into a DBSP circuit
from external data sources and to stream the outputs of the circuit to
external consumers.

The framework defines the APIs to integrate different transport
technologies (files, Kafka streams, database connections, etc.) and data
formats (CSV, bincode, JSON, etc.) into the DBSP input and output
pipelines.  This PR only the input half of the API.

Overview
========

 The data ingestion pipeline consists of two kinds of adapters: *data
 transport* adapters and **data format** adapters.

```text
                                                       ┌──────────────┐
                                                       │  controller  │
                                                       │  ┌────────┐  │
                                                       │  │ catalog│  │
                                                       │  ├────────┤  │
                                                       │  │ config │  │
                                                       │  ├────────┤  │
                         control commands              │  │  stats │  │
                  ┌──┬────────┬─┬──────────────────────┤  └────────┘  │
                  │  │        │ │                      │              │
                  │  │        │ │                      └──────────────┘
                  │  │        │ │
                  │  │        │ │
                  │  │        │ │                        ┌───────────┐
                  ▼  │        │ │                        │           │
                ┌────┴───┐    ▼ │   ┌──────┐        ┌────┴─┐         │
          ─────►│endpoint├──────┼──►│parser├───────►│handle│         ├───►
                └────────┘      │   └──────┘        └────┬─┘         │
                     ▼          │                        │  circuit  │
 transport-     ┌────────┐bytes ▼   ┌──────┐records ┌────┴─┐         │
 specific ─────►│endpoint├─────────►│parser├───────►│handle│         │
 protocol       └────────┘          └──────┘        └────┬─┘         │
                    ▲                  ▲                 │           │
                    │                  │                 └───────────┘
               ┌────┴────┐         ┌───┴────┐
               │  input  │         │ input  │
               │transport│         │ format │
               └─────────┘         └────────┘
```

A data transport implements support for a specific streaming technology like
Kafka. It provides an API to create transport **endpoints**, that connect to
specified data sources, e.g., Kafka topics.  An endpoint reads raw binary
data from the source and provides basic flow control and error reporting
facilities, but is agnostic of the contents or format of the data.

A data format adapter implements support for a data encapsulation format
like CSV, JSON, or bincode.  It provides an API to create **parsers**, which
transform raw binary data into a stream of **records** and push this data to
the DBSP circuit.

The Controller component serves as a centralized control plane that
coordinates the creation, reconfiguration, teardown of the pipeline, and
implements runtime flow control.  It instantiates the pipeline according to
a user-provided configuration (see below) and exposes an API to reconfigure
and monitor the pipeline at runtime.

Adapter API
===========

The transport adapter API consists of two traits:

* `InputTransport` is a factory trait that creates `InputEndpoint`
  instances.
* `InputEndpoint` represents an individual data connection, e.g., a file,
  an S3 bucket or a Kafka topic.

Similarly, the format adapter API consists of:

* `InputFormat` - a factory trait that creates `Parser` instances
* `Parser` - a parser that consumes a raw binary stream and outputs a
  stream of records.

------

This PR introduces a workspace, with the I/O framework implemented as a
separate crate within this workspace.

Author: ryzhyk

.github/workflows/main.yml

@@ -89,19 +89,19 @@ jobs:
         uses: actions-rs/cargo@v1
         with:
           command: test
-          args: --no-run --all-features  --target ${{ matrix.target }}
+          args: --workspace --no-run --all-features  --target ${{ matrix.target }}
 
       - name: Build tests with no features
         uses: actions-rs/cargo@v1
         with:
           command: test
-          args: --no-run --no-default-features  --target ${{ matrix.target }}
+          args: --workspace --no-run --no-default-features  --target ${{ matrix.target }}
 
       - name: Run tests
         uses: actions-rs/cargo@v1
         with:
           command: test
-          args: ${{ env.ALMOST_ALL_FEATURES }} --target ${{ matrix.target }}
+          args: --workspace ${{ env.ALMOST_ALL_FEATURES }} --target ${{ matrix.target }}
 
   #   miri:
   #     name: Miri
@@ -215,11 +215,11 @@ jobs:
           # of test threads.  Crashes happen in random tests, but always at the
           # same location in the sanitizer code:
           # https://github.com/llvm-mirror/compiler-rt/blob/69445f095c22aac2388f939bedebf224a6efcdaf/lib/sanitizer_common/sanitizer_thread_registry.h#L104.
-          # So we run tests with only 1 threads at a time to mitigate this.
+          # So we run tests one at a time to mitigate this.
           THREADS: "${{ matrix.sanitizer == 'leak' && '--test-threads=1' || '' }}"
         with:
           command: test
-          args: ${{ env.ALMOST_ALL_FEATURES }} --target ${{ matrix.target }} -Z build-std -- --skip 'proptest' --skip 'persistent' ${{ env.THREADS }}
+          args: --workspace ${{ env.ALMOST_ALL_FEATURES }} --target ${{ matrix.target }} -Z build-std -- --skip 'proptest' --skip 'persistent' ${{ env.THREADS }}
   clippy:
     name: Clippy
     runs-on: ubuntu-latest
@@ -245,7 +245,7 @@ jobs:
         uses: actions-rs/clippy-check@v1
         with:
           token: ${{ secrets.GITHUB_TOKEN }}
-          args: --all-features --all -- -D warnings
+          args: --all-features --workspace -- -D warnings
 
   fmt:
     name: Rustfmt
@@ -293,9 +293,11 @@ jobs:
 
       - name: Check links
         uses: actions-rs/cargo@v1
+        env:
+          RUSTRDOCFLAGS: "-D warnings --cfg docsrs"
         with:
-          command: rustdoc
-          args: ${{ env.ALMOST_ALL_FEATURES }} -- -D warnings --cfg docsrs
+          command: doc
+          args: --workspace ${{ env.ALMOST_ALL_FEATURES }}
 
   udeps:
     name: Unused Dependencies
@@ -327,4 +329,4 @@ jobs:
         uses: actions-rs/cargo@v1
         with:
           command: udeps
-          args: --all --all-features --all-targets
+          args: --workspace --all-features --all-targets