more docs

site/content/docs/guide/plugin/for-developers.md

@@ -0,0 +1,230 @@
+## Creating a New Plugin
+
+A Hipcheck plugin is a separate executable artifact that Hipcheck downloads,
+starts, and communicates with over a gRPC protocol to request data. A plugin's
+executable artifact is the binary, set of executable program files, Docker
+container, or other artifact which can be run as a command line interface
+program through a singular "start command" defined in the plugin's [manifest
+file](@Todo).
+
+The benefit of the executable-and-gRPC plugin design is that plugins can be
+written in any of the many languages that have a gRPC library. One drawback is
+that plugin authors have to at least be aware of the target platform(s) they
+compile their plugin for, and more likely will need to support a handful of
+target platforms. This can be simplified through the optional use of container
+files as the plugin executable artifact.
+
+Once a plugin author writes their plugin, compiles, packages, and distribute it (@Todo - add link to
+distributing section), Hipcheck users can specify the plugin in their policy
+file for Hipcheck to fetch and use in analysis.
+
+#### Plugin CLI
+
+Hipcheck requires that plugins provide a CLI which accepts a `--port <PORT>`
+argument, enabling Hipcheck to centrally manage the ports plugins are listening
+on. The port provided via this CLI argument must be the port the running plugin
+process listens on for gRPC requests, and on which it returns responses.
+
+Once started, the plugin should continue running, listening for gRPC requests
+from Hipcheck, until shut down by the Hipcheck process. For information on the
+Hipcheck gRPC protocol, see [here](@Todo).
+
+### The Rust SDK
+
+The Hipcheck team maintains a library crate `hipcheck-sdk` which provides
+developers with tools for greatly simplifying plugin development in Rust. This
+section will describe at a high level how a plugin author can use the SDK, but
+for more detailed information please see the [API docs](@Todo).
+
+The first step is to add `hipcheck-sdk` as a dependency to your Rust project.
+Next, the SDK provides `prelude` module which authors can import to get access
+to all the essential types it exposes. If you want to manage your imports to
+avoid potential type name collisions you may do so, otherwise simply write `use
+hipcheck_sdk::prelude::*`.
+
+#### Defining Queries
+
+The Hipcheck plugin communication protocol allows a plugin to expose multiple
+named query endpoints that can be called by Hipcheck core or other plugins. For
+each query endpoint you want to define, you must create a struct that implements
+the `Query` trait from the `prelude`. `Query` is declared as such:
+
+```rust
+#[tonic::async_trait]
+trait Query: Send {
+	fn input_schema(&self) -> JsonSchema;
+
+	fn output_schema(&self) -> JsonSchema;
+
+	async fn run(&self, engine: &mut PluginEngine, input: JsonValue) -> Result<JsonValue>;
+}
+```
+
+The `input_schema()` and `output_schema()` function calls allow you to declare
+the signature of the query (what type of JSON value it takes and returns,
+respectively) as a `schemars::schema::Schema` object. Since schemas are
+themselves JSON objects, we recommend you store these as separate `.json`
+files that you reference in  `include_str!()` macro calls to copy the contents
+into your binary at compile time as a `&'static str`. For example:
+
+```rust
+static MY_QUERY_KEY_SCHEMA: &str = include_str!("../schema/my_query_key_schema.json");
+static MY_QUERY_OUTPUT_SCHEMA: &str = include_str!("../schema/my_query_output_schema.json");
+```
+
+##### The `Query::run()` Function
+
+The `run()` function is the place where your actual query logic wil go. Let's
+look at it in more detail. It's an `async` function since the underlying SDK
+may execute the `run()` functions of different `impl Query` structs in parallel
+as queries from Hipcheck come in, and `async` allows for simple and efficient
+concurrency. The function takes a (mutable) reference to a `PluginEngine`
+struct. We will discuss `PluginEngine` below, but for now just know that
+this struct exposes an `async query()` function that allows your
+query endpoint to in turn request information from other plugins. With that complexity
+out of the way, all that's left is a simple function that takes a JSON object as
+input and returns a JSON object of its own, wrapped in a `Result` to allow for failure.
+
+The first step of your `run()` function implementation will likely be to parse the JSON
+value in to primitive typed data that you can manipulate. This could involve
+deserializing to a struct or `match`ing on the `JsonValue` variants manually.
+If the value of `input` does not match what your query endpoint expects in its
+input schema, you can return an `Err(Error::UnexpectedPluginQueryInputFormat)`,
+where `Error` is the `enum` type from the SDK `prelude`.  For more information on the
+different error variants, see [here](@Todo).
+
+If your query endpoint can complete with just the input data, then you can
+simply perform the calculations, serialize the output type to a JSON value, and
+return it wrapped in `Ok`. However, many plugins will rely on additional data from other
+plugins. In the next subsection we will describe how to do that in more detail.
+
+##### Querying Other Plugins
+
+As mentioned above, the `run()` function receives a handle to a `PluginEngine` instance which exposes
+the following generic function:
+
+```rust
+async fn query<T, V>(&mut self, target: T, input: V) -> Result<JsonValue>
+where
+	T: TryInto<QueryTarget, Error: Into<Error>>,
+	V: Into<JsonValue>;
+
+struct QueryTarget {
+	publisher: String,
+	plugin: String,
+	query: Option<String>,
+}
+```
+
+At a high-level, this function simply takes a value that identifies the target
+plugin and query endpoint, and passes the `input` value to give to that query
+endpoint's `run()` function, then returns the forwarded result of that
+operation.
+
+The "target query endpoint" identifier is anything that implements
+`TryInto<QueryTarget>`. The SDK implements this trait for `String`, so you can
+pass a string of the format `publisher/plugin[/query]` where the bracketed
+substring is optional. Each plugin is allowed to declare an unnamed "default"
+query; by omitting the `/query` from your target string, you are targetting the
+default query endpoint for the plugin. If you don't want to pass a `String` to
+`target`, you can always instantiate a `QueryTarget` yourself and pass that.
+
+#### The `Plugin` Trait
+
+At this point, you should have one struct that implements `Query` for each
+query endpoint you want your plugin to expose. Now, you need to implement the
+`Plugin` trait which will tie everything together and expose some additional
+information about your plugin to Hipcheck. The `Plugin` trait is as follows:
+
+```
+trait Plugin: Send + Sync + 'static {
+
+	const PUBLISHER: &'static str;
+
+	const NAME: &'static str;
+
+	fn set_config(&self, config: JsonValue) -> StdResult<(), ConfigError>;
+
+	fn queries(&self) -> impl Iterator<Item = NamedQuery>;
+
+	fn explain_default_query(&self) -> Result<Option<String>>;
+
+	fn default_policy_expr(&self) -> Result<String>;
+}
+
+pub struct NamedQuery {
+	name: &'static str,
+	inner: DynQuery,
+}
+
+type DynQuery = Box<dyn Query>;
+```
+
+The `PUBLISHER` and `NAME` associated `str`s allow you to declare the publisher
+and name of the plugin, respectively.
+
+The `set_config()` function allows Hipcheck users to pass a set of `String`
+key-value pairs to your plugin as a configuration step before any endpoints are
+queried. On success, simply return `Ok(())`. If the contents of the `config`
+JSON value do not match what you expect, return a `ConfigError` enum variant to
+describe why.
+
+Your implementation of `queries()` is what actually binds each of your `impl
+Query` structs to the plugin. As briefly mentioned above, query endpoints have
+names, with up to one query allowed be unnamed (`name` is an empty string) and
+thus designated as the "default" query for the plugin. Due to limitations of
+Rust, the SDK must introduce a `NamedQuery` struct to bind a name to the query
+structs. Your implementation of `queries()` will, for each `impl Query` struct,
+instantiate that struct, then use that to create a `NamedQuery` instance with
+the appropriate `name` field. Finally, return an iterator of all the
+`NamedQuery` instances.
+
+Plugins are not required to declare a default query endpoint, but plugins
+designed for "top-level" analysis (namely those that are not explicitly
+designed to provide data to other plugins) are highly encouraged to do so.
+Furthermore, it is highly suggested that the default query endpoint is designed
+to take the `Target` schema (@Todo - link to it), as this is the object type
+passed to the designated query endpoints of all "top-level" plugins declared in
+the Hipcheck policy file.
+
+If you do define a default query endpoint, `Plugin::explain_default_query()` should return a `Ok(Some(_))` containing a string that explains the default query.
+
+Lastly, if yours is an analysis plugin, users will need to write [policy
+expressions](@Todo - link) to interpret your plugin's output. In many cases, it
+may be appropriate to define a default policy expression associated with your
+default query endpoint so that users do not have to write one themselves. This
+is the purpose of `default_policy_expr()`. This function will only ever be
+called by the SDK after `set_config()` has completed, so you can also take
+configuration parameters to influence the value returned by
+`default_policy_expr().` For example, if the output of your plugin will
+generally will be compared against an integer/float threshold, you can return a
+`(lte $ <THRESHOLD>)` where `<THRESHOLD>` may be a value received from
+`set_config()`.
+
+#### Running Your Plugin
+
+At this point you now have a struct that implements `Plugin`. The last thing to do is write some boilerplate code for starting the plugin server. The Rust SDK exposes a `PluginServer` type as follows:
+
+```rust
+pub struct PluginServer<P> {
+	plugin: Arc<P>,
+}
+
+impl<P: Plugin> PluginServer<P> {
+	pub fn register(plugin: P) -> PluginServer<P> {
+		...
+	}
+
+	pub async fn listen(self, port: u16) -> Result<()> {
+		...
+	}
+}
+```
+
+So, once you have parsed the port from the CLI `--port <PORT>` flag that
+Hipcheck passes to your plugin, you simply pass an instance of your `impl
+Plugin` struct to `PluginServer::register()`, then call `listen(<PORT>)` on the
+returned `PluginServer` instance. This function will not return until the gRPC
+channel with Hipcheck core is closed.
+
+And that's all there is to it! Happy plugin development!

site/content/docs/guide/plugin/index.md

@@ -150,8 +150,8 @@ other plugins could be written to consume and process their output. However,
 the scoring system of Hipcheck relies on turning the output of each top-level
 plugin into a pass/fail evalution. In order to facilitate transforming plugin
 data into a boolean value, Hipcheck provides "policy expressions", which are a
-small expression language. See [here](#policy-expression-reference) for a
-reference on the policy expression language.
+small expression language. See [here](policy-expr) for a reference on the policy
+expression language.
 
 Users can define the pass/fail policy for an analysis node in the score tree
 with a `policy` key. As described in more detail in the policy expression

site/content/docs/guide/plugin/policy-expr.md

@@ -61,10 +61,11 @@ weeks.
 
 #### Arrays
 
-Arrays are vectors of homogeneously-type primitives. For example, an array of
-integers cannot contain any other type. Similarly, arrays cannot contain other
- expression types, which includes other arrays. Square brackets
- represent the array boundaries and elements are separated by whitespace. Examples:
+Arrays are vectors of homogeneously-type primitives. This means that all
+elements of an array must be the same type, and that type must be a primitive
+(integer, float, boolean, datetime, span). Arrays cannot contain expression
+types like other arrays, functions, or lambdas. Square brackets represent the
+array boundaries and elements are separated by whitespace. Examples:
 
  ```
  [1 1 2 3 5 8]
@@ -74,7 +75,7 @@ integers cannot contain any other type. Similarly, arrays cannot contain other
 
 #### Function
 
-Functions are lisp-like expressions, meaning that they are bounded by
+Functions are Lisp-like expressions, meaning that they are bounded by
 parentheses, and the function name comes first followed by whitespace-delimited
 operands. Examples:
 
@@ -110,10 +111,11 @@ following functions:
 
 #### Lambdas
 
-A lambda is an incomplete function that is missing an operand. Lambdas are
-powerful as some functions take a lambda and an array as operands, and then
-evaluate the lambda for each element of the array. For example, `(lte 8.0)`
-is an incomplete `lte` function. When we do the following:
+A lambda is an incomplete function invocation that is missing an operand. In the
+standard policy expression environment, there are multiple functions that take
+as operands a lambda and an array, and then evaluate the lambda
+for each element of the array. For example, `(lte 8.0)` is an incomplete `lte`
+function call. When we do the following:
 
 ```
 (foreach (lte 8.0) [0.3, 9.4, 5.1])
@@ -148,4 +150,44 @@ Some examples:
 
 #### JSON Pointers
 
+As a reminder, the purpose of the policy expression language is to allow us to
+manipulate data from plugins and produce a boolean pass/fail determination. Each
+policy expression in a Hipcheck policy file needs to contain one or more
+locations at which to "receive" part or all of the JSON data from a plugin
+(otherwise the policy would be independent of the data and could be evaluated
+immediately). This is where JSON pointers come in.
 
+A JSON pointer is a replacement for an expression or function operand in a
+policy expression. They are prefixed with a `$`. If the JSON value is an object,
+fields can be recursively accessed by appending `/<FIELD_NAME>`. For example,
+to extract the float at field "baz" below, we would use `$/bar/baz`:
+
+```
+{
+	"foo": [1, 2, 3, 4],
+	"bar": {
+		"bee": false,
+		"baz": 0.01
+	}
+}
+```
+
+Examples:
+
+|Plugin Output | Goal | Policy Expression
+|----|----|----|
+| A boolean value | Forward the value as the pass/fail determination | `$` |
+| A JSON array | Pass if all elements less than 10 | `(all (gt 10) $)` |
+| An object containing a boolean field "fail" | Invert the field | `(not $/fail)` |
+
+As mentioned above, a policy expression can contain multiple JSON
+pointers. As an example, this can be useful if you want to calculate the
+percentage of elements of an array that pass a filter:
+
+```
+(lt (divz (count (filter (gt 10) $) (count $))) 0.5)
+```
+
+This policy expression will check that less than half of the elements in `$` are
+less than 10.  It uses JSON pointers twice, once to get the total element count,
+again to count the number of elements filtered by the lambda.