rewrite README & verify_host docstring

README.md

@@ -3,34 +3,62 @@
 [![Build Status](https://travis-ci.org/JuliaLang/NetworkOptions.jl.svg?branch=master)](https://travis-ci.org/JuliaLang/NetworkOptions.jl)
 [![Codecov](https://codecov.io/gh/JuliaLang/NetworkOptions.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/JuliaLang/NetworkOptions.jl)
 
-The `NetworkOptions` package provides a since utility function `verify_hosts`
-which allows the caller to inderectly make host verification decisions based on
-the values of three environment variables:
+The `NetworkOptions` package acts as a mediator between ways of configuring
+network transport mechanisms (SSL/TLS, SSH, proxies, etc.) and Julia packages
+that provide access to transport mechanisms. This allows the a common interface
+to configuring things like TLS and SSH host verification and proxies via
+environment variables (currently) and other configuration mechanisms (in the
+future), while packages that need to configure these mechanisms can simply
+ask `NetworkOptions` what to do in specific situations without worrying about
+how that configuration is expressed.
+
+## API
+
+### verify_host
+
+```jl
+verify_host(url::AbstractString, transport::AbstractString) -> Bool
+verify_host(url::AbstractString) -> Bool
+```
+The `verify_host` function tells the caller whether the identity of a host
+should be verified when communicating over secure transports like TLS or SSH.
+The `url` argument may be:
+
+1. a proper URL staring with `proto://`
+2. an `ssh`-style bare host name or host name prefixed with `user@`
+3. an `scp`-style host as above, followed by `:` and a path location
+
+In each case the host name part is parsed out and the decision about whether to
+verify or not is made based solely on the host name, not anything else about the
+input URL. In particular, the protocol of the URL does not matter (more below).
+
+The `transport` argument indicates the kind of transport that the query is
+about. The currently known values are `SSL` (alias `TLS`) and `SSH`. If the
+transport is ommitted, the query will return `true` only if the host name should
+not be verified regardless of transport.
+
+The host name is matched against the host patterns in the relavent environment
+variables depending on whether `transport` is supplied and what its value is:
 
 - `JULIA_NO_VERIFY_HOSTS` — hosts that should not be verified for any transport
 - `JULIA_SSL_NO_VERIFY_HOSTS` — hosts that should not be verified for SSL/TLS
 - `JULIA_SSH_NO_VERIFY_HOSTS` — hosts that should not be verified for SSH
 
-Each of these variables can be set to a comma-separated list of host patterns
-which determines a set of host names that should not be verified for various
-secure transport mechanisms. The `JULIA_NO_VERIFY_HOSTS` describes a set of host
-names whose identities should not be verified for any transport mechanism, while
-the `JULIA_SSL_NO_VERIFY_HOSTS` and `JULIA_SSH_NO_VERIFY_HOSTS` variables
-describe sets of host names whose identities should not be verified for the
-SSL/TLS and SSH transport mechanisms, respectively.
-
 The values of each of these variables is a comma-separated list of host name
-patterns with the following syntax: each pattern is split on `.` into parts and
+patterns with the following syntax — each pattern is split on `.` into parts and
 each part must one of:
 
 1. A literal domain name component consisting of one or more ASCII letter,
-   digit,  hyphen or underscore (technically not part of a legal host name, but
+   digit, hyphen or underscore (technically not part of a legal host name, but
    sometimes used). A literal domain name component matches only itself.
 2. A `**`, which matches zero or more domain name components.
 3. A `*`, which match any one domain name component.
 
-To match a pattern list, an entire host name must match one of the patterns.
-For example:
+When matching a host name against a pattern list in one of these variables, the
+host name is split on `.` into components and that sequence of words is matched
+against the pattern: a literal pattern matches exactly one host name component
+with that value; a `*` pattern matches exactly one host name component with any
+value; a `**` pattern matches any number of host name components. For example:
 
 - `**` matches any host name
 - `**.org` matches any host name in the `.org` top-level domain
@@ -40,31 +68,26 @@ For example:
 - `**.example.com` matches any domain under `example.com`, including
   `example.com` itself, `api.example.com` and `v1.api.example.com`
 
-If you want to skip host verification all domains under `safe.example.com` for
-all protocols, skip SSL host verification for `ssl.example.com`, and skip SSH
+#### Example scenario
+
+Suppose you want to not verify any hosts under `safe.example.com` for all
+protocols, skip SSL host verification for just `ssl.example.com`, and skip SSH
 host verification for `ssh.example.com` and its immediate first level
-subdomains, you could set the following environment variable values:
+subdomains. Then you could set the following environment variable values:
 ```sh
 export JULIA_NO_VERIFY_HOSTS="**.safe.example.com"
 export JULIA_SSL_NO_VERIFY_HOSTS="ssl.example.com"
 export JULIA_SSH_NO_VERIFY_HOSTS="ssh.example.com,*.ssh.example.com"
 ```
+With this configuration:
 
-## API
-
-```jl
-verify_host(url::AbstractString, transport::AbstractString) -> Bool
-verify_host(url::AbstractString) -> Bool
-```
-This is a utility function that can be used to check if the identity of a host
-should be verified when communicating over secure transports like HTTPS or SSH.
-The `url` argument may be a bare host name, a host name prefixed with `user@` in
-the style of SSH, or a URL, in which case the host name is parsed out of the
-`url`. The `transport` argument indicates the kind of transport. The currently
-known values are `SSL` (alias `TLS`) and `SSH`. If the transport is ommitted,
-the query will only return `true` for URLs for which the host should not be
-verified regardless of transport. The value of `transport` is case insensitive,
-so `ssh` and `SSH` both indicate a query for the SSH transport protocol.
+- `example.com` would be verified for all protocols
+- `safe.example.com`, `api.safe.example.com`, `v1.api.safe.example.com` and so
+  on would be unverified for all transports
+- `ssl.example.com` would be unverified for SSL/TLS transport
+- `sub.ssl.example.com` would be verified for all transports, including SSL/TLS
+- `ssh.example.com` and `sub.ssh.example.com` would be unverified for SSH only
+- `sub.sub.ssh.example.com` would be verified for all transports
 
 Note that the protocol of `url` need not match the transport mechanism being
 queried: the protocol of the URL is entirely discarded. The reason for this is

src/NetworkOptions.jl

@@ -3,30 +3,45 @@ module NetworkOptions
 export verify_host
 
 """
-    verify_host(url::AbstractString, transport::AbstractString) -> Bool
-    verify_host(url::AbstractString) -> Bool
-
-This is a utility function that can be used to check if the identity of a host
-should be verified when communicating over secure transports like HTTPS or SSH.
-The `url` argument may be a bare host name, a host name prefixed with `user@` in
-the style of SSH, or a URL, in which case the host name is parsed out of the
-`url`. The `transport` argument indicates the kind of transport. The currently
-known values are `SSL` (alias `TLS`) and `SSH`. If the transport is ommitted,
-the query will only return `true` for URLs for which the host should not be
-verified regardless of transport. The host name is matched against the host
-pattern in the relavent environment variables:
+The `verify_host` function tells the caller whether the identity of a host
+should be verified when communicating over secure transports like TLS or SSH.
+The `url` argument may be:
+
+1. a proper URL staring with `proto://`
+2. an `ssh`-style bare host name or host name prefixed with `user@`
+3. an `scp`-style host as above, followed by `:` and a path location
+
+In each case the host name part is parsed out and the decision about whether to
+verify or not is made based solely on the host name, not anything else about the
+input URL. In particular, the protocol of the URL does not matter (more below).
+
+The `transport` argument indicates the kind of transport that the query is
+about. The currently known values are `SSL` (alias `TLS`) and `SSH`. If the
+transport is ommitted, the query will return `true` only if the host name should
+not be verified regardless of transport.
+
+The host name is matched against the host patterns in the relavent environment
+variables depending on whether `transport` is supplied and what its value is:
 
 - `JULIA_NO_VERIFY_HOSTS` — hosts that should not be verified for any transport
 - `JULIA_SSL_NO_VERIFY_HOSTS` — hosts that should not be verified for SSL/TLS
 - `JULIA_SSH_NO_VERIFY_HOSTS` — hosts that should not be verified for SSH
 
-The value of these variables is a comma-separated list of host patterns. Each
-host pattern consists of one or more parts, each of which may be a literal
-domain name part (letters, numbers and dashes), `*` or `**`. A literal domain
-name part matches only itself; a `*` part is a wildcard matching exactly one
-host name part; a `**` part is a wildcard matching zero or more host name parts.
-To match a pattern list, an entire host name must match one of the patterns.
-Some examples:
+The values of each of these variables is a comma-separated list of host name
+patterns with the following syntax — each pattern is split on `.` into parts and
+each part must one of:
+
+1. A literal domain name component consisting of one or more ASCII letter,
+   digit, hyphen or underscore (technically not part of a legal host name, but
+   sometimes used). A literal domain name component matches only itself.
+2. A `**`, which matches zero or more domain name components.
+3. A `*`, which match any one domain name component.
+
+When matching a host name against a pattern list in one of these variables, the
+host name is split on `.` into components and that sequence of words is matched
+against the pattern: a literal pattern matches exactly one host name component
+with that value; a `*` pattern matches exactly one host name component with any
+value; a `**` pattern matches any number of host name components. For example:
 
 - `**` matches any host name
 - `**.org` matches any host name in the `.org` top-level domain
@@ -35,15 +50,6 @@ Some examples:
   `v1.api.example.com`
 - `**.example.com` matches any domain under `example.com`, including
   `example.com` itself, `api.example.com` and `v1.api.example.com`
-
-If you want to skip host verification all domains under `safe.example.com` for
-all protocols, skip SSL host verification for `ssl.example.com`, and skip SSH
-host verification for `ssh.example.com` and its immediate first level
-subdomains, you could set the following environment variable values:
-```sh
-export JULIA_NO_VERIFY_HOSTS="**.safe.example.com"
-export JULIA_SSL_NO_VERIFY_HOSTS="ssl.example.com"
-export JULIA_SSH_NO_VERIFY_HOSTS="ssh.example.com,*.ssh.example.com"
 ```
 """
 function verify_host(