Update Documentation for Suite Directories

docs/advanced.rst

@@ -16,8 +16,8 @@ In addition to host config files, you can provide mode config files that
 you can apply to any test when you run it. They have the same format as
 the host configs, but multiple can be provided per test.
 
-Unlike host configs, mode configs apply _last_ in the process, overriding
-values set be host configs and the test itself.
+Unlike host configs, mode configs apply *last* in the process, overriding
+values set by host configs and the test itself.
 
 For example, the following mode file could be used to set a particular
 set of slurm vars:
@@ -38,7 +38,7 @@ Test Series
 A test series is a well defined group of tests that are designated to be
 run together. A series is created automatically whenever you use ``pav run``,
 but you can also explicitly define series using a
-``configs/series/<series_name>.yaml`` file. This allows you defined
+``configs/series/<series_name>.yaml`` file. This allows you to define
 relationships and dependencies between the tests, among other things.
 
 See :ref:`tutorials.series`.

docs/basics.rst

@@ -77,8 +77,7 @@ Pavilion doesn't come with any tests itself (though it does come with
 example tests in ``examples/``); it's just a system for
 running them on HPC clusters. Each test needs a configuration script,
 and most will need some source files. Both of these will live in one of
-your :ref:`config.config_dirs` under the ``tests/`` and ``test_src/``
-sub-directories.
+your :ref:`config.config_dirs` under the ``suites/`` sub-directory.
 
 Test configs tell pavilion what environment it needs to build and run
 your test, the commands to run it, how to schedule it on a cluster, and
@@ -144,6 +143,11 @@ current host, which is provided via a built-in
 providing your own sys\_var plugin, typically to use more colloquial names for
 systems.
 
+.. note::
+    It is also possible to specify host configs on a per-suite basis using a slightly
+    different format. For more information on suite-specific host configs, see the
+    section on :ref:`tests.suites`.
+
 Running tests
 ~~~~~~~~~~~~~
 
@@ -165,7 +169,7 @@ suite). Did you forget what you named them? That's ok! Just ask Pavilion.
     1 tests started as test series s33.
 
 If you want to run every test in the suite, you can just give the suite
-name. You can also run whatever combinations of tests you want. You also
+name. You can also run whatever combinations of tests you want. You can also
 list tests in a file and have Pavilion read that.
 
 .. code:: bash
@@ -183,7 +187,7 @@ Test Status
 ^^^^^^^^^^^
 
 If you want to know what's going on with your tests, just use the
-``pav  status`` command.
+``pav status`` command.
 
 .. code:: bash
 
@@ -201,7 +205,7 @@ Test Series and ID's
 ~~~~~~~~~~~~~~~~~~~~
 
 From the above, you may have noticed that each test gets a series id
-like ``s24`` and a test id like ``41``. You can use these id's to
+like ``s24`` and a test id like ``41``. You can use these ID's to
 reference tests or suites of tests to get their status, results, and
 logs through the pavilion interface. The ID's are unique for a given
 Pavilion :ref:`config.working_dir` but they will
@@ -214,6 +218,9 @@ Pavilion builds a mapping of result keys and values for every test that
 runs. You can view the results of any tests using the ``pav results``
 command.
 
+runs. You can view the results of any tests using the ``pav results``
+command.
+
 .. code:: bash
 
     $ pav results
@@ -239,7 +246,7 @@ Every test has a results object that contains a variety of useful,
 automatically populated keys. Additional keys can be defined through
 :ref:`result parsing and result evaluations <results.basics>`.
 
-Results are saved alongside each test, as well being written to a
+Results are saved alongside each test, as well as being written to a
 central result log that is suitable for import into Splunk or other
 tools.
 

docs/config.rst

@@ -21,10 +21,10 @@ Finding the Pavilion.yaml file
 Pavilion looks for its main ``pavilion.yaml`` config in the following hierarchy
 and uses the first one it finds.
 
--  The user's `~/.pavilion` directory.
+-  The user's ``~/.pavilion`` directory.
 -  The directory given via the ``PAV_CONFIG_DIR`` environment variable.
 
-The pavilion.yaml file can configure additional locations to look for test,
+The ``pavilion.yaml`` file can configure additional locations to look for test,
 mode, and host configs, as well as plugins using the ``config_dirs`` option.
 The ``~/.pavilion`` directory is only searched for ``pavilion.yaml`` by
 default, but searches for other configs there can be turned on in ``pavilion

docs/install.rst

@@ -76,7 +76,7 @@ We recommend the following directory structure for Pavilion.
 The ``<root>/pavilion/config`` directory will contain all of your site
 specific tests, plugins, and test source files. If you organize your tests in
 separate repositories, you may check them out here, and simply link the
-appropriate files into ``config/tests`` and ``config/test_src``.
+appropriate files into ``config/suites``.
 Alternatively, you can make the entire config directory, source tarballs and
 all, its own repository.
 

docs/test_run_lifecycle.rst

@@ -17,8 +17,8 @@ You'll often run a test using a test command like this:
 
   $ pav run mytests.last_login
 
-In this case, we have a ``mytests.yaml`` test suite file in our tests
-directory, that looks like this:
+In this case, we have a ``mytests.yaml`` test suite file in our ``suites``
+directory that looks like this:
 
 .. code-block:: yaml
 
@@ -37,7 +37,7 @@ directory, that looks like this:
       # Five hours in seconds
       time_limit: '{{ 60^2 * hours }}'
 
-    # Always skip gary.
+    # Always skip jerry.
     not_if:
         '{{user}}': 'jerry'
 
@@ -109,7 +109,7 @@ Finding Suite, Host, and Mode Configs
 
 Pavilion can be configured to look in multiple places for test configs
 (see :ref:`config.config_dirs`), and uses the first matching file found. In
-case, we need a suite config named ``tests/mytests.yaml`` and a host config
+our case, we need a suite config named ``suites/mytests.yaml`` and a host config
 named ``hosts/tester.yaml``. Pavilion will load the yaml from each of these
 files, and use it to construct our raw test config.
 
@@ -239,7 +239,7 @@ permutations, and generates test run objects and directories.
 1. The available variable values are collected for each of the four variable
    types and put in a single *variable manager* for each test
    (:ref:`tests.variables`).
-2. These, along with the ``permute_on`` value for a test, is used to compute
+2. These, along with the ``permute_on`` value for a test, are used to compute
    a unique collection of variable values for each
    :ref:`Test Permutation<tests.permutations>`. Each of these will result
    in a separate *Test Run*.
@@ -301,7 +301,7 @@ Deferred Variable Errors
 ########################
 
 You may also see errors involving :ref:`tests.variables.deferred`. Some
-sections, like 'build' and scheduler configuration sections, don't allow them.
+sections, like ``build`` and ``scheduler`` configuration sections, don't allow them.
 
 .. code-block:: yaml
 
@@ -329,8 +329,8 @@ that object and run the test.
    :alt: Creating a Test Run
 
 1. The *Test Run* object is created from the config, which immediately grabs
-   the next available test_id number. The test run directory is then created
-   in a directory named for that number under `<working_dir>/test_runs/`.
+   the next available test ID number. The test run directory is then created
+   in a directory named for that number under ``<working_dir>/test_runs/``.
 2. Everything needed to create the test run object is saved to the test's run
    directory, including the config, test variables, and any other attributes
    of the test.
@@ -365,7 +365,7 @@ The following steps will be taken:
 
 1) A ``kickoff`` script will be generated by the scheduler for each test run.
 
-   1) The kickoff script will run `pav _run <test_run_id>` and set up
+   1) The kickoff script will run ``pav _run <test_run_id>`` and set up
       the basic Pavilion environment.
    #) The extension of the kickoff script is scheduler dependent.
 2) The scheduler plugin will run the kickoff script such that
@@ -398,9 +398,9 @@ Generating the Run Script
 Run scripts are generated mostly identically to build scripts, and consist
 of the same basic components.
 
-1) Manage modules as described in the 'run.modules' options.
-2) Manipulate environment variables as set in the 'run.env' options.
-3) Run all the commands in `run.cmds`.
+1) Manage modules as described in the ``run.modules`` options.
+2) Manipulate environment variables as set in the ``run.env`` options.
+3) Run all the commands in ``run.cmds``.
 
 For example:
 
@@ -441,7 +441,7 @@ At this point, Pavilion simply runs the test's ``run.sh`` script. Like with
 building, Pavilion will only timeout a test if it doesn't produce output at
 least once every ``run.timeout`` seconds.
 
-The return value from the ``run.sh`` script is saved in the 'return_value'
+The return value from the ``run.sh`` script is saved in the ``return_value``
 result, and may be used later when we gather the test results.
 
 If this step seems overly simple, it is! Most of the work running Pavilion
@@ -455,6 +455,6 @@ This is described fully in :ref:`results.basics`.
 Mark the Test as Complete
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Finally, the test is marked as complete by saving a 'RUN_COMPLETE' file in the
+Finally, the test is marked as complete by saving a ``RUN_COMPLETE`` file in the
 test's run directory. Pavilion uses this to quickly determine which tests
-might still be running.
+might still be running.

docs/tests/build.rst

@@ -72,9 +72,13 @@ This build option tells Pavilion when to download test source.
 Finding Source Files
 ~~~~~~~~~~~~~~~~~~~~
 
-The primary way to specify source in Pavilion is with the ``source_path``
-build option. Additionally you can bring additional files into your build
-directory with the ``extra files`` option.
+If a suite is organized as directory, all references to files within a test are interpreted as
+relative to the suite directory. If your tests requires source code or other files, this is the
+recommended method of organizing your test suite.
+
+It is also possible to use source code outside of the suite directory using the ``source_path``
+build option. Additionally you can bring additional files into your build directory with the 
+``extra files`` option.
 
 .. code:: yaml
 

docs/tests/format.rst

@@ -11,7 +11,7 @@ This page contains in-depth documentation on the test format.
 Tests and Suites
 ----------------
 
-Each Suite is a yaml file (with a ``.yaml`` extension) which can contain
+Each suite is a yaml file (with a ``.yaml`` extension) which can contain
 multiple tests. Suite files must reside in ``<config_dir>/suites/``,
 where ``<config_dir>`` is one of your :ref:`config.config_dirs`. Tests
 in a suite can be run as a group or independently, and can even inherit
@@ -136,10 +136,10 @@ All Pavilion (non-structural) test config values are interpreted as
 strings.
 
 YAML provides several different data types, but Pavilion forcibly
-converts all of them to strings. The bool True becomes "True", 5 becomes
-the string "5", and so on. This done mostly because it enables Pavilion
+converts all of them to strings. The bool ``True`` becomes ``"True"``, ``5`` becomes
+the string ``"5"``, and so on. This is done mostly because it enables Pavilion
 variable substitution in any config value. Some Pavilion scheduler and
-result parser plugins ask for integer or other specific data types in
+result parser plugins ask for integers or other specific data types in
 their configs. It's up to those plugins to interpret those values and
 report errors.
 
@@ -196,7 +196,7 @@ Items in the config that can take a list can be extended by a list variable.
 Hidden Tests
 ------------
 
-Tests can be hidden by starting their name with an underscore '_' character.
+Tests can be hidden by starting their name with an underscore ``_`` character.
 This is often useful when you have a base test that others inherit from, but
 the base test is never supposed to run on its own.
 
@@ -239,6 +239,10 @@ You can specify the OS config with the ``'-o'`` option to the ``pav run``.
 
     pav run -o another_os my_tests
 
+.. note::
+    It is also possible to define host, OS, and mode configs on a per-suite
+    basis. See the :ref:`tests.suites` section for an overview of this feature.
+
 Host Configs
 ------------
 
@@ -259,10 +263,6 @@ You can specify the host config with the ``-H`` option to the
 
     pav run -H another_host my_tests
 
-.. note::
-    An optional alternate format for host, OS, and mode configs exists when using suite
-    directories. See the following section for an overview of this format.
-
 Format
 ~~~~~~
 
@@ -325,12 +325,12 @@ Rules of Inheritance
 ~~~~~~~~~~~~~~~~~~~~
 
 1. Every field in a test config can be inherited (except for
-   inherits\_from).
-2. A field that takes a list (modules, cmds, etc.) are completely
+   ``inherits\_from``).
+2. A field that takes a list (``modules``, ``cmds``, etc.) is completely
    overwritten by a new list by default. (In the above example, the single command
    in the fs test command list overwrites the entire original command
    list.)
-3. You can add a plus to any 'list' key to extend that list with all The
+3. You can add a plus to any list key to extend that list with all the
    given items instead.
 4. A test can inherit from a test, which inherits from a test, and so
    on.
@@ -381,17 +381,17 @@ order.
    6. Any mode configs in the order specified.
 
 2. Tests are filtered down to only those requested.
-3. Command line overrides ('-c') are applied.
+3. Command line overrides (``-c``) are applied.
 4. Permutations are resolved.
 5. Variables in the chosen scheduler config section are resolved. (You
-   should't have ``sched`` variables in these sections.)
+   shouldn't have ``sched`` variables in these sections.)
 6. Variables are resolved throughout the rest of the config.
 
 This results in the semi-final test config. :ref:`tests.variables.deferred`
 can't be resolved until we're on the allocation. Once there, we'll finish
 resolving those, and resolve any parts of the config that used them. Parts of
-the config that are required before kicking off the test (like the build and
-scheduler sections), can't use deferred variables.
+the config that are required before kicking off the test (like the ``build`` and
+``schedule`` sections) can't use deferred variables.
 
 Top Level Test Config Keys
 --------------------------
@@ -400,8 +400,8 @@ inherits\_from
 ~~~~~~~~~~~~~~
 
 Sets the test (by test base name) that this test inherits from *which must be*
-*a test from this file*. The resulting test will be composed of all
-keys in the test it inherits from, plus any specified in this test
+*a test from the same file*. The resulting test will be composed of all
+keys in the test it inherits from, plus any keys specified in this test
 config. See :ref:`tests.format.inheritance`.
 
 subtitle
@@ -436,7 +436,7 @@ list of mappings (with the same keys) of strings. See the
 scheduler
 ~~~~~~~~~
 
-Sets the scheduler for this test. Defaults to 'raw'. It's recommended to
+Sets the scheduler for this test. Defaults to ``raw``. It's recommended to
 set this in your host configs.
 
 build
@@ -474,7 +474,7 @@ These sub-sections defines conditions under which tests are skipped.
 
 See :ref:`tests.skip_conditions` for the sub-section keys and usage.
 
-<schedule>
+schedule
 ~~~~~~~~~~
 
 Settings for controlling test scheduling. See ``pav show sched --config`` for