Major overhaul of the internal workings of riak_test

* Upgrade versions have moved to being a list not just a single
  version. `upgrade_version` is removed in favor of
  `upgrade_path`. `upgrade_path` is a comma-separated list
  representing an upgrade sequence *e.g.* `1.3.4,1.4.10,2.0.0`
* Unification of node deployment code: common code and work from the
  individual harness modules has been brought into the framework where
  possible.
* Further decouple and distinguish between framework setup and prep,
  test setup, and test execution.
* Streamline the configuration for versions and root path. Move away
  from convention of *current*, *previous*, and *legacy*. Too
  restrictive and not enough benefit and *current* especially is
  ambiguous. Config must specify `root_path` and all versions under
  root path are represented as release directories. A subdirectory of
  `2.0.0` means that *2.0.0* can now be used as a version the same as
  *current*, *previous*, or *legacy* could previously, but the upside
  is that it requires no extra configuration.
* Upgrade transitions can be specified as a list with no bound so if
  we want to test upgrading to each major release from `1.0.0` to
  `2.0.0` then that is possible.
* Node deployment and teardown for a particular test execution is more
  isolated. The framework attempts to stop **all** nodes during its
  setup phase, but doing so after each test execution is
  unnecessary. Node deployment is now a matter of requesting the
  number of required nodes and the versions involved in the test. The
  framework determines if the number of nodes to cover the
  requirements of the test are available and returns success or
  failure on that criteria. As the last statement implies, concurrent
  test execution is now attempted where possible. If there are 8 nodes
  available for the *2.0.0* version and 4 tests queued to be run that
  require only 2 nodes each and do not require upgrade testing then
  there is no reason to block on serial execution.
* Responsibility for management of test execution is more clearly
  delineated. Previously it was hard to account for responsibility of
  tasks between `riak_test_escript`, `riak_test_runner`, and
  `rt`. `riak_test_escript` has been heavily refactored and the work
  it does has been minimized to command line argument parsing and
  spawning the workhorse processes to execute the tests. It also made
  sense to make use of OTP behaviors for the implementation of some of
  the execution helpers. The `riak_test_executor`, an `gen_fsm`, is
  introduced to manage the scheduling of tests and handle reporting
  results. It is a named process and only one runs at a
  time. Individual test execution is managed by a `riak_test_runner`
  process. These are spawned by the `riak_test_executor` and there is
  one for each test that executes. `riak_test_runner` is also
  implemented as a `gen_fsm`. Finally there is the `node_manager`
  process. It is a `gen_server` that handles all node manipulation and
  manages access to the nodes for testing. The `riak_test_executor`
  requests to reserve `N` number of nodes from the node manager and if
  the reservation can be fulfilled the `node_manager` responds with
  the list of nodes for the requesting test to use. If the requested
  number of nodes is not available the execution of the test is
  deferred. The `node_manager` is aware if the current series of test
  executions involves upgrades and deploys nodes initially using the
  correct version based on that information. Thus when a test receives
  a list of nodes there is no need to take any action and test
  execution can begin immediately. * Maximize resource efficiency and
  length of execution duration by seeking to avoid unnecessary node
  starts or cycles. *e.g.* The `node_manager` is initialized with a
  list of nodes and the versions involved in test execution, but no
  nodes are deployed until the first call to
  `node_manager:reserve_nodes` that requires those nodes. If only one
  test slated for execution and it only requires 3 nodes there is no
  reason to start or stop more than 3 nodes.
* Facilitate replication testing setup and eliminate crufty setup code
  duplicated in replication test with new properties `cluster_count`
  (defaults to 1) and `cluster_weights`, a list of weights that
  determine distribution of available nodes among requested clusters
  (defaults to `undefined`). Setting up multiple clusters for testing
  replication should not require any kludgy steps, it should have full
  support in the framework.
* Update backend setup so that backend configuration can be done for
  selected nodes only instead of all nodes
* Change setup scripts to use only version numbers as directory names.
* Change setup scripts to avoid the unnecessary `dev` directory when
  installing devrel releases. *e.g.* Instead of `~/rt/riak/2.0.0/dev/`
  being the path to the `dev*` releases, the path is just
  `~/rt/riak/2.0.0/`. This just removes an unnecessary subdirectory
  and removes the need for some complications in node deployment.
* Refactor properties to distinguish between node name and id: The
  helper functions used by the framework have different input
  requirements.  Some that use rpc require the actual node name which
  may be different depending on the harness used. Others use the node
  identifier to form strings representing shell commands to
  execute. The node property has been replaced by a node_id property
  and a node_map structure that maps a node identifier to a full node
  name.
* Add new rtdev-install script that is intended to replace most of the
  other setup scripts.
* Allow distinction between console and file logging levels: Move from
  a single lager_level configuration option to lager_console_level and
  lager_file_level in order to be able to control these
  independently. The default level for the console output is notice to
  minimize the output display during a run. The default level for the
  file output is info in order to capture all of the logging that
  previously has been output to the console.
* Use `riak_cli` table generation for display of test result details
  when using the `-v` option.
* Update rebar to 2.5.1
* Update Makefile to use tools.mk

Makefile

@@ -1,23 +1,12 @@
 .PHONY: deps
 
-APPS = kernel stdlib sasl erts ssl tools os_mon runtime_tools crypto inets \
-	xmerl webtool eunit syntax_tools compiler hipe mnesia public_key \
-	observer wx gs
-PLT = $(HOME)/.riak-test_dialyzer_plt
-
 all: deps compile
 	./rebar skip_deps=true escriptize
 	SMOKE_TEST=1 ./rebar skip_deps=true escriptize
 
 deps:
 	./rebar get-deps
 
-docs:
-	./rebar skip_deps=true doc
-
-docsclean:
-	@rm -rf doc/*.png doc/*.html doc/*.css edoc-info
-
 compile: deps
 	./rebar compile
 
@@ -31,30 +20,8 @@ quickbuild:
 	./rebar skip_deps=true compile
 	./rebar escriptize
 
-##################
-# Dialyzer targets
-##################
-
-# public targets
-dialyzer: compile $(PLT)
-	dialyzer -Wno_return -Wunderspecs -Wunmatched_returns --plt $(PLT) ebin deps/*/ebin | \
-		egrep -v -f ./dialyzer.ignore-warnings
-
-clean_plt:
-	@echo
-	@echo "Are you sure?  It takes about 1/2 hour to re-build."
-	@echo Deleting $(PLT) in 5 seconds.
-	@echo
-	sleep 5
-	rm $(PLT)
-
-# internal targets
-# build plt file. assumes 'compile' was already run, e.g. from 'dialyzer' target
-$(PLT):
-	@echo
-	@echo "Building dialyzer's plt file. This can take 1/2 hour."
-	@echo "  Because it wasn't here:" $(PLT)
-	@echo "  Consider using R15B03 or later for 100x faster build time!"
-	@echo
-	@sleep 1
-	dialyzer --build_plt --output_plt $(PLT) --apps $(APPS)
+DIALYZER_APPS = kernel stdlib sasl erts ssl tools os_mon runtime_tools crypto inets \
+	xmerl webtool eunit syntax_tools compiler hipe mnesia public_key \
+	observer wx gs
+
+include tools.mk

bin/rtdev-current.sh

@@ -1,32 +1,33 @@
 #!/usr/bin/env bash
 
-# bail out if things go south
+# just bail out if things go south
 set -e
 
 : ${RT_DEST_DIR:="$HOME/rt/riak"}
 
-echo "Making $(pwd) the current release:"
+echo "Making $(pwd) a tagged release:"
 cwd=$(pwd)
 echo -n " - Determining version: "
 if [ -f $cwd/dependency_manifest.git ]; then
     VERSION=`cat $cwd/dependency_manifest.git | awk '/^-/ { print $NF }'`
 else
-    VERSION="$(git describe --tags)-$(git branch | awk '/\*/ {print $2}')"
+    VERSION=`git describe --tags | awk '{sub(/riak-/,"",$0);print}'`
 fi
 echo $VERSION
 cd $RT_DEST_DIR
 echo " - Resetting existing $RT_DEST_DIR"
-git reset HEAD --hard > /dev/null 
-git clean -fd > /dev/null 
-echo " - Removing and recreating $RT_DEST_DIR/current"
-rm -rf $RT_DEST_DIR/current
-mkdir $RT_DEST_DIR/current
+git reset HEAD --hard > /dev/null 2>&1
+git clean -fd > /dev/null 2>&1
+echo " - Removing and recreating $RT_DEST_DIR/$VERSION"
+rm -rf $RT_DEST_DIR/$VERSION
+mkdir $RT_DEST_DIR/$VERSION
 cd $cwd
-echo " - Copying devrel to $RT_DEST_DIR/current"
-cp -p -P -R dev $RT_DEST_DIR/current
-echo " - Writing $RT_DEST_DIR/current/VERSION"
-echo -n $VERSION > $RT_DEST_DIR/current/VERSION
+echo " - Copying devrel to $RT_DEST_DIR/$VERSION"
+cd dev
+for i in `ls`; do cp -p -P -R $i $RT_DEST_DIR/$VERSION/; done
+echo " - Writing $RT_DEST_DIR/$VERSION/VERSION"
+echo -n $VERSION > $RT_DEST_DIR/$VERSION/VERSION
 cd $RT_DEST_DIR
 echo " - Reinitializing git state"
-git add .
-git commit -a -m "riak_test init" --amend > /dev/null
+git add -f .
+git commit -a -m "riak_test init" --amend > /dev/null 2>&1

bin/rtdev-install.sh

@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+
+# just bail out if things go south
+set -e
+
+: ${RT_DEST_DIR:="$HOME/rt/riak"}
+
+cwd=$(pwd)
+echo -n " - Determining version: "
+if [ -z "${VERSION+xxx}" ] || ([ -z "$VERSION" ] && [ "${VERSION+xxx}" = "xxx" ]); then
+       if [ -f $cwd/dependency_manifest.git ]; then
+           VERSION=`cat $cwd/dependency_manifest.git | awk '/^-/ { print $NF }'`
+       else
+           echo "Making $(pwd) a tagged release:"
+           VERSION=`git describe --tags | awk '{sub(/riak-/,"",$0);print}'`
+       fi
+fi
+echo $VERSION
+if [ ! -d $RT_DEST_DIR ]; then
+    mkdir $RT_DEST_DIR
+fi
+cd $RT_DEST_DIR
+if [ -d ".git" ]; then
+    echo " - Resetting existing $RT_DEST_DIR"
+    git reset HEAD --hard > /dev/null 2>&1
+    git clean -fd > /dev/null 2>&1
+fi
+echo " - Removing and recreating $RT_DEST_DIR/$VERSION"
+rm -rf $RT_DEST_DIR/$VERSION
+mkdir $RT_DEST_DIR/$VERSION
+cd $cwd
+echo " - Copying devrel to $RT_DEST_DIR/$VERSION"
+if [ ! -d "dev" ]; then
+    echo "You need to run \"make devrel\" or \"make stagedevrel\" first"
+    exit 1
+fi
+cd dev
+for i in `ls`; do cp -p -P -R $i $RT_DEST_DIR/$VERSION/; done
+echo " - Writing $RT_DEST_DIR/$VERSION/VERSION"
+echo -n $VERSION > $RT_DEST_DIR/$VERSION/VERSION
+cd $RT_DEST_DIR
+if [ -d ".git" ]; then
+    echo " - Reinitializing git state"
+    git add -f .
+    git commit -a -m "riak_test init" --amend > /dev/null 2>&1
+else
+    git init
+
+    ## Some versions of git and/or OS require these fields
+    git config user.name "Riak Test"
+    git config user.email "[email protected]"
+
+    git add .
+    git commit -a -m "riak_test init" > /dev/null
+    echo " - Successfully completed initial git commit of $RT_DEST_DIR"
+fi

bin/rtdev-setup-releases.sh

@@ -16,10 +16,10 @@ echo " - Creating $RT_DEST_DIR"
 rm -rf $RT_DEST_DIR
 mkdir -p $RT_DEST_DIR
 
-count=$(ls */dev 2> /dev/null | wc -l)
+count=$(ls * 2> /dev/null | wc -l)
 if [ "$count" -ne "0" ]
 then
-    for rel in */dev; do
+    for rel in *; do
         vsn=$(dirname "$rel")
         echo " - Initializing $RT_DEST_DIR/$vsn"
         mkdir -p "$RT_DEST_DIR/$vsn"
@@ -34,7 +34,7 @@ else
 fi
 
 cd $RT_DEST_DIR
-git init 
+git init
 
 ## Some versions of git and/or OS require these fields
 git config user.name "Riak Test"

rebar.config

@@ -16,7 +16,8 @@
         {riakc, ".*", {git, "git://github.com/basho/riak-erlang-client", {branch, "master"}}},
         {riakhttpc, ".*", {git, "git://github.com/basho/riak-erlang-http-client", {branch, "master"}}},
         {kvc, "1.3.0", {git, "https://github.com/etrepum/kvc", {tag, "v1.3.0"}}},
-        {druuid, ".*", {git, "git://github.com/kellymclaughlin/druuid.git", {tag, "0.2"}}}
+        {druuid, ".*", {git, "git://github.com/kellymclaughlin/druuid.git", {tag, "0.2"}}},
+        {riak_cli, ".*", {git, "[email protected]:basho/riak_cli", {branch, "master"}}}
        ]}.
 
 {escript_incl_apps, [goldrush, lager, getopt, riakhttpc, riakc, ibrowse, mochiweb, kvc]}.