Skip to content

Commit

Permalink
Add support for fixtures
Browse files Browse the repository at this point in the history
Resolves amperity#52.

This is a first pass for this feature so design input is requested.

Initial decisions:

- Use `use-fixtures` from `clojure.test` for fixture functions that
  apply to all of the tests in a namespace. Note that this does not
  support `:once` fixtures as of yet because the runner does not run all
  tests in a namespace as a group that can be `:once`d. This will
  require a change to `greenlight.runner`.
- Add `::test/each` as an option to the `deftest` attribute map. This
  needs to be a collection of fixture functions. This could not quite be
  expressed via the `use-fixtures` method because it would unilaterally
  apply a fixture function to all steps in all tests in a namespace.
  This could be an additional feature as well.

Open questions:
- Are functions expressive enough, or should this introduce a new
  namespace that allows for composable definitions of fixtures in the
  way that steps are composable? Something like `deffixture`, a
  greenlight-specific `use-fixture` for top-level, and then fixture
  configuration at the test/step level.
- Should context be passed into the fixture functions, and should it be
  modifiable by the fixture functions? e.g., should downstream steps see
  the changed context and should the test output contain that changed
  context. This gets a bit sticky and pushes on the `deffixture` idea,
  in that greenlight tries to be explicit about its data dependencies
  and outputs, but with fixtures that can take/update context, declaring
  data dependencies gets a bit hairier. I can imagine that `deffixture`
  would _also_ declare inputs and outputs, which could then get merged
  into the step inputs/outputs. Depends on how complex this needs to be.
  • Loading branch information
robhanlon22 committed Jun 9, 2022
1 parent 783e6fe commit 92dc6ac
Show file tree
Hide file tree
Showing 3 changed files with 73 additions and 23 deletions.
58 changes: 36 additions & 22 deletions src/greenlight/test.clj
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
specific usage scenario."
(:require
[clojure.spec.alpha :as s]
[clojure.test :as ctest]
[clojure.string :as str]
[greenlight.step :as step])
(:import
Expand Down Expand Up @@ -152,27 +153,33 @@
"Executes a sequence of test steps by running them in order until one fails.
Returns a tuple with the enriched vector of steps run and the final context
map."
[system options ctx steps]
(loop [history []
ctx ctx
steps steps]
(if-let [step (first steps)]
; Run next step to advance the test.
(let [step (step/initialize step ctx)
_ (*report* {:type :step-start
:step step})
[step' ctx'] (step/advance! system step ctx)
history' (conj history step')]
(*report* {:type :step-end
:step step'})
; Continue while steps pass.
(if (= :pass (::step/outcome step'))
(recur history' ctx' (next steps))
(if (retry-step? options step')
(recur history ctx steps)
[(vec (concat history' (rest steps))) ctx'])))
; No more steps.
[history ctx])))
[system options ctx test-case]
(let [each-fixture-fn (ctest/join-fixtures (::each test-case))]
(loop [history []
ctx ctx
steps (::steps test-case)]
(if-let [step (first steps)]
;; Run next step to advance the test.
(let [step (-> step
(step/initialize ctx)
(update ::step/test
(fn [test-fn]
(fn [inputs]
(each-fixture-fn #(test-fn inputs))))))
_ (*report* {:type :step-start
:step step})
[step' ctx'] (step/advance! system step ctx)
history' (conj history step')]
(*report* {:type :step-end
:step step'})
;; Continue while steps pass.
(if (= :pass (::step/outcome step'))
(recur history' ctx' (next steps))
(if (retry-step? options step')
(recur history ctx steps)
[(vec (concat history' (rest steps))) ctx'])))
;; No more steps.
[history ctx]))))


(defn- run-cleanup!
Expand Down Expand Up @@ -203,7 +210,14 @@
:test test-case})
(let [started-at (Instant/now)
ctx (::context test-case {})
[history ctx] (run-steps! system options ctx (::steps test-case))
each-fixture-fn (-> test-case
::ns
the-ns
meta
::ctest/each
ctest/join-fixtures)
[history ctx] (each-fixture-fn
#(run-steps! system options ctx test-case))
_ (run-cleanup! system history)
ended-at (Instant/now)
test-case (assoc test-case
Expand Down
27 changes: 26 additions & 1 deletion test/greenlight/test_suite/blue.clj
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
(ns greenlight.test-suite.blue
(:require
[clojure.test :refer [is]]
[clojure.test :refer [is use-fixtures]]
[com.stuartsierra.component :as component]
[greenlight.step :as step :refer [defstep]]
[greenlight.test :as test :refer [deftest]]))
Expand Down Expand Up @@ -51,6 +51,31 @@
:test (fn [_]
(is (= 1 1)))})

(def counter
(atom 0))

(use-fixtures :each
(fn [f]
(reset! counter 0)
(f)))

(deftest with-fixtures
#::test{:description "foobar"
:context {:foo :bar}
:each [(fn [f]
(swap! counter inc)
(f))
(fn [f]
(swap! counter inc)
(f))]}
#::step{:name 'step-1
:title "step-1"
:test (fn [_]
(is (= 1 1)))}
#::step{:name 'step-2
:title "step-2"
:test (fn [_]
(is (= 1 1)))})

(deftest sample-test
"A sample greenlight test in the blue test suite"
Expand Down
11 changes: 11 additions & 0 deletions test/greenlight/test_test.clj
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,17 @@
(mapv ::step/title (::test/steps test-result))))
(is (= 1 (count (::test/steps test-result))))))

(deftest fixtures-test
(let [system (component/system-map :greenlight.test-test/component 6)
attr-test (blue/with-fixtures)
test-result (test/run-test! system attr-test)]
(is (= (::test/description test-result) "foobar"))
(is (= (::test/context test-result) {:foo :bar}))
(is (= :pass (::test/outcome test-result)))
(is (= 4 @blue/counter))
(is (= ["step-1" "step-2"]
(mapv ::step/title (::test/steps test-result))))
(is (= 2 (count (::test/steps test-result))))))

(defmacro with-io
[input & forms]
Expand Down

0 comments on commit 92dc6ac

Please sign in to comment.