Skip to content

Commit

Permalink
Rewrite tutorial for 0.2.1-SNAPSHOT
Browse files Browse the repository at this point in the history
  • Loading branch information
@aphyr
aphyr committed Sep 22, 2020
1 parent 2fe9473 commit 7b91052
Show file tree
Hide file tree
Showing 8 changed files with 168 additions and 152 deletions.
3 changes: 2 additions & 1 deletion doc/tutorial/01-scaffolding.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Verschlimmbesserung: a library for talking to etcd.
:url "http://www.eclipse.org/legal/epl-v10.html"}
:main jepsen.etcdemo
:dependencies [[org.clojure/clojure "1.10.0"]
[jepsen "0.1.13"]
[jepsen "0.2.1-SNAPSHOT"]
[verschlimmbesserung "0.1.3"]])
```

Expand Down Expand Up @@ -87,6 +87,7 @@ errors, logging, etc. Let's pull in the `jepsen.cli` namespace, call it `cli` fo
:concurrency, ...), constructs a test map."
[opts]
(merge tests/noop-test
{:pure-generators true}
opts))

(defn -main
Expand Down
3 changes: 2 additions & 1 deletion doc/tutorial/02-db.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,8 @@ construct using the `db` function we just wrote. We'll test etcd version
opts
{:name "etcd"
:os debian/os
:db (db "v3.1.5")}))
:db (db "v3.1.5")
:pure-generators true}))
```

`noop-test`, like all Jepsen tests, is a map with keys like `:os`, `:name`,
Expand Down
62 changes: 33 additions & 29 deletions doc/tutorial/03-client.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,15 @@ connected clients later.

```clj
(defn etcd-test
"Given an options map from the command-line runner (e.g. :nodes, :ssh,
:concurrency, ...), constructs a test map."
"Given an options map from the command line runner (e.g. :nodes, :ssh,
:concurrency ...), constructs a test map."
[opts]
(merge tests/noop-test
opts
{:name "etcd"
:os debian/os
:db (db "v3.1.5")
{:pure-generators true
:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)}))
```

Expand Down Expand Up @@ -170,14 +171,15 @@ duties), and stop after 15 seconds.
[opts]
(merge tests/noop-test
opts
{:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)
:generator (->> r
(gen/stagger 1)
(gen/nemesis nil)
(gen/time-limit 15))}))
{:pure-generators true
:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)
:generator (->> r
(gen/stagger 1)
(gen/nemesis nil)
(gen/time-limit 15))}))
```

This throws a bunch of errors, because we haven't told the client *how* to
Expand All @@ -186,17 +188,18 @@ intepret these reads yet.
```bash
$ lein run test
...
WARN [2018-02-02 15:00:54,338] jepsen worker 3 - jepsen.core Error running worker 3
java.lang.AssertionError: Assert failed: Expected client/invoke! to return a map with :type :ok, :fail, or :info, but received {:time 107596475} instead
WARN [2020-09-21 20:16:33,150] jepsen worker 0 - jepsen.generator.interpreter Process 0 crashed
clojure.lang.ExceptionInfo: throw+: {:type :jepsen.client/invalid-completion, :op {:type :invoke, :f :read, :value nil, :time 26387538, :process 0}, :op' nil, :problems ["should be a map" ":type should be :ok, :info, or :fail" ":process should be the same" ":f should be the same"]}
```
The client's `invoke!` function takes an invocation operation, and right now,
does nothing with it, returning `nil`. Jepsen adds a timestamp, which is where
`{:time ...}` comes from, but we have to tell it what else happened to the
operation. We'll construct a corresponding completion op, with type `:ok` if
the operation succeeded, `:fail` if it didn't take place, or `:info` if we're
not sure. `invoke!` can also throw an exception, which is automatically
converted to an `:info`.
does nothing with it, returning `nil`. Jepsen is telling us that it should be a
map, and specifically one with a `:type` field, a matching `:process`, and a
matching `:f`. In short, we have to construct a completion operation which
*concludes* the invocation operation. We'll construct this completion op with
type `:ok` if the operation succeeded, `:fail` if it didn't take place, or
`:info` if we're not sure. `invoke!` can also throw an exception, which is
automatically converted to an `:info`.
Let's start by handling reads. We'll use `v/get` to read the value of a single
key. We can pick any name we like--let's call it "foo" for now.
Expand Down Expand Up @@ -237,14 +240,15 @@ We'll change our generator to take a random mixture of reads and writes, using
[opts]
(merge tests/noop-test
opts
{:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)
:generator (->> (gen/mix [r w])
(gen/stagger 1)
(gen/nemesis nil)
(gen/time-limit 15))}))
{:pure-generators true
:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)
:generator (->> (gen/mix [r w])
(gen/stagger 1)
(gen/nemesis nil)
(gen/time-limit 15))}))
```

To handle those writes, we'll use `v/reset!`, and return the op with
Expand Down
49 changes: 26 additions & 23 deletions doc/tutorial/04-checker.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ like locks and registers. Here's one for [a compare-and-set register](https://gi
(inconsistent (str "can't read " (:value op)
" from register " value))))))
```

We don't need to write these in our tests, as long as `knossos` provides a
model for the type of thing we're checking. This is just so you can see how
things work under the hood.
Expand All @@ -89,11 +90,11 @@ through. This allows us to satisfy histories which include reads that never
returned.

To analyze the history, we'll specify a `:checker` for the test, and provide a
`:model` to specify how the system *should* behave.
`checker/linearizable` uses the Knossos linearizability checker to verify that
every operation appears to take place atomically between its invocation and
completion. The linearizable checker requires a model and to specify a particular
algorithm which we pass to it in an options map.
`:model` to specify how the system *should* behave. `checker/linearizable` uses
the Knossos linearizability checker to verify that every operation appears to
take place atomically between its invocation and completion. The linearizable
checker requires a model and to specify a particular algorithm which we pass to
it in an options map.

```clj
(defn etcd-test
Expand All @@ -102,17 +103,18 @@ algorithm which we pass to it in an options map.
[opts]
(merge tests/noop-test
opts
{:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)
:checker (checker/linearizable {:model (model/cas-register)
:algorithm :linear})
:generator (->> (gen/mix [r w cas])
(gen/stagger 1)
(gen/nemesis nil)
(gen/time-limit 15))}))

{:pure-generators true
:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)
:checker (checker/linearizable
{:model (model/cas-register)
:algorithm :linear})
:generator (->> (gen/mix [r w cas])
(gen/stagger 1)
(gen/nemesis nil)
(gen/time-limit 15))}))
```

Running the test, we can confirm the checker's results:
Expand All @@ -125,14 +127,14 @@ INFO [2019-04-17 17:38:16,861] jepsen worker 0 - jepsen.util 0 :ok :write 1
...
INFO [2019-04-18 03:53:32,714] jepsen test runner - jepsen.core {:valid? true,
:configs
({:model {:value 1},
({:model #knossos.model.CASRegister{:value 3},
:last-op
{:process 3,
{:process 1,
:type :ok,
:f :write,
:value 1,
:index 151,
:time 14796541900},
:value 3,
:index 29,
:time 14105346871},
:pending []}),
:analyzer :linear,
:final-paths ()}
Expand Down Expand Up @@ -179,8 +181,9 @@ And add that checker to the test:
```clj
:checker (checker/compose
{:perf (checker/perf)
:linear (checker/linearizable {:model (model/cas-register)
:algorithm :linear})
:linear (checker/linearizable
{:model (model/cas-register)
:algorithm :linear})
:timeline (timeline/html)})
```
Expand Down
56 changes: 29 additions & 27 deletions doc/tutorial/05-nemesis.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,27 +29,29 @@ test. This one partitions the network into two halves, selected randomly, when
it receives a `:start` op, and heals the network when it receives a `:stop`.

```clj

(defn etcd-test
"Given an options map from the command line runner (e.g. :nodes, :ssh,
:concurrency ...), constructs a test map."
[opts]
(merge tests/noop-test
opts
{:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)
:nemesis (nemesis/partition-random-halves)
:checker (checker/compose
{:perf (checker/perf)
:linear (checker/linearizable {:model (model/cas-register)
:algorithm :linear})
:timeline (timeline/html)})
:generator (->> (gen/mix [r w cas])
(gen/stagger 1)
(gen/nemesis nil)
(gen/time-limit 15))}))

{:pure-generators true
:name "etcd"
:os debian/os
:db (db "v3.1.5")
:client (Client. nil)
:nemesis (nemesis/partition-random-halves)
:checker (checker/compose
{:perf (checker/perf)
:linear (checker/linearizable
{:model (model/cas-register)
:algorithm :linear})
:timeline (timeline/html)})
:generator (->> (gen/mix [r w cas])
(gen/stagger 1)
(gen/nemesis nil)
(gen/time-limit 15))}))
```

Like regular clients, the nemesis draws operations from the generator. Right
Expand All @@ -61,16 +63,16 @@ dedicated generator for nemesis operations. We're also going to increase the tim
:generator (->> (gen/mix [r w cas])
(gen/stagger 1)
(gen/nemesis
(gen/seq (cycle [(gen/sleep 5)
{:type :info, :f :start}
(gen/sleep 5)
{:type :info, :f :stop}])))
(cycle [(gen/sleep 5)
{:type :info, :f :start}
(gen/sleep 5)
{:type :info, :f :stop}]))
(gen/time-limit 30))}
```

`gen/seq` takes a sequence of generators and emits a single op from each one.
We use `cycle` to construct an infinite loop of sleep, start, sleep, stop, ...,
which ends once the time limit is up.
Clojure sequences can act as generators, so we can use regular Clojure
functions to construct them. Here, we use `cycle` to construct an infinite loop
of sleep, start, sleep, stop, ..., which ends once the time limit is up.

The network partition causes some operations to crash:

Expand Down Expand Up @@ -112,12 +114,12 @@ Now that we can run tests for shorter or longer, let's speed up the request rate

```clj
:generator (->> (gen/mix [r w cas])
(gen/stagger 1/10)
(gen/stagger 1/50)
(gen/nemesis
(gen/seq (cycle [(gen/sleep 5)
{:type :info, :f :start}
(gen/sleep 5)
{:type :info, :f :stop}])))
(cycle [(gen/sleep 5)
{:type :info, :f :start}
(gen/sleep 5)
{:type :info, :f :stop}]))
(gen/time-limit (:time-limit opts)))}
```

Expand Down
10 changes: 5 additions & 5 deletions doc/tutorial/06-refining.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -136,13 +136,13 @@ We have a generator that emits operations on a single key, like `{:type :invoke,
(range)
(fn [k]
(->> (gen/mix [r w cas])
(gen/stagger 1/10)
(gen/stagger 1/50)
(gen/limit 100))))
(gen/nemesis
(gen/seq (cycle [(gen/sleep 5)
{:type :info, :f :start}
(gen/sleep 5)
{:type :info, :f :stop}])))
(cycle [(gen/sleep 5)
{:type :info, :f :start}
(gen/sleep 5)
{:type :info, :f :stop}]))
(gen/time-limit (:time-limit opts)))}))
```

Expand Down
Loading

0 comments on commit 7b91052

Please sign in to comment.