Skip to content

Commit

Permalink
simplify docs + improve refs
Browse files Browse the repository at this point in the history
  • Loading branch information
Filip Schouwenaars committed Aug 15, 2018
1 parent 2bc6f92 commit ce7788e
Show file tree
Hide file tree
Showing 8 changed files with 26 additions and 144 deletions.
Binary file removed docs/ast_example.png
Binary file not shown.
Binary file removed docs/ast_example2.png
Binary file not shown.
Binary file removed docs/ast_example3_combi.png
Binary file not shown.
Binary file removed docs/ast_example3a.png
Binary file not shown.
Binary file removed docs/ast_example3b.png
Binary file not shown.
146 changes: 12 additions & 134 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,145 +1,23 @@
protowhat
---------

``protowhat`` is a utility package required by ``sqlwhat`` and ``shellwhat`` packages,
used for writing Submission Correctness Tests SCTs for interactive Shell and SQL exercises on DataCamp.
It contains shared functionality related to SCT syntax and selectors, state manipulation and commonly used functions.
protowhat is a utility package required by

- If you are new to teaching on DataCamp, check out https://authoring.datacamp.com.
- If you want to learn what SCTs are and how they work, visit `this article <https://authoring.datacamp.com/courses/exercises/technical-details/sct.html>`_ specifically.
- `sqlwhat <https://sqlwhat.readthedocs.io>`_ to write SCTs for SQL exercises, and
- `protowhat <https://shellwhat.readthedocs.io>`_ to write SCTs for Shell exercises.

This documentation takes a deeper dive into the chaining syntax employed for SCTs and
contains reference documentation of all relevant SCT functions.
If you are new to writing SCSTs for SQL or Shell exercises,
read the Syntax section below first before embarking onto
the reference documentation or the ``sqlwhat`` (`link <https://sqlwhat.readthedocs.io>`_) and ``shellwhat`` (`link <https://shellwhat.readthedocs.io>`_) dedicated docs.
protowhat contains functionality that is shared between these packages, including:

Syntax
======
- SCT function chaining and syntactical sugar,
- State management,
- AST element selection, dispatching and message generation,
- Basic SCT functions such as ``success_msg()`` and ``has_chosen()``.

``protowhat`` (and by extension ``sqlwhat`` and ``shellwhat``, as they depend on ``protowhat``) start from the concept of an exercise state.
This exercise state can be accessed with ``Ex()`` and contains all the information that is required to check if an exercise is correct:

1. the student submission and the solution as text and their corresponding abstract syntax trees.
2. a reference to the student coding session and the solution coding session.
3. the results, output and errors that were generated when executing the student code.

Currently, as coding sessions and result formatting and parsing is very different between SQL an Shell,
``protowhat`` is only concerned with the student submission and solution code as text and their corresponding syntax trees (AST).
For SCT functions related to checking results and output, consult the docs of ``sqlwhat`` and ``shellwhat``.

Abastract Syntax tree (AST)
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The abstract syntax tree is a parsed tree representations of a code submission. To better understand what this means, visit `this AST viewer app <https://ast-viewer.datacamp.com>`_
In editor mode, you can include a snippet of code, specify the parser it should use, and generate the parse tree. As an example, this PostgreSQL code

.. code:: sql
SELECT id FROM artists WHERE id > 100
is first parsed into a complex-looking parse tree, and then restructured into an abstract syntax tree that looks like this:

.. image:: ast_example.png
:align: center

Notice how the statement is neatly chopped up into its consituents: the ``SELECT`` statement is chopped up into three parts:
the ``target_list`` (which columns to select), the ``from_clause`` (from which table to select) and the ``where_clause`` (the condition that has to be satisfied).
Next, the ``where_caluse`` is a ``BinaryExpr`` that is further chopped up.

Chaining
~~~~~~~~

The root exercise state, that contains the entire student submission and solution and their corresponding ASTs, is accessed with ``Ex()``.
From this ``Ex()`` call, you can 'chain together' SCT functions with the ``.`` operator.
As SCT functions are chained, the ``Ex()`` state is copied and adapted into so-called child states that zoom in on particular parts of the code.
In other words, by chaining calls, you're instructing ``protowhat`` how to walk down the AST.

Example
~~~~~~~

Consider the example above again and suppose you want to check whether students have correctly specified the table from which to select columns (the ``FROM artists`` part).
This SCT script does that:

.. code:
Ex().check_node(state, "SelectStmt").check_field("from_clause").has_equal_ast()
We'll now explain step by step what happens when a student submits the following (incorrect) code:

.. code::
SELECT id FROM producers WHERE id > 100
When the SCT executes:

- ``Ex()`` runs first, and fetches the root state that considers the entire student submission and solution:

.. code::
-- solution
SELECT id FROM artists WHERE id > 100
-- student
SELECT id FROM producers WHERE id > 100
This is the corresponding AST of the solution.
This is the same tree as included earlier in this article.
The AST for the student submission will look very similar.

.. image:: ast_example2.png
:align: center
:scale: 80%

- Next, ``check_node()`` chains off of the state produced by ``Ex()`` and
produces a child state that focuses on the ``SelectStmt`` portion of the submission and solution:

.. code::
-- solution
SELECT id FROM artists WHERE id > 100
-- student
SELECT id FROM producers WHERE id > 100
The corresponding AST of the solution is the following. Notice that although the textual representation is the same as ``Ex()``,
the AST representation no longer includes the ``Script`` node. The AST for the student submission will look very similar.

.. image:: ast_example2.png
:align: center
:scale: 80%

- Next, ``check_field()`` chains off of the state produced by ``check_node()`` and zooms in on the ``from_clause`` branch of the AST:

.. code::
-- solution
artists
-- student
producers
The corresponding ASTs for solution and student are as follows:

.. image:: ast_example3_combi.png
:align: center
:width: 300px

- Finally, ``has_equal_ast()`` chains off of the state produced by ``check_field()`` and
checks whether the student submission and solution sub-ASTs correspond.
As the solution expects ``artists`` while the student specified ``producers`` the SCT fails
and ``protowhat`` will generate a meaningful feedback message.

.. note::

Notice that we are using two different functions here: ``check_node()`` and ``check_field()``.
As a rule, ``check_node()`` is used to select a `node` of the AST tree (a circle in the image).
while ``check_field()`` is used to walk down a `branch` of the AST tree (a line in the image).

Reference Documentation
=======================
All relevent documentation to write SCTs for SQL and Shell exercises,
including functions that reside in ``protowhat``, can be found in the `sqlwhat <https://sqlwhat.readthedocs.io>`_ and `protowhat <https://shellwhat.readthedocs.io>`_ documentation.

.. toctree::
:maxdepth: 2
:caption: Reference

reference
reference
14 changes: 7 additions & 7 deletions docs/reference.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
Simple tests
------------
Ex()
----

.. automodule:: protowhat.checks.check_simple
:members:
.. automethod:: protowhat.sct_syntax.Ex.__call__

AST Checks
----------
Expand All @@ -16,7 +15,8 @@ Logic
.. automodule:: protowhat.checks.check_logic
:members:

Ex()
----
Simple tests
------------

.. automethod:: protowhat.sct_syntax.Ex.__call__
.. automodule:: protowhat.checks.check_simple
:members:
10 changes: 7 additions & 3 deletions protowhat/checks/check_simple.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
def has_chosen(state, correct, msgs):
"""
Note uses 1-based indexing in order to work with Datacamp's campus app.
"""Verify exercises of the type MultipleChoiceExercise
Args:
state: State instance describing student and solution code. Can be omitted if used with Ex().
correct: index of correct option, where 1 is the first option.
Expand All @@ -12,6 +11,11 @@ def has_chosen(state, correct, msgs):
of which is correct.::
Ex().has_chosen(1, ['Correct!', 'Incorrect. Try again!'])
Note:
``has_chosen()`` uses 1-based indexing in order to work with Datacamp's campus app.
"""

ctxt = {}
Expand Down

0 comments on commit ce7788e

Please sign in to comment.