Document uniformisation of tests (new API)

requirements.txt

@@ -1,4 +1,4 @@
---editable git+https://github.com/openfisca/openfisca-core.git@master#egg=OpenFisca-Core[dev]
+--editable git+https://github.com/openfisca/openfisca-core.git@uniformize-tests-api#egg=OpenFisca-Core[dev]
 docutils==0.12
 guzzle_sphinx_theme
 recommonmark

source/coding-the-legislation/10_basic_example.md

@@ -25,7 +25,7 @@ Let's explain in details the different parts of the code:
 ### The variable attributes
 
 All variables have a set of attributes.
-* `value_type` defines the type of the formula output. Possible types are the basic python types. 
+* `value_type` defines the type of the formula output. Possible types are the basic python types.
 Note however that OpenFisca uses NumPy to [run calculations vectorially](25_vectorial_computing.md),
 so the actual type of data may be slightly different from the builtin Python ones.
 Available types are :
@@ -35,7 +35,7 @@ Available types are :
     - `float`: float (Note that to reduce memory usage, float are stored on 32 bits using NumPy's `float32`)
     - `int`: integer
     - `str`: string
-* `entity` defines who or what group the variable concerns, e.g. individuals, households, families. 
+* `entity` defines who or what group the variable concerns, e.g. individuals, households, families.
 * `definition_period` defines the period on which the variable is calculated. It can be `MONTH` (e.g. salary), `YEAR` (e.g. income taxes), or ETERNITY (e.g. date of birth)
 * `label` is a human friendly way to describe the variable
 * `reference` is a list of relevant legislative reference for this variables (usually URLs the text of the law or another trustworthy source)
@@ -46,24 +46,24 @@ Available types are :
   - `salary = person('salary', period)` calculates the salary of the person, for the given month. This will, of course, work only if `salary` is another variable in the tax and benefit system.
   - `return salary * 0.25` returns the result for the given period.
   - [Dated Formulas](40_legislation_evolutions.md) have a start and/or an end date.
-  
+
 ## Testing a formula
 
 To make sure that the formula you have just written works the way you expect, you have to test it. Tests about legislation are written in a [YAML syntax](writing_yaml_tests.md). The `flat_tax_on_salary` formula can for instance be tested with the following test file:
 
 ```yaml
 - name: "Flax tax on salary - No income"
   period: 2017-01
-  input_variables:
+  input:
     salary: 0
-  output_variables:
+  output:
     flat_tax_on_salary: 0
 
 - name: "Flax tax on salary - With income"
   period: 2017-01
-  input_variables:
+  input:
     salary: 2000
-  output_variables:
+  output:
     flat_tax_on_salary: 500
 ```
 

source/coding-the-legislation/20_input_variables.md

@@ -125,12 +125,12 @@ You can now test the formula in a YAML test:
 ```yaml
 - name: Household with free lodger status living in a 100 sq.meters accomodation
   period: 2017
-  input_variables:
+  input:
     accomodation_size:
       2017-01: 100
     housing_occupancy_status:
       2017-01: free_lodger
-  output_variables:
+  output:
     housing_tax: 0
 ```
 

source/coding-the-legislation/35_periods.md

@@ -33,15 +33,15 @@ This [YAML test](writing_yaml_tests.md) on `income_tax` evolution over time show
 
 - name: Income tax over time
   period: 2016-01
-  input_variables:
+  input:
     salary:
       year:2014:3: 100000 # This person earned 100,000 between 2014 and 2016
-  output_variables:
+  output:
     income_tax:
       2014-01: 388.8889
       2015-01: 416.6667 # The income tax rate changes in 2015
       2016-01: 416.6667
-      2017-01: 0 # The salary is not set for this period and defaults to 0 
+      2017-01: 0 # The salary is not set for this period and defaults to 0
 
 ```
 
@@ -58,7 +58,7 @@ class salary(Variable):
         ...
 ```
 
-Most of the values calculated in OpenFisca, such as `income_tax`, and `housing_allowance`, can change over time. 
+Most of the values calculated in OpenFisca, such as `income_tax`, and `housing_allowance`, can change over time.
 
 Therefore, all OpenFisca variables have a `definition_period` attribute:
   - `definition_period = MONTH`: The variable may have a different value each month. *For example*, the salary of a person. When `formula` is executed, the parameter `period` will always be a whole month. Trying to compute `salary` with a period that is not a month will raise an error before entering `formula`.

source/coding-the-legislation/writing_yaml_tests.md

@@ -14,9 +14,9 @@ In [`irpp.yaml`](https://github.com/openfisca/openfisca-france/blob/29.3.7/tests
 - name: "IRPP - Célibataire ayant des revenus salariaux (1AJ) de 20 000 €"
   period: 2012
   absolute_error_margin: 0.5
-  input_variables:
+  input:
     salaire_imposable: 20000
-  output_variables:
+  output:
     irpp: -1181
 ```
 
@@ -60,10 +60,10 @@ For instance:
 - name: "IRPP - Célibataire ayant des revenus salariaux (1AJ) de 20 000 €"
   period: 2012
   absolute_error_margin: 0.5
-  input_variables:
+  input:
     salaire_imposable: 20000
     salaire_brut: 20000
-  output_variables:
+  output:
     irpp: -1181
 ```
 
@@ -125,7 +125,7 @@ In this case, there is another convention:
 - finally, define a dictionnary of the expected values of the output variables. Each output variable takes a list of length equal to the number of individuals defined in the test. E.g, for a family of four individuals with two working parents and two unemployed children, the output variable salaire_super_brut is defined as follows:
 
     ```yaml
-    output_variables:
+    output:
         salaire_super_brut: [3500, 2500, 0, 0]
     ```
 
@@ -146,10 +146,10 @@ Values can be arithmetic expressions too.
 
 ## Running a test
 
-To run YAML tests, use the command line tool `openfisca-run-test`, documented [here](https://openfisca.readthedocs.io/en/latest/openfisca-run-test.html):
+To run YAML tests, use the command line tool `openfisca test`, documented [here](https://openfisca.readthedocs.io/en/latest/openfisca-run-test.html):
 
 ```sh
-  openfisca-run-test path/to/file.yaml
+openfisca test path/to/file.yaml
 ```
 
 >You can also run tests programatically using the [`test_runner` module](https://openfisca.readthedocs.io/en/latest/test_runner.html).

source/openfisca-python-api/index.rst

@@ -11,6 +11,7 @@ Modules:
    parameters
    reforms
    simulations
+   simulation_builder
    entities
    test_runner
    tracer

source/openfisca-python-api/openfisca-run-test.rst

@@ -1,11 +1,12 @@
-==================
-openfisca-run-test
-==================
+===============
+openfiscan test
+===============
 
 .. argparse::
-   :module: openfisca_core.scripts.run_test
-   :func: build_parser
-   :prog: openfisca-run-test
+   :module: openfisca_core.scripts.openfisca_command
+   :func: get_parser
+   :prog: openfisca
+   :path: test
 
 Examples
 --------
@@ -21,19 +22,19 @@ Basic use
 
     - name: "Basic test"
       period: 2015
-      input_variables:
+      input:
         gross_salary: 2000
-      output_variables:
+      output:
         net_salary: 2000 * 0.8
 
 **Command line:**
 
 .. code-block:: shell
 
-  openfisca-run-test -c openfisca_france test.yaml
+  openfisca test -c openfisca_france test.yaml
   # Success
 
-  openfisca-run-test test.yaml
+  openfisca test test.yaml
   # Success: the country package is automatically detected.
   # May fail if several country packages are installed in your environment.
   # In that case, specify which package to use with the --country_package option
@@ -49,17 +50,17 @@ Error margin
     - name: "Test defining a relative error margin"
       period: 2015
       relative_error_margin: 0.05
-      input_variables:
+      input:
         gross_salary: 1000
-      output_variables:
+      output:
         net_salary: 780 # the right value is 800
 
     - name: "Test defining an absolute error margin"
       absolute_error_margin: 10
       period: 2015
-      input_variables:
+      input:
         gross_salary: 1000
-      output_variables:
+      output:
         net_salary: 790 # the right value is 800
 
 **test_3.yaml:**
@@ -68,20 +69,20 @@ Error margin
 
     - name: "Test not defining any error margin"
       period: 2015
-      input_variables:
+      input:
         gross_salary: 1000
-      output_variables:
+      output:
         net_salary: 795 # the right value is 800
 
 
 **Command line:**
 
 .. code-block:: shell
 
-  openfisca-run-test test_2.yaml
+  openfisca test test_2.yaml
   # Success: the test pass, as the actual results are within the error margins
 
-  openfisca-run-test test_3.yaml
+  openfisca test test_3.yaml
   # Failure: the test does not pass, as its error margin is by default 0
 
 
@@ -94,36 +95,36 @@ Name filter
 
     - name: "Test containing the word openfisca in its name"
       period: 2015
-      input_variables:
+      input:
         gross_salary: 1000
-      output_variables:
+      output:
         net_salary: 800
 
     - name: "Test that contains the magic word in its keywords"
       keywords:
         - some keyword
         - openfisca
       period: 2015
-      input_variables:
+      input:
         gross_salary: 1000
-      output_variables:
+      output:
         net_salary: 800
 
     - name: "Some other test that fails"
       period: 2015
-      input_variables:
+      input:
         gross_salary: 1000
-      output_variables:
+      output:
         net_salary: 0
 
 **Command line:**
 
 .. code-block:: shell
 
-  openfisca-run-test test_4.yaml
+  openfisca test test_4.yaml
   # Failure: the third test does not pass
 
-  openfisca-run-test -n openfisca test_4.yaml
+  openfisca test -n openfisca test_4.yaml
   # Success: the third test is not executed, as it doesn't contain the word 'openfisca'
 
 Note that if a test file name contains the name filter, all the inner tests will be executed.
@@ -141,21 +142,21 @@ Let's now assume an extension to ``openfisca_france``, ``openfisca_paris`` is in
 
     - name: "Test using an extension"
       period: 2015
-      input_variables:
+      input:
         net_salary: 0
-      output_variables:
+      output:
         paris_housing_benefit: 200
 
 
 **Command line:**
 
 .. code-block:: shell
 
-  openfisca-run-test test_5.yaml
+  openfisca test test_5.yaml
   # Failure: the test returns an error:
   # the country package openfisca_france does not references a variable named paris_housing_benefit
 
-  openfisca-run-test -e openfisca_paris test_5.yaml
+  openfisca test -e openfisca_paris test_5.yaml
   # Success: The test passes, as the extension is loaded in the tax benefit system before running the test
 
 
@@ -173,18 +174,18 @@ This reform is called ``increase_cotisation`` and available in the python module
 
     - name: "Test on a reform"
       period: 2015
-      input_variables:
+      input:
         gross_salary: 1000
-      output_variables:
+      output:
         net_salary: 600
 
 
 **Command line:**
 
 .. code-block:: shell
 
-  openfisca-run-test test_6.yaml
+  openfisca test test_6.yaml
   # Failure: the test does not pass, as the regular openfisca_france is used
 
-  openfisca-run-test -r openfisca_france.reforms.increase_cotisation.increase_cotisation test_5.yaml
+  openfisca test -r openfisca_france.reforms.increase_cotisation.increase_cotisation test_5.yaml
   # Success: The test passes, as the increase_cotisation reform is applied

source/openfisca-python-api/simulation_builder.rst

@@ -0,0 +1,6 @@
+=================
+SimulationBuilder
+=================
+
+.. autoclass:: openfisca_core.simulation_builder.SimulationBuilder
+    :members: build_from_dict, build_from_entities, build_from_variables, build_default_simulation, explicit_singular_entities