Monitoring

Monitoring

Part of the simulation design process is to decide which results should be captured with a monitoring statement. Not all the data produced by the simulation are needed to interpret the results: mosquito-to-human transmission intensity, parasite density per infection per human together with human infectiousness to mosquitoes, bouts of sickness and treatments affecting each human, etc. A summary of those events over all the simulated population may be more important than the detailed experience of each simulated individual.

The monitoring method is described by an XML element of the form below. The continuous element is optional and describes continuous reporting, similarly the cohorts element is required only if output is required by sub-population. The other three elements are required and describe surveys.

<monitoring name="(some description or name)">
  <continuous ... />
  <SurveyOptions ... />
  <surveys ... />
  <ageGroup ... />
  <cohorts ... />
</monitoring>

Surveys

Surveys are periodic summaries at predefined time-steps. At those time-steps, information is saved about the quantity of interest, this could be a measurable quantity such as the EIR, or number of events since the last survey time point. Where this data concerns humans, it can be segregated into age groups, and can be sampled either across all simulated individuals or from selected subpopulations.

Survey measures

Standard outputs from OpenMalaria are intended to simulate the collection of data at a cross-sectional survey of the whole population. The measures output may include cross-sectional data from the time of the survey and/or values summed over the population since the previous survey. Exactly which measures will be output by a simulation depends on the SurveyOptions element. This contains a list of options, for example:

<SurveyOptions>
  <option name="nHost" value="true"/>
  <option name="nPatent" value="true"/>
  <option name="nMassVaccinations"/>
</SurveyOptions>

(The value attribute is optional and can be omitted as in the last example. It can also be explicitly set to "false" which is the same as omitting the option entirely.)

A list of all outputs currently implemented can be found here.

Survey time points

Surveys can take place at any time point, starting from the beginning of the intervention period. Surveys only report events which happened from the beginning of the time-step of the last survey until the end of the time-step before the current survey time-step, and measures of the current state (such as the number of patent hosts) from the beginning of the survey time-step.

The timing of these is described as in the XML fragment below. The first valid time-point for a survey is time-step 0; however, any events happening before time-step 0 are not reported, so measures of events (such as infectious bites received) will be zero.

<surveys detectionLimit="40">
  <surveyTime>18</surveyTime>
  <surveyTime>37</surveyTime>
  <surveyTime>55</surveyTime>
  <surveyTime>73</surveyTime>
</surveys>

This example describes four quarterly surveys (assuming a 5-day time-step); these are reported in the output.txt file as surveys 1, 2, 3 and 4. (The detectionLimit attribute describes the minimum parasite density (parasites per µl/microlitre) at which a human is counted as having a patent infection.)

Timestep 0 will always correspond to the first of January (up to 5th Jan with a 5-day timestep), except that, prior to schema 22, when the maximumAgeYrs attribute of demography was not a whole number of years timestep values may well have been offset. Years are always modelled as 365 days long, hence (n 365 / days_per_timestep) is always the beginning of the n-th year of the intervention period (excepting above offset).

Survey age groups

The ageGroup element describes which age-groups human-specific data is segregated into. Examples:

<ageGroup lowerbound="0.0">
  <group upperbound="0.25"/>
  <group upperbound="0.5"/>
  <group upperbound="0.75"/>
  <group upperbound="1"/>
  <group upperbound="1.5"/>
  <group upperbound="2"/>
  <group upperbound="3"/>
  <group upperbound="4"/>
  <group upperbound="5"/>
  <group upperbound="6"/>
  <group upperbound="7"/>
  <group upperbound="8"/>
  <group upperbound="9"/>
  <group upperbound="10"/>
  <group upperbound="12"/>
  <group upperbound="14"/>
  <group upperbound="16"/>
  <group upperbound="18"/>
  <group upperbound="20"/>
  <group upperbound="25"/>
  <group upperbound="30"/>
  <group upperbound="35"/>
  <group upperbound="40"/>
  <group upperbound="45"/>
  <group upperbound="50"/>
  <group upperbound="55"/>
  <group upperbound="60"/>
  <group upperbound="65"/>
  <group upperbound="70"/>
  <group upperbound="99"/>
</ageGroup>

<ageGroup lowerbound="0.0">
  <group upperbound="99"/>
</ageGroup>

Here the upper bound is inclusive, so exact age 1 is included under age group 4 (with upper bound 1), but exact age 0 is always included in the first age group. Note also that this is no implicit catch-all last age group, so if the upper bound on the last age group is less than the oldest age allowed, then humans who are older than the last upper bound will simply not be included in surveys.

By default, survey data comes from the whole simulated population. It is also possible to specify monitoring only for defined subpopulations.

Continuous

Whereas survey reporting is designed to aggregate data into configurable-size lumps, the continuous reporting mechanism is designed to report some data at high frequencies (but generally without segregation by age group and with less configuration potential).

To enable continuous reporting, add a continuous sub-element to the monitoring element, of the following form.

<continuous duringInit="false" period="1">
  <option name="input EIR" value="true"/>
  <option name="simulated EIR" value="true"/>
  <option name="human infectiousness" value="true"/>
</continuous>

The period attribute specifies the number of time-steps between reports. duringInit is mostly used for debugging and can be omitted entirely; if set to true it enables reporting during the warm-up period and an extra column in the output (simulation time, the time-step counting from the beginning of the simulation rather than the beginning of the intervention period).

A list of available continuous outputs is here.

Monitoring of sub-populations and cohorts

Monitoring of sub-populations (version 32 on)

As of schema 32, multiple subpopulations may be monitored in surveys (but not in continuous output).
Each of the monitored subpopulations must first be defined, either implicitly, by an intervention within the element, or explicitly by the recruitment only pseudo-intervention.

Each of the subpopulations needs to be specified in the "cohorts" element, which links a subpopulation number to the id assigned when the subpopulation was declared in the "component" element:

    <cohorts>
      <subPop id="StudycohortA" number="1"/>
      <subPop id="StudycohortB" number="2"/>
      <subPop id="StudycohortB" number="4"/>
      ....
    </cohorts>

The number must be a power of 2 (i.e. 1, 2, 4, 8, ...). These numbers are used to label the output in the output.txt file.

If only a single subpopulation is listed, the corresponding output set in the output.txt file is identified by multiplying the number by 1000. This increment is then added to the values in the column usually reserved for age Group, corresponding to this subpopulation. The values for the complementary sub-population (those not included) retain the standard values in the age-group column.

Where multiple sub-populations are listed, output is segregated according to all combinations of membership: e.g. if sub-populations A (number=1) and B (number=2) are listed, there will be outputs for "member of A and B"(increment=3000); "member of A but not B"(increment=1000);, "B but not A"(increment=2000); and "not a member of A or B"(increment=0). A corresponding rule is used to assign increment values to all 8 combinations if three sub-populations are listed (each is further segregated by age groups, survey times and enabled output measures, which could lead to excessive program memory usage and output file size if many subpopulations are listed).

Monitoring of cohorts (Versions before 32)

To monitor a defined cohort only, in Versions before 32, cohortOnly is set to "true" as follows:

<monitoring name="(some description or name)" cohortOnly="true">
  <continuous ... />
  <SurveyOptions ... />
  <surveys ... />
  <ageGroup ... />
</monitoring>

cohortOnly works only for versions up to 31 and has been removed in schema version 32. It may be assigned values true or false or, if not using cohorts (with Versions 32 or above), omitted entirely.

If cohortOnly is set to true, then the output of most measures are restricted to the cohort (although not quite all; see the list of measures for details); if it is set to false or omitted, then these measures act on the entire simulated population. To obtain outputs for both the cohort and the entire population prior to version 32 the simulation must be run twice.