Skip to content

Latest commit

 

History

History
35 lines (26 loc) · 5.64 KB

Randomization.md

File metadata and controls

35 lines (26 loc) · 5.64 KB

There are three different levels where randomization can be applied (see below)

All randomization is seeded with the participant id, so with the same participant ID, you will always see the same order. Also, the study phase index is used, so two exactly identical study phases (following each other) will have different orders for the same participant.

Study Factor

  • Type: Within vs. Between:
    • Within: participants see all levels of this factor
    • Between: participants only see one level of this factor each
  • Mixing Order:
    • RandomOrder: Using Balanced Latin Squares to balance orders between participants, so every participant sees another order of the conditions. If all factors are RandomOrder orders are repeated after [product of all factor level] many participants, so the number of participants should ideally be a multiple of that (e.g., for a 2x3 design [one factor with 2 levels, and a second with 3 levels] it should be a multiple of 6).
    • En Block: All conditions with the same level of this factor will be shown en block so right after each other (at most one factor can be En Block).
    • In Order: The order of the factors' levels will not be randomized (Latingquare) between subjects. So all subjects see this factor's levels in the order specified for the factor.
    • ⚠️ Careful when using En Block and In Order together: En Block is considered first and then the In Order factors are kept in there given order. Note that the order of defining the factors matters, e.g., when using multiple In Order factors (see Randomization/Examples for clarification).
  • Non-combined: This means that this factor is not used to generate the different conditions, but rather as randomness in the task or repetitions. So it is not combined with the other factors but just randomized in parallel. For example, if the task is drawing different animals and other factors like brush size and drawing method should be examined, the type of animal to draw should be non-combined so that ideally each condition (combination of brush size and drawing technique) is evaluated with another animal for all participants to avoid effects of some animals being easier to draw. non-combined factors are either
    • Random using a random sequence (seeded with the participantID) to avoid confounding with other LatinSquare randomized factors
    • or InOrder, so that they are potentially perfectly randomly mapped to the conditions if those use LAtinSquare randomization. The only caveat is that the order of the levels of this factor is always the same, which should be carefully considered.
    • levels are potentially repeating if more than given are needed for the conditions computed from the other factors.
    • At least one factor per phase has to not be non-combined!
  • Repetitions: if conditions should be repeated multiple times, it is easiest to add an additional Factor, e.g. Repetition with Levels {1,2,3} for three repetitions, and then use Random (repetitions are done fully random), EnBlock (the second repetitions are only begun once all conditions were seen once, and so on), or InOrder (e.g., in combination with another EnBlock Factor of which all repetitions are shown right after each other). Examples of this can be found in Randomization/Examples. In case more involved repetition patterns are required, the ConditionSortingCallback() function should be used (see below).

Study Setup

  • PhasesToOrderRandomize can be used to specify phase names of phases that should be randomized in order between participants. So if a study has the phases Warmup, Phase1, Break, Phase2 and PhasesToOrderRandomize = { Phase1, Phase2} is given. Then participants will alternately see orders Warmup, Phase1, Break, Phase2 and Warmup, Phase2, Break, Phase1.
  • ConditionSortingCallback(): this is a function in the ASFStudySetup class, that in its default implementation does nothing. It can be overridden in C++ (currently blueprints are not supported for this, since that complicated matters a lot). The function receives an TArray<USFCondition*> (containing the order of condition as created by all specification possibilities mentioned above) and should return a reordered version of this. You can simply change the order of all of these conditions. However, removing or adding entries can have undefined behavior! Also ParticipantRunningNumber (from 0 to the number of participants) is provided, which can be used to implement counterbalancing.

Number of Participants

Since randomization is to avoid order effects you should make sure that all orders are seen the same number of times for the best possible statistic validity. That means:

  • The number of participants should be a multiple of the number of levels of all between-subjects factors
  • The number of participants should be a multiple of the number of conditions (per phase). Excluding en block factors since they reduce the number of orders shown). So, e.g., with 2 random factors with 2 and 3 levels respectively, the number of participants should be a multiple of 6.
  • If you are unsure. Use the Generate Test Study Runs button of the Study Setup (in the Study Setup Debug section) and then check in the generated runs (StudyFramework/StudyRuns) after which the number of participants the orders repeat.