Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: scoring for multistep ORA configuration #6

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

mariajgrimaldi
Copy link
Member

@mariajgrimaldi mariajgrimaldi commented Sep 15, 2023

TL;DR - This PR implements behind a systems feature flag a grading strategy of accumulating scores obtained on each workflow step.

The changes I have made are intended to allow a grade that accumulates the final score of each step each time the grade changes. That is when the instructor grades the student, then the student's final grade is the sum of the steps graded so far (for now only staff). When the peers grade my submission, then the student's final grade is the sum of the steps graded so far (staff + peer). Thus, we do not modify how the score is calculated, but what the final score is.

What changed?

Now, the accumulative scoring strategy can be turned on/off system-wide. When off, the ORA scoring system works as usual.

Developer Checklist

Testing Instructions

  1. Install this version into your environment.

  2. Turn on the feature flag for the LMS:
    FEATURES['ENABLE_ACCUMULATIVE_GRADING'] = True

  3. Configure your ORA with the default criteria

  4. Configure the ORA steps as follows:
    image

  5. Now try replicating the following:

  • As student1, I submit my essay.
  • As student2, I submit my essay and grade student1's essay.
  • As instructor, I grade student2's essay.
  • As student2, I see my grade. If my instructor in criteria 1 gave me 5/5 and in criteria 2 gave me 1/5, I will have 6/10 and I will get a message indicating that I am waiting for a grade from my peers.
  • As student1, I grade student2's essay.
  • As student2, I check my grades. If the median of my peers gives 3/5 for criterion 1, and for criterion 2 is 4/5, then I will have:
    5 (median peer) + 5 (staff) = 10/10.
    3 (median peer) + 1 (staff) = 4/10
    And the total: 14/20
  • As instructor, I grade student1's essay.
  • As student1, I check my grade:
    3 (median peer) + 0 (staff) = 3/10.
    3 (median peer) + 1 (staff) = 4/10
    And the total: 7/20

Reviewer Checklist

Collectively, these should be completed by reviewers of this PR:

  • I've done a visual code review
  • I've tested the new functionality

@mariajgrimaldi mariajgrimaldi changed the title feat: modify current get_score to always return the accumulation feat: scoring for multistep ORA configuration Sep 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant