Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'Test' categories with total weights each #2

Open
Bakerbakura opened this issue Jul 14, 2020 · 1 comment
Open

'Test' categories with total weights each #2

Bakerbakura opened this issue Jul 14, 2020 · 1 comment

Comments

@Bakerbakura
Copy link
Contributor

So after chatting with Herman about the weights for Stellenbosch, April, and IMO selection, it seems like we're going to go with a scheme wherein, taking the April Camp selection as an example, 50% is allocated to Stellenbosch Senior tests, 30% to monthlies and 20% to the SAMO 3rd round. So I think we should have each test/source of scores being allocated to a category, and the weights specified on a category level, with each category weight divided equally among the tests in that category to calculate the test weights which are used in the objective function.

@dlnnlsn
Copy link
Owner

dlnnlsn commented Jan 11, 2021

I think that the normal way that we think of test weights of course doesn't make sense in the Magic Method context. Usually, we take a weighted sum (or average) of all of the test scores. If test j has weight w_j, then student i's overall score is sum_j (w_j a_{ij}). Now the whole point of the Magic Method is to interpolate scores where they are not available, so we would then replace a_{ij} with s_i d_j, making the student's overall score sum_j (w_j d_j s_i) = (sum_j w_j d_j) s_i. But (sum_j w_j d_j) is just a constant, so ranking students according to this total is then equivalent to ranking them according to the ideal score s_i produced by the Magic Method, so weights in the traditional sense are meaningless.

I think that what Herman's intuition when we say something like "50% is allocated to Stellenbosch Senior tests, 30% to monthlies" is that Senior tests are 5/3 as important as monthlies. If we actually go by 50% as a category for Senior tests that is then divided equally among the tests, then each Stellenbosch test becomes 1/3 as important as the monthlies as a whole. In this case to capture our intuition about what weights mean, I think that every Stellenbosch test should be weighted 50, and every monthly should be weighted 30, so that each Stellenbosch test is 5/3 as important as each given monthly.

On the other hand, if "50% is allocated to Stellenbosch Senior tests, 30% to monthlies" should mean that the total for the Stellenbosch tests is 5/3 as important as the total for the monthlies, then I think that the reasonable interpretation is to divide the 50% among the Stellenbosch tests and weight each one 10%. Though in this case, if we have 3 monthlies that count (approximately like usual), then this division would make each individual Stellenbosch test as important as each individual monthly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants