Create dim_platform model

[rachellougee]

What are the relevant tickets?

#1353

Description (What does it do?)

Creating dim_platform seeded by a csv file

How can this be tested?

dbt build --select +dim_platform

[rachellougee]

@pdpinch Do you have any suggestions on the platform_description?

I grabbed some text from https://openlearning.mit.edu/, but it needs some editing

[KatelynGit]

Should we include OCW?

[KatelynGit]

This looks good to me. One thing is should the platform id be a numeric data type? Usually numeric types are better for id's for size and speed reasons. And does the platform info in the csv match the info coming form the applications?

[rachellougee]

@KatelynGit Thanks for the review.

Should we include OCW?

Sure. I added OCW although we don't include OCW into the combined mart. It's good to have that in place.

One thing is should the platform id be a numeric data type? Usually numeric types are better for id's for size and speed reasons.

I believe we have discussed this in RFC https://github.com/mitodl/hq/pull/5749#discussion_r1805182430
and the adopted decision https://github.com/mitodl/hq/blob/main/rfcs/0004-data-warehouse-dimensional-marts.md#schema-considerations. The hash key makes sense to me as we need a non-numeric primary key for tables like combined users. I thought we were on the same page, but let me know if we need to revisit this discussion.

And does the platform info in the csv match the info coming form the applications?

Can you say more about this? Not sure if I understand what you asking.

[KatelynGit]

@KatelynGit Thanks for the review.

Should we include OCW?

Sure. I added OCW although we don't include OCW into the combined mart. It's good to have that in place.

Thanks for adding OCW.

One thing is should the platform id be a numeric data type? Usually numeric types are better for id's for size and speed reasons.

I believe we have discussed this in RFC mitodl/hq#5749 (comment) and the adopted decision https://github.com/mitodl/hq/blob/main/rfcs/0004-data-warehouse-dimensional-marts.md#schema-considerations. The hash key makes sense to me as we need a non-numeric primary key for tables like combined users. I thought we were on the same page, but let me know if we need to revisit this discussion.

ok yes a hash key makes sense

And does the platform info in the csv match the info coming form the applications?

Can you say more about this? Not sure if I understand what you asking.

I just meant for things like the platform names this new table lists the platform as "MIT xPRO" but in the marts it is displayed differently like "xPro" in marts__combined__orders etc. Do you think we should change it to match what we have in the marts or applications currently?

[rachellougee]

I just meant for things like the platform names this new table lists the platform as "MIT xPRO" but in the marts it is displayed differently like "xPro" in marts__combined__orders etc. Do you think we should change it to match what we have in the marts or applications currently?

Thank you for clarifying that. For clarity, I intended to spell out MIT xPRO instead of xPRO in the new model to match with https://openlearning.mit.edu/. Currently, xPRO in the mart is pulled from a variable defined in https://github.com/mitodl/ol-data-platform/blob/main/src/ol_dbt/dbt_project.yml#L36.

Unless we hardcoded the platform name somewhere on the superset dataset, this new value should flow to the mart and dashboard without any side effects. I can match it to what we currently have if needed. I believe these platform names are just for display and filtering.

[KatelynGit]

I just meant for things like the platform names this new table lists the platform as "MIT xPRO" but in the marts it is displayed differently like "xPro" in marts__combined__orders etc. Do you think we should change it to match what we have in the marts or applications currently?

Thank you for clarifying that. For clarity, I intended to spell out MIT xPRO instead of xPRO in the new model to match with https://openlearning.mit.edu/. Currently, xPRO in the mart is pulled from a variable defined in https://github.com/mitodl/ol-data-platform/blob/main/src/ol_dbt/dbt_project.yml#L36.

Unless we hardcoded the platform name somewhere on the superset dataset, this new value should flow to the mart and dashboard without any side effects. I can match it to what we currently have if needed. I believe these platform names are just for display and filtering.

ok that sounds good! I'll go in and approve