About pivot_wider/longer functions

[msquiroga89]

Hi!

I would like to raise a question about the way pivot_wider and pivot_longer are presented in the R for Social Sciences lesson. It can be a very challenging set of functions, but the fact that we first have to separate rows and create logical variables can be a distracting moment for learners. The transformation needed is difficult, and maybe it’s not easily understandable why we need to do that.

We talked about it in the CAC, and since there seem to be other aspects of the dataset to be discussed, we thought that it could be a good opportunity to include a different approach to these functions, like chosing variables that don't need a previous transformation.

Thanks!

[juanfung]

@msquiroga89 thank you that's great feedback! would you be interested in submitting a PR or providing some suggestions to improve the illustration of the two functions?

[juanfung]

Per discussion with @eirini-zormpa perhaps make this section optional, in addition to improving the example

[bbartholdy]

This is also something we have discussed in our group, since we rarely have time to make it through all of the code in Data Wrangling with tidyr. And this is a problem since the last part is crucial for continuing with Data Visualisation.

I think the tidyr part could be trimmed down by removing the example of making the respondent_wall_type wider (it's unrealistic that you would want to do this anyway). Instead add a step-by-step walkthrough of pivoting items_owned wider. Then maybe add an exercise to do the same to months_lack_food just to make sure the process is clear to the learners.
Pivoting months_lack_food could also be dropped completely since it's not used in subsequent episodes.

[juanfung]

@bbartholdy these are great suggestions! i have assigned this issue to myself, but if you are willing to submit a PR with these suggestions, it would be super appreciated :)

[bbartholdy]

@juanfung sure! I'll see what I can come up with. On this branch https://github.com/bbartholdy/r-socialsci/tree/tidyr-adjust (for anyone interested).

[msquiroga89]

Hi, sorry I disappeared, I got caught up in my work. I would like to collaborate with this section of the lesson, are you developing an alternative?

[bbartholdy]

Hi @msquiroga89, no worries! Yes, I am working on modifications to the episode. It's tricky because it still needs to align with the next episode on data visualisation. My plan was to add a step-by-step walkthrough of pivoting items_owned wider, instead of respondent_wall_type (since it's doesn't really make sense and actually makes the data 'untidy') . items_owned is the only modified variable that is actually used in a later episode (as far as I can tell). It might also be nice to add an exercise, maybe on months_lack_food.

Let me know what your thoughts are.

[juanfung]

@bbartholdy @msquiroga89 just checking in on this issue...let us (maintainers) know if you need any help!

[bbartholdy]

@bbartholdy @msquiroga89 just checking in on this issue...let us (maintainers) know if you need any help!

Thanks @juanfung! I hit a bit of a busy period but should be able to pick it up again after next week.

[juanfung]

@bbartholdy no worries, that sounds good and your contributions are much appreciated!

[bbartholdy]

I have reorganised the episode as discussed above. I had to deviate a bit from the original plan in order for the episode to make sense (bbartholdy@a9182ce).

To summarise:

• The transformation of respondent_wall_type has been removed
• There is, instead, a step-by-step guide to transform items_owned
• The transformed items_owned data frame is used to demonstrate pivot_longer
• Exercises now involve creating summary tables from the longer data frame (to demonstrate how it differs from the wider format)
• Creating interviews_plotting is now directly integrated in the episode (before it seemed more like an afterthought)

Still needed:

• The figures showing the reshaping of respondent_wall_type need to be updated to depict items_owned
• ??? an optional exercise at the end could be going back to the original data frame. This would involve introducing reframe()

I wanted to get some input on the new layout before changing the figures (@juanfung , @msquiroga89 ?).

[juanfung]

@bbartholdy thank you for the incredible amount of work you've done on this! apologies for not replying sooner, but i finally had a chance to serve your changes locally and have two comments:

• this is a fantastic new layout and i support modifying the accompanying figures
• when i serve locally, i am getting the following error:

Error in separate_longer_delim(., items_owned, delim = ";"): could not find function "separate_longer_delim"

i checked and i have tidyr 1.3.0 installed, so i'm puzzled. i'm not well versed in how workbench works, and whether there is a local environment enforcing package versions (but if there were, i wouldn't be getting an error). just flagging to make sure you double check this all works as intended. i am guessing it's a local problem on my end, but can't figure it out

[bbartholdy]

It sounds like a local problem, I didn't have any issues, including on a fresh Ubuntu install. I know there were problems with some previous versions of {renv}, so maybe worth updating? It might also be an issue with the way the workbench uses {renv}, but I also don't really know enough about it.