-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Major update of Care Home script #945
Conversation
see document on sharepoint for description also: - added in type of admission description - updated care home contact in fill_ch_name script
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks ok to me. Happy to merge it when it is ready.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
@check-spelling-bot Report🔴 Please reviewSee the 📂 files view, the 📜action log, or 📝 job summary for details. Unrecognized words (15)
Previously acknowledged words that are now absentadpe canx ccyy cnws comhairle lcho matern mcbride ppas returnsthe smra 🫥To accept these unrecognized words as correct and remove the previously acknowledged and now absent words, you could run the following commands... in a clone of the [email protected]:Public-Health-Scotland/source-linkage-files.git repository curl -s -S -L 'https://raw.githubusercontent.com/check-spelling/check-spelling/main/apply.pl' |
perl - 'https://github.com/Public-Health-Scotland/source-linkage-files/actions/runs/10283303502/attempts/1' OR To have the bot accept them for you, reply quoting the following line: Available 📚 dictionaries could cover words (expected and unrecognized) not in the 📘 dictionaryThis includes both expected items (307) from .github/actions/spelling/expect.txt and unrecognized words (15)
Consider adding them (in with:
extra_dictionaries:
cspell:fullstack/dict/fullstack.txt
cspell:k8s/dict/k8s.txt
cspell:php/dict/php.txt
cspell:node/dict/node.txt
cspell:npm/dict/npm.txt To stop checking additional dictionaries, add (in check_extra_dictionaries: '' Pattern suggestions ✂️ (1)You could add these patterns to
Errors (2)See the 📂 files view, the 📜action log, or 📝 job summary for details.
See ❌ Event descriptions for more information. If the flagged items are 🤯 false positivesIf items relate to a ...
|
Pushed a few changes for the deaths methodology. Ready to merge |
In the care home methodology update #945 the cis markers were working but not for episodes nested within a stay, it would wrongly assign the 'next' episode counter. This was due to the lag function assigning NA to the first row. This update fixes the nested cases and i think it is working properly now for all scenarios. Where the first row is NA for `previous_discharge_date` this will use the `ch_discharge_date` which will be the first episode. We also use the following code, adapted from Social care scripts to create the cis markers. This simplifies the code and seems to work for all scenarios. e.g. when the first episode linked to a chi is NA and the next two episodes are continuous, this will return 1,2,2 correctly. Previously this would return 1,2,3. ``` mutate(continuous_marker = accumulate(new_episode[-1], .init = 1, ~ if (.y == "Y") {.x + 1} else {.x})) ```
* update documentation * Update sc connection name * Update documentation * 936 - Update parameters with file paths (#939) Specify file paths in sc function parameters * remove and merge overlapping records in GP OoHs * Style code * update spelling to lowercases * update spelling * Create tests for social care sandpit extracts (#943) * Update `write_tests_xlsx` * Update documentation * Add in sandpit tests where the extract is saved * Setup tests for sandpit Further checks needed for writing to disk * Update documentation * Amend case_when statement * rename function to include 'sc' * Update documentation * Use `is.null` instead of `missing` * Update documentation * Add `year` as a parameter * Update documentation * Setup for writing sandpit tests to disk * Update parameters for sandpit tests * Update documentation * Use `process_tests_sc_sandpit` * Apply styling * Style code * update documentation Co-authored-by: Zihao Li <[email protected]> * Rename variable sc_id Co-authored-by: Zihao Li <[email protected]> * Rename variable Co-authored-by: Zihao Li <[email protected]> * Rename variable Co-authored-by: Zihao Li <[email protected]> * Update documentation * [check-spelling] Update metadata Update for https://github.com/Public-Health-Scotland/source-linkage-files/actions/runs/8689503990/attempts/1 Accepted in #943 (comment) Signed-off-by: check-spelling-bot <[email protected]> on-behalf-of: @check-spelling <[email protected]> * update spelling * update spelling expect variant --------- Signed-off-by: check-spelling-bot <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Zihao Li <[email protected]> Co-authored-by: Zihao Li <[email protected]> * only select columns we want in ltc raw data * [check-spelling] Update metadata Update for https://github.com/Public-Health-Scotland/source-linkage-files/actions/runs/8897746003/attempts/1 Accepted in #947 (comment) Signed-off-by: check-spelling-bot <[email protected]> on-behalf-of: @check-spelling <[email protected]> * for some reason the latest scid code was overwritten after the march update?? anyway, now it is fixed. * Style code * Merge May24 NI update into June update branch (#949) Collect data before manipulations * Update NEWS.md * link GP-OoH with CUP markers * Style code * update gp ooh cup * link cup to acute * Update documentation * adding the death dates to activity after death cases (#972) adding the death dates to the cases where there is activity after death Co-authored-by: marjom02 <[email protected]> * Add sys time to functions (#971) * adding in syst_time alerts for all functions in create episode and create individual. So that when it runs manually I can see where it is and where it's getting stuck * Style code --------- Co-authored-by: marjom02 <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: Jennit07 <[email protected]> * Update slf deaths lookup function name * automate combined deaths lookup * Update documentation * Update targets script * Update years * Update running process manually * re-arrange brackets * Style code * Update run targets scripts * Set up all-targets script * Style code * Update documentation * Update targets script * Update years * Update running process manually * re-arrange brackets * Style code * Update run targets scripts * Set up all-targets script * Style code * Style code * Automate the combined slf deaths lookup (#973) Closes #957 and #968 * Update documentation * Update Run_SLF_Files_targets/run_all_targets.R Co-authored-by: Zihao Li <[email protected]> * Style code * remove combined_deaths_lookup from targets * Style code * fix acute_cup and gp_ooh_cup paths * Update documentation * fix typo * adapt acute_cup for anon_chi * Style code * minor changes * Style code * Update documentation * Person id sds (#981) * added back in missing person_id for SDS. also added latest_flag back in to client lookup in targets * Update documentation * added back in missing person_id for SDS. also added latest_flag back in to client lookup in targets * Update documentation * change as suggested by Jen * Update documentation --------- Co-authored-by: marjom02 <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: Jennifer Thom <[email protected]> Co-authored-by: Jennit07 <[email protected]> * Update NEWS.md * unify file names for cup files * Add client flags (#979) * New methodology for social care client data. - removed code that wasn't needed. - updated housing codes - latest social care ID - changed "mental health problems" to "mental health disorders" in line with PHS style guide * Update documentation * Style code * add person ID to client so it carries through to match on to all cases * New methodology for social care client data. - removed code that wasn't needed. - updated housing codes - latest social care ID - changed "mental health problems" to "mental health disorders" in line with PHS style guide * Update documentation * Style code * add person ID to client so it carries through to match on to all cases --------- Co-authored-by: marjom02 <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: Jennifer Thom <[email protected]> Co-authored-by: Jennit07 <[email protected]> * Update lookup to use anon-chi * Remove redundant code This uses the NRS Weekly dates and if this is blank use the chi death date. This methodology is wrong. We want to use the monthly nrs boxi date by default and chi date if there is an issue * Update documentation * remove weekly nrs date variable * Use boxi nrs date or chi death date * Use `get_combined_slf_deaths_path` * add catch for NAs * add notes * Fix typo * remove redundant code * Style code * add a function of combine nrs and it_chi death * Update documentation * Style code * minor changes * remove process_slf_deaths_lookup * Update documentation * Major update of Care Home script (#945) * # major changes to care home script see document on sharepoint for description also: - added in type of admission description - updated care home contact in fill_ch_name script * minor note updates * Style code * Update documentation * couple of note updates * Update R/process_sc_all_care_home.R Co-authored-by: Jennit07 <[email protected]> * Update R/process_sc_all_care_home.R Co-authored-by: Jennit07 <[email protected]> * Style code * Update R/process_sc_all_care_home.R * change to ch name lookup * Update documentation * remove fill ch provider fill line * update fill ch names so it works with new ch methodology * Style code * Update documentation * Style code * Update documentation * Remove redundant variable `latest_sc_id` * use slfhelper::get_chi * new section for sc_ch_id_markers * Style code * Update documentation * Remove extra text and white space * add rename to use death_date_chi * use `read_excel` function * Update documentation * Return the paths only for SPD and ch name * Update documentation * Remove rename - no longer needed * fix typo * remove variables that dont exist --------- Co-authored-by: marjom02 <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Jennifer Thom <[email protected]> * update `ch_chi_cis` methodology * update `ch_sc_id_cis` methodology * Update notes * Use `right_join` * Update process_sc_all_care_home.R added in missing variable at the end * Add new nsu (#991) * Add NSU code to github Includes extracting the service user cohort to send to the chili team and then NSU extraction. * Style code * Add compression and package library * Style code * pick up latest geography file, and save out with compression (#983) * pick up latest geography file, and save out with compression * Style code * use `get_spd_path` Co-authored-by: James McMahon <[email protected]> --------- Co-authored-by: marjom02 <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: James McMahon <[email protected]> --------- Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Megan McNicol <[email protected]> Co-authored-by: marjom02 <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: James McMahon <[email protected]> * update reference * Reduce dependencies (#984) * removing packages that I don't think get used anywhere. and removing references to fst and spss files * Update documentation * Update authors in description * [check-spelling] Update metadata Update for https://github.com/Public-Health-Scotland/source-linkage-files/actions/runs/10469297653/attempts/1 Accepted in #984 (comment) Signed-off-by: check-spelling-bot <[email protected]> on-behalf-of: @check-spelling <[email protected]> * [check-spelling] Update metadata Update for https://github.com/Public-Health-Scotland/source-linkage-files/actions/runs/10469690723/attempts/1https://github.com/Public-Health-Scotland/source-linkage-files/actions/runs/10469690723/attempts/1 Accepted in #984 (comment) Signed-off-by: check-spelling-bot <[email protected]> on-behalf-of: @check-spelling <[email protected]> --------- Signed-off-by: check-spelling-bot <[email protected]> Co-authored-by: marjom02 <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: Jennifer Thom <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Jennit07 <[email protected]> * minor changes to social care code * Style code * Update process_sc_all_care_home.R * Update NEWS.md * merge Sep2024 fix into sep24 branch (#1003) * update write_tests_xlsx * update process_refined_death * fix tests by removing get_chi * add 2425 * Style code * fix NA matches in refined_death * move latest_cost_year() to cost_uplift() * improve automation * Update documentation * fix `cij_ppa` in DD data * fix bugs of dd and populate cij_delay back to episodes * Style code * keep all variable for delayed discharge episodes * remove dummy variable names from dd_date * Style code * remove `deceased_boxi` variable - bug * remove `create_person_id`. Its matched in client * remove `create_person_id` * Update `run_slf_manually` scripts * further remove person_id * fix duplicate row introduced by adding death * remove duplicated chi when joining death data * TODO: check distinct death data by chi while keeping chi==NA records * add parameter for year * fix duplicate in add_activity_after_death_flag * Update `check_year_valid` * Declare DN variables * Style code * remove redundant variables --------- Co-authored-by: Zihao Li <[email protected]> Co-authored-by: lizihao-anu <[email protected]> Co-authored-by: Jennit07 <[email protected]> * update copy_to_hscdiip.R * Update older years to bring the data in line with our newest processes. (#988) * fix sc_client_lookup sc_send_lca * fix an issue of get_pop_path * Style code * fix the rest of get_pop_path from get_datazone_pop_path * Update documentation * fix sc_send_lca * add missing year column * Remove redundant code * Update documentation * Style code * explicitly specify the argument year to avoid corruption of targets * Update documentation * Reorder when we match on client variables This was causing NSUs to show a social care id. This now resolves this. * Update documentation * Style code * Add chi parameter to `create_demog_test_flags` * Style code * Use CHI parameter for ep/indiv tests * Use CHI parameter for extract tests (chi) * Change test sheet names to lowercase * Change date to lowercase * Update documentation * new data pipeline with targets remove create_individual_files from targets and append it to run_targets script * minor changes * Style code * Update documentation * Update documentation * Style code * undo sc_send_lca bit * Add code for running years available * Update `_targets.R` script for running old years * Style code * Update `check_year_valid` for running old years * Use `check_year_valid` where no data for old yrs * Style code * Fix pick variables This was not taking the correct variables, leading to NSUs being assigned psychiatry * SC Demographics and SDS (#900) * Style code * # read in sc demographics different variables - removed extract date as not accurate, using chi over upi after discussion with social care data management. Added in date of death just for fun. * social care demographics first draft removed a lot of the submitted variables and instead using chi variables from chi seeding. Other changes: - Fill in missing values, - create flag for latest social care id (one from database is not accurate), this makes sure that each chi only has ONE sc id as the latest to stop it creating duplicates - change postcode to choose chi over submitted * Style code * had a github error? Not sure what happened but commiting first draft of sc demographics * Style code * first draft sds. No major changes - only how demographics is matched on and how latest social care id is selected * Update documentation * demographics - add sending location to group by * Style code * Update documentation * Added ungroup() * Remove comments * Remove comments * Style code --------- Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: marjom02 <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Zihao Li <[email protected]> * Sc all at speedup (#904) * speed up process_sc_all_alarms_telecare function with data.table package * Update documentation --------- Co-authored-by: lizihao-anu <[email protected]> Co-authored-by: Megan McNicol <[email protected]> Co-authored-by: Jennit07 <[email protected]> * Add case_when statement for `high_cc` cohort * Bug - `high_cc` in demographic cohort showing `NAs` instead of `TRUE/FALSE` (#911) Add case_when statement for `high_cc` cohort * added a casewhen to update property type description for homelessness * Update documentation * Style code * Bug - deal with missing variables (#914) * Add missing sc variables for no sc data * Fix code for including `_inc_dna` variables * Remove commented line * Bug - Fix get pop path failing and preventing the indiv file from running. (#913) Fix bug - pop file paths breaking indiv file * correct file hscp file path * Declare missing variables for older years * setup targets scripts for old years * Style code * Include `check_year_valid` for sc client path * Add check year valid to join sc client * Add if else statement * WIP - TO DO - fix dummy path for `get_chi()` * Style code * update dummy data file to read empty tibble * Update `check_year_valid` * Update declared `NA` variables * Update documentation * declare `count_not_known` as NA * supply year as default in `aggregate_by_chi` * Decalre unused variables * Style code * Update sc client with sept update new code * Specify code for running older years * Style code * Add Running SLF files manually scripts * Style code * update write_tests_xlsx * update process_refined_death * fix tests by removing get_chi * add 2425 * Style code * fix NA matches in refined_death * move latest_cost_year() to cost_uplift() * improve automation * Update documentation * fix `cij_ppa` in DD data * fix bugs of dd and populate cij_delay back to episodes * Style code * keep all variable for delayed discharge episodes * remove dummy variable names from dd_date * Style code * remove `deceased_boxi` variable - bug * remove `create_person_id`. Its matched in client * remove `create_person_id` * Update `run_slf_manually` scripts * further remove person_id * fix duplicate row introduced by adding death * remove duplicated chi when joining death data * TODO: check distinct death data by chi while keeping chi==NA records * add parameter for year * fix duplicate in add_activity_after_death_flag * Update `check_year_valid` * Declare DN variables * Style code * Declare client variables * remove extra dd variables * remove redundant variables * remove fy variable * Remove redundant variable `count_not_known` * Remove duplicate code * revert commit - remove fy * update manual run * declare missing sc variables indiv file * Style code --------- Co-authored-by: Zihao Li <[email protected]> Co-authored-by: lizihao-anu <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Megan McNicol <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: Zihao Li <[email protected]> Co-authored-by: marjom02 <[email protected]> * Update `replace_sc_id_with_latest` function * Style code --------- Signed-off-by: check-spelling-bot <[email protected]> Co-authored-by: Jennit07 <[email protected]> Co-authored-by: Zihao Li <[email protected]> Co-authored-by: Zihao Li <[email protected]> Co-authored-by: lizihao-anu <[email protected]> Co-authored-by: marjom02 <[email protected]> Co-authored-by: Megan McNicol <[email protected]> Co-authored-by: SwiftySalmon <[email protected]> Co-authored-by: rchlv <[email protected]> Co-authored-by: James McMahon <[email protected]>
lots of changes to the care home processing script. Details in the document on sharepoint. Needs thouroughly checked!!!