Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test rolling SF sale-price feature #95

Closed
wants to merge 26 commits into from
+477 −133
Closed

Test rolling SF sale-price feature #95

Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
83f3212
First commit
wrridgeway Jan 28, 2025
ef9f870
Linting
wrridgeway Jan 28, 2025
54c9dcb
Linting
wrridgeway Jan 28, 2025
25f7846
Re-knit readme
wrridgeway Jan 28, 2025
8d8fef0
Linting
wrridgeway Jan 28, 2025
a279ccc
Linting
wrridgeway Jan 28, 2025
bdebe0a
Commenting
wrridgeway Jan 28, 2025
abc5b52
Update dvc
wrridgeway Jan 28, 2025
c41e13e
Fill NAs in training set
wrridgeway Jan 29, 2025
729f171
Fix assess fill
wrridgeway Jan 29, 2025
8b10c39
Commenting
wrridgeway Jan 29, 2025
66b6f42
Use time-weighted for interpolation
wrridgeway Jan 29, 2025
d27a7fe
Lint
wrridgeway Jan 29, 2025
a324ae4
Remove NA filling
wrridgeway Jan 29, 2025
f64d667
Merge branch '2025-assessment-year' into 94-test-rolling-sf-sale-pric…
wrridgeway Jan 29, 2025
480b45a
Update dvc
wrridgeway Jan 29, 2025
0a0eec2
Improve commenting, data table handling
wrridgeway Jan 30, 2025
2448188
Update data
wrridgeway Jan 30, 2025
adebd76
Merge branch '2025-assessment-year' into 94-test-rolling-sf-sale-pric…
wrridgeway Feb 4, 2025
536a288
Update dvc data
wrridgeway Feb 4, 2025
769f1d3
Remove strata for testing
wrridgeway Feb 4, 2025
835acf0
Reknit without strata
wrridgeway Feb 4, 2025
3834659
Add strata back
wrridgeway Feb 4, 2025
3d3ac5f
Add back strata
wrridgeway Feb 4, 2025
49f7ea6
Merge branch '2025-assessment-year' into 94-test-rolling-sf-sale-pric…
wrridgeway Feb 4, 2025
7456b03
Update dvc
wrridgeway Feb 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions R/setup.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ suppressPackageStartupMessages({
# Resolve package namespace conflicts, preferring the library::function pair
# shown over other functions with the same name from different libraries
conflicts_prefer(
data.table::`:=`,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inconsistently got an error that data.table and rlang were conflicting.

dplyr::filter,
dplyr::first,
dplyr::lag,
Expand Down
23 changes: 23 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,15 @@ ones used in the most recent assessment model.
| Total Condominium Building Non-Livable Parcels | char_building_non_units | Count of non-livable 14-digit PINs | Characteristic | numeric | X |
| Condominium Building Is Mixed Use | char_bldg_is_mixed_use | The 10-digit PIN (building) contains a 14-digit PIN that is neither class 299 nor 399 | Characteristic | logical | X |
| Total Condominium Building Square Footage | char_building_sf | Square footage of the *building* (PIN10) containing this unit | Characteristic | numeric | X |
| Building Square Footage | char_building_sf | Square footage of the *building* (PIN10) containing this unit | Characteristic | numeric | X |
| Condominium Unit Square Footage | char_unit_sf | Square footage of the condominium unit associated with this PIN | Characteristic | numeric | X |
| Unit Square Footage | char_unit_sf | Square footage of the condominium unit associated with this PIN | Characteristic | numeric | X |
| Condominium Unit Bedrooms | char_bedrooms | Number of bedrooms in the building | Characteristic | numeric | X |
| Bedrooms | char_bedrooms | Number of bedrooms in the building | Characteristic | numeric | X |
| Condominium Unit Half Baths | char_half_baths | Number of half baths | Characteristic | numeric | X |
| Half Baths | char_half_baths | Number of half baths | Characteristic | numeric | X |
| Condominium Unit Full Baths | char_full_baths | Number of full bathrooms | Characteristic | numeric | X |
| Full Baths | char_full_baths | Number of full bathrooms | Characteristic | numeric | X |
| Condominium % Ownership | meta_tieback_proration_rate | Proration rate applied to the PIN | Meta | numeric | X |
| Condominium Building Strata 1 | meta_strata_1 | Condominium Building Strata - 10 Levels | Meta | character | X |
| Condominium Building Strata 2 | meta_strata_2 | Condominium Building Strata - 100 Levels | Meta | character | X |
Expand All @@ -135,6 +140,24 @@ ones used in the most recent assessment model.
| Average Daily Traffic Count on Nearest Collector Road | prox_nearest_road_collector_daily_traffic | Daily traffic of nearest collector road | Proximity | numeric | X |
| Nearest New Construction (Feet) | prox_nearest_new_construction_dist_ft | Nearest new construction distance (feet) | Proximity | numeric | X |
| Nearest Major Stadium (Feet) | prox_nearest_stadium_dist_ft | Nearest stadium distance (feet) | Proximity | numeric | X |
| NA | time_sale_roll_mean_nbhd_sf_t0_w1 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_sf_t0_w2 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_sf_t0_w3 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_sf_t1_w1 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_sf_t1_w2 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_sf_t1_w3 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_sf_t2_w1 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_sf_t2_w2 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_sf_t2_w3 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t0_w1 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t0_w2 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t0_w3 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t1_w1 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t1_w2 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t1_w3 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t2_w1 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t2_w2 | | NA | NA | X |
| NA | time_sale_roll_mean_nbhd_condo_t2_w3 | | NA | NA | X |
| Percent Population Age, Under 19 Years Old | acs5_percent_age_children | Percent of the people 17 years or younger | ACS5 | numeric | |
| Percent Population Age, Over 65 Years Old | acs5_percent_age_senior | Percent of the people 65 years or older | ACS5 | numeric | |
| Median Population Age | acs5_median_age_total | Median age for whole population | ACS5 | numeric | |
Expand Down
23 changes: 23 additions & 0 deletions docs/data-dict.csv
View file
Open in desktop
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can fill this in if we plan to move forward with the rolling means.

Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,15 @@ Total Condominium Building Livable Parcels,char_building_units,Count of livable
Total Condominium Building Non-Livable Parcels,char_building_non_units,Count of non-livable 14-digit PINs,Characteristic,numeric,TRUE
Condominium Building Is Mixed Use,char_bldg_is_mixed_use,The 10-digit PIN (building) contains a 14-digit PIN that is neither class 299 nor 399,Characteristic,logical,TRUE
Total Condominium Building Square Footage,char_building_sf,Square footage of the _building_ (PIN10) containing this unit,Characteristic,numeric,TRUE
Building Square Footage,char_building_sf,Square footage of the _building_ (PIN10) containing this unit,Characteristic,numeric,TRUE
Condominium Unit Square Footage,char_unit_sf,Square footage of the condominium unit associated with this PIN,Characteristic,numeric,TRUE
Unit Square Footage,char_unit_sf,Square footage of the condominium unit associated with this PIN,Characteristic,numeric,TRUE
Condominium Unit Bedrooms,char_bedrooms,Number of bedrooms in the building,Characteristic,numeric,TRUE
Bedrooms,char_bedrooms,Number of bedrooms in the building,Characteristic,numeric,TRUE
Condominium Unit Half Baths,char_half_baths,Number of half baths,Characteristic,numeric,TRUE
Half Baths,char_half_baths,Number of half baths,Characteristic,numeric,TRUE
Condominium Unit Full Baths,char_full_baths,Number of full bathrooms,Characteristic,numeric,TRUE
Full Baths,char_full_baths,Number of full bathrooms,Characteristic,numeric,TRUE
Condominium % Ownership,meta_tieback_proration_rate,Proration rate applied to the PIN,Meta,numeric,TRUE
Condominium Building Strata 1,meta_strata_1,Condominium Building Strata - 10 Levels,Meta,character,TRUE
Condominium Building Strata 2,meta_strata_2,Condominium Building Strata - 100 Levels,Meta,character,TRUE
Expand All @@ -25,6 +30,24 @@ Average Daily Traffic Count on Nearest Arterial Road,prox_nearest_road_arterial_
Average Daily Traffic Count on Nearest Collector Road,prox_nearest_road_collector_daily_traffic,Daily traffic of nearest collector road,Proximity,numeric,TRUE
Nearest New Construction (Feet),prox_nearest_new_construction_dist_ft,Nearest new construction distance (feet),Proximity,numeric,TRUE
Nearest Major Stadium (Feet),prox_nearest_stadium_dist_ft,Nearest stadium distance (feet),Proximity,numeric,TRUE
NA,time_sale_roll_mean_nbhd_sf_t0_w1,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_sf_t0_w2,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_sf_t0_w3,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_sf_t1_w1,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_sf_t1_w2,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_sf_t1_w3,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_sf_t2_w1,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_sf_t2_w2,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_sf_t2_w3,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t0_w1,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t0_w2,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t0_w3,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t1_w1,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t1_w2,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t1_w3,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t2_w1,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t2_w2,,NA,NA,TRUE
NA,time_sale_roll_mean_nbhd_condo_t2_w3,,NA,NA,TRUE
"Percent Population Age, Under 19 Years Old",acs5_percent_age_children,Percent of the people 17 years or younger,ACS5,numeric,FALSE
"Percent Population Age, Over 65 Years Old",acs5_percent_age_senior,Percent of the people 65 years or older,ACS5,numeric,FALSE
Median Population Age,acs5_median_age_total,Median age for whole population,ACS5,numeric,FALSE
Expand Down
16 changes: 8 additions & 8 deletions dvc.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ stages:
deps:
- path: pipeline/00-ingest.R
hash: md5
md5: f758cc2d2c8dbe928806ffb0a46ab821
size: 24134
md5: 5e5a7064ad082589edb27a08497ecc68
size: 33538
params:
params.yaml:
assessment:
Expand All @@ -31,12 +31,12 @@ stages:
outs:
- path: input/assessment_data.parquet
hash: md5
md5: b1462cc55efa7d8beb5ec2af9a649a9b
size: 76103136
md5: 4e45e642eb57513bf2e9e8e5aa7062df
size: 78834247
- path: input/char_data.parquet
hash: md5
md5: 09a842b0910fa84c9fa7834593ee488c
size: 149301395
md5: 8d16027948929f5944f5db86b5b43b7f
size: 148499762
- path: input/condo_strata_data.parquet
hash: md5
md5: ded3ecde590af57e6b98a8935fae0215
Expand All @@ -47,8 +47,8 @@ stages:
size: 6019
- path: input/training_data.parquet
hash: md5
md5: ef87ceb9be93d8ae85118991ab5269f2
size: 76713007
md5: 936eea2a250d1222936eb9f0f81c9d84
size: 100482987
train:
cmd: Rscript pipeline/01-train.R
deps:
Expand Down
Loading