Tesla internship validation issue

[johndpjr]

Context

I've noticed that when scraping Tesla, some invalid attributes get written to the db. Sometimes, the Tesla internship title gets put into the period and year. See image below.

TODO

• Fix?

Notes

[JeremyEastham]

I know why this is happening. When the regex tries to pull the period and year out of the title, it falls back to the entire title. It should fall back to an empty string or None/null. I was having trouble specifying an empty string or None/null as the default value for the regex when I was finalizing the big add websites PR. Pydantic doesn't make a meaningful distinction between a non-existent value, an empty value, and a null value in the config, so I tried to write a custom validator for it. Unfortunately, Pydantic validators are unintuitive. It appears that the current config validation is not working correctly.

[johndpjr]

Hopefully this will be solved with #136 , so I might hold off on completing this ticket until we get that implemented

[karammasad]

@johndpjr Now that ticket #136 won't be done at this time. Should I go ahead and attempt to solve this bug? Or is this being put off further?

[johndpjr]

Does this run config work: python3 -m backend --dev --scrape-only --include-companies Tesla --save-jobs

[johndpjr]

With the new backend, this issue will (hopefully) go away!