Broken/Inconsistent Date Parsing

[triplingual]

I think I'm seeing some strange date parsing in the OneClick web version. In a small test where the date value in jsoncatalog.txt is '2016-m-01' for each of 10 entries with appropriate changes to m for each entry, the hover values in the GUI are all 'M [last or next-to-last day of month] 2015'. For instance, search for 'Karachi', and the only item that has it hovers a date of 'Jan 30 2015'

Using the Congress sample data in a new Bookworm, dates in the hover are significantly off, though not by as much as in the above test case. Search for 'election day' and hover over the final peak. It shows a date of 'Dec 30, 2012' but the date in the jsoncatalog for that bill is '2014-1-3'.

[bmschmidt]

Thanks for posting this. I've seen this in some other place as well. This is a pretty big problem we should be fixing.

Just a few notes here for bookworm project people, preliminary to fixing it:

1. Bookworm internally stores days in an integer format represented as 'days since zero.' When used with the MYSQL FROM_DAYS function, these integers return the right dates.
2. This is kind of stupid, because there is no year zero. In most languages, the number is actually 'days since January 1, 1 AD' plus 366. (Because mysql sets the nonexistent year 0 as a leap year. Why not, I guess.)
3. This standard seems to be a switch in 2014 from storing it in a more pythonic manner.
4. A project member committed a bug in the code in spring 2014 that, I think, may have been inconsistent in how it applied the date format, and also looks to me not to have handled leap years properly. I fixed some of these in 2014, and then switched to dateutil in 2015.
5. Some spot checks on the current code appear to me to show proper behavior now with the latest branches of bookwormDB and bookwormGUI, but it's enough of a mess that we need more robust testing. The new-ish unit tests include day-level dates for the federalist papers, and so should be enough to write unit tests.
6. So here's my tag question: @sa501428 Is it possible the one-click server is using that old, buggy code and still hasn't pulled the corrections from 2015? What's your upgrade procedure over there? Or is the problem that someone (probably me?) made changes to the linechart repo in 2014 to accommodate the bug and then never switched them out?
7. But really, we probably should not be exposing this integer format at all, and the GUI should instead by returning ISO 8661 strings not funky MySQL integers. That's a change that will cause some major breakage.