Storing historic data

[badrihippo]

We need to decide how much data we want to save over a long-term, and how to save it.

The Borrowing/PastBorrowing models are one example of past data being stored differently from current data, but what are we going to do for past users and groups? What happens when the users leave? Are we still going to keep a record of what books they borrowed? (Merlin does.) And if a book gets deleted/discarded from the library, do we want to continue storing data about who borrowed it? Do we also want to know which group the user was in when they borrowed that book (makes sense if, eg., we want to see the trend of that-age-group's preferred books over the years)? What if there are many copies of the book that have been borrowed, and only some copies get discarded? Do we want to know everyone who borrowed that particular book, or only people who borrowed that particular copy?

If we're not going to keep records for past users but still track borrowing of books, then we'll have to make PastBorrowing records contain the user name as text instead of an FK. Or make a PastUser model, but that'll make things too complicated because PastBorrowing will need to know whether to reference a User or a PastUser.... It doesn't make sense to store ALL the data (contact, email, birthday,...) for a past user, but it'll still be nice to have a complete list of everybody who borrowed the book (like we have for Merlin now).

Similarly, we'll also need to decide on how to store past Groups. As for book removals, what Merlin does right now is that it doesn't remove a book record completely, but only marks it as "deleted". So it doesn't show up in current searches, but (I think) can be used for, eg. listing details in a user's past borrowings. Don't know if this is the best way to do it; could they be moved to a separate table so that they don't need to be considered in current-record searches?

We need to discuss this further, as it may also affect how we design our database right now (in fact, it already has: see commit da807d3).