Clean up DP storage options, add public/private option

[gijzelaerr]

at 2010-02-12 15:28:37 Oleg Smirnov reported:

Clean up DP storage options, add public/private option

[gijzelaerr]

Original comment thread migrated from bugzilla

at 2010-02-12 15:28:37 Oleg Smirnov replied:

The redesign implied by bug 791 will necessarily make the copy/move DP option
independent. Along with this change, I need to introduce two types of data
product archival: private and public.

Consider my latest 3C147/pipelines purrlog, which is fast approaching 10 Gb
because I'm also using it as an archive of intermediate results. The only way to
share such a beast is to go and strip out all the archived FITS images and MEP
tables before distribution. (The SSSC had this problem too.) On the other hand,
there's a small set of FITS images that I very much do want to distribute! (the
final maps, etc.)

What I need is an option to mark data "public" (so that full FITS images or
whatever gets shipped with the log when you share it) or "private" (data is
stored in the purrlog but not shipped -- only thumbnails, etc. are) A simple
option in the Entry Editor GUI will do. Implementation-wise, perhaps have
purrlogname/purrlogname.public and purrlogname/purrlogname.private directories?
The former would store HTML index files and public DPs, the latter would hold
private DPs. Public distribution is then simply a matter of shipping the public
directory.

The default public/private setting needs to be carefully considered. MEP tables
and FITS images should be private by default, scripts, annotations etc. public
by default. Or will a simple "if size > 10K, default is private" rule will do?

GUI-wise, I think I need to go with a single icon per option: one for policy,
one for copy/move, one for public/private.