Downloading timeout for big runs

[FaroutYLq]

This is a problem in the vanilla admix-download in stable branch. When a run is too huge, for example 050073-raw_records-rfzvpzj4mf, you will have problems downloading because of a timeout issue. A patch was proposed before and didn't solve the problem.

However, a brutal way to bypass this is to use rucio download. It might be acceptable to find a way to insert this functionality into admix stable branch.

def _meta_admix_download(st, run, target, rse = "UC_OSG_USERDISK"):
    codes = ""
    filename = str(st.key_for(run, target))
    straxhash = filename.split('-')[-1]
    did = "xnt_"+run+":"+target+"-"+straxhash

    # make directory
    codes += "mkdir -p %s"%(filename) + "\n"
    
    # enter directory
    codes += "cd %s"%(filename) + "\n"

    # rucio download
    codes += "rucio download %s --rse %s"%(did, rse) + "\n"

    # move things out
    codes += "mv %s/* ."%(did.split(':')[-1]) + "\n"

    # remove empty folder 
    codes += "rm -rf %s"%(did.split(':')[-1]) + "\n"

    # cd out back to parent directory
    codes += "cd ../" + "\n"

    print(codes)

def meta_admix_download(runs, target = "raw_records", rse = "UC_OSG_USERDISK"):
    import cutax
    st = cutax.xenonnt_offline()
    for r in runs:
        if type(r) == str:
            pass
        else:
            r = str(r).zfill(6)
        _meta_admix_download(st, r, target, rse)

An example will be

meta_admix_download(['049198', '049327', '050045', '050073'])

and this will give you the output you are expected to paste into terminal:

mkdir -p 049198-raw_records-rfzvpzj4mf
cd 049198-raw_records-rfzvpzj4mf
rucio download xnt_049198:raw_records-rfzvpzj4mf --rse UC_OSG_USERDISK
mv raw_records-rfzvpzj4mf/* .
rm -rf raw_records-rfzvpzj4mf
cd ../

mkdir -p 049327-raw_records-rfzvpzj4mf
cd 049327-raw_records-rfzvpzj4mf
rucio download xnt_049327:raw_records-rfzvpzj4mf --rse UC_OSG_USERDISK
mv raw_records-rfzvpzj4mf/* .
rm -rf raw_records-rfzvpzj4mf
cd ../

mkdir -p 050045-raw_records-rfzvpzj4mf
cd 050045-raw_records-rfzvpzj4mf
rucio download xnt_050045:raw_records-rfzvpzj4mf --rse UC_OSG_USERDISK
mv raw_records-rfzvpzj4mf/* .
rm -rf raw_records-rfzvpzj4mf
cd ../

mkdir -p 050073-raw_records-rfzvpzj4mf
cd 050073-raw_records-rfzvpzj4mf
rucio download xnt_050073:raw_records-rfzvpzj4mf --rse UC_OSG_USERDISK
mv raw_records-rfzvpzj4mf/* .
rm -rf raw_records-rfzvpzj4mf
cd ../