rest: client cannot get file locations after latest deployment

[tiborsimko]

Current behaviour

After latest deployment, the cernopendata-client cannot get file locations for records that have file indexes:

$ cernopendata-client get-file-locations --recid 1 --protocol xrootd
$ echo $?
0

There is no output and there is no error code passed onto the user.

Expected behaviour

The output should look like:

$ cernopendata-client get-file-locations --recid 1 --protocol xrootd
root://eospublic.cern.ch//eos/opendata/cms/Run2010B/BTau/AOD/Apr21ReReco-v1/0000/00E16FBB-9071-E011-83D3-003048673F12.root
root://eospublic.cern.ch//eos/opendata/cms/Run2010B/BTau/AOD/Apr21ReReco-v1/0000/0248915F-EE71-E011-8894-0025902009E8.root
root://eospublic.cern.ch//eos/opendata/cms/Run2010B/BTau/AOD/Apr21ReReco-v1/0000/0268F635-B671-E011-9090-002481E14E00.root
root://eospublic.cern.ch//eos/opendata/cms/Run2010B/BTau/AOD/Apr21ReReco-v1/0000/0278F65A-9A71-E011-A5C0-0025902008A8.root
...

Notes

1. Note that for directly attached files, the client works nicely:

$ cernopendata-client get-file-locations --recid 5500 --protocol xrootd
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/BuildFile.xml
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/HiggsDemoAnalyzer.cc
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/List_indexfile.txt
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/M4Lnormdatall.cc
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/M4Lnormdatall_lvl3.cc
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/demoanalyzer_cfg_level3MC.py
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/demoanalyzer_cfg_level3data.py
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/demoanalyzer_cfg_level4MC.py
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/demoanalyzer_cfg_level4data.py
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/mass4l_combine.pdf
root://eospublic.cern.ch//eos/opendata/cms/software/HiggsExample20112012/mass4l_combine.png

2. The problem is only with files attached via file indexes. Would it be possible to amend the repository's REST API replies so that the client will work again? Alternatively, we could adapt the client to the new APIs and make a new release, but it would be better to keep the old APIs alive, if possible, so that even older versions of clients (that people might have around) continue working.

3. We should try to enrich the test suite to cover these early. The cernopendata-client has its own test suite, so this error is caught there; but we do not run it periodically, and since there are no test cases in the cernopendata-portal repository for this functionality, the problem was not caught before deployment.

[tiborsimko]

P.S. Note also the --no-expand client option that should list index files themselves, e.g. for record 1 the output should contain CMS_Run2010B_BTau_AOD_Apr21ReReco-v1_0001_file_index.json. Something to beware of when fixing this issue.

See the cernopendata-client documentation at https://cernopendata-client.readthedocs.io/en/latest/cliapi.html#cernopendata-client-get-file-locations and the test cases at https://github.com/cernopendata/cernopendata-client/blob/master/tests/test_cli_get_file_locations.py#L58