Catalog

[stroblme]

This PR resolves #11 by

• moving IO interaction to kedro by introducing various datasets in the catalog
• removing file actions in various nodes and pipelines
  • adiabatic maxcut
  • adiabatic trackrec
  • anneal schedule
  • generation
  • qaoa maxcut
  • qaoa trackrec
  • qubo
  • visualization
• cleaning up variable dependencies
• copied the sample events into 01_raw (persistent in git now). In the long run we should utilize e.g. the trackml library to load arbitrary event data (e.g. based on a parameter)
• added cleanup.sh to remove any data that nodes created
• update docstring (once concept is approved by @majafranz )

Note1: When merging from main, I broke the geometric_index stuff, because of conflicts. Should be fixed before merging back again. Fixed it, parameter is now considered when building qubo.
Note2: I added a mean value calc. in the anneal schedules method, because the length of the vectors did not add up. However, I might have messed sth. up with the datatype when saving, so please check that.
Note3: The visualization pipeline is only roughly implemented and lacks more details, but we should discuss that in a meeting how to proceed imo. Because the "approach" slightly changed with this PR from "put all in one CSV and have parameters embedded there" to "separate result files and parameters are being inferred from the config", context is always required when visualizing from the result files. One quick-and-dirty solution would be to have a final "collection" node that gathers together all the results and required parameter and then builds up the solution.csv file which is being used for plotting right now. A much cleaner solution would be the plotting of the individual results directly as kedro node, which, however, would require switching from R to Python or using the ggplot2 port.
Note4: In the long run we could utilize namespaces to allow switching between track-reconstruction and maxcut and thus reusing existing nodes to be a bit closer to the "spirit"

[majafranz]

Thanks for the effort and the great work! Looks good to me. 🚀

But, I would prefer CSV files instead of JSON in the results, as this is easier to plot with R.

I think pandas dataframes are easy to export as a pandas.CSVDataset, but we have to make sure the columns are properly set. I can try to fix that.

[stroblme]

CSV is fine for me, just keep in mind what I wrote in Note3; tldr: eventually we don't have to read from the "intermediate" results with R :) maybe you need to tweak the load_args or save_args

[majafranz]

CSV is fine for me, just keep in mind what I wrote in Note3; tldr: eventually we don't have to read from the "intermediate" results with R :) maybe you need to tweak the load_args or save_args

Ah I think there might be a misunderstanding. I am not only using the previous "solution.csv" but also the other files for plotting (and also for plotting different things). But maybe let's discuss this later.

[majafranz]

I think the mean should not be necessary. Maybe the beta and gamma values are somehow lists of lists now? beta[i] and gamma[i] should just be a float.

[stroblme]

as discussed, let's go with max_p, can you fix that @majafranz ?

[majafranz]

done in 1e13311

[majafranz]

Did you add an additional build_model function? The qallse version does not like the "doublets"

[stroblme]

fixed with 807836a

[stroblme]

• @stroblme check how to connect config params and output files in kedro (eventually create export config node)
• @majafranz convert json output to csv

[stroblme]

added export parameters pipeline (merged with viz)