Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can plotting recipies for TH1D, TGraph, TGraphError be a part of the package? #10

Open
mmikhasenko opened this issue Apr 2, 2020 · 18 comments

Comments

@mmikhasenko
Copy link
Member

Here is how I plot TGraphError extracted from TFile,

f = TFile("test.root")
c = f["myTGE"]  # typo of object PyObject <b'TGraphErrors' b'myTGE' 0x00003a8740f0>
plot(
    vcat(c.xvalues...),
    vcat(c.yvalues...),
    xerr = vcat(c.xerrors...), yerr = vcat(c.yerrors...),
    lab="",
    xlab = latexstring(replace(c.xlabel,"text" => "mathrm")),
    ylab = latexstring(replace(c.ylabel,"text" => "mathrm")),
    title = c.title
)

I think it would make a lot of sense to write a few recipes. I am just not sure how to check subtypes of these PyObject without writing the parser

@mmikhasenko mmikhasenko changed the title Can plotting recipy for TH1D, TGraph, TGraphError be a part of the package? Can plotting recipies for TH1D, TGraph, TGraphError be a part of the package? Apr 2, 2020
@oschulz
Copy link
Member

oschulz commented Apr 2, 2020

Well, we should definitely add support for THxx and implement a conversion to StatsBase.Histogram.

And you're right, Plots recipes for some things would be useful, too.

@mmikhasenko
Copy link
Member Author

do you know how to check subtypes of the PyObject? I am afraid it cannot be done by the dispatch, i.e. requires conversion to Julia structure upfront.

I saw you do something similar in the py2jl function
https://github.com/JuliaHEP/UpROOT.jl/blob/master/src/pyjlconv.jl#L45

how will this work for TH*, TGraph*?

@oschulz
Copy link
Member

oschulz commented Apr 3, 2020

I think it will just be done via further additions to py2jl, checking the type on the python side and then selecting an appropriate Julia wrapper or type to convert to.

@mmikhasenko
Copy link
Member Author

mmikhasenko commented May 8, 2020

any experience with reading TCanvas in TFile using uproot?
I cannot find the way to extract content from PyObject <Undefined_TCanvas at 0x00005067a0f0>.

o._fields # Vector{Any} with 0 elements
o._members # PyObject <bound method ROOTStreamedObject._members of <class 'uproot.rootio.Undefined_TCanvas'>>
o.__getattribute__ # PyObject <method-wrapper '__getattribute__' of Undefined_TCanvas object at 0x000000005067A0F0>

@oschulz
Copy link
Member

oschulz commented May 8, 2020

any experience with reading TCanvas in TFile using uproot?

Haven't tried yet - but if Python can read it, there must be a way to get at it. :-)

@mmikhasenko
Copy link
Member Author

I have got the next opportunity (an urgent need) to get the extraction of the canvas data working.

julia> c0 = TFile("file.root")["page0"]
PyObject <Undefined_TCanvas at 0x00000256f6d0>
julia> keys(c0)
39-element Vector{Symbol}:
 :__class__
 :__delattr__
 :__dict__
 :__dir__
 :__doc__
 :__eq__
 :__format__
 :__ge__
 :__getattribute__
 :__gt__
 :__hash__
 :__init__
 :__init_subclass__
 :__le__
 :__lt__
 :__metaclass__
 :__module__
 :__ne__
 :__new__
 :__reduce__
 :__reduce_ex__
 :__repr__
 :__setattr__
 :__sizeof__
 :__str__
 :__subclasshook__
 :__weakref__
 :_classname
 :_classversion
 :_copycontext
 :_cursor
 :_fields
 :_members
 :_postprocess
 :_readinto
 :_recarray
 :_recarray_dtype
 :classname
 :read

any idea where to look for the right methods?

@oschulz
Copy link
Member

oschulz commented Nov 29, 2021

Oh dear ... uhm, not really, without digging depper into ROOT myself ...

@tamasgal , @Moelf you're much deeper in ROOT's internals than me right now - any idea? Or can UnROOT do something that help Misha with this already (or provide an easier path to it at least)?

@tamasgal
Copy link
Member

I am a bit confused. What is the main issue? Accessing the data from a TCanvas object?

@mmikhasenko
Copy link
Member Author

yes, indeed. To get the data points and the curves from the plot
image

@tamasgal
Copy link
Member

Is it possible to upload the ROOT-file somewhere? (e.g. here, as a ZIP file)

@mmikhasenko
Copy link
Member Author

mmikhasenko commented Nov 29, 2021

sure
plot.ReIm.pre.att0911.pdf.zip
you get the canvas by

c0 = TFile("plot.ReIm.pre.att0911.pdf.root")["page0"]

@mmikhasenko
Copy link
Member Author

mmikhasenko commented Nov 29, 2021

In the ROOT I would do something like this:

root> c1 = (TCanvas*)page0->GetListOfPrimitives()->First()
root> l1 = c1->GetListOfPrimitives()->FirstLink()
root> m1 = (TMultiGraph*)l1->Next()->GetObject()
root> m1->GetListOfGraphs()->GetEntries()
(int) 11
root> g1 = (TGraph*)m1->GetListOfGraphs()->First()
root> N = g1->GetN()
(int) 85
root> xv = g1->GetX()
root> yv = g1->GetY()

ah, root

@tamasgal
Copy link
Member

I fear that this stuff is deeply connected with the underlying C++ function mapping. While it might be possible to retrieve the data points if there is valid streamer info (it seems uproot already fails to understand the primitives, that's why it's an Undefined_TCanvas, but that's more of a guess), I am pretty sure that the other information (curves) is pretty much useless without the C++ origins.

I assume it's worthless to ask if there is a possibility to change the output ROOT format to something which is less dependent of the C++ framework...

@mmikhasenko
Copy link
Member Author

mmikhasenko commented Nov 29, 2021

I guess, this points to the same direction

julia> pycall(c0, PyObject)[:GetListOfPrimitives]()
PyError ($(Expr(:escape, :(ccall(#= .julia\packages\PyCall\3fwVL\src\pyfncall.jl:43 =# @pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw))))) <class 'TypeError'>                                                                                                                         
TypeError("'Undefined_TCanvas' object is not callable")                                                                                                               

Does it?

I assume it's worthless to ask if there is a possibility to change the output ROOT format to something which is less dependent of the C++ framework...
haha.

No way around unwrapping the objects in the root terminal ... :(

@Moelf
Copy link
Member

Moelf commented Nov 29, 2021

unwrapping the objects in the root terminal

call root from shell> and parse the stdout :(((

@mmikhasenko
Copy link
Member Author

mmikhasenko commented Nov 29, 2021

AAA! TBufferJSON::ToJSON(page0) works
as well as
TBufferJSON::ExportToFile("page0.json", page0)

@mmikhasenko
Copy link
Member Author

mmikhasenko commented Nov 30, 2021

BTW, the root structures in JSON look so simple.
I thought julia structures can hold it very well. One can even generate the structures automatically from json file.

            }, {
              "_typename" : "TGraph",
              "fUniqueID" : 0,
              "fBits" : 1032,
              "fName" : "Graph",
              "fTitle" : "",
              "fLineColor" : 2,
              "fLineStyle" : 2,
              "fLineWidth" : 1,
              "fFillColor" : 1,
              "fFillStyle" : 1001,
              "fMarkerColor" : 1,
              "fMarkerStyle" : 1,
              "fMarkerSize" : 1,
              "fNpoints" : 100,
              "fX" : [0.81, 0.826969696969697, 0.843939393939394, 0.860909090909091, 0.877878787878788, 0.894848484848485, 0.911818181818182, 0.928787878787879, 0.945757575757576, 0.962727272727273, 0.97969696969697, 0.996666666666667, 1.01363636363636, 1.03060606060606, 1.04757575757576, 1.06454545454545, 1.08151515151515, 1.09848484848485, 1.11545454545455, 1.13242424242424, 1.14939393939394, 1.16636363636364, 1.18333333333333, 1.20030303030303, 1.21727272727273, 1.23424242424242, 1.25121212121212, 1.26818181818182, 1.28515151515152, 1.30212121212121, 1.31909090909091, 1.33606060606061, 1.3530303030303, 1.37, 1.3869696969697, 1.40393939393939, 1.42090909090909, 1.43787878787879, 1.45484848484849, 1.47181818181818, 1.48878787878788, 1.50575757575758, 1.52272727272727, 1.53969696969697, 1.55666666666667, 1.57363636363636, 1.59060606060606, 1.60757575757576, 1.62454545454545, 1.64151515151515, 1.65848484848485, 1.67545454545455, 1.69242424242424, 1.70939393939394, 1.72636363636364, 1.74333333333333, 1.76030303030303, 1.77727272727273, 1.79424242424242, 1.81121212121212, 1.82818181818182, 1.84515151515152, 1.86212121212121, 1.87909090909091, 1.89606060606061, 1.9130303030303, 1.93, 1.9469696969697, 1.96393939393939, 1.98090909090909, 1.99787878787879, 2.01484848484849, 2.03181818181818, 2.04878787878788, 2.06575757575758, 2.08272727272727, 2.09969696969697, 2.11666666666667, 2.13363636363636, 2.15060606060606, 2.16757575757576, 2.18454545454545, 2.20151515151515, 2.21848484848485, 2.23545454545455, 2.25242424242424, 2.26939393939394, 2.28636363636364, 2.30333333333333, 2.32030303030303, 2.33727272727273, 2.35424242424242, 2.37121212121212, 2.38818181818182, 2.40515151515152, 2.42212121212121, 2.43909090909091, 2.45606060606061, 2.4730303030303, 2.49],
              "fY" : [12.9682208066244, 12.2094997483641, 11.4953288592651, 10.6288623242888, 9.72012339765948, 9.13094068009057, 8.61332999571456, 8.2844855461918, 8.40400332795204, 8.94647394275909, 9.99194057534541, 11.8358371437289, 14.5303443527173, 18.1650509327863, 23.2639603667477, 29.8207803869633, 38.0170630705138, 48.9372324281355, 62.2211665324215, 78.8905861575526, 99.9940144865317, 125.257394457864, 158.260701113804, 198.190520728579, 248.27298647972, 312.274175721447, 389.123385331609, 494.343851590691, 621.108034680253, 793.036787487114, 1010.28276127011, 1299.76649271051, 1685.09530180919, 2185.00350639352, 2874.72900835389, 3738.74127009556, 4914.01136960223, 6343.42029830824, 8166.09847664562, 10331.3897775627, 12912.6300304604, 15905.4079903201, 19300.4365309511, 23096.6131718114, 27170.6422072014, 31425.8047139083, 35539.159595438, 39263.7643207574, 42100.977290294, 43796.2938246386, 44075.7183365252, 42966.4422322125, 40721.5586158681, 37625.9638532562, 34086.3892159137, 30321.3388155636, 26545.8464041568, 22839.3549352592, 19267.2598756726, 15935.5812340163, 12976.276746148, 10554.4969779581, 8761.0310039344, 7565.83540280415, 6841.49325553332, 6429.43173106948, 6194.21758970629, 6044.95849612353, 5926.77485317235, 5809.50465962095, 5679.59357571744, 5531.54077350329, 5364.45272664233, 5179.745501022, 4980.11037861504, 4768.72837959997, 4548.64458271685, 4323.20975051944, 4095.35370699651, 3867.81672514108, 3642.62411929312, 3422.03400541244, 3207.79112899683, 3001.07132780383, 2802.95685231493, 2614.34877244856, 2435.80212226489, 2267.64370902316, 2109.8181280774, 1962.44290466261, 1825.41550485699, 1698.73959196222, 1581.28791397172, 1473.58373042074, 1374.86265862507, 1283.7273882579, 1201.24301989937, 1126.2575418734, 1058.33190043502, 997.02766957764],
              "fFunctions" : {
                "_typename" : "TList",
                "name" : "TList",
                "arr" : [],
                "opt" : []
              },
              "fHistogram" : {
                "_typename" : "TH1F",
                "fUniqueID" : 0,
                "fBits" : 512,
                "fName" : "Graph",
                "fTitle" : "",
                "fLineColor" : 602,
                "fLineStyle" : 1,
                "fLineWidth" : 1,
                "fFillColor" : 0,
                "fFillStyle" : 1001,
                "fMarkerColor" : 1,
                "fMarkerStyle" : 1,
                "fMarkerSize" : 1,
                "fNcells" : 102,
                "fXaxis" : {
                  "_typename" : "TAxis",
                  "fUniqueID" : 0,
                  "fBits" : 0,
                  "fName" : "xaxis",
                  "fTitle" : "",
                  "fNdivisions" : 510,
                  "fAxisColor" : 1,
                  "fLabelColor" : 1,
                  "fLabelFont" : 42,
                  "fLabelOffset" : 0.005,
                  "fLabelSize" : 0.035,
                  "fTickLength" : 0.03,
                  "fTitleOffset" : 1,
                  "fTitleSize" : 0.035,
                  "fTitleColor" : 1,
                  "fTitleFont" : 42,
                  "fNbins" : 100,
                  "fXmin" : 0.642,
                  "fXmax" : 2.658,
                  "fXbins" : [],
                  "fFirst" : 0,
                  "fLast" : 0,
                  "fBits2" : 0,
                  "fTimeDisplay" : false,
                  "fTimeFormat" : "",
                  "fLabels" : null,
                  "fModLabs" : null
                },
                "fYaxis" : {
                  "_typename" : "TAxis",
                  "fUniqueID" : 0,
                  "fBits" : 0,
                  "fName" : "yaxis",
                  "fTitle" : "",
                  "fNdivisions" : 510,
                  "fAxisColor" : 1,
                  "fLabelColor" : 1,
                  "fLabelFont" : 42,
                  "fLabelOffset" : 0.005,
                  "fLabelSize" : 0.035,
                  "fTickLength" : 0.03,
                  "fTitleOffset" : 1,
                  "fTitleSize" : 0.035,
                  "fTitleColor" : 1,
                  "fTitleFont" : 42,
                  "fNbins" : 1,
                  "fXmin" : 0,
                  "fXmax" : 48482.4617216231,
                  "fXbins" : [],
                  "fFirst" : 0,
                  "fLast" : 0,
                  "fBits2" : 0,
                  "fTimeDisplay" : false,
                  "fTimeFormat" : "",
                  "fLabels" : null,
                  "fModLabs" : null
                },
                "fZaxis" : {
                  "_typename" : "TAxis",
                  "fUniqueID" : 0,
                  "fBits" : 0,
                  "fName" : "zaxis",
                  "fTitle" : "",
                  "fNdivisions" : 510,
                  "fAxisColor" : 1,
                  "fLabelColor" : 1,
                  "fLabelFont" : 42,
                  "fLabelOffset" : 0.005,
                  "fLabelSize" : 0.035,
                  "fTickLength" : 0.03,
                  "fTitleOffset" : 1,
                  "fTitleSize" : 0.035,
                  "fTitleColor" : 1,
                  "fTitleFont" : 42,
                  "fNbins" : 1,
                  "fXmin" : 0,
                  "fXmax" : 1,
                  "fXbins" : [],
                  "fFirst" : 0,
                  "fLast" : 0,
                  "fBits2" : 0,
                  "fTimeDisplay" : false,
                  "fTimeFormat" : "",
                  "fLabels" : null,
                  "fModLabs" : null
                },
                "fBarOffset" : 0,
                "fBarWidth" : 1000,
                "fEntries" : 0,
                "fTsumw" : 0,
                "fTsumw2" : 0,
                "fTsumwx" : 0,
                "fTsumwx2" : 0,
                "fMaximum" : 48482.4617216231,
                "fMinimum" : 0,
                "fNormFactor" : 0,
                "fContour" : [],
                "fSumw2" : [],
                "fOption" : "",
                "fFunctions" : {
                  "_typename" : "TList",
                  "name" : "TList",
                  "arr" : [],
                  "opt" : []
                },
                "fBufferSize" : 0,
                "fBuffer" : [],
                "fBinStatErrOpt" : 0,
                "fStatOverflows" : 2,
                "fArray" : [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
              },
              "fMinimum" : -1111,
              "fMaximum" : -1111
            }, {

@Moelf
Copy link
Member

Moelf commented Nov 30, 2021

It's not the data structure that is the problem, it's the parsing of nested C++ classes that lead to the final result

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants