-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
xrootd: improve read performances 10-fold #923
Conversation
This CL reduces the lock contention on xrootd.File operations. ``` $> benchstat ./ref.txt ./new.txt name old time/op new time/op delta Read-8 67.2s ± 1% 5.4s ± 7% -91.93% (p=0.000 n=9+28) name old alloc/op new alloc/op delta Read-8 343MB ± 0% 341MB ± 0% -0.78% (p=0.000 n=8+30) name old allocs/op new allocs/op delta Read-8 277k ± 0% 288k ± 0% +3.84% (p=0.000 n=7+29) ``` and now: ``` $> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null real 0m7.279s user 0m8.221s sys 0m1.256s ``` compared to: ``` $> time root-dump https://cern.ch/binet/big-file.root > /dev/null real 0m5.454s user 0m6.156s sys 0m0.228s ``` Updates go-hep#920.
Codecov Report
@@ Coverage Diff @@
## main #923 +/- ##
==========================================
+ Coverage 73.75% 73.77% +0.01%
==========================================
Files 404 404
Lines 48554 48541 -13
==========================================
Hits 35812 35812
+ Misses 10460 10451 -9
+ Partials 2282 2278 -4
Continue to review full report at Codecov.
|
I want to understand this mutex lock a bit more. Is it not possible to read from a xrootd file handler in parallel?
I think the answer is yes. But I don't know how it plays with the |
it should. (IIRC, there's a test for that, with the |
depending on how you call into go-hep/xrootd (presumably by generating a .so that wraps a Go type and exposes C-funcs), it shouldn't be an issue. |
thanks. I would call this function from different threads. I will check if it works by looking at performance I guess |
This CL reduces the lock contention on xrootd.File operations.
and now:
compared to:
Updates #920.