Skip to content

Commit c24b489

Browse files
Mark-SimulacrumMark-Simulacrum
authored
Merge pull request #1368 from pnkfelix/triage-2022-07-27
triage for this week.
2 parents 4998f79 + 52e010e commit c24b489

File tree

1 file changed

+239
-0
lines changed

1 file changed

+239
-0
lines changed

‎triage/2022-07-27.md

+239
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
# 2022-07-27 Triage Log
2+
3+
Overall it was a mostly good week, with some very significant wins among the
4+
secondary benchmarks. Rollups continue to complicate triage process.
5+
6+
Triage done by **@pnkfelix**.
7+
Revision range: [8bd12e8c..50166d5e](https://perf.rust-lang.org/?start=8bd12e8cca3f28f302b9cc0f1f47bb64bd1f98fd&end=50166d5e5e82ca795306824decbe4ffabcc23d3d&absolute=false&stat=instructions%3Au)
8+
9+
**Summary**:
10+
11+
| | mean | max | count |
12+
|:----------:|:----:|:---:|:-----:|
13+
| Regressions 😿 <br /> (primary) | N/A | N/A | 0 |
14+
| Regressions 😿 <br /> (secondary) | 2.2% | 3.2% | 6 |
15+
| Improvements 🎉 <br /> (primary) | -1.8% | -21.2% | 199 |
16+
| Improvements 🎉 <br /> (secondary) | -2.6% | -9.0% | 124 |
17+
| All 😿🎉 (primary) | -1.8% | -21.2% | 199 |
18+
19+
20+
5 Regressions, 4 Improvements, 4 Mixed; 4 of them in rollups
21+
61 artifact comparisons made in total
22+
23+
#### Regressions
24+
25+
Rollup of 9 pull requests [#99520](https://github.com/rust-lang/rust/pull/99520) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=a7468c60f8dbf5feb23ad840b174d7e57113a846&end=d68e7ebc38cb42b8b237392b28045edeec761503&stat=instructions:u)
26+
27+
| | mean | max | count |
28+
|:----------:|:----:|:---:|:-----:|
29+
| Regressions 😿 <br /> (primary) | 2.0% | 2.7% | 4 |
30+
| Regressions 😿 <br /> (secondary) | 1.3% | 2.5% | 29 |
31+
| Improvements 🎉 <br /> (primary) | N/A | N/A | 0 |
32+
| Improvements 🎉 <br /> (secondary) | N/A | N/A | 0 |
33+
| All 😿🎉 (primary) | 2.0% | 2.7% | 4 |
34+
35+
* The 4 primary regressions, 3 are helloworld check, regressing by 2.5% to 2.7% on various incr scenarios. The last is ripgrep check but that only regressed by 0.36%.
36+
* From looking at the graph of helloworld-check over time, the regression to helloworld-check that was injected here was legitimate, as it plateaued up there for 4 or 5 days until it jumped back down due to PR [#99677](https://github.com/rust-lang/rust/pull/99677)
37+
* PR #99677 was put in to address regressions injected by PR #97786, which was rolled up in PR #98656. Looking at the data from that rollup, it appears that helloworld-check there also regressed by 2.6%; so it seems to me like the regression injected by #99520 is probably still persisting; its presence is just masked by the effect of PR #98656...
38+
* Perhaps the regression is coming from the following queries/functions: stability_implications, metadata_decode_entry_stability_implications, defined_lib_features, metadata_decode_entry_defined_lib_features, all of which are present in the new commit but not the base commit. Were all of those added as part of PRs in this rollup?
39+
* If the above queries are indeed to blame for the regression here, then I think that would be tied to [PR #99212](https://github.com/rust-lang/rust/pull/99212), "introduce implied_by in #[unstable] attribute".
40+
* Not marking as triaged. I'm leaving the perf-regression marker in place until we at least confirm which PR was the cause; then we can better evaluate whether the regression is an acceptable price to pay.
41+
42+
move `considering_regions` to the infcx [#99501](https://github.com/rust-lang/rust/pull/99501) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=af7ab3447079fddc51c6c7749b160d24769f7c16&end=62b272d25c5bb8b6bb8ac73797d82b8b9a1eabda&stat=instructions:u)
43+
44+
| | mean | max | count |
45+
|:----------:|:----:|:---:|:-----:|
46+
| Regressions 😿 <br /> (primary) | 0.4% | 0.4% | 2 |
47+
| Regressions 😿 <br /> (secondary) | 0.4% | 0.5% | 5 |
48+
| Improvements 🎉 <br /> (primary) | N/A | N/A | 0 |
49+
| Improvements 🎉 <br /> (secondary) | N/A | N/A | 0 |
50+
| All 😿🎉 (primary) | 0.4% | 0.4% | 2 |
51+
52+
* The secondary regressions were [already anticipated](https://github.com/rust-lang/rust/pull/99501#issuecomment-1191647607) by the PR reviewer. The primary regressions are both diesel and they look like blips in the data to me from the graph.
53+
* Marking as triaged.
54+
55+
Sync in portable-simd subtree [#99491](https://github.com/rust-lang/rust/pull/99491) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=e7a9c1141698bc4557b9da3d3fce2bf75339427f&end=41419e70366962c9a878bfe673ef4df38db6f7f1&stat=instructions:u)
56+
57+
| | mean | max | count |
58+
|:----------:|:----:|:---:|:-----:|
59+
| Regressions 😿 <br /> (primary) | 0.5% | 1.0% | 11 |
60+
| Regressions 😿 <br /> (secondary) | 0.8% | 1.3% | 20 |
61+
| Improvements 🎉 <br /> (primary) | -0.2% | -0.2% | 1 |
62+
| Improvements 🎉 <br /> (secondary) | N/A | N/A | 0 |
63+
| All 😿🎉 (primary) | 0.4% | 1.0% | 12 |
64+
65+
* All of the regressions here are on doc profiles. I don't think its worth us spending time trying to figure out 1% regressions to rustdoc performance.
66+
* Marking as triaged.
67+
68+
Fix hack that remaps env constness. [#99521](https://github.com/rust-lang/rust/pull/99521) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=41419e70366962c9a878bfe673ef4df38db6f7f1&end=22d25f21dc008785f52e7c2833de4f4236b1066b&stat=instructions:u)
69+
70+
| | mean | max | count |
71+
|:----------:|:----:|:---:|:-----:|
72+
| Regressions 😿 <br /> (primary) | 0.5% | 0.8% | 7 |
73+
| Regressions 😿 <br /> (secondary) | 0.6% | 0.6% | 1 |
74+
| Improvements 🎉 <br /> (primary) | N/A | N/A | 0 |
75+
| Improvements 🎉 <br /> (secondary) | N/A | N/A | 0 |
76+
| All 😿🎉 (primary) | 0.5% | 0.8% | 7 |
77+
78+
* This regression was anticipated by the PR author and [analyzed](https://github.com/rust-lang/rust/pull/99521#issuecomment-1191511904) by the reviewer.
79+
* marking as triaged.
80+
81+
82+
Rollup of 8 pull requests [#99792](https://github.com/rust-lang/rust/pull/99792) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=b573e10d21b69ebfadf41aa9c2f0a27919fe4480&end=e33cc71a61c91e1d510bf283e9d345067e64eed2&stat=instructions:u)
83+
84+
| | mean | max | count |
85+
|:----------:|:----:|:---:|:-----:|
86+
| Regressions 😿 <br /> (primary) | 0.5% | 0.8% | 9 |
87+
| Regressions 😿 <br /> (secondary) | 1.8% | 2.9% | 6 |
88+
| Improvements 🎉 <br /> (primary) | N/A | N/A | 0 |
89+
| Improvements 🎉 <br /> (secondary) | N/A | N/A | 0 |
90+
| All 😿🎉 (primary) | 0.5% | 0.8% | 9 |
91+
92+
* Primary regressions were to clap (check full, check incr-full, and doc full), libc (doc full), hyper (check full, check incr-full, and doc full), image (doc full), and webrender (doc full).
93+
* The significance factor points mostly to the clap cases (with 4.13x, 3.25x, and 7.15x respectively to each of the scenarios I listed above for clap).
94+
* The [detailed query data](https://perf.rust-lang.org/detailed-query.html?sort_idx=-11&commit=e33cc71a61c91e1d510bf283e9d345067e64eed2&base_commit=b573e10d21b69ebfadf41aa9c2f0a27919fe4480&benchmark=clap-3.1.6-check&scenario=full) for clap check full indicates that the problem is mostly in `metadata_decode_entry_item_attrs` and `visible_parent_map`; those are the ones that had a significant time delta that end up explaining the overall time delta (0.003 + 0.003 > 0.005).
95+
* `visible_parent_map` slowdown may be due to [PR #99698](https://github.com/rust-lang/rust/issues/99698).
96+
* The slowdown to `metadata_decode_entry_Item_attrs` may be due to [PR #99712](https://github.com/rust-lang/rust/issues/99712) ? Hard to say.
97+
* The secondary regressions are all to the projection-caching benchmark, which regressed by 1.2% to 2.9% in various scenarios. That regression seems to be to due [a combination](https://perf.rust-lang.org/detailed-query.html?sort_idx=-11&commit=e33cc71a61c91e1d510bf283e9d345067e64eed2&base_commit=b573e10d21b69ebfadf41aa9c2f0a27919fe4480&benchmark=projection-caching-check&scenario=full) of both the `metadata_decode_entry_item_attrs` and `visible_parent_map` regressions, as well as a little bit more time spent in `type_op_prove_predicate`, `evaluate_obligation`, and `normalize_projection_ty`. Not sure why though, I don't think those got touched by this rollup. Maybe just different execution paths from the stdlib changes that *did* come in with this rollup?
98+
* Leaving comments on both the rollup PR and the two suspect PRs from the rollup. Not marking as triaged.
99+
100+
#### Improvements
101+
102+
Revert "Rollup merge of #98582 - oli-obk:unconstrained_opaque_type, r… [#99495](https://github.com/rust-lang/rust/pull/99495) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=748cb1f01d623f2afd0d8b84fda7e2c8f7a11c7b&end=d60d88fe5cd55496b9ccb1511a9af4994b7c43d0&stat=instructions:u)
103+
104+
| | mean | max | count |
105+
|:----------:|:----:|:---:|:-----:|
106+
| Regressions 😿 <br /> (primary) | N/A | N/A | 0 |
107+
| Regressions 😿 <br /> (secondary) | N/A | N/A | 0 |
108+
| Improvements 🎉 <br /> (primary) | -0.6% | -2.6% | 136 |
109+
| Improvements 🎉 <br /> (secondary) | -1.0% | -5.5% | 93 |
110+
| All 😿🎉 (primary) | -0.6% | -2.6% | 136 |
111+
112+
113+
Rollup of 7 pull requests [#99506](https://github.com/rust-lang/rust/pull/99506) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=d60d88fe5cd55496b9ccb1511a9af4994b7c43d0&end=14dbfebfa25a0e626ad827526934381b2545cbb4&stat=instructions:u)
114+
115+
| | mean | max | count |
116+
|:----------:|:----:|:---:|:-----:|
117+
| Regressions 😿 <br /> (primary) | N/A | N/A | 0 |
118+
| Regressions 😿 <br /> (secondary) | N/A | N/A | 0 |
119+
| Improvements 🎉 <br /> (primary) | -1.4% | -20.7% | 35 |
120+
| Improvements 🎉 <br /> (secondary) | -1.1% | -2.8% | 19 |
121+
| All 😿🎉 (primary) | -1.4% | -20.7% | 35 |
122+
123+
* The -20.7% improvement was to webrender-2022 (check profile, incr-patched:println scenario).
124+
* Not quite sure which PR in the rollup yielded that kind of improvement. Maybe [PR #99486](https://github.com/rust-lang/rust/pull/99486) sidestepped some pathological string construction(s) and comparison(s) in webrender?
125+
* The primary benchmarks other than webrender all observed <1% improvement.
126+
127+
Tweak `SubstFolder` implementation [#99600](https://github.com/rust-lang/rust/pull/99600) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=7f93d4aa0dc4ac071c617e0e07d2758e3bb388f9&end=2f320a224e827b400be25966755a621779f797cc&stat=instructions:u)
128+
129+
| | mean | max | count |
130+
|:----------:|:----:|:---:|:-----:|
131+
| Regressions 😿 <br /> (primary) | N/A | N/A | 0 |
132+
| Regressions 😿 <br /> (secondary) | 1.6% | 1.6% | 1 |
133+
| Improvements 🎉 <br /> (primary) | -0.4% | -0.6% | 22 |
134+
| Improvements 🎉 <br /> (secondary) | -1.6% | -3.6% | 14 |
135+
| All 😿🎉 (primary) | -0.4% | -0.6% | 22 |
136+
137+
138+
Remove new allocations from `imported_source_files` [#99677](https://github.com/rust-lang/rust/pull/99677) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=b629c85bd74dfb730a3e9308312b007c0bf027cb&end=96b9bb4620f4d48aa25c381c7ea77e0cab48ac5b&stat=instructions:u)
139+
140+
| | mean | max | count |
141+
|:----------:|:----:|:---:|:-----:|
142+
| Regressions 😿 <br /> (primary) | N/A | N/A | 0 |
143+
| Regressions 😿 <br /> (secondary) | N/A | N/A | 0 |
144+
| Improvements 🎉 <br /> (primary) | -1.5% | -9.9% | 132 |
145+
| Improvements 🎉 <br /> (secondary) | -3.2% | -9.8% | 77 |
146+
| All 😿🎉 (primary) | -1.5% | -9.9% | 132 |
147+
148+
149+
#### Mixed
150+
151+
Improve the function pointer docs [#98180](https://github.com/rust-lang/rust/pull/98180) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=29c5a028b0c92aa5da6a8eb6d6585a389fcf1035&end=9a7b7d5e50ab0b59c6d349bbf005680a7c880e98&stat=instructions:u)
152+
153+
| | mean | max | count |
154+
|:----------:|:----:|:---:|:-----:|
155+
| Regressions 😿 <br /> (primary) | 0.2% | 0.3% | 3 |
156+
| Regressions 😿 <br /> (secondary) | 0.4% | 0.4% | 8 |
157+
| Improvements 🎉 <br /> (primary) | N/A | N/A | 0 |
158+
| Improvements 🎉 <br /> (secondary) | -1.2% | -1.2% | 1 |
159+
| All 😿🎉 (primary) | 0.2% | 0.3% | 3 |
160+
161+
* The regressions above are all in doc generation, and they are all minor.
162+
* Marked as triaged.
163+
164+
165+
Rollup of 11 pull requests [#99567](https://github.com/rust-lang/rust/pull/99567) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=1673f1450eeaf4a5452e086db0fe2ae274a0144f&end=af7ab3447079fddc51c6c7749b160d24769f7c16&stat=instructions:u)
166+
167+
| | mean | max | count |
168+
|:----------:|:----:|:---:|:-----:|
169+
| Regressions 😿 <br /> (primary) | N/A | N/A | 0 |
170+
| Regressions 😿 <br /> (secondary) | 0.5% | 0.5% | 1 |
171+
| Improvements 🎉 <br /> (primary) | -0.3% | -0.3% | 4 |
172+
| Improvements 🎉 <br /> (secondary) | -0.7% | -1.0% | 5 |
173+
| All 😿🎉 (primary) | -0.3% | -0.3% | 4 |
174+
175+
* Sole (small) regression was to secondary benchmark wg-grammar (doc full scenario), of 0.54%.
176+
* Not worth trying to tease that out of a rollup.
177+
178+
179+
rustc_expand: Switch FxHashMap to FxIndexMap where iteration is used [#99320](https://github.com/rust-lang/rust/pull/99320) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=7d0a55bcdc262d12447942e028c480b2387076ea&end=47ba93596586783efd41df7b8ea84f4f1e37f923&stat=instructions:u)
180+
181+
| | mean | max | count |
182+
|:----------:|:----:|:---:|:-----:|
183+
| Regressions 😿 <br /> (primary) | N/A | N/A | 0 |
184+
| Regressions 😿 <br /> (secondary) | 0.4% | 0.4% | 1 |
185+
| Improvements 🎉 <br /> (primary) | -1.1% | -1.8% | 11 |
186+
| Improvements 🎉 <br /> (secondary) | N/A | N/A | 0 |
187+
| All 😿🎉 (primary) | -1.1% | -1.8% | 11 |
188+
189+
* Sole (small) regression was to secondary benchmark tt-muncher (check incr-unchanged scenario), of 0.41%
190+
* Seems like a justifiable cost given that 11 primary benchmarks were improved by a mean -1.1%.
191+
192+
193+
Upgrade indexmap and thorin-dwp to use hashbrown 0.12 [#99251](https://github.com/rust-lang/rust/pull/99251) [(Comparison Link)](https://perf.rust-lang.org/compare.html?start=db8086eb6056f022d3bb719f777307e9daa3f8d8&end=4dbc89de3f160f4fd91a1e20b72fc6b3157b2e04&stat=instructions:u)
194+
195+
| | mean | max | count |
196+
|:----------:|:----:|:---:|:-----:|
197+
| Regressions 😿 <br /> (primary) | 0.2% | 0.2% | 3 |
198+
| Regressions 😿 <br /> (secondary) | N/A | N/A | 0 |
199+
| Improvements 🎉 <br /> (primary) | -0.4% | -0.5% | 7 |
200+
| Improvements 🎉 <br /> (secondary) | -1.4% | -1.4% | 2 |
201+
| All 😿🎉 (primary) | -0.2% | -0.5% | 10 |
202+
203+
204+
* 7 primary improvements, eight on diesel in check+opt full+incr-full profiles, in the range -0.31% to -0.47%; 3 primary regressions, two on diesel in debug+opt incr-unchanged, all roughly 0.23%.
205+
* The change here was in part motivated by a soundness fix. So the relatively small regression here is easily outweighed by the soundness fix (and the fact that there were more significant improvements to boot is icing on the cake).
206+
* marking as triaged.
207+
208+
#### Untriaged Pull Requests
209+
210+
- [#99792 Rollup of 8 pull requests](https://github.com/rust-lang/rust/pull/99792)
211+
- [#99521 Fix hack that remaps env constness.](https://github.com/rust-lang/rust/pull/99521)
212+
- [#99520 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/99520)
213+
- [#99501 move `considering_regions` to the infcx](https://github.com/rust-lang/rust/pull/99501)
214+
- [#99491 Sync in portable-simd subtree](https://github.com/rust-lang/rust/pull/99491)
215+
- [#99251 Upgrade indexmap and thorin-dwp to use hashbrown 0.12](https://github.com/rust-lang/rust/pull/99251)
216+
- [#99231 Rollup of 5 pull requests](https://github.com/rust-lang/rust/pull/99231)
217+
- [#99210 Rollup of 5 pull requests](https://github.com/rust-lang/rust/pull/99210)
218+
- [#99126 remove allow(rustc::potential_query_instability) in rustc_span](https://github.com/rust-lang/rust/pull/99126)
219+
- [#99047 Rollup of 6 pull requests](https://github.com/rust-lang/rust/pull/99047)
220+
- [#99014 Rollup of 8 pull requests](https://github.com/rust-lang/rust/pull/99014)
221+
- [#98987 Rollup of 7 pull requests](https://github.com/rust-lang/rust/pull/98987)
222+
- [#98957 don't allow ZST in ScalarInt ](https://github.com/rust-lang/rust/pull/98957)
223+
- [#98904 Rollup of 8 pull requests](https://github.com/rust-lang/rust/pull/98904)
224+
- [#98874 Rollup of 5 pull requests](https://github.com/rust-lang/rust/pull/98874)
225+
- [#98612 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/98612)
226+
- [#98591 Rollup of 9 pull requests](https://github.com/rust-lang/rust/pull/98591)
227+
- [#98180 Improve the function pointer docs](https://github.com/rust-lang/rust/pull/98180)
228+
- [#98178 btree: avoid forcing the allocator to be a reference](https://github.com/rust-lang/rust/pull/98178)
229+
- [#98145 Pull Derefer before ElaborateDrops](https://github.com/rust-lang/rust/pull/98145)
230+
- [#97786 Account for `-Z simulate-remapped-rust-src-base` when resolving remapped paths](https://github.com/rust-lang/rust/pull/97786)
231+
- [#97019 Transition to valtrees pt1](https://github.com/rust-lang/rust/pull/97019)
232+
- [#97004 Proc macro tweaks](https://github.com/rust-lang/rust/pull/97004)
233+
- [#96883 Add EarlyBinder](https://github.com/rust-lang/rust/pull/96883)
234+
- [#96825 Retire `ItemLikeVisitor` trait](https://github.com/rust-lang/rust/pull/96825)
235+
- [#96010 Implement `core::ptr::Unique` on top of `NonNull`](https://github.com/rust-lang/rust/pull/96010)
236+
- [#95990 Rollup of 7 pull requests](https://github.com/rust-lang/rust/pull/95990)
237+
- [#95956 Support unstable moves via stable in unstable items](https://github.com/rust-lang/rust/pull/95956)
238+
- [#95899 rustc_metadata: Do not encode unnecessary module children](https://github.com/rust-lang/rust/pull/95899)
239+
- [#95893 Respect -Z verify-llvm-ir and other flags that add extra passes when combined with -C no-prepopulate-passes in the new LLVM Pass Manager.](https://github.com/rust-lang/rust/pull/95893)

0 commit comments

Comments
 (0)