Skip to content

Commit

Permalink
Generator performance (#2200)
Browse files Browse the repository at this point in the history 
As part of profiling efforts in a customer projects we identified a couple of hotspots during generation.

Most of them were typesystem related. To cache the expensive typesystem calls I introduced TypeCacheHelper which essentially applies memoization. It is used to wrap all call to the typesystem and will return a result from it's cache when it encounters the same node twice.

The new typesystem cache is used in:

handleArrayStructUnionInitExprAndAssignments
TransparentStructHelper
From my testing caching there doesn't break anything but please do some extensive testing in other codebases to see if this really holds.

The second hotspot we identified was sorting the module content. Here as well we apply caching. Since the sortied relies on function signatures the function to a given signature is cached in the Dependencies class. The class itself is also no longer static so that we can determine the lifecycle of the cache.

There are a couple of smaller optimisations in various places of the generator where multiple calls to .type are replaced with a local variable to store the result.

All in all these changes cut our generation by 40-50%.
  • Loading branch information
@coolya
coolya committed Dec 3, 2021
1 parent 71f44c9 commit 591162d
Show file tree
Hide file tree
Showing 8 changed files with 924 additions and 530 deletions.
5 changes: 5 additions & 0 deletions code/languages/com.mbeddr.build/solutions/com.mbeddr.build/models/com/mbeddr/build/build.mps
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3700,6 +3700,11 @@
<ref role="3bR37D" node="6y2eE4zZsnH" resolve="com.mbeddr.core.stdlib" />
</node>
</node>
<node concept="1SiIV0" id="4LUDPiajR77" role="3bR37C">
<node concept="3bR9La" id="4LUDPiajR78" role="1SiIV1">
<ref role="3bR37D" to="ffeo:1H905DlDUSw" resolve="MPS.OpenAPI" />
</node>
</node>
</node>
<node concept="1SiIV0" id="6hLPhdD9ewl" role="3bR37C">
<node concept="1Busua" id="6hLPhdD9ewm" role="1SiIV1">
Expand Down
1,004 changes: 596 additions & 408 deletions ...r.core.modules.gen/generator/template/com/mbeddr/core/modules/gen/generator/main/util.mps
View file

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions ...les/generator/template/com.mbeddr.core.modules.generator.com.mbeddr.core.modules.util.mps
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
<import index="mj1l" ref="r:c371cf98-dcc8-4a43-8eb8-8a8096de18b2(com.mbeddr.core.expressions.structure)" />
<import index="tpck" ref="r:00000000-0000-4000-0000-011c89590288(jetbrains.mps.lang.core.structure)" />
<import index="yq40" ref="r:152b3fc0-83a1-4bab-a8cd-565eb8483785(com.mbeddr.core.pointers.structure)" />
<import index="1ka" ref="6ed54515-acc8-4d1e-a16c-9fd6cfe951ea/java:jetbrains.mps.typechecking(MPS.Core/)" />
<import index="mhbf" ref="8865b7a8-5271-43d3-884c-6fd1d9cfdd34/java:org.jetbrains.mps.openapi.model(MPS.OpenAPI/)" />
<import index="wyt6" ref="6354ebe7-c22a-4a0f-ac54-50b52ab9b065/java:java.lang(JDK/)" implicit="true" />
<import index="1s42" ref="r:d482a2e6-b3ef-4c45-883b-cf624a56b653(com.mbeddr.core.pointers.behavior)" implicit="true" />
<import index="xlxw" ref="6354ebe7-c22a-4a0f-ac54-50b52ab9b065/java:java.math(JDK/)" implicit="true" />
Expand Down
252 changes: 164 additions & 88 deletions ...s/com.mbeddr.core/languages/com.mbeddr.core.modules/generator/template/[email protected]
View file

Large diffs are not rendered by default.

119 changes: 119 additions & 0 deletions code/languages/com.mbeddr.core/languages/com.mbeddr.core.modules/languageModels/plugin.mps
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@
<use id="c0080a47-7e37-4558-bee9-9ae18e690549" name="jetbrains.mps.lang.extension" version="-1" />
<use id="ef7bf5ac-d06c-4342-b11d-e42104eb9343" name="jetbrains.mps.lang.plugin.standalone" version="-1" />
<use id="63650c59-16c8-498a-99c8-005c7ee9515d" name="jetbrains.mps.lang.access" version="-1" />
<use id="f3061a53-9226-4cc5-a443-f952ceaf5816" name="jetbrains.mps.baseLanguage" version="11" />
<use id="83888646-71ce-4f1c-9c53-c54016f6ad4f" name="jetbrains.mps.baseLanguage.collections" version="1" />
<use id="7866978e-a0f0-4cc7-81bc-4d213d9375e1" name="jetbrains.mps.lang.smodel" version="17" />
<devkit ref="fbc25dd2-5da4-483a-8b19-70928e1b62d7(jetbrains.mps.devkit.general-purpose)" />
</languages>
<imports>
Expand All @@ -35,6 +38,7 @@
<import index="5nvm" ref="r:27bc780b-59b2-4d26-9db5-a38b63c35884(jetbrains.mps.refactoring.participant)" />
<import index="4ugc" ref="r:bd779a69-a10b-4882-b646-c1303f2dd4f7(jetbrains.mps.refactoring.participant.plugin)" />
<import index="w1kc" ref="6ed54515-acc8-4d1e-a16c-9fd6cfe951ea/java:jetbrains.mps.smodel(MPS.Core/)" />
<import index="1ka" ref="6ed54515-acc8-4d1e-a16c-9fd6cfe951ea/java:jetbrains.mps.typechecking(MPS.Core/)" />
<import index="qd6m" ref="r:c4c3f7d3-0acf-4671-a134-5fab66c4e637(com.mbeddr.core.modules.behavior)" implicit="true" />
<import index="wyt6" ref="6354ebe7-c22a-4a0f-ac54-50b52ab9b065/java:java.lang(JDK/)" implicit="true" />
<import index="vs0r" ref="r:f7764ca4-8c75-4049-922b-08516400a727(com.mbeddr.core.base.structure)" implicit="true" />
Expand Down Expand Up @@ -161,6 +165,11 @@
<reference id="1107535924139" name="classifier" index="3uigEE" />
<child id="1109201940907" name="parameter" index="11_B2D" />
</concept>
<concept id="1081773326031" name="jetbrains.mps.baseLanguage.structure.BinaryOperation" flags="nn" index="3uHJSO">
<child id="1081773367579" name="rightExpression" index="3uHU7w" />
<child id="1081773367580" name="leftExpression" index="3uHU7B" />
</concept>
<concept id="1073239437375" name="jetbrains.mps.baseLanguage.structure.NotEqualsExpression" flags="nn" index="3y3z36" />
<concept id="1178549954367" name="jetbrains.mps.baseLanguage.structure.IVisible" flags="ng" index="1B3ioH">
<child id="1178549979242" name="visibility" index="1B3o_S" />
</concept>
Expand Down Expand Up @@ -255,6 +264,18 @@
<concept id="1203518072036" name="jetbrains.mps.baseLanguage.collections.structure.SmartClosureParameterDeclaration" flags="ig" index="Rh6nW" />
<concept id="1160600644654" name="jetbrains.mps.baseLanguage.collections.structure.ListCreatorWithInit" flags="nn" index="Tc6Ow" />
<concept id="1160612413312" name="jetbrains.mps.baseLanguage.collections.structure.AddElementOperation" flags="nn" index="TSZUe" />
<concept id="1197683403723" name="jetbrains.mps.baseLanguage.collections.structure.MapType" flags="in" index="3rvAFt">
<child id="1197683466920" name="keyType" index="3rvQeY" />
<child id="1197683475734" name="valueType" index="3rvSg0" />
</concept>
<concept id="1197686869805" name="jetbrains.mps.baseLanguage.collections.structure.HashMapCreator" flags="nn" index="3rGOSV">
<child id="1197687026896" name="keyType" index="3rHrn6" />
<child id="1197687035757" name="valueType" index="3rHtpV" />
</concept>
<concept id="1197932370469" name="jetbrains.mps.baseLanguage.collections.structure.MapElement" flags="nn" index="3EllGN">
<child id="1197932505799" name="map" index="3ElQJh" />
<child id="1197932525128" name="key" index="3ElVtu" />
</concept>
<concept id="1172254888721" name="jetbrains.mps.baseLanguage.collections.structure.ContainsOperation" flags="nn" index="3JPx81" />
</language>
</registry>
Expand Down Expand Up @@ -1148,5 +1169,103 @@
<ref role="3uigEE" to="lfzw:1F5g4zQqVSR" resolve="MoveNodesActionBase" />
</node>
</node>
<node concept="312cEu" id="4LUDPiac$re">
<property role="TrG5h" value="TypeCacheHelper" />
<node concept="312cEg" id="4LUDPiacA4g" role="jymVt">
<property role="TrG5h" value="cache" />
<node concept="3Tm6S6" id="4LUDPiacA3b" role="1B3o_S" />
<node concept="3rvAFt" id="4LUDPiacA3R" role="1tU5fm">
<node concept="3Tqbb2" id="4LUDPiacA4a" role="3rvQeY" />
<node concept="3Tqbb2" id="4LUDPiacA4d" role="3rvSg0" />
</node>
<node concept="2ShNRf" id="4LUDPiacA79" role="33vP2m">
<node concept="3rGOSV" id="4LUDPiacA6P" role="2ShVmc">
<node concept="3Tqbb2" id="4LUDPiacA6Q" role="3rHrn6" />
<node concept="3Tqbb2" id="4LUDPiacA6R" role="3rHtpV" />
</node>
</node>
</node>
<node concept="3clFb_" id="4LUDPiac$sG" role="jymVt">
<property role="TrG5h" value="typeOf" />
<node concept="3clFbS" id="4LUDPiac$sJ" role="3clF47">
<node concept="3cpWs8" id="4LUDPiacAfu" role="3cqZAp">
<node concept="3cpWsn" id="4LUDPiacAfx" role="3cpWs9">
<property role="TrG5h" value="fromCache" />
<node concept="3Tqbb2" id="4LUDPiacAfs" role="1tU5fm" />
<node concept="3EllGN" id="4LUDPiacAI9" role="33vP2m">
<node concept="37vLTw" id="4LUDPiacAJG" role="3ElVtu">
<ref role="3cqZAo" node="4LUDPiac$t7" resolve="n" />
</node>
<node concept="37vLTw" id="4LUDPiacAiU" role="3ElQJh">
<ref role="3cqZAo" node="4LUDPiacA4g" resolve="cache" />
</node>
</node>
</node>
</node>
<node concept="3clFbJ" id="4LUDPiacAMG" role="3cqZAp">
<node concept="3clFbS" id="4LUDPiacAMI" role="3clFbx">
<node concept="3cpWs6" id="4LUDPiacB7s" role="3cqZAp">
<node concept="37vLTw" id="4LUDPiacB8d" role="3cqZAk">
<ref role="3cqZAo" node="4LUDPiacAfx" resolve="fromCache" />
</node>
</node>
</node>
<node concept="3y3z36" id="4LUDPiacAZ_" role="3clFbw">
<node concept="10Nm6u" id="4LUDPiacB5y" role="3uHU7w" />
<node concept="37vLTw" id="4LUDPiacAOP" role="3uHU7B">
<ref role="3cqZAo" node="4LUDPiacAfx" resolve="fromCache" />
</node>
</node>
</node>
<node concept="3cpWs8" id="4LUDPiacBcJ" role="3cqZAp">
<node concept="3cpWsn" id="4LUDPiacBcK" role="3cpWs9">
<property role="TrG5h" value="typeOf" />
<node concept="3uibUv" id="4LUDPiacBak" role="1tU5fm">
<ref role="3uigEE" to="mhbf:~SNode" resolve="SNode" />
</node>
<node concept="2OqwBi" id="4LUDPiacBcL" role="33vP2m">
<node concept="2YIFZM" id="4LUDPiacBcM" role="2Oq$k0">
<ref role="37wK5l" to="1ka:~TypecheckingFacade.getFromContext()" resolve="getFromContext" />
<ref role="1Pybhc" to="1ka:~TypecheckingFacade" resolve="TypecheckingFacade" />
</node>
<node concept="liA8E" id="4LUDPiacBcN" role="2OqNvi">
<ref role="37wK5l" to="1ka:~TypecheckingFacade.getTypeOf(org.jetbrains.mps.openapi.model.SNode)" resolve="getTypeOf" />
<node concept="37vLTw" id="4LUDPiacBcO" role="37wK5m">
<ref role="3cqZAo" node="4LUDPiac$t7" resolve="n" />
</node>
</node>
</node>
</node>
</node>
<node concept="3clFbF" id="4LUDPiacBoA" role="3cqZAp">
<node concept="37vLTI" id="4LUDPiacBZr" role="3clFbG">
<node concept="37vLTw" id="4LUDPiacC2V" role="37vLTx">
<ref role="3cqZAo" node="4LUDPiacBcK" resolve="typeOf" />
</node>
<node concept="3EllGN" id="4LUDPiacBNs" role="37vLTJ">
<node concept="37vLTw" id="4LUDPiacBQr" role="3ElVtu">
<ref role="3cqZAo" node="4LUDPiac$t7" resolve="n" />
</node>
<node concept="37vLTw" id="4LUDPiacBo$" role="3ElQJh">
<ref role="3cqZAo" node="4LUDPiacA4g" resolve="cache" />
</node>
</node>
</node>
</node>
<node concept="3cpWs6" id="4LUDPiacC97" role="3cqZAp">
<node concept="37vLTw" id="4LUDPiacCi0" role="3cqZAk">
<ref role="3cqZAo" node="4LUDPiacBcK" resolve="typeOf" />
</node>
</node>
</node>
<node concept="3Tm1VV" id="4LUDPiac$si" role="1B3o_S" />
<node concept="3Tqbb2" id="4LUDPiac$sx" role="3clF45" />
<node concept="37vLTG" id="4LUDPiac$t7" role="3clF46">
<property role="TrG5h" value="n" />
<node concept="3Tqbb2" id="4LUDPiac$t6" role="1tU5fm" />
</node>
</node>
<node concept="3Tm1VV" id="4LUDPiac$rf" role="1B3o_S" />
</node>
</model>

1 change: 1 addition & 0 deletions code/languages/com.mbeddr.core/languages/com.mbeddr.core.modules/modules.mpl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
<dependency reexport="false">6d11763d-483d-4b2b-8efc-09336c1b0001(com.mbeddr.core.modules)</dependency>
<dependency reexport="false">be090156-a678-454c-bd8f-89ebd4a0e6af(com.mbeddr.core.modules.gen#1758019824472882132)</dependency>
<dependency reexport="false">2ed50273-af07-4e30-9004-b1f89545178a(com.mbeddr.core.stdlib)</dependency>
<dependency reexport="false">8865b7a8-5271-43d3-884c-6fd1d9cfdd34(MPS.OpenAPI)</dependency>
</dependencies>
<languageVersions>
<language slang="l:d4280a54-f6df-4383-aa41-d1b2bffa7eb1:com.mbeddr.core.base" version="6" />
Expand Down
45 changes: 21 additions & 24 deletions .../com.mbeddr.core/languages/com.mbeddr.core.unittest/generator/template/[email protected]
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1857,18 +1857,27 @@
<ref role="30HIoZ" to="yz9a:6Iyv0noEbti" resolve="StructuredBinOpAssertStatement" />
<node concept="30G5F_" id="O5IvPFxxjZ" role="30HLyM">
<node concept="3clFbS" id="O5IvPFxxk0" role="2VODD2">
<node concept="3cpWs8" id="4LUDPiakCma" role="3cqZAp">
<node concept="3cpWsn" id="4LUDPiakCmb" role="3cpWs9">
<property role="TrG5h" value="type" />
<node concept="3Tqbb2" id="4LUDPiakxr2" role="1tU5fm" />
<node concept="2OqwBi" id="4LUDPiakCmc" role="33vP2m">
<node concept="2OqwBi" id="4LUDPiakCmd" role="2Oq$k0">
<node concept="3TrEf2" id="4LUDPiakCme" role="2OqNvi">
<ref role="3Tt5mk" to="yz9a:6Iyv0noEbtl" resolve="actual" />
</node>
<node concept="30H73N" id="4LUDPiakCmf" role="2Oq$k0" />
</node>
<node concept="3JvlWi" id="4LUDPiakCmg" role="2OqNvi" />
</node>
</node>
</node>
<node concept="3clFbF" id="O5IvPFxyrv" role="3cqZAp">
<node concept="1Wc70l" id="shBE9KlgXq" role="3clFbG">
<node concept="3fqX7Q" id="shBE9Klhhr" role="3uHU7w">
<node concept="2OqwBi" id="shBE9KlnXo" role="3fr31v">
<node concept="2OqwBi" id="shBE9KlltY" role="2Oq$k0">
<node concept="2OqwBi" id="shBE9Kli69" role="2Oq$k0">
<node concept="30H73N" id="shBE9KlhzX" role="2Oq$k0" />
<node concept="3TrEf2" id="shBE9Klk3k" role="2OqNvi">
<ref role="3Tt5mk" to="yz9a:6Iyv0noEbtl" resolve="actual" />
</node>
</node>
<node concept="3JvlWi" id="shBE9KlmQE" role="2OqNvi" />
<node concept="37vLTw" id="4LUDPiakCmi" role="2Oq$k0">
<ref role="3cqZAo" node="4LUDPiakCmb" resolve="type" />
</node>
<node concept="1mIQ4w" id="shBE9KloXK" role="2OqNvi">
<node concept="chp4Y" id="shBE9KlpgB" role="cj9EA">
Expand All @@ -1880,14 +1889,8 @@
<node concept="1Wc70l" id="5yEkkqTj3sy" role="3uHU7B">
<node concept="3fqX7Q" id="O5IvPFxAS4" role="3uHU7B">
<node concept="2OqwBi" id="O5IvPFxAS6" role="3fr31v">
<node concept="2OqwBi" id="O5IvPFxAS7" role="2Oq$k0">
<node concept="2OqwBi" id="O5IvPFxAS8" role="2Oq$k0">
<node concept="3TrEf2" id="5D$IuLxkv8i" role="2OqNvi">
<ref role="3Tt5mk" to="yz9a:6Iyv0noEbtl" resolve="actual" />
</node>
<node concept="30H73N" id="O5IvPFxAS9" role="2Oq$k0" />
</node>
<node concept="3JvlWi" id="O5IvPFxASb" role="2OqNvi" />
<node concept="37vLTw" id="4LUDPiakCmh" role="2Oq$k0">
<ref role="3cqZAo" node="4LUDPiakCmb" resolve="type" />
</node>
<node concept="1mIQ4w" id="O5IvPFxASc" role="2OqNvi">
<node concept="chp4Y" id="O5IvPFxASd" role="cj9EA">
Expand All @@ -1898,14 +1901,8 @@
</node>
<node concept="3fqX7Q" id="5yEkkqTj3yo" role="3uHU7w">
<node concept="2OqwBi" id="5yEkkqTj3yp" role="3fr31v">
<node concept="2OqwBi" id="5yEkkqTj3yq" role="2Oq$k0">
<node concept="2OqwBi" id="5yEkkqTj3yr" role="2Oq$k0">
<node concept="3TrEf2" id="5yEkkqTj3ys" role="2OqNvi">
<ref role="3Tt5mk" to="yz9a:6Iyv0noEbtl" resolve="actual" />
</node>
<node concept="30H73N" id="5yEkkqTj3yt" role="2Oq$k0" />
</node>
<node concept="3JvlWi" id="5yEkkqTj3yu" role="2OqNvi" />
<node concept="37vLTw" id="4LUDPiakCmj" role="2Oq$k0">
<ref role="3cqZAo" node="4LUDPiakCmb" resolve="type" />
</node>
<node concept="1mIQ4w" id="5yEkkqTj3yv" role="2OqNvi">
<node concept="chp4Y" id="5yEkkqTj3Cu" role="cj9EA">
Expand Down
26 changes: 16 additions & 10 deletions ...ages/com.mbeddr.core/languages/com.mbeddr.core.util/generator/template/[email protected]
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26485,6 +26485,18 @@
</node>
</node>
</node>
<node concept="3cpWs8" id="4LUDPiakDe5" role="3cqZAp">
<node concept="3cpWsn" id="4LUDPiakDe6" role="3cpWs9">
<property role="TrG5h" value="closureType" />
<node concept="3Tqbb2" id="4LUDPiaknCv" role="1tU5fm" />
<node concept="2OqwBi" id="4LUDPiakDe7" role="33vP2m">
<node concept="37vLTw" id="4LUDPiakDe8" role="2Oq$k0">
<ref role="3cqZAo" node="1$YD8rkT2zE" resolve="closure" />
</node>
<node concept="3JvlWi" id="4LUDPiakDe9" role="2OqNvi" />
</node>
</node>
</node>
<node concept="3cpWs6" id="7XRsj4rjZs9" role="3cqZAp">
<node concept="1Wc70l" id="7XRsj4rjWUc" role="3cqZAk">
<node concept="3clFbC" id="7XRsj4rjXlj" role="3uHU7w">
Expand All @@ -26501,11 +26513,8 @@
<node concept="1eOMI4" id="7XRsj4rjY2t" role="3uHU7B">
<node concept="1Wc70l" id="7XRsj4rjPKa" role="1eOMHV">
<node concept="2OqwBi" id="7XRsj4rjOBI" role="3uHU7B">
<node concept="2OqwBi" id="7XRsj4rjNJM" role="2Oq$k0">
<node concept="37vLTw" id="7XRsj4rjNJN" role="2Oq$k0">
<ref role="3cqZAo" node="1$YD8rkT2zE" resolve="closure" />
</node>
<node concept="3JvlWi" id="7XRsj4rjNJO" role="2OqNvi" />
<node concept="37vLTw" id="4LUDPiakDea" role="2Oq$k0">
<ref role="3cqZAo" node="4LUDPiakDe6" resolve="closureType" />
</node>
<node concept="1mIQ4w" id="7XRsj4rjP0V" role="2OqNvi">
<node concept="chp4Y" id="7XRsj4rjPlr" role="cj9EB">
Expand All @@ -26517,11 +26526,8 @@
<node concept="2OqwBi" id="7XRsj4rjTQb" role="3fr31v">
<node concept="2OqwBi" id="7XRsj4rjS4_" role="2Oq$k0">
<node concept="1PxgMI" id="7XRsj4rjRm8" role="2Oq$k0">
<node concept="2OqwBi" id="7XRsj4rjQ4Y" role="1m5AlR">
<node concept="37vLTw" id="7XRsj4rjQ4Z" role="2Oq$k0">
<ref role="3cqZAo" node="1$YD8rkT2zE" resolve="closure" />
</node>
<node concept="3JvlWi" id="7XRsj4rjQ50" role="2OqNvi" />
<node concept="37vLTw" id="4LUDPiakDeb" role="1m5AlR">
<ref role="3cqZAo" node="4LUDPiakDe6" resolve="closureType" />
</node>
<node concept="chp4Y" id="1SbcsM$FvRY" role="3oSUPX">
<ref role="cht4Q" to="x27k:7qHzltJ03ir" resolve="FunctionRefType" />
Expand Down

0 comments on commit 591162d

Please sign in to comment.