Seeking advice for setting up solver for finite element

[kiendangtor]

Environment information:

OS: Windows 11
CUDA runtime: CUDA 12.4.
AMGX version or commit hash: 2.5.0
NVIDIA driver: 551.78
NVIDIA GPU: NVIDIA RTX 3060 Ti

AMGX solver configuration: This is one of AMG configuration that I tried to use but failed to converged.
{
"config_version": 2,
"solver": {
"matrix_coloring_scheme": "MIN_MAX",
"max_uncolored_percentage": 0.15,
"algorithm": "AGGREGATION",
"obtain_timings": 1,
"solver": "AMG",
"smoother": "MULTICOLOR_DILU",
"print_solve_stats": 1,
"presweeps": 1,
"selector": "SIZE_2",
"coarsest_sweeps": 2,
"max_iters": 1000,
"monitor_residual": 1,
"scope": "main",
"max_levels": 1000,
"postsweeps": 1,
"tolerance": 0.1,
"print_grid_stats": 1,
"norm": "L1",
"cycle": "V"
}
}

Matrix is attached

Reproduction steps

I ran this with the command line as in the quick start document :"amgx_capi -m identity_system.mtx -c AGGREGATION_DILU.json

It got diverged. Samte thing with AMG_CLASSICAL_CG.json. Using AMG_AGGRREGATION_CG it manage to converge but very slow

If I use PCG_AGGREGATION_JACOBI i got the best convergence speed.

Can you please tell me which option should I choose for the type of matrix that I attached? This is basically a 10 nodes tet mesh generated from a 3D solid dishes.

Thanks,

Kien

coarse.zip

coarse.zip

[marsaev]

@kiendangtor I apologize for the slow response.

This matrix is quite small to exploit full GPU power. Would you consider direct solve approach here?

Other than that, configurations from provided examples are typical for this type of meshes (of course also depending on what are you trying to solve). I had in mind something like "config scanner" to find best solver options for given matrix, but no ETA on that.

[kiendangtor]

Thank you for your reply. "config scanner" could be awesome. I had bigger mesh with 20mil of non zero but to make the diagnosis easier I attached a very coarse mesh. But the behavior are the same. I was able to successfully run the solver with many configs but all the one that has "MULTICOLOR_DILU" did not work either as a smoother or as preconditioner. Do we need to provide graph separately? I look at the code and it doesnt seem like it need that.

[marsaev]

Understood, thanks. Right, starting with smaller matrix is often easier to put aside what's clearly not working.
Just wanted to clarify some things

1. Do we need to provide graph separately?
   At this moment the linear system (A, rhs, optional x_0) is the only input AMGX is working with.
2. I was able to successfully run the solver with many configs
   So for the larger system (i.e. 20m NNZ, as you have mentioned above) many configs work, but they are still not fast enough? Or what kind of problem you experience there? Can you provide output of the best performing config for your larger case?
3. all the one that has "MULTICOLOR_DILU" did not work either as a smoother or as preconditioner.
   Can you elaborate what you mean here? In general, there is no reason to always use stronger smoother (like ILU), since they can be significantly slower than other smoothers, and we can try to compensate convergence rate by adjusting multigrid structure.

Thanks,

[kiendangtor]

Thank you for your fast response. I have upload the larger system here: https://we.tl/t-ymNm1kBuan
2. My issue is that the CG solver without preconditioner is the best option so far in my case.. Here is my best configuration (CG without Precond, took ~ 6s)
{
"config_version": 2,
"solver": {
"solver": "CG",
"print_solve_stats": 1,
"obtain_timings": 1,
"max_iters": 2000,
"monitor_residual": 1,
"convergence": "RELATIVE_INI_CORE",
"scope": "main",
"tolerance": 1e-6,
"norm": "L2"
}
}

Here is the output:
AMGX version 2.5.0
Built on Apr 25 2024, 13:20:11
Compiled with CUDA Runtime 12.4, using CUDA driver 12.4
iter Mem Usage (GB) residual rate
----------------------------------------------------------------------
Ini 0 1.353886e+01
0 0 1.262998e+01 0.9329
1 0.0000 1.344963e+01 1.0649
2 0.0000 1.341992e+01 0.9978
3 0.0000 1.287418e+01 0.9593
4 0.0000 1.291408e+01 1.0031
5 0.0000 1.116468e+01 0.8645
6 0.0000 1.006132e+01 0.9012
7 0.0000 9.782631e+00 0.9723
8 0.0000 9.901049e+00 1.0121
9 0.0000 9.289660e+00 0.9383
10 0.0000 8.494891e+00 0.9144
11 0.0000 8.746236e+00 1.0296
12 0.0000 9.184710e+00 1.0501
13 0.0000 8.967623e+00 0.9764
14 0.0000 8.210301e+00 0.9155
15 0.0000 7.656633e+00 0.9326
16 0.0000 7.515435e+00 0.9816
17 0.0000 7.752882e+00 1.0316
18 0.0000 7.573552e+00 0.9769
19 0.0000 6.960823e+00 0.9191
20 0.0000 6.493094e+00 0.9328
21 0.0000 6.856289e+00 1.0559
22 0.0000 7.807127e+00 1.1387
23 0.0000 7.382842e+00 0.9457
24 0.0000 6.450630e+00 0.8737
25 0.0000 6.198494e+00 0.9609
26 0.0000 6.606438e+00 1.0658
27 0.0000 6.915229e+00 1.0467
28 0.0000 6.720466e+00 0.9718
29 0.0000 6.636989e+00 0.9876
30 0.0000 6.287094e+00 0.9473
31 0.0000 5.595264e+00 0.8900
32 0.0000 5.520952e+00 0.9867
33 0.0000 5.446880e+00 0.9866
34 0.0000 5.337425e+00 0.9799
35 0.0000 5.085706e+00 0.9528
36 0.0000 4.873419e+00 0.9583
37 0.0000 4.674906e+00 0.9593
38 0.0000 4.506826e+00 0.9640
39 0.0000 4.389167e+00 0.9739
40 0.0000 4.533550e+00 1.0329
41 0.0000 4.536435e+00 1.0006
42 0.0000 4.155694e+00 0.9161
43 0.0000 4.163612e+00 1.0019
44 0.0000 4.321235e+00 1.0379
45 0.0000 3.934455e+00 0.9105
46 0.0000 3.598542e+00 0.9146
47 0.0000 3.606063e+00 1.0021
48 0.0000 3.828375e+00 1.0616
49 0.0000 3.652129e+00 0.9540
50 0.0000 3.340484e+00 0.9147
51 0.0000 3.341184e+00 1.0002
52 0.0000 3.360057e+00 1.0056
53 0.0000 3.161396e+00 0.9409
54 0.0000 3.171314e+00 1.0031
55 0.0000 3.069862e+00 0.9680
56 0.0000 2.932093e+00 0.9551
57 0.0000 2.925210e+00 0.9977
58 0.0000 2.768549e+00 0.9464
59 0.0000 2.684689e+00 0.9697
60 0.0000 2.733094e+00 1.0180
61 0.0000 2.531500e+00 0.9262
62 0.0000 2.409799e+00 0.9519
63 0.0000 2.575164e+00 1.0686
64 0.0000 2.486035e+00 0.9654
65 0.0000 2.201733e+00 0.8856
66 0.0000 2.209115e+00 1.0034
67 0.0000 2.206543e+00 0.9988
68 0.0000 2.200250e+00 0.9971
69 0.0000 2.254593e+00 1.0247
70 0.0000 2.201090e+00 0.9763
71 0.0000 2.113318e+00 0.9601
72 0.0000 2.080655e+00 0.9845
73 0.0000 2.179618e+00 1.0476
74 0.0000 2.060706e+00 0.9454
75 0.0000 2.057177e+00 0.9983
76 0.0000 2.110698e+00 1.0260
77 0.0000 1.945716e+00 0.9218
78 0.0000 1.900318e+00 0.9767
79 0.0000 1.922642e+00 1.0117
80 0.0000 1.836647e+00 0.9553
81 0.0000 1.834999e+00 0.9991
82 0.0000 1.910327e+00 1.0411
83 0.0000 1.781650e+00 0.9326
84 0.0000 1.714393e+00 0.9622
85 0.0000 1.695226e+00 0.9888
86 0.0000 1.698762e+00 1.0021
87 0.0000 1.756629e+00 1.0341
88 0.0000 1.720690e+00 0.9795
89 0.0000 1.655724e+00 0.9622
90 0.0000 1.667694e+00 1.0072
91 0.0000 1.695344e+00 1.0166
92 0.0000 1.588701e+00 0.9371
93 0.0000 1.541870e+00 0.9705
94 0.0000 1.582440e+00 1.0263
95 0.0000 1.581971e+00 0.9997
96 0.0000 1.522868e+00 0.9626
97 0.0000 1.524990e+00 1.0014
98 0.0000 1.475319e+00 0.9674
99 0.0000 1.471768e+00 0.9976
100 0.0000 1.501872e+00 1.0205
101 0.0000 1.425984e+00 0.9495
102 0.0000 1.397138e+00 0.9798
103 0.0000 1.411637e+00 1.0104
104 0.0000 1.376674e+00 0.9752
105 0.0000 1.334801e+00 0.9696
106 0.0000 1.331577e+00 0.9976
107 0.0000 1.352925e+00 1.0160
108 0.0000 1.282638e+00 0.9480
109 0.0000 1.289760e+00 1.0056
110 0.0000 1.311367e+00 1.0168
111 0.0000 1.271219e+00 0.9694
112 0.0000 1.256974e+00 0.9888
113 0.0000 1.279412e+00 1.0179
114 0.0000 1.257423e+00 0.9828
115 0.0000 1.231465e+00 0.9794
116 0.0000 1.235452e+00 1.0032
117 0.0000 1.186845e+00 0.9607
118 0.0000 1.158272e+00 0.9759
119 0.0000 1.163486e+00 1.0045
120 0.0000 1.146347e+00 0.9853
121 0.0000 1.128234e+00 0.9842
122 0.0000 1.151797e+00 1.0209
123 0.0000 1.163217e+00 1.0099
124 0.0000 1.141058e+00 0.9810
125 0.0000 1.130028e+00 0.9903
126 0.0000 1.090592e+00 0.9651
127 0.0000 1.084568e+00 0.9945
128 0.0000 1.052836e+00 0.9707
129 0.0000 1.031181e+00 0.9794
130 0.0000 1.057783e+00 1.0258
131 0.0000 1.037753e+00 0.9811
132 0.0000 1.003208e+00 0.9667
133 0.0000 1.009502e+00 1.0063
134 0.0000 1.015455e+00 1.0059
135 0.0000 9.903361e-01 0.9753
136 0.0000 9.782567e-01 0.9878
137 0.0000 9.792711e-01 1.0010
138 0.0000 9.522720e-01 0.9724
139 0.0000 9.373981e-01 0.9844
140 0.0000 9.393499e-01 1.0021
141 0.0000 9.349063e-01 0.9953
142 0.0000 9.307010e-01 0.9955
143 0.0000 9.104812e-01 0.9783
144 0.0000 9.000397e-01 0.9885
145 0.0000 8.931340e-01 0.9923
146 0.0000 8.813223e-01 0.9868
147 0.0000 8.854138e-01 1.0046
148 0.0000 8.714043e-01 0.9842
149 0.0000 8.596665e-01 0.9865
150 0.0000 8.388330e-01 0.9758
151 0.0000 8.474289e-01 1.0102
152 0.0000 8.478095e-01 1.0004
153 0.0000 8.403001e-01 0.9911
154 0.0000 8.205706e-01 0.9765
155 0.0000 8.303758e-01 1.0119
156 0.0000 8.393402e-01 1.0108
157 0.0000 8.239402e-01 0.9817
158 0.0000 8.310023e-01 1.0086
159 0.0000 8.476804e-01 1.0201
160 0.0000 8.022486e-01 0.9464
161 0.0000 7.757737e-01 0.9670
162 0.0000 7.889112e-01 1.0169
163 0.0000 7.860369e-01 0.9964
164 0.0000 7.505643e-01 0.9549
165 0.0000 7.480393e-01 0.9966
166 0.0000 7.573304e-01 1.0124
167 0.0000 7.290384e-01 0.9626
168 0.0000 7.477997e-01 1.0257
169 0.0000 7.526946e-01 1.0065
170 0.0000 7.233232e-01 0.9610
171 0.0000 7.462722e-01 1.0317
172 0.0000 7.483545e-01 1.0028
173 0.0000 7.120085e-01 0.9514
174 0.0000 7.181118e-01 1.0086
175 0.0000 7.167547e-01 0.9981
176 0.0000 7.207807e-01 1.0056
177 0.0000 7.161876e-01 0.9936
178 0.0000 6.931766e-01 0.9679
179 0.0000 6.684345e-01 0.9643
180 0.0000 6.566061e-01 0.9823
181 0.0000 6.584297e-01 1.0028
182 0.0000 6.757532e-01 1.0263
183 0.0000 6.572882e-01 0.9727
184 0.0000 6.698679e-01 1.0191
185 0.0000 6.785304e-01 1.0129
186 0.0000 6.390069e-01 0.9418
187 0.0000 6.447940e-01 1.0091
188 0.0000 6.370597e-01 0.9880
189 0.0000 6.382035e-01 1.0018
190 0.0000 6.326952e-01 0.9914
191 0.0000 6.173935e-01 0.9758
192 0.0000 6.126585e-01 0.9923
193 0.0000 6.200593e-01 1.0121
194 0.0000 6.183008e-01 0.9972
195 0.0000 6.195359e-01 1.0020
196 0.0000 6.215696e-01 1.0033
197 0.0000 6.009018e-01 0.9667
198 0.0000 6.136047e-01 1.0211
199 0.0000 6.118257e-01 0.9971
200 0.0000 6.168415e-01 1.0082
201 0.0000 6.159987e-01 0.9986
202 0.0000 5.853574e-01 0.9503
203 0.0000 5.884180e-01 1.0052
204 0.0000 6.030999e-01 1.0250
205 0.0000 5.890910e-01 0.9768
206 0.0000 5.823547e-01 0.9886
207 0.0000 5.855304e-01 1.0055
208 0.0000 5.724553e-01 0.9777
209 0.0000 5.614496e-01 0.9808
210 0.0000 5.762443e-01 1.0264
211 0.0000 5.673486e-01 0.9846
212 0.0000 5.650837e-01 0.9960
213 0.0000 5.709320e-01 1.0103
214 0.0000 5.884823e-01 1.0307
215 0.0000 5.737960e-01 0.9750
216 0.0000 5.553936e-01 0.9679
217 0.0000 5.511891e-01 0.9924
218 0.0000 5.436915e-01 0.9864
219 0.0000 5.523357e-01 1.0159
220 0.0000 5.481563e-01 0.9924
221 0.0000 5.295897e-01 0.9661
222 0.0000 5.276222e-01 0.9963
223 0.0000 5.382623e-01 1.0202
224 0.0000 5.356678e-01 0.9952
225 0.0000 5.328757e-01 0.9948
226 0.0000 5.263735e-01 0.9878
227 0.0000 5.179160e-01 0.9839
228 0.0000 5.029985e-01 0.9712
229 0.0000 4.939617e-01 0.9820
230 0.0000 4.967452e-01 1.0056
231 0.0000 4.767040e-01 0.9597
232 0.0000 4.719018e-01 0.9899
233 0.0000 4.827788e-01 1.0230
234 0.0000 4.641570e-01 0.9614
235 0.0000 4.580175e-01 0.9868
236 0.0000 4.626222e-01 1.0101
237 0.0000 4.577362e-01 0.9894
238 0.0000 4.635887e-01 1.0128
239 0.0000 4.624222e-01 0.9975
240 0.0000 4.438046e-01 0.9597
241 0.0000 4.463173e-01 1.0057
242 0.0000 4.554095e-01 1.0204
243 0.0000 4.301062e-01 0.9444
244 0.0000 4.336583e-01 1.0083
245 0.0000 4.284497e-01 0.9880
246 0.0000 4.189278e-01 0.9778
247 0.0000 4.373496e-01 1.0440
248 0.0000 4.332623e-01 0.9907
249 0.0000 4.345460e-01 1.0030
250 0.0000 4.393239e-01 1.0110
251 0.0000 4.207781e-01 0.9578
252 0.0000 4.332030e-01 1.0295
253 0.0000 4.239820e-01 0.9787
254 0.0000 4.211489e-01 0.9933
255 0.0000 4.192285e-01 0.9954
256 0.0000 4.034146e-01 0.9623
257 0.0000 4.089019e-01 1.0136
258 0.0000 4.069869e-01 0.9953
259 0.0000 4.020965e-01 0.9880
260 0.0000 4.101955e-01 1.0201
261 0.0000 4.054649e-01 0.9885
262 0.0000 3.963023e-01 0.9774
263 0.0000 3.990551e-01 1.0069
264 0.0000 3.803718e-01 0.9532
265 0.0000 3.792435e-01 0.9970
266 0.0000 3.828624e-01 1.0095
267 0.0000 3.575243e-01 0.9338
268 0.0000 3.442248e-01 0.9628
269 0.0000 3.424394e-01 0.9948
270 0.0000 3.271007e-01 0.9552
271 0.0000 3.143270e-01 0.9609
272 0.0000 3.144425e-01 1.0004
273 0.0000 3.037625e-01 0.9660
274 0.0000 2.861743e-01 0.9421
275 0.0000 2.769139e-01 0.9676
276 0.0000 2.671295e-01 0.9647
277 0.0000 2.532142e-01 0.9479
278 0.0000 2.375813e-01 0.9383
279 0.0000 2.234230e-01 0.9404
280 0.0000 2.069208e-01 0.9261
281 0.0000 1.929073e-01 0.9323
282 0.0000 1.848349e-01 0.9582
283 0.0000 1.809836e-01 0.9792
284 0.0000 1.736374e-01 0.9594
285 0.0000 1.527659e-01 0.8798
286 0.0000 1.417858e-01 0.9281
287 0.0000 1.315394e-01 0.9277
288 0.0000 1.235922e-01 0.9396
289 0.0000 1.162488e-01 0.9406
290 0.0000 1.119325e-01 0.9629
291 0.0000 1.068759e-01 0.9548
292 0.0000 9.955425e-02 0.9315
293 0.0000 9.073929e-02 0.9115
294 0.0000 8.643774e-02 0.9526
295 0.0000 8.299320e-02 0.9602
296 0.0000 8.080298e-02 0.9736
297 0.0000 7.763006e-02 0.9607
298 0.0000 7.273561e-02 0.9370
299 0.0000 7.232574e-02 0.9944
300 0.0000 6.658044e-02 0.9206
301 0.0000 6.364191e-02 0.9559
302 0.0000 6.233454e-02 0.9795
303 0.0000 6.132779e-02 0.9838
304 0.0000 5.801469e-02 0.9460
305 0.0000 5.795389e-02 0.9990
306 0.0000 5.633932e-02 0.9721
307 0.0000 5.434833e-02 0.9647
308 0.0000 5.382773e-02 0.9904
309 0.0000 5.311001e-02 0.9867
310 0.0000 5.027714e-02 0.9467
311 0.0000 4.726738e-02 0.9401
312 0.0000 4.629796e-02 0.9795
313 0.0000 4.521235e-02 0.9766
314 0.0000 4.273949e-02 0.9453
315 0.0000 4.155467e-02 0.9723
316 0.0000 4.217253e-02 1.0149
317 0.0000 3.988379e-02 0.9457
318 0.0000 3.859674e-02 0.9677
319 0.0000 3.897464e-02 1.0098
320 0.0000 3.746127e-02 0.9612
321 0.0000 3.647192e-02 0.9736
322 0.0000 3.558145e-02 0.9756
323 0.0000 3.408042e-02 0.9578
324 0.0000 3.336385e-02 0.9790
325 0.0000 3.280365e-02 0.9832
326 0.0000 3.194300e-02 0.9738
327 0.0000 3.162545e-02 0.9901
328 0.0000 3.207835e-02 1.0143
329 0.0000 3.056155e-02 0.9527
330 0.0000 2.968799e-02 0.9714
331 0.0000 2.903696e-02 0.9781
332 0.0000 2.764549e-02 0.9521
333 0.0000 2.757215e-02 0.9973
334 0.0000 2.787487e-02 1.0110
335 0.0000 2.687063e-02 0.9640
336 0.0000 2.648189e-02 0.9855
337 0.0000 2.687394e-02 1.0148
338 0.0000 2.693819e-02 1.0024
339 0.0000 2.627692e-02 0.9755
340 0.0000 2.556245e-02 0.9728
341 0.0000 2.536271e-02 0.9922
342 0.0000 2.513220e-02 0.9909
343 0.0000 2.541615e-02 1.0113
344 0.0000 2.598969e-02 1.0226
345 0.0000 2.565233e-02 0.9870
346 0.0000 2.455921e-02 0.9574
347 0.0000 2.338488e-02 0.9522
348 0.0000 2.328266e-02 0.9956
349 0.0000 2.367848e-02 1.0170
350 0.0000 2.278399e-02 0.9622
351 0.0000 2.238886e-02 0.9827
352 0.0000 2.245145e-02 1.0028
353 0.0000 2.184556e-02 0.9730
354 0.0000 2.186281e-02 1.0008
355 0.0000 2.197417e-02 1.0051
356 0.0000 2.193714e-02 0.9983
357 0.0000 2.215427e-02 1.0099
358 0.0000 2.194692e-02 0.9906
359 0.0000 2.112797e-02 0.9627
360 0.0000 2.072562e-02 0.9810
361 0.0000 2.087301e-02 1.0071
362 0.0000 2.067195e-02 0.9904
363 0.0000 2.124003e-02 1.0275
364 0.0000 2.074068e-02 0.9765
365 0.0000 1.932746e-02 0.9319
366 0.0000 1.964174e-02 1.0163
367 0.0000 1.952642e-02 0.9941
368 0.0000 1.912590e-02 0.9795
369 0.0000 1.956800e-02 1.0231
370 0.0000 1.892145e-02 0.9670
371 0.0000 1.880505e-02 0.9938
372 0.0000 1.935910e-02 1.0295
373 0.0000 1.886556e-02 0.9745
374 0.0000 1.892023e-02 1.0029
375 0.0000 1.862439e-02 0.9844
376 0.0000 1.825186e-02 0.9800
377 0.0000 1.813391e-02 0.9935
378 0.0000 1.811987e-02 0.9992
379 0.0000 1.783458e-02 0.9843
380 0.0000 1.785637e-02 1.0012
381 0.0000 1.832706e-02 1.0264
382 0.0000 1.819248e-02 0.9927
383 0.0000 1.717503e-02 0.9441
384 0.0000 1.694985e-02 0.9869
385 0.0000 1.674220e-02 0.9877
386 0.0000 1.661223e-02 0.9922
387 0.0000 1.686434e-02 1.0152
388 0.0000 1.687965e-02 1.0009
389 0.0000 1.698021e-02 1.0060
390 0.0000 1.645316e-02 0.9690
391 0.0000 1.632727e-02 0.9923
392 0.0000 1.677217e-02 1.0272
393 0.0000 1.567514e-02 0.9346
394 0.0000 1.549359e-02 0.9884
395 0.0000 1.568206e-02 1.0122
396 0.0000 1.544631e-02 0.9850
397 0.0000 1.578358e-02 1.0218
398 0.0000 1.582254e-02 1.0025
399 0.0000 1.573691e-02 0.9946
400 0.0000 1.550577e-02 0.9853
401 0.0000 1.548083e-02 0.9984
402 0.0000 1.535429e-02 0.9918
403 0.0000 1.572514e-02 1.0242
404 0.0000 1.604480e-02 1.0203
405 0.0000 1.601420e-02 0.9981
406 0.0000 1.585728e-02 0.9902
407 0.0000 1.531246e-02 0.9656
408 0.0000 1.513600e-02 0.9885
409 0.0000 1.563596e-02 1.0330
410 0.0000 1.568023e-02 1.0028
411 0.0000 1.608473e-02 1.0258
412 0.0000 1.529724e-02 0.9510
413 0.0000 1.491083e-02 0.9747
414 0.0000 1.598739e-02 1.0722
415 0.0000 1.578798e-02 0.9875
416 0.0000 1.515374e-02 0.9598
417 0.0000 1.520244e-02 1.0032
418 0.0000 1.470155e-02 0.9671
419 0.0000 1.456586e-02 0.9908
420 0.0000 1.469726e-02 1.0090
421 0.0000 1.468928e-02 0.9995
422 0.0000 1.437507e-02 0.9786
423 0.0000 1.442039e-02 1.0032
424 0.0000 1.458873e-02 1.0117
425 0.0000 1.410793e-02 0.9670
426 0.0000 1.374792e-02 0.9745
427 0.0000 1.376393e-02 1.0012
428 0.0000 1.362299e-02 0.9898
429 0.0000 1.357906e-02 0.9968
430 0.0000 1.374911e-02 1.0125
431 0.0000 1.360199e-02 0.9893
432 0.0000 1.310474e-02 0.9634
433 0.0000 1.307918e-02 0.9980
434 0.0000 1.313347e-02 1.0042
435 0.0000 1.338300e-02 1.0190
436 0.0000 1.370956e-02 1.0244
437 0.0000 1.287422e-02 0.9391
438 0.0000 1.235340e-02 0.9595
439 0.0000 1.245901e-02 1.0085
440 0.0000 1.242344e-02 0.9971
441 0.0000 1.247323e-02 1.0040
442 0.0000 1.298080e-02 1.0407
443 0.0000 1.291163e-02 0.9947
444 0.0000 1.274599e-02 0.9872
445 0.0000 1.255987e-02 0.9854
446 0.0000 1.237453e-02 0.9852
447 0.0000 1.247371e-02 1.0080
448 0.0000 1.263250e-02 1.0127
449 0.0000 1.271840e-02 1.0068
450 0.0000 1.222111e-02 0.9609
451 0.0000 1.199371e-02 0.9814
452 0.0000 1.223683e-02 1.0203
453 0.0000 1.206892e-02 0.9863
454 0.0000 1.180749e-02 0.9783
455 0.0000 1.122894e-02 0.9510
456 0.0000 1.112267e-02 0.9905
457 0.0000 1.132348e-02 1.0181
458 0.0000 1.109869e-02 0.9801
459 0.0000 1.095065e-02 0.9867
460 0.0000 1.078360e-02 0.9847
461 0.0000 1.050138e-02 0.9738
462 0.0000 1.037224e-02 0.9877
463 0.0000 1.042789e-02 1.0054
464 0.0000 1.052245e-02 1.0091
465 0.0000 1.026729e-02 0.9758
466 0.0000 9.842529e-03 0.9586
467 0.0000 9.732925e-03 0.9889
468 0.0000 9.525361e-03 0.9787
469 0.0000 9.552203e-03 1.0028
470 0.0000 9.502360e-03 0.9948
471 0.0000 9.599928e-03 1.0103
472 0.0000 9.534268e-03 0.9932
473 0.0000 9.076386e-03 0.9520
474 0.0000 8.959994e-03 0.9872
475 0.0000 8.939976e-03 0.9978
476 0.0000 8.795774e-03 0.9839
477 0.0000 8.727889e-03 0.9923
478 0.0000 8.782370e-03 1.0062
479 0.0000 8.983462e-03 1.0229
480 0.0000 9.075476e-03 1.0102
481 0.0000 8.419634e-03 0.9277
482 0.0000 8.401494e-03 0.9978
483 0.0000 8.477547e-03 1.0091
484 0.0000 8.349500e-03 0.9849
485 0.0000 8.223551e-03 0.9849
486 0.0000 8.072738e-03 0.9817
487 0.0000 7.854028e-03 0.9729
488 0.0000 7.722498e-03 0.9833
489 0.0000 7.667248e-03 0.9928
490 0.0000 7.634887e-03 0.9958
491 0.0000 7.656882e-03 1.0029
492 0.0000 7.697057e-03 1.0052
493 0.0000 7.669751e-03 0.9965
494 0.0000 7.357037e-03 0.9592
495 0.0000 7.097171e-03 0.9647
496 0.0000 7.206420e-03 1.0154
497 0.0000 7.076045e-03 0.9819
498 0.0000 6.735162e-03 0.9518
499 0.0000 6.765319e-03 1.0045
500 0.0000 6.794066e-03 1.0042
501 0.0000 6.688231e-03 0.9844
502 0.0000 6.849964e-03 1.0242
503 0.0000 6.645858e-03 0.9702
504 0.0000 6.440888e-03 0.9692
505 0.0000 6.320286e-03 0.9813
506 0.0000 6.248014e-03 0.9886
507 0.0000 6.337786e-03 1.0144
508 0.0000 6.349945e-03 1.0019
509 0.0000 6.349234e-03 0.9999
510 0.0000 6.388300e-03 1.0062
511 0.0000 6.395770e-03 1.0012
512 0.0000 6.289377e-03 0.9834
513 0.0000 6.241988e-03 0.9925
514 0.0000 6.170202e-03 0.9885
515 0.0000 6.312940e-03 1.0231
516 0.0000 6.636119e-03 1.0512
517 0.0000 6.341295e-03 0.9556
518 0.0000 6.353455e-03 1.0019
519 0.0000 6.296889e-03 0.9911
520 0.0000 6.241926e-03 0.9913
521 0.0000 6.386142e-03 1.0231
522 0.0000 6.247452e-03 0.9783
523 0.0000 6.186063e-03 0.9902
524 0.0000 6.187890e-03 1.0003
525 0.0000 6.105420e-03 0.9867
526 0.0000 6.226842e-03 1.0199
527 0.0000 6.319307e-03 1.0148
528 0.0000 6.334320e-03 1.0024
529 0.0000 6.464382e-03 1.0205
530 0.0000 6.301481e-03 0.9748
531 0.0000 6.273641e-03 0.9956
532 0.0000 6.388512e-03 1.0183
533 0.0000 6.345122e-03 0.9932
534 0.0000 6.208837e-03 0.9785
535 0.0000 6.146809e-03 0.9900
536 0.0000 6.202854e-03 1.0091
537 0.0000 6.234117e-03 1.0050
538 0.0000 6.113430e-03 0.9806
539 0.0000 6.066137e-03 0.9923
540 0.0000 6.066997e-03 1.0001
541 0.0000 6.048647e-03 0.9970
542 0.0000 6.134174e-03 1.0141
543 0.0000 6.160458e-03 1.0043
544 0.0000 6.227700e-03 1.0109
545 0.0000 6.318068e-03 1.0145
546 0.0000 6.169826e-03 0.9765
547 0.0000 6.221217e-03 1.0083
548 0.0000 6.264204e-03 1.0069
549 0.0000 6.186179e-03 0.9875
550 0.0000 6.309242e-03 1.0199
551 0.0000 6.353479e-03 1.0070
552 0.0000 6.357793e-03 1.0007
553 0.0000 6.353085e-03 0.9993
554 0.0000 6.362545e-03 1.0015
555 0.0000 6.307434e-03 0.9913
556 0.0000 6.260310e-03 0.9925
557 0.0000 6.375725e-03 1.0184
558 0.0000 6.262039e-03 0.9822
559 0.0000 6.339788e-03 1.0124
560 0.0000 6.373469e-03 1.0053
561 0.0000 6.169795e-03 0.9680
562 0.0000 6.452991e-03 1.0459
563 0.0000 6.528671e-03 1.0117
564 0.0000 6.262205e-03 0.9592
565 0.0000 6.303491e-03 1.0066
566 0.0000 6.412179e-03 1.0172
567 0.0000 6.303144e-03 0.9830
568 0.0000 6.147397e-03 0.9753
569 0.0000 6.136071e-03 0.9982
570 0.0000 6.294733e-03 1.0259
571 0.0000 6.012360e-03 0.9551
572 0.0000 5.891027e-03 0.9798
573 0.0000 5.988166e-03 1.0165
574 0.0000 5.891136e-03 0.9838
575 0.0000 5.977501e-03 1.0147
576 0.0000 5.716100e-03 0.9563
577 0.0000 5.635954e-03 0.9860
578 0.0000 5.594633e-03 0.9927
579 0.0000 5.477058e-03 0.9790
580 0.0000 5.440232e-03 0.9933
581 0.0000 5.456996e-03 1.0031
582 0.0000 5.406281e-03 0.9907
583 0.0000 5.364208e-03 0.9922
584 0.0000 5.343921e-03 0.9962
585 0.0000 5.195428e-03 0.9722
586 0.0000 5.294369e-03 1.0190
587 0.0000 5.331546e-03 1.0070
588 0.0000 5.083982e-03 0.9536
589 0.0000 5.033618e-03 0.9901
590 0.0000 4.946860e-03 0.9828
591 0.0000 4.898466e-03 0.9902
592 0.0000 4.911797e-03 1.0027
593 0.0000 4.852804e-03 0.9880
594 0.0000 4.802611e-03 0.9897
595 0.0000 4.875350e-03 1.0151
596 0.0000 4.836140e-03 0.9920
597 0.0000 4.760769e-03 0.9844
598 0.0000 4.619881e-03 0.9704
599 0.0000 4.380142e-03 0.9481
600 0.0000 4.388399e-03 1.0019
601 0.0000 4.401163e-03 1.0029
602 0.0000 4.292870e-03 0.9754
603 0.0000 4.236514e-03 0.9869
604 0.0000 4.266072e-03 1.0070
605 0.0000 4.292588e-03 1.0062
606 0.0000 4.262486e-03 0.9930
607 0.0000 4.249779e-03 0.9970
608 0.0000 4.102022e-03 0.9652
609 0.0000 4.095270e-03 0.9984
610 0.0000 4.047164e-03 0.9883
611 0.0000 3.910163e-03 0.9661
612 0.0000 3.914555e-03 1.0011
613 0.0000 3.781342e-03 0.9660
614 0.0000 3.725926e-03 0.9853
615 0.0000 3.809098e-03 1.0223
616 0.0000 3.795154e-03 0.9963
617 0.0000 3.777809e-03 0.9954
618 0.0000 3.726416e-03 0.9864
619 0.0000 3.552350e-03 0.9533
620 0.0000 3.618827e-03 1.0187
621 0.0000 3.681537e-03 1.0173
622 0.0000 3.627929e-03 0.9854
623 0.0000 3.571412e-03 0.9844
624 0.0000 3.417510e-03 0.9569
625 0.0000 3.426221e-03 1.0025
626 0.0000 3.458219e-03 1.0093
627 0.0000 3.438220e-03 0.9942
628 0.0000 3.536513e-03 1.0286
629 0.0000 3.362617e-03 0.9508
630 0.0000 3.245816e-03 0.9653
631 0.0000 3.292259e-03 1.0143
632 0.0000 3.292518e-03 1.0001
633 0.0000 3.260045e-03 0.9901
634 0.0000 3.379410e-03 1.0366
635 0.0000 3.318488e-03 0.9820
636 0.0000 3.299100e-03 0.9942
637 0.0000 3.323998e-03 1.0075
638 0.0000 3.165984e-03 0.9525
639 0.0000 3.162043e-03 0.9988
640 0.0000 3.219586e-03 1.0182
641 0.0000 3.144344e-03 0.9766
642 0.0000 3.141328e-03 0.9990
643 0.0000 3.223903e-03 1.0263
644 0.0000 3.233494e-03 1.0030
645 0.0000 3.247744e-03 1.0044
646 0.0000 3.141434e-03 0.9673
647 0.0000 3.092526e-03 0.9844
648 0.0000 3.080675e-03 0.9962
649 0.0000 3.050095e-03 0.9901
650 0.0000 3.121482e-03 1.0234
651 0.0000 3.205082e-03 1.0268
652 0.0000 3.096339e-03 0.9661
653 0.0000 3.014102e-03 0.9734
654 0.0000 2.980229e-03 0.9888
655 0.0000 3.016261e-03 1.0121
656 0.0000 3.047112e-03 1.0102
657 0.0000 3.096890e-03 1.0163
658 0.0000 3.090594e-03 0.9980
659 0.0000 3.049689e-03 0.9868
660 0.0000 3.119639e-03 1.0229
661 0.0000 3.104548e-03 0.9952
662 0.0000 3.063124e-03 0.9867
663 0.0000 3.071626e-03 1.0028
664 0.0000 2.994386e-03 0.9749
665 0.0000 3.074427e-03 1.0267
666 0.0000 3.042785e-03 0.9897
667 0.0000 3.002771e-03 0.9868
668 0.0000 3.071673e-03 1.0229
669 0.0000 3.020934e-03 0.9835
670 0.0000 3.068377e-03 1.0157
671 0.0000 3.032896e-03 0.9884
672 0.0000 2.933976e-03 0.9674
673 0.0000 2.964433e-03 1.0104
674 0.0000 2.952689e-03 0.9960
675 0.0000 2.885523e-03 0.9773
676 0.0000 2.882944e-03 0.9991
677 0.0000 2.935961e-03 1.0184
678 0.0000 2.920453e-03 0.9947
679 0.0000 2.858204e-03 0.9787
680 0.0000 2.762709e-03 0.9666
681 0.0000 2.707928e-03 0.9802
682 0.0000 2.716042e-03 1.0030
683 0.0000 2.721017e-03 1.0018
684 0.0000 2.720301e-03 0.9997
685 0.0000 2.696040e-03 0.9911
686 0.0000 2.657746e-03 0.9858
687 0.0000 2.664420e-03 1.0025
688 0.0000 2.610177e-03 0.9796
689 0.0000 2.645394e-03 1.0135
690 0.0000 2.699657e-03 1.0205
691 0.0000 2.609284e-03 0.9665
692 0.0000 2.629226e-03 1.0076
693 0.0000 2.611216e-03 0.9932
694 0.0000 2.630112e-03 1.0072
695 0.0000 2.653442e-03 1.0089
696 0.0000 2.538814e-03 0.9568
697 0.0000 2.502645e-03 0.9858
698 0.0000 2.504065e-03 1.0006
699 0.0000 2.687520e-03 1.0733
700 0.0000 2.551698e-03 0.9495
701 0.0000 2.461522e-03 0.9647
702 0.0000 2.531003e-03 1.0282
703 0.0000 2.430953e-03 0.9605
704 0.0000 2.430052e-03 0.9996
705 0.0000 2.504488e-03 1.0306
706 0.0000 2.466978e-03 0.9850
707 0.0000 2.456033e-03 0.9956
708 0.0000 2.491324e-03 1.0144
709 0.0000 2.426681e-03 0.9741
710 0.0000 2.391890e-03 0.9857
711 0.0000 2.478016e-03 1.0360
712 0.0000 2.511699e-03 1.0136
713 0.0000 2.449575e-03 0.9753
714 0.0000 2.481302e-03 1.0130
715 0.0000 2.505996e-03 1.0100
716 0.0000 2.501104e-03 0.9980
717 0.0000 2.451500e-03 0.9802
718 0.0000 2.477735e-03 1.0107
719 0.0000 2.442478e-03 0.9858
720 0.0000 2.383028e-03 0.9757
721 0.0000 2.382710e-03 0.9999
722 0.0000 2.370649e-03 0.9949
723 0.0000 2.415416e-03 1.0189
724 0.0000 2.438737e-03 1.0097
725 0.0000 2.407252e-03 0.9871
726 0.0000 2.346697e-03 0.9748
727 0.0000 2.321115e-03 0.9891
728 0.0000 2.333413e-03 1.0053
729 0.0000 2.351854e-03 1.0079
730 0.0000 2.409359e-03 1.0245
731 0.0000 2.344550e-03 0.9731
732 0.0000 2.293627e-03 0.9783
733 0.0000 2.371450e-03 1.0339
734 0.0000 2.287999e-03 0.9648
735 0.0000 2.313765e-03 1.0113
736 0.0000 2.260754e-03 0.9771
737 0.0000 2.188349e-03 0.9680
738 0.0000 2.238853e-03 1.0231
739 0.0000 2.291927e-03 1.0237
740 0.0000 2.258121e-03 0.9853
741 0.0000 2.213533e-03 0.9803
742 0.0000 2.192415e-03 0.9905
743 0.0000 2.159613e-03 0.9850
744 0.0000 2.152427e-03 0.9967
745 0.0000 2.170189e-03 1.0083
746 0.0000 2.223195e-03 1.0244
747 0.0000 2.307541e-03 1.0379
748 0.0000 2.181823e-03 0.9455
749 0.0000 2.163917e-03 0.9918
750 0.0000 2.167458e-03 1.0016
751 0.0000 2.156657e-03 0.9950
752 0.0000 2.125397e-03 0.9855
753 0.0000 2.142835e-03 1.0082
754 0.0000 2.217114e-03 1.0347
755 0.0000 2.107002e-03 0.9503
756 0.0000 2.074550e-03 0.9846
757 0.0000 2.047449e-03 0.9869
758 0.0000 2.003972e-03 0.9788
759 0.0000 2.014668e-03 1.0053
760 0.0000 1.975443e-03 0.9805
761 0.0000 1.945485e-03 0.9848
762 0.0000 1.952831e-03 1.0038
763 0.0000 1.937567e-03 0.9922
764 0.0000 1.894101e-03 0.9776
765 0.0000 1.868663e-03 0.9866
766 0.0000 1.876975e-03 1.0044
767 0.0000 1.855223e-03 0.9884
768 0.0000 1.856640e-03 1.0008
769 0.0000 1.873003e-03 1.0088
770 0.0000 1.825525e-03 0.9747
771 0.0000 1.772884e-03 0.9712
772 0.0000 1.791424e-03 1.0105
773 0.0000 1.761388e-03 0.9832
774 0.0000 1.754499e-03 0.9961
775 0.0000 1.749571e-03 0.9972
776 0.0000 1.732596e-03 0.9903
777 0.0000 1.727874e-03 0.9973
778 0.0000 1.732001e-03 1.0024
779 0.0000 1.727282e-03 0.9973
780 0.0000 1.700106e-03 0.9843
781 0.0000 1.703887e-03 1.0022
782 0.0000 1.680089e-03 0.9860
783 0.0000 1.628074e-03 0.9690
784 0.0000 1.628501e-03 1.0003
785 0.0000 1.636634e-03 1.0050
786 0.0000 1.628870e-03 0.9953
787 0.0000 1.641730e-03 1.0079
788 0.0000 1.625949e-03 0.9904
789 0.0000 1.581212e-03 0.9725
790 0.0000 1.532030e-03 0.9689
791 0.0000 1.519634e-03 0.9919
792 0.0000 1.535160e-03 1.0102
793 0.0000 1.550009e-03 1.0097
794 0.0000 1.563274e-03 1.0086
795 0.0000 1.493288e-03 0.9552
796 0.0000 1.459549e-03 0.9774
797 0.0000 1.509726e-03 1.0344
798 0.0000 1.470356e-03 0.9739
799 0.0000 1.436431e-03 0.9769
800 0.0000 1.411141e-03 0.9824
801 0.0000 1.403195e-03 0.9944
802 0.0000 1.402421e-03 0.9994
803 0.0000 1.385873e-03 0.9882
804 0.0000 1.353841e-03 0.9769
805 0.0000 1.340951e-03 0.9905
806 0.0000 1.341311e-03 1.0003
807 0.0000 1.349036e-03 1.0058
808 0.0000 1.350563e-03 1.0011
809 0.0000 1.285433e-03 0.9518
810 0.0000 1.302456e-03 1.0132
811 0.0000 1.317619e-03 1.0116
812 0.0000 1.323645e-03 1.0046
813 0.0000 1.322994e-03 0.9995
814 0.0000 1.356128e-03 1.0250
815 0.0000 1.317021e-03 0.9712
816 0.0000 1.286394e-03 0.9767
817 0.0000 1.293315e-03 1.0054
818 0.0000 1.301398e-03 1.0062
819 0.0000 1.347228e-03 1.0352
820 0.0000 1.294844e-03 0.9611
821 0.0000 1.260692e-03 0.9736
822 0.0000 1.269896e-03 1.0073
823 0.0000 1.234638e-03 0.9722
824 0.0000 1.231516e-03 0.9975
825 0.0000 1.250195e-03 1.0152
826 0.0000 1.237134e-03 0.9896
827 0.0000 1.258198e-03 1.0170
828 0.0000 1.279474e-03 1.0169
829 0.0000 1.274285e-03 0.9959
830 0.0000 1.252230e-03 0.9827
831 0.0000 1.216491e-03 0.9715
832 0.0000 1.204917e-03 0.9905
833 0.0000 1.167398e-03 0.9689
834 0.0000 1.161045e-03 0.9946
835 0.0000 1.173960e-03 1.0111
836 0.0000 1.182561e-03 1.0073
837 0.0000 1.192346e-03 1.0083
838 0.0000 1.148196e-03 0.9630
839 0.0000 1.092183e-03 0.9512
840 0.0000 1.058793e-03 0.9694
841 0.0000 1.045892e-03 0.9878
842 0.0000 1.038956e-03 0.9934
843 0.0000 1.037247e-03 0.9984
844 0.0000 1.042432e-03 1.0050
845 0.0000 1.049370e-03 1.0067
846 0.0000 1.064085e-03 1.0140
847 0.0000 1.021784e-03 0.9602
848 0.0000 9.960429e-04 0.9748
849 0.0000 9.829912e-04 0.9869
850 0.0000 9.908452e-04 1.0080
851 0.0000 1.005408e-03 1.0147
852 0.0000 9.850166e-04 0.9797
853 0.0000 9.731450e-04 0.9879
854 0.0000 9.615552e-04 0.9881
855 0.0000 9.421366e-04 0.9798
856 0.0000 9.616912e-04 1.0208
857 0.0000 9.832599e-04 1.0224
858 0.0000 9.875020e-04 1.0043
859 0.0000 9.857665e-04 0.9982
860 0.0000 9.712259e-04 0.9852
861 0.0000 9.714164e-04 1.0002
862 0.0000 9.580410e-04 0.9862
863 0.0000 9.725060e-04 1.0151
864 0.0000 9.726826e-04 1.0002
865 0.0000 9.394663e-04 0.9659
866 0.0000 9.413713e-04 1.0020
867 0.0000 9.396075e-04 0.9981
868 0.0000 9.409370e-04 1.0014
869 0.0000 9.498303e-04 1.0095
870 0.0000 9.261593e-04 0.9751
871 0.0000 9.199120e-04 0.9933
872 0.0000 9.200373e-04 1.0001
873 0.0000 9.041397e-04 0.9827
874 0.0000 9.123642e-04 1.0091
875 0.0000 9.192381e-04 1.0075
876 0.0000 9.188214e-04 0.9995
877 0.0000 9.000381e-04 0.9796
878 0.0000 8.806142e-04 0.9784
879 0.0000 8.855791e-04 1.0056
880 0.0000 8.676648e-04 0.9798
881 0.0000 8.677054e-04 1.0000
882 0.0000 8.807259e-04 1.0150
883 0.0000 8.676048e-04 0.9851
884 0.0000 8.676534e-04 1.0001
885 0.0000 8.309741e-04 0.9577
886 0.0000 8.163647e-04 0.9824
887 0.0000 8.268292e-04 1.0128
888 0.0000 8.249113e-04 0.9977
889 0.0000 8.431913e-04 1.0222
890 0.0000 8.093790e-04 0.9599
891 0.0000 8.053845e-04 0.9951
892 0.0000 8.313124e-04 1.0322
893 0.0000 7.973871e-04 0.9592
894 0.0000 7.883449e-04 0.9887
895 0.0000 7.711701e-04 0.9782
896 0.0000 7.718840e-04 1.0009
897 0.0000 7.925186e-04 1.0267
898 0.0000 8.114793e-04 1.0239
899 0.0000 8.119894e-04 1.0006
900 0.0000 7.745538e-04 0.9539
901 0.0000 7.748123e-04 1.0003
902 0.0000 7.678743e-04 0.9910
903 0.0000 7.640441e-04 0.9950
904 0.0000 7.755048e-04 1.0150
905 0.0000 7.891542e-04 1.0176
906 0.0000 7.742804e-04 0.9812
907 0.0000 7.525522e-04 0.9719
908 0.0000 7.566220e-04 1.0054
909 0.0000 7.469665e-04 0.9872
910 0.0000 7.290903e-04 0.9761
911 0.0000 7.522938e-04 1.0318
912 0.0000 7.442352e-04 0.9893
913 0.0000 7.108979e-04 0.9552
914 0.0000 7.095701e-04 0.9981
915 0.0000 7.070365e-04 0.9964
916 0.0000 7.088656e-04 1.0026
917 0.0000 6.982154e-04 0.9850
918 0.0000 6.753450e-04 0.9672
919 0.0000 6.675135e-04 0.9884
920 0.0000 6.671596e-04 0.9995
921 0.0000 6.708132e-04 1.0055
922 0.0000 6.444531e-04 0.9607
923 0.0000 6.226790e-04 0.9662
924 0.0000 6.230278e-04 1.0006
925 0.0000 6.163252e-04 0.9892
926 0.0000 6.022176e-04 0.9771
927 0.0000 5.991566e-04 0.9949
928 0.0000 5.912835e-04 0.9869
929 0.0000 5.831130e-04 0.9862
930 0.0000 5.700193e-04 0.9775
931 0.0000 5.571408e-04 0.9774
932 0.0000 5.691536e-04 1.0216
933 0.0000 5.579242e-04 0.9803
934 0.0000 5.528541e-04 0.9909
935 0.0000 5.490729e-04 0.9932
936 0.0000 5.333079e-04 0.9713
937 0.0000 5.326629e-04 0.9988
938 0.0000 5.286821e-04 0.9925
939 0.0000 5.300768e-04 1.0026
940 0.0000 5.217623e-04 0.9843
941 0.0000 5.058423e-04 0.9695
942 0.0000 4.945977e-04 0.9778
943 0.0000 4.943199e-04 0.9994
944 0.0000 5.014849e-04 1.0145
945 0.0000 4.948979e-04 0.9869
946 0.0000 4.834331e-04 0.9768
947 0.0000 4.703484e-04 0.9729
948 0.0000 4.760264e-04 1.0121
949 0.0000 4.698934e-04 0.9871
950 0.0000 4.603654e-04 0.9797
951 0.0000 4.505246e-04 0.9786
952 0.0000 4.449649e-04 0.9877
953 0.0000 4.477592e-04 1.0063
954 0.0000 4.486911e-04 1.0021
955 0.0000 4.348044e-04 0.9691
956 0.0000 4.218062e-04 0.9701
957 0.0000 4.101086e-04 0.9723
958 0.0000 4.066455e-04 0.9916
959 0.0000 4.043486e-04 0.9944
960 0.0000 3.920222e-04 0.9695
961 0.0000 3.847938e-04 0.9816
962 0.0000 3.794788e-04 0.9862
963 0.0000 3.799792e-04 1.0013
964 0.0000 3.839192e-04 1.0104
965 0.0000 3.715595e-04 0.9678
966 0.0000 3.647828e-04 0.9818
967 0.0000 3.574942e-04 0.9800
968 0.0000 3.566596e-04 0.9977
969 0.0000 3.477963e-04 0.9751
970 0.0000 3.457679e-04 0.9942
971 0.0000 3.517328e-04 1.0173
972 0.0000 3.450002e-04 0.9809
973 0.0000 3.401140e-04 0.9858
974 0.0000 3.331621e-04 0.9796
975 0.0000 3.234024e-04 0.9707
976 0.0000 3.258192e-04 1.0075
977 0.0000 3.350614e-04 1.0284
978 0.0000 3.334387e-04 0.9952
979 0.0000 3.293452e-04 0.9877
980 0.0000 3.211299e-04 0.9751
981 0.0000 3.210761e-04 0.9998
982 0.0000 3.164181e-04 0.9855
983 0.0000 3.050857e-04 0.9642
984 0.0000 3.012009e-04 0.9873
985 0.0000 3.024822e-04 1.0043
986 0.0000 3.026344e-04 1.0005
987 0.0000 3.086870e-04 1.0200
988 0.0000 3.049964e-04 0.9880
989 0.0000 2.873506e-04 0.9421
990 0.0000 2.834352e-04 0.9864
991 0.0000 2.808062e-04 0.9907
992 0.0000 2.793854e-04 0.9949
993 0.0000 2.865596e-04 1.0257
994 0.0000 2.866176e-04 1.0002
995 0.0000 2.788491e-04 0.9729
996 0.0000 2.758293e-04 0.9892
997 0.0000 2.763111e-04 1.0017
998 0.0000 2.682861e-04 0.9710
999 0.0000 2.601792e-04 0.9698
1000 0.0000 2.596169e-04 0.9978
1001 0.0000 2.616959e-04 1.0080
1002 0.0000 2.535893e-04 0.9690
1003 0.0000 2.456781e-04 0.9688
1004 0.0000 2.442945e-04 0.9944
1005 0.0000 2.418638e-04 0.9900
1006 0.0000 2.453883e-04 1.0146
1007 0.0000 2.476002e-04 1.0090
1008 0.0000 2.420985e-04 0.9778
1009 0.0000 2.323620e-04 0.9598
1010 0.0000 2.233616e-04 0.9613
1011 0.0000 2.289190e-04 1.0249
1012 0.0000 2.321657e-04 1.0142
1013 0.0000 2.274925e-04 0.9799
1014 0.0000 2.191199e-04 0.9632
1015 0.0000 2.175438e-04 0.9928
1016 0.0000 2.220332e-04 1.0206
1017 0.0000 2.172046e-04 0.9783
1018 0.0000 2.097659e-04 0.9658
1019 0.0000 2.036157e-04 0.9707
1020 0.0000 1.994712e-04 0.9796
1021 0.0000 1.959702e-04 0.9824
1022 0.0000 1.930542e-04 0.9851
1023 0.0000 1.909361e-04 0.9890
1024 0.0000 1.903897e-04 0.9971
1025 0.0000 1.917316e-04 1.0070
1026 0.0000 1.910562e-04 0.9965
1027 0.0000 1.833002e-04 0.9594
1028 0.0000 1.776087e-04 0.9689
1029 0.0000 1.754119e-04 0.9876
1030 0.0000 1.745686e-04 0.9952
1031 0.0000 1.718461e-04 0.9844
1032 0.0000 1.664330e-04 0.9685
1033 0.0000 1.644253e-04 0.9879
1034 0.0000 1.606550e-04 0.9771
1035 0.0000 1.561615e-04 0.9720
1036 0.0000 1.522687e-04 0.9751
1037 0.0000 1.486872e-04 0.9765
1038 0.0000 1.492920e-04 1.0041
1039 0.0000 1.525888e-04 1.0221
1040 0.0000 1.458966e-04 0.9561
1041 0.0000 1.426591e-04 0.9778
1042 0.0000 1.399706e-04 0.9812
1043 0.0000 1.379055e-04 0.9852
1044 0.0000 1.389959e-04 1.0079
1045 0.0000 1.386503e-04 0.9975
1046 0.0000 1.380247e-04 0.9955
1047 0.0000 1.357530e-04 0.9835
1048 0.0000 1.356600e-04 0.9993
1049 0.0000 1.340695e-04 0.9883
1050 0.0000 1.305877e-04 0.9740
1051 0.0000 1.261571e-04 0.9661
1052 0.0000 1.238009e-04 0.9813
1053 0.0000 1.238562e-04 1.0004
1054 0.0000 1.280320e-04 1.0337
1055 0.0000 1.288540e-04 1.0064
1056 0.0000 1.192187e-04 0.9252
1057 0.0000 1.171484e-04 0.9826
1058 0.0000 1.177575e-04 1.0052
1059 0.0000 1.160407e-04 0.9854
1060 0.0000 1.192555e-04 1.0277
1061 0.0000 1.145663e-04 0.9607
1062 0.0000 1.119191e-04 0.9769
1063 0.0000 1.113072e-04 0.9945
1064 0.0000 1.066629e-04 0.9583
1065 0.0000 1.082836e-04 1.0152
1066 0.0000 1.094147e-04 1.0104
1067 0.0000 1.078406e-04 0.9856
1068 0.0000 1.059389e-04 0.9824
1069 0.0000 1.025855e-04 0.9683
1070 0.0000 1.065688e-04 1.0388
1071 0.0000 1.038323e-04 0.9743
1072 0.0000 1.005078e-04 0.9680
1073 0.0000 9.960116e-05 0.9910
1074 0.0000 9.555465e-05 0.9594
1075 0.0000 9.389676e-05 0.9826
1076 0.0000 9.329190e-05 0.9936
1077 0.0000 9.168408e-05 0.9828
1078 0.0000 9.045320e-05 0.9866
1079 0.0000 9.012966e-05 0.9964
1080 0.0000 8.915993e-05 0.9892
1081 0.0000 8.807006e-05 0.9878
1082 0.0000 8.689666e-05 0.9867
1083 0.0000 8.928345e-05 1.0275
1084 0.0000 8.881869e-05 0.9948
1085 0.0000 8.305333e-05 0.9351
1086 0.0000 8.164746e-05 0.9831
1087 0.0000 8.361787e-05 1.0241
1088 0.0000 8.482257e-05 1.0144
1089 0.0000 8.102409e-05 0.9552
1090 0.0000 7.804316e-05 0.9632
1091 0.0000 7.777590e-05 0.9966
1092 0.0000 7.698070e-05 0.9898
1093 0.0000 7.617721e-05 0.9896
1094 0.0000 7.785015e-05 1.0220
1095 0.0000 7.908359e-05 1.0158
1096 0.0000 7.586426e-05 0.9593
1097 0.0000 7.311162e-05 0.9637
1098 0.0000 7.077980e-05 0.9681
1099 0.0000 7.057252e-05 0.9971
1100 0.0000 7.018112e-05 0.9945
1101 0.0000 6.975981e-05 0.9940
1102 0.0000 7.069918e-05 1.0135
1103 0.0000 6.917076e-05 0.9784
1104 0.0000 6.798418e-05 0.9828
1105 0.0000 6.724114e-05 0.9891
1106 0.0000 6.722599e-05 0.9998
1107 0.0000 6.703655e-05 0.9972
1108 0.0000 6.766579e-05 1.0094
1109 0.0000 6.728036e-05 0.9943
1110 0.0000 6.337604e-05 0.9420
1111 0.0000 6.185643e-05 0.9760
1112 0.0000 6.180634e-05 0.9992
1113 0.0000 6.237878e-05 1.0093
1114 0.0000 6.059123e-05 0.9713
1115 0.0000 5.896654e-05 0.9732
1116 0.0000 5.900299e-05 1.0006
1117 0.0000 5.988532e-05 1.0150
1118 0.0000 5.821689e-05 0.9721
1119 0.0000 5.568219e-05 0.9565
1120 0.0000 5.459460e-05 0.9805
1121 0.0000 5.404070e-05 0.9899
1122 0.0000 5.327976e-05 0.9859
1123 0.0000 5.329041e-05 1.0002
1124 0.0000 5.310982e-05 0.9966
1125 0.0000 5.209220e-05 0.9808
1126 0.0000 5.205710e-05 0.9993
1127 0.0000 5.152420e-05 0.9898
1128 0.0000 5.079353e-05 0.9858
1129 0.0000 4.902484e-05 0.9652
1130 0.0000 4.714705e-05 0.9617
1131 0.0000 4.671814e-05 0.9909
1132 0.0000 4.722716e-05 1.0109
1133 0.0000 4.672086e-05 0.9893
1134 0.0000 4.591729e-05 0.9828
1135 0.0000 4.543921e-05 0.9896
1136 0.0000 4.455681e-05 0.9806
1137 0.0000 4.512239e-05 1.0127
1138 0.0000 4.406280e-05 0.9765
1139 0.0000 4.304828e-05 0.9770
1140 0.0000 4.228005e-05 0.9822
1141 0.0000 4.117294e-05 0.9738
1142 0.0000 4.094077e-05 0.9944
1143 0.0000 4.121272e-05 1.0066
1144 0.0000 4.008860e-05 0.9727
1145 0.0000 3.926912e-05 0.9796
1146 0.0000 3.884349e-05 0.9892
1147 0.0000 3.774906e-05 0.9718
1148 0.0000 3.682124e-05 0.9754
1149 0.0000 3.645388e-05 0.9900
1150 0.0000 3.626818e-05 0.9949
1151 0.0000 3.691003e-05 1.0177
1152 0.0000 3.568361e-05 0.9668
1153 0.0000 3.417539e-05 0.9577
1154 0.0000 3.359662e-05 0.9831
1155 0.0000 3.366826e-05 1.0021
1156 0.0000 3.365647e-05 0.9996
1157 0.0000 3.293674e-05 0.9786
1158 0.0000 3.176295e-05 0.9644
1159 0.0000 3.052760e-05 0.9611
1160 0.0000 3.032254e-05 0.9933
1161 0.0000 2.970485e-05 0.9796
1162 0.0000 2.880702e-05 0.9698
1163 0.0000 2.869126e-05 0.9960
1164 0.0000 2.866974e-05 0.9992
1165 0.0000 2.925789e-05 1.0205
1166 0.0000 2.978360e-05 1.0180
1167 0.0000 2.862327e-05 0.9610
1168 0.0000 2.773773e-05 0.9691
1169 0.0000 2.713071e-05 0.9781
1170 0.0000 2.739368e-05 1.0097
1171 0.0000 2.752526e-05 1.0048
1172 0.0000 2.726573e-05 0.9906
1173 0.0000 2.669564e-05 0.9791
1174 0.0000 2.588392e-05 0.9696
1175 0.0000 2.542371e-05 0.9822
1176 0.0000 2.491846e-05 0.9801
1177 0.0000 2.492559e-05 1.0003
1178 0.0000 2.416940e-05 0.9697
1179 0.0000 2.377181e-05 0.9835
1180 0.0000 2.363671e-05 0.9943
1181 0.0000 2.398205e-05 1.0146
1182 0.0000 2.405632e-05 1.0031
1183 0.0000 2.271367e-05 0.9442
1184 0.0000 2.249473e-05 0.9904
1185 0.0000 2.231221e-05 0.9919
1186 0.0000 2.193788e-05 0.9832
1187 0.0000 2.178802e-05 0.9932
1188 0.0000 2.121793e-05 0.9738
1189 0.0000 2.104937e-05 0.9921
1190 0.0000 2.149159e-05 1.0210
1191 0.0000 2.074685e-05 0.9653
1192 0.0000 2.000348e-05 0.9642
1193 0.0000 2.010386e-05 1.0050
1194 0.0000 1.991699e-05 0.9907
1195 0.0000 1.964748e-05 0.9865
1196 0.0000 1.946989e-05 0.9910
1197 0.0000 1.925539e-05 0.9890
1198 0.0000 1.917586e-05 0.9959
1199 0.0000 1.894290e-05 0.9879
1200 0.0000 1.899356e-05 1.0027
1201 0.0000 1.884265e-05 0.9921
1202 0.0000 1.807620e-05 0.9593
1203 0.0000 1.796752e-05 0.9940
1204 0.0000 1.803827e-05 1.0039
1205 0.0000 1.774020e-05 0.9835
1206 0.0000 1.721043e-05 0.9701
1207 0.0000 1.674415e-05 0.9729
1208 0.0000 1.658830e-05 0.9907
1209 0.0000 1.667041e-05 1.0050
1210 0.0000 1.703125e-05 1.0216
1211 0.0000 1.649411e-05 0.9685
1212 0.0000 1.617734e-05 0.9808
1213 0.0000 1.657556e-05 1.0246
1214 0.0000 1.634708e-05 0.9862
1215 0.0000 1.587665e-05 0.9712
1216 0.0000 1.564958e-05 0.9857
1217 0.0000 1.550977e-05 0.9911
1218 0.0000 1.568440e-05 1.0113
1219 0.0000 1.520927e-05 0.9697
1220 0.0000 1.467077e-05 0.9646
1221 0.0000 1.451754e-05 0.9896
1222 0.0000 1.433151e-05 0.9872
1223 0.0000 1.443223e-05 1.0070
1224 0.0000 1.450418e-05 1.0050
1225 0.0000 1.426128e-05 0.9833
1226 0.0000 1.393596e-05 0.9772
1227 0.0000 1.368879e-05 0.9823
1228 0.0000 1.374039e-05 1.0038
1229 0.0000 1.369593e-05 0.9968
1230 0.0000 1.353448e-05 0.9882
----------------------------------------------------------------------
Total Iterations: 1231
Avg Convergence Rate: 0.9888
Final Residual: 1.353448e-05
Total Reduction in Residual: 9.996766e-07
Maximum Memory Usage: 0.000 GB
----------------------------------------------------------------------
Total Time: 5.72287
setup: 0.000170048 s
solve: 5.7227 s
solve(per iteration): 0.00464882 s

If I use GMRES_AMG_D2.json it took 16s
AMGX version 2.5.0
Built on Apr 25 2024, 13:20:11
Compiled with CUDA Runtime 12.4, using CUDA driver 12.4
AMG Grid:
Number of Levels: 9
LVL ROWS NNZ PARTS SPRSTY Mem (GB)
----------------------------------------------------------------------
0(D) 1507876 113305856 1 4.98e-05 1.3
1(D) 281692 31103364 1 0.000392 0.706
2(D) 39859 4883517 1 0.00307 0.111
3(D) 4793 328295 1 0.0143 0.00753
4(D) 762 35364 1 0.0609 0.000822
5(D) 198 7160 1 0.183 0.000168
6(D) 60 1622 1 0.451 3.88e-05
7(D) 18 254 1 0.784 6.45e-06
8(D) 6 36 1 1 9.98e-07
----------------------------------------------------------------------
Grid Complexity: 1.21712
Operator Complexity: 1.3209
Total Memory Usage: 2.12977 GB
----------------------------------------------------------------------
iter Mem Usage (GB) residual rate
----------------------------------------------------------------------
Ini 4.50732 1.353886e+01
0 4.50732 9.552636e+00 0.7056
1 4.5073 6.826744e+00 0.7146
2 4.5073 5.451789e+00 0.7986
3 4.5073 4.193551e+00 0.7692
4 4.5073 3.318682e+00 0.7914
5 4.5073 2.696656e+00 0.8126
6 4.5073 2.253779e+00 0.8358
7 4.5073 1.897404e+00 0.8419
8 4.5073 1.605763e+00 0.8463
9 4.5073 1.371794e+00 0.8543
10 4.5073 1.280317e+00 0.9333
11 4.5073 1.165342e+00 0.9102
12 4.5073 1.044937e+00 0.8967
13 4.5073 9.311771e-01 0.8911
14 4.5073 8.317691e-01 0.8932
15 4.5073 7.104873e-01 0.8542
16 4.5073 6.342603e-01 0.8927
17 4.5073 5.645147e-01 0.8900
18 4.5073 5.062215e-01 0.8967
19 4.5073 4.567174e-01 0.9022
20 4.5073 4.388230e-01 0.9608
21 4.5073 4.211402e-01 0.9597
22 4.5073 3.973845e-01 0.9436
23 4.5073 3.710982e-01 0.9339
24 4.5073 3.424965e-01 0.9229
25 4.5073 3.155958e-01 0.9215
26 4.5073 2.966112e-01 0.9398
27 4.5073 2.795076e-01 0.9423
28 4.5073 2.635899e-01 0.9431
29 4.5073 2.461252e-01 0.9337
30 4.5073 2.365946e-01 0.9613
31 4.5073 2.264181e-01 0.9570
32 4.5073 2.144454e-01 0.9471
33 4.5073 2.034822e-01 0.9489
34 4.5073 1.939666e-01 0.9532
35 4.5073 1.830933e-01 0.9439
36 4.5073 1.759580e-01 0.9610
37 4.5073 1.689407e-01 0.9601
38 4.5073 1.625273e-01 0.9620
39 4.5073 1.567911e-01 0.9647
40 4.5073 1.523330e-01 0.9716
41 4.5073 1.482169e-01 0.9730
42 4.5073 1.430960e-01 0.9655
43 4.5073 1.379123e-01 0.9638
44 4.5073 1.318838e-01 0.9563
45 4.5073 1.262892e-01 0.9576
46 4.5073 1.216729e-01 0.9634
47 4.5073 1.172642e-01 0.9638
48 4.5073 1.136271e-01 0.9690
49 4.5073 1.098468e-01 0.9667
50 4.5073 1.073779e-01 0.9775
51 4.5073 1.044496e-01 0.9727
52 4.5073 1.007739e-01 0.9648
53 4.5073 9.737754e-02 0.9663
54 4.5073 9.470861e-02 0.9726
55 4.5073 9.166303e-02 0.9678
56 4.5073 8.969872e-02 0.9786
57 4.5073 8.761253e-02 0.9767
58 4.5073 8.539046e-02 0.9746
59 4.5073 8.346770e-02 0.9775
60 4.5073 8.175577e-02 0.9795
61 4.5073 8.011439e-02 0.9799
62 4.5073 7.851187e-02 0.9800
63 4.5073 7.699035e-02 0.9806
64 4.5073 7.499845e-02 0.9741
65 4.5073 7.315360e-02 0.9754
66 4.5073 7.119191e-02 0.9732
67 4.5073 6.927591e-02 0.9731
68 4.5073 6.783679e-02 0.9792
69 4.5073 6.647737e-02 0.9800
70 4.5073 6.545250e-02 0.9846
71 4.5073 6.426370e-02 0.9818
72 4.5073 6.257041e-02 0.9737
73 4.5073 6.100275e-02 0.9749
74 4.5073 5.987852e-02 0.9816
75 4.5073 5.862508e-02 0.9791
76 4.5073 5.780494e-02 0.9860
77 4.5073 5.687329e-02 0.9839
78 4.5073 5.563071e-02 0.9782
79 4.5073 5.456461e-02 0.9808
80 4.5073 5.351356e-02 0.9807
81 4.5073 5.255514e-02 0.9821
82 4.5073 5.177287e-02 0.9851
83 4.5073 5.106999e-02 0.9864
84 4.5073 5.012493e-02 0.9815
85 4.5073 4.921204e-02 0.9818
86 4.5073 4.799043e-02 0.9752
87 4.5073 4.670630e-02 0.9732
88 4.5073 4.583710e-02 0.9814
89 4.5073 4.502878e-02 0.9824
90 4.5073 4.445102e-02 0.9872
91 4.5073 4.367765e-02 0.9826
92 4.5073 4.254507e-02 0.9741
93 4.5073 4.147386e-02 0.9748
94 4.5073 4.078047e-02 0.9833
95 4.5073 4.004918e-02 0.9821
96 4.5073 3.958986e-02 0.9885
97 4.5073 3.904670e-02 0.9863
98 4.5073 3.819285e-02 0.9781
99 4.5073 3.743150e-02 0.9801
100 4.5073 3.671366e-02 0.9808
101 4.5073 3.597912e-02 0.9800
102 4.5073 3.552583e-02 0.9874
103 4.5073 3.510992e-02 0.9883
104 4.5073 3.452025e-02 0.9832
105 4.5073 3.394837e-02 0.9834
106 4.5073 3.301806e-02 0.9726
107 4.5073 3.206440e-02 0.9711
108 4.5073 3.147588e-02 0.9816
109 4.5073 3.095063e-02 0.9833
110 4.5073 3.051699e-02 0.9860
111 4.5073 3.003388e-02 0.9842
112 4.5073 2.918968e-02 0.9719
113 4.5073 2.837800e-02 0.9722
114 4.5073 2.792080e-02 0.9839
115 4.5073 2.745213e-02 0.9832
116 4.5073 2.715536e-02 0.9892
117 4.5073 2.684906e-02 0.9887
118 4.5073 2.618969e-02 0.9754
119 4.5073 2.561850e-02 0.9782
120 4.5073 2.506056e-02 0.9782
121 4.5073 2.451296e-02 0.9781
122 4.5073 2.423323e-02 0.9886
123 4.5073 2.397209e-02 0.9892
124 4.5073 2.358096e-02 0.9837
125 4.5073 2.317335e-02 0.9827
126 4.5073 2.252357e-02 0.9720
127 4.5073 2.182169e-02 0.9688
128 4.5073 2.142175e-02 0.9817
129 4.5073 2.106657e-02 0.9834
130 4.5073 2.076983e-02 0.9859
131 4.5073 2.043719e-02 0.9840
132 4.5073 1.982875e-02 0.9702
133 4.5073 1.925887e-02 0.9713
134 4.5073 1.892543e-02 0.9827
135 4.5073 1.861500e-02 0.9836
136 4.5073 1.841600e-02 0.9893
137 4.5073 1.822721e-02 0.9897
138 4.5073 1.774987e-02 0.9738
139 4.5073 1.723753e-02 0.9711
140 4.5073 1.675373e-02 0.9719
141 4.5073 1.636719e-02 0.9769
142 4.5073 1.617550e-02 0.9883
143 4.5073 1.600921e-02 0.9897
144 4.5073 1.578544e-02 0.9860
145 4.5073 1.549496e-02 0.9816
146 4.5073 1.508306e-02 0.9734
147 4.5073 1.456481e-02 0.9656
148 4.5073 1.427043e-02 0.9798
149 4.5073 1.402647e-02 0.9829
150 4.5073 1.384524e-02 0.9871
151 4.5073 1.359538e-02 0.9820
152 4.5073 1.315202e-02 0.9674
153 4.5073 1.278672e-02 0.9722
154 4.5073 1.253887e-02 0.9806
155 4.5073 1.235779e-02 0.9856
156 4.5073 1.223345e-02 0.9899
157 4.5073 1.210488e-02 0.9895
158 4.5073 1.180894e-02 0.9756
159 4.5073 1.147988e-02 0.9721
160 4.5073 1.116957e-02 0.9730
161 4.5073 1.092455e-02 0.9781
162 4.5073 1.079219e-02 0.9879
163 4.5073 1.069048e-02 0.9906
164 4.5073 1.054792e-02 0.9867
165 4.5073 1.033906e-02 0.9802
166 4.5073 1.006528e-02 0.9735
167 4.5073 9.688645e-03 0.9626
168 4.5073 9.485966e-03 0.9791
169 4.5073 9.321227e-03 0.9826
170 4.5073 9.195419e-03 0.9865
171 4.5073 9.021929e-03 0.9811
172 4.5073 8.718005e-03 0.9663
173 4.5073 8.466200e-03 0.9711
174 4.5073 8.285718e-03 0.9787
175 4.5073 8.173952e-03 0.9865
176 4.5073 8.095984e-03 0.9905
177 4.5073 8.004101e-03 0.9887
178 4.5073 7.829951e-03 0.9782
179 4.5073 7.643146e-03 0.9761
180 4.5073 7.462735e-03 0.9764
181 4.5073 7.309771e-03 0.9795
182 4.5073 7.223806e-03 0.9882
183 4.5073 7.155744e-03 0.9906
184 4.5073 7.058219e-03 0.9864
185 4.5073 6.910895e-03 0.9791
186 4.5073 6.705845e-03 0.9703
187 4.5073 6.452930e-03 0.9623
188 4.5073 6.324550e-03 0.9801
189 4.5073 6.209451e-03 0.9818
190 4.5073 6.117331e-03 0.9852
191 4.5073 6.002762e-03 0.9813
192 4.5073 5.795177e-03 0.9654
193 4.5073 5.617591e-03 0.9694
194 4.5073 5.492606e-03 0.9778
195 4.5073 5.417069e-03 0.9862
196 4.5073 5.365974e-03 0.9906
197 4.5073 5.306792e-03 0.9890
198 4.5073 5.199231e-03 0.9797
199 4.5073 5.089314e-03 0.9789
200 4.5073 4.984597e-03 0.9794
201 4.5073 4.885234e-03 0.9801
202 4.5073 4.831392e-03 0.9890
203 4.5073 4.785620e-03 0.9905
204 4.5073 4.720878e-03 0.9865
205 4.5073 4.617083e-03 0.9780
206 4.5073 4.473098e-03 0.9688
207 4.5073 4.298896e-03 0.9611
208 4.5073 4.211524e-03 0.9797
209 4.5073 4.126273e-03 0.9798
210 4.5073 4.054650e-03 0.9826
211 4.5073 3.976860e-03 0.9808
212 4.5073 3.832365e-03 0.9637
213 4.5073 3.712143e-03 0.9686
214 4.5073 3.627296e-03 0.9771
215 4.5073 3.576847e-03 0.9861
216 4.5073 3.542947e-03 0.9905
217 4.5073 3.507197e-03 0.9899
218 4.5073 3.438890e-03 0.9805
219 4.5073 3.373210e-03 0.9809
220 4.5073 3.309594e-03 0.9811
221 4.5073 3.246149e-03 0.9808
222 4.5073 3.212992e-03 0.9898
223 4.5073 3.182623e-03 0.9905
224 4.5073 3.138007e-03 0.9860
225 4.5073 3.063484e-03 0.9763
226 4.5073 2.962185e-03 0.9669
227 4.5073 2.842631e-03 0.9596
228 4.5073 2.782643e-03 0.9789
229 4.5073 2.721328e-03 0.9780
230 4.5073 2.669796e-03 0.9811
231 4.5073 2.615614e-03 0.9797
232 4.5073 2.516624e-03 0.9622
233 4.5073 2.433075e-03 0.9668
234 4.5073 2.372982e-03 0.9753
235 4.5073 2.338936e-03 0.9857
236 4.5073 2.316631e-03 0.9905
237 4.5073 2.293694e-03 0.9901
238 4.5073 2.248138e-03 0.9801
239 4.5073 2.206995e-03 0.9817
240 4.5073 2.167269e-03 0.9820
241 4.5073 2.124825e-03 0.9804
242 4.5073 2.103450e-03 0.9899
243 4.5073 2.083674e-03 0.9906
244 4.5073 2.054267e-03 0.9859
245 4.5073 2.001011e-03 0.9741
246 4.5073 1.930307e-03 0.9647
247 4.5073 1.848721e-03 0.9577
248 4.5073 1.808268e-03 0.9781
249 4.5073 1.763726e-03 0.9754
250 4.5073 1.726539e-03 0.9789
251 4.5073 1.689947e-03 0.9788
252 4.5073 1.624000e-03 0.9610
253 4.5073 1.565009e-03 0.9637
254 4.5073 1.522738e-03 0.9730
255 4.5073 1.500731e-03 0.9855
256 4.5073 1.486607e-03 0.9906
257 4.5073 1.472336e-03 0.9904
258 4.5073 1.442995e-03 0.9801
259 4.5073 1.401880e-03 0.9715
260 4.5073 1.363245e-03 0.9724
261 4.5073 1.338839e-03 0.9821
262 4.5073 1.324971e-03 0.9896
263 4.5073 1.314374e-03 0.9920
264 4.5073 1.297219e-03 0.9869
265 4.5073 1.265595e-03 0.9756
266 4.5073 1.219403e-03 0.9635
267 4.5073 1.165827e-03 0.9561
268 4.5073 1.136158e-03 0.9746
269 4.5073 1.106047e-03 0.9735
270 4.5073 1.081060e-03 0.9774
271 4.5073 1.057849e-03 0.9785
272 4.5073 1.011653e-03 0.9563
273 4.5073 9.744879e-04 0.9633
274 4.5073 9.499882e-04 0.9749
275 4.5073 9.369585e-04 0.9863
276 4.5073 9.290940e-04 0.9916
277 4.5073 9.204583e-04 0.9907
278 4.5073 9.048393e-04 0.9830
279 4.5073 8.898536e-04 0.9834
280 4.5073 8.755330e-04 0.9839
281 4.5073 8.611554e-04 0.9836
282 4.5073 8.530492e-04 0.9906
283 4.5073 8.459542e-04 0.9917
284 4.5073 8.339620e-04 0.9858
285 4.5073 8.123608e-04 0.9741
286 4.5073 7.823079e-04 0.9630
287 4.5073 7.475207e-04 0.9555
288 4.5073 7.289601e-04 0.9752
289 4.5073 7.082735e-04 0.9716
290 4.5073 6.906207e-04 0.9751
291 4.5073 6.748595e-04 0.9772
292 4.5073 6.470612e-04 0.9588
293 4.5073 6.235908e-04 0.9637
294 4.5073 6.062682e-04 0.9722
295 4.5073 5.972097e-04 0.9851
296 4.5073 5.916863e-04 0.9908
297 4.5073 5.861331e-04 0.9906
298 4.5073 5.771284e-04 0.9846
299 4.5073 5.698246e-04 0.9873
300 4.5073 5.629160e-04 0.9879
301 4.5073 5.548604e-04 0.9857
302 4.5073 5.496628e-04 0.9906
303 4.5073 5.443566e-04 0.9903
304 4.5073 5.351942e-04 0.9832
305 4.5073 5.197481e-04 0.9711
306 4.5073 5.013518e-04 0.9646
307 4.5073 4.820704e-04 0.9615
308 4.5073 4.703863e-04 0.9758
309 4.5073 4.551120e-04 0.9675
310 4.5073 4.420310e-04 0.9713
311 4.5073 4.321963e-04 0.9778
312 4.5073 4.173260e-04 0.9656
313 4.5073 4.027740e-04 0.9651
314 4.5073 3.902635e-04 0.9689
315 4.5073 3.834007e-04 0.9824
316 4.5073 3.794524e-04 0.9897
317 4.5073 3.759048e-04 0.9907
318 4.5073 3.706544e-04 0.9860
319 4.5073 3.662881e-04 0.9882
320 4.5073 3.621661e-04 0.9887
321 4.5073 3.575503e-04 0.9873
322 4.5073 3.542326e-04 0.9907
323 4.5073 3.505049e-04 0.9895
324 4.5073 3.438591e-04 0.9810
325 4.5073 3.327386e-04 0.9677
326 4.5073 3.209327e-04 0.9645
327 4.5073 3.097232e-04 0.9651
328 4.5073 3.025474e-04 0.9768
329 4.5073 2.915968e-04 0.9638
330 4.5073 2.822276e-04 0.9679
331 4.5073 2.758958e-04 0.9776
332 4.5073 2.670500e-04 0.9679
333 4.5073 2.577056e-04 0.9650
334 4.5073 2.490600e-04 0.9665
335 4.5073 2.442267e-04 0.9806
336 4.5073 2.416747e-04 0.9896
337 4.5073 2.395729e-04 0.9913
338 4.5073 2.366283e-04 0.9877
339 4.5073 2.329755e-04 0.9846
340 4.5073 2.295243e-04 0.9852
341 4.5073 2.271336e-04 0.9896
342 4.5073 2.249478e-04 0.9904
343 4.5073 2.225149e-04 0.9892
344 4.5073 2.183825e-04 0.9814
345 4.5073 2.115974e-04 0.9689
346 4.5073 2.037608e-04 0.9630
347 4.5073 1.961096e-04 0.9624
348 4.5073 1.915623e-04 0.9768
349 4.5073 1.844891e-04 0.9631
350 4.5073 1.789415e-04 0.9699
351 4.5073 1.747135e-04 0.9764
352 4.5073 1.685326e-04 0.9646
353 4.5073 1.625619e-04 0.9646
354 4.5073 1.573789e-04 0.9681
355 4.5073 1.543450e-04 0.9807
356 4.5073 1.527411e-04 0.9896
357 4.5073 1.514816e-04 0.9918
358 4.5073 1.498667e-04 0.9893
359 4.5073 1.482822e-04 0.9894
360 4.5073 1.467254e-04 0.9895
361 4.5073 1.453226e-04 0.9904
362 4.5073 1.440473e-04 0.9912
363 4.5073 1.425693e-04 0.9897
364 4.5073 1.399659e-04 0.9817
365 4.5073 1.354891e-04 0.9680
366 4.5073 1.304395e-04 0.9627
367 4.5073 1.255801e-04 0.9627
368 4.5073 1.224384e-04 0.9750
369 4.5073 1.178715e-04 0.9627
370 4.5073 1.141439e-04 0.9684
371 4.5073 1.114363e-04 0.9763
372 4.5073 1.074004e-04 0.9638
373 4.5073 1.037559e-04 0.9661
374 4.5073 1.005130e-04 0.9687
375 4.5073 9.864468e-05 0.9814
376 4.5073 9.765982e-05 0.9900
377 4.5073 9.693787e-05 0.9926
378 4.5073 9.603671e-05 0.9907
379 4.5073 9.536281e-05 0.9930
380 4.5073 9.470601e-05 0.9931
381 4.5073 9.386835e-05 0.9912
382 4.5073 9.312684e-05 0.9921
383 4.5073 9.220860e-05 0.9901
384 4.5073 9.051789e-05 0.9817
385 4.5073 8.757986e-05 0.9675
386 4.5073 8.441697e-05 0.9639
387 4.5073 8.117176e-05 0.9616
388 4.5073 7.917326e-05 0.9754
389 4.5073 7.650414e-05 0.9663
390 4.5073 7.415814e-05 0.9693
391 4.5073 7.243978e-05 0.9768
392 4.5073 6.992122e-05 0.9652
393 4.5073 6.755464e-05 0.9662
394 4.5073 6.550298e-05 0.9696
395 4.5073 6.442420e-05 0.9835
396 4.5073 6.386114e-05 0.9913
397 4.5073 6.343917e-05 0.9934
398 4.5073 6.291857e-05 0.9918
399 4.5073 6.255101e-05 0.9942
400 4.5073 6.218764e-05 0.9942
401 4.5073 6.169533e-05 0.9921
402 4.5073 6.126233e-05 0.9930
403 4.5073 6.074052e-05 0.9915
404 4.5073 5.976100e-05 0.9839
405 4.5073 5.783132e-05 0.9677
406 4.5073 5.582332e-05 0.9653
407 4.5073 5.381541e-05 0.9640
408 4.5073 5.256528e-05 0.9768
409 4.5073 5.088652e-05 0.9681
410 4.5073 4.942387e-05 0.9713
411 4.5073 4.836018e-05 0.9785
412 4.5073 4.681841e-05 0.9681
413 4.5073 4.530922e-05 0.9678
414 4.5073 4.403902e-05 0.9720
415 4.5073 4.342640e-05 0.9861
416 4.5073 4.311669e-05 0.9929
417 4.5073 4.288333e-05 0.9946
418 4.5073 4.258149e-05 0.9930
419 4.5073 4.234146e-05 0.9944
420 4.5073 4.210422e-05 0.9944
421 4.5073 4.182142e-05 0.9933
422 4.5073 4.157963e-05 0.9942
423 4.5073 4.129297e-05 0.9931
424 4.5073 4.075008e-05 0.9869
425 4.5073 3.955404e-05 0.9706
426 4.5073 3.826284e-05 0.9674
427 4.5073 3.703658e-05 0.9680
428 4.5073 3.627819e-05 0.9795
429 4.5073 3.518842e-05 0.9700
430 4.5073 3.427321e-05 0.9740
431 4.5073 3.360933e-05 0.9806
432 4.5073 3.266358e-05 0.9719
433 4.5073 3.170647e-05 0.9707
434 4.5073 3.092739e-05 0.9754
435 4.5073 3.057961e-05 0.9888
436 4.5073 3.040358e-05 0.9942
437 4.5073 3.027417e-05 0.9957
438 4.5073 3.010033e-05 0.9943
439 4.5073 2.991853e-05 0.9940
440 4.5073 2.973998e-05 0.9940
441 4.5073 2.958330e-05 0.9947
442 4.5073 2.944665e-05 0.9954
443 4.5073 2.928091e-05 0.9944
444 4.5073 2.897940e-05 0.9897
445 4.5073 2.826589e-05 0.9754
446 4.5073 2.743132e-05 0.9705
447 4.5073 2.667133e-05 0.9723
448 4.5073 2.615545e-05 0.9807
449 4.5073 2.540920e-05 0.9715
450 4.5073 2.485503e-05 0.9782
451 4.5073 2.436528e-05 0.9803
452 4.5073 2.373976e-05 0.9743
453 4.5073 2.310008e-05 0.9731
454 4.5073 2.261866e-05 0.9792
455 4.5073 2.240689e-05 0.9906
456 4.5073 2.229318e-05 0.9949
457 4.5073 2.221321e-05 0.9964
458 4.5073 2.210959e-05 0.9953
459 4.5073 2.200217e-05 0.9951
460 4.5073 2.189792e-05 0.9953
461 4.5073 2.180671e-05 0.9958
462 4.5073 2.171675e-05 0.9959
463 4.5073 2.161412e-05 0.9953
464 4.5073 2.143114e-05 0.9915
465 4.5073 2.097284e-05 0.9786
466 4.5073 2.038027e-05 0.9717
467 4.5073 1.986920e-05 0.9749
468 4.5073 1.948462e-05 0.9806
469 4.5073 1.898508e-05 0.9744
470 4.5073 1.857710e-05 0.9785
471 4.5073 1.821628e-05 0.9806
472 4.5073 1.782268e-05 0.9784
473 4.5073 1.736033e-05 0.9741
474 4.5073 1.698073e-05 0.9781
475 4.5073 1.684377e-05 0.9919
476 4.5073 1.676710e-05 0.9954
477 4.5073 1.670770e-05 0.9965
478 4.5073 1.663904e-05 0.9959
479 4.5073 1.657661e-05 0.9962
480 4.5073 1.651758e-05 0.9964
481 4.5073 1.646211e-05 0.9966
482 4.5073 1.639271e-05 0.9958
483 4.5073 1.633027e-05 0.9962
484 4.5073 1.622154e-05 0.9933
485 4.5073 1.587925e-05 0.9789
486 4.5073 1.541874e-05 0.9710
487 4.5073 1.509442e-05 0.9790
488 4.5073 1.477210e-05 0.9786
489 4.5073 1.440651e-05 0.9753
490 4.5073 1.411757e-05 0.9799
491 4.5073 1.379559e-05 0.9772
492 4.5073 1.354723e-05 0.9820
493 4.5073 1.314833e-05 0.9706
----------------------------------------------------------------------
Total Iterations: 494
Avg Convergence Rate: 0.9724
Final Residual: 1.314833e-05
Total Reduction in Residual: 9.711549e-07
Maximum Memory Usage: 4.507 GB
----------------------------------------------------------------------
Total Time: 16.6462
setup: 0.555882 s
solve: 16.0903 s
solve(per iteration): 0.0325714 s

Output for PCG_AGGREGATION_JACOBI.json (~16s as well):
AMGX version 2.5.0
Built on Apr 25 2024, 13:20:11
Compiled with CUDA Runtime 12.4, using CUDA driver 12.4
AMG Grid:
Number of Levels: 9
LVL ROWS NNZ PARTS SPRSTY Mem (GB)
----------------------------------------------------------------------
0(D) 1507876 113305856 1 4.98e-05 1.3
1(D) 281692 31103364 1 0.000392 0.706
2(D) 39859 4883517 1 0.00307 0.111
3(D) 4793 328295 1 0.0143 0.00753
4(D) 762 35364 1 0.0609 0.000822
5(D) 198 7160 1 0.183 0.000168
6(D) 60 1622 1 0.451 3.88e-05
7(D) 18 254 1 0.784 6.45e-06
8(D) 6 36 1 1 9.98e-07
----------------------------------------------------------------------
Grid Complexity: 1.21712
Operator Complexity: 1.3209
Total Memory Usage: 2.12977 GB
----------------------------------------------------------------------
iter Mem Usage (GB) residual rate
----------------------------------------------------------------------
Ini 4.50732 1.353886e+01
0 4.50732 9.552636e+00 0.7056
1 4.5073 6.826744e+00 0.7146
2 4.5073 5.451789e+00 0.7986
3 4.5073 4.193551e+00 0.7692
4 4.5073 3.318682e+00 0.7914
5 4.5073 2.696656e+00 0.8126
6 4.5073 2.253779e+00 0.8358
7 4.5073 1.897404e+00 0.8419
8 4.5073 1.605763e+00 0.8463
9 4.5073 1.371794e+00 0.8543
10 4.5073 1.280317e+00 0.9333
11 4.5073 1.165342e+00 0.9102
12 4.5073 1.044937e+00 0.8967
13 4.5073 9.311771e-01 0.8911
14 4.5073 8.317691e-01 0.8932
15 4.5073 7.104873e-01 0.8542
16 4.5073 6.342603e-01 0.8927
17 4.5073 5.645147e-01 0.8900
18 4.5073 5.062215e-01 0.8967
19 4.5073 4.567174e-01 0.9022
20 4.5073 4.388230e-01 0.9608
21 4.5073 4.211402e-01 0.9597
22 4.5073 3.973845e-01 0.9436
23 4.5073 3.710982e-01 0.9339
24 4.5073 3.424965e-01 0.9229
25 4.5073 3.155958e-01 0.9215
26 4.5073 2.966112e-01 0.9398
27 4.5073 2.795076e-01 0.9423
28 4.5073 2.635899e-01 0.9431
29 4.5073 2.461252e-01 0.9337
30 4.5073 2.365946e-01 0.9613
31 4.5073 2.264181e-01 0.9570
32 4.5073 2.144454e-01 0.9471
33 4.5073 2.034822e-01 0.9489
34 4.5073 1.939666e-01 0.9532
35 4.5073 1.830933e-01 0.9439
36 4.5073 1.759580e-01 0.9610
37 4.5073 1.689407e-01 0.9601
38 4.5073 1.625273e-01 0.9620
39 4.5073 1.567911e-01 0.9647
40 4.5073 1.523330e-01 0.9716
41 4.5073 1.482169e-01 0.9730
42 4.5073 1.430960e-01 0.9655
43 4.5073 1.379123e-01 0.9638
44 4.5073 1.318838e-01 0.9563
45 4.5073 1.262892e-01 0.9576
46 4.5073 1.216729e-01 0.9634
47 4.5073 1.172642e-01 0.9638
48 4.5073 1.136271e-01 0.9690
49 4.5073 1.098468e-01 0.9667
50 4.5073 1.073779e-01 0.9775
51 4.5073 1.044496e-01 0.9727
52 4.5073 1.007739e-01 0.9648
53 4.5073 9.737754e-02 0.9663
54 4.5073 9.470861e-02 0.9726
55 4.5073 9.166303e-02 0.9678
56 4.5073 8.969872e-02 0.9786
57 4.5073 8.761253e-02 0.9767
58 4.5073 8.539046e-02 0.9746
59 4.5073 8.346770e-02 0.9775
60 4.5073 8.175577e-02 0.9795
61 4.5073 8.011439e-02 0.9799
62 4.5073 7.851187e-02 0.9800
63 4.5073 7.699035e-02 0.9806
64 4.5073 7.499845e-02 0.9741
65 4.5073 7.315360e-02 0.9754
66 4.5073 7.119191e-02 0.9732
67 4.5073 6.927591e-02 0.9731
68 4.5073 6.783679e-02 0.9792
69 4.5073 6.647737e-02 0.9800
70 4.5073 6.545250e-02 0.9846
71 4.5073 6.426370e-02 0.9818
72 4.5073 6.257041e-02 0.9737
73 4.5073 6.100275e-02 0.9749
74 4.5073 5.987852e-02 0.9816
75 4.5073 5.862508e-02 0.9791
76 4.5073 5.780494e-02 0.9860
77 4.5073 5.687329e-02 0.9839
78 4.5073 5.563071e-02 0.9782
79 4.5073 5.456461e-02 0.9808
80 4.5073 5.351356e-02 0.9807
81 4.5073 5.255514e-02 0.9821
82 4.5073 5.177287e-02 0.9851
83 4.5073 5.106999e-02 0.9864
84 4.5073 5.012493e-02 0.9815
85 4.5073 4.921204e-02 0.9818
86 4.5073 4.799043e-02 0.9752
87 4.5073 4.670630e-02 0.9732
88 4.5073 4.583710e-02 0.9814
89 4.5073 4.502878e-02 0.9824
90 4.5073 4.445102e-02 0.9872
91 4.5073 4.367765e-02 0.9826
92 4.5073 4.254507e-02 0.9741
93 4.5073 4.147386e-02 0.9748
94 4.5073 4.078047e-02 0.9833
95 4.5073 4.004918e-02 0.9821
96 4.5073 3.958986e-02 0.9885
97 4.5073 3.904670e-02 0.9863
98 4.5073 3.819285e-02 0.9781
99 4.5073 3.743150e-02 0.9801
100 4.5073 3.671366e-02 0.9808
101 4.5073 3.597912e-02 0.9800
102 4.5073 3.552583e-02 0.9874
103 4.5073 3.510992e-02 0.9883
104 4.5073 3.452025e-02 0.9832
105 4.5073 3.394837e-02 0.9834
106 4.5073 3.301806e-02 0.9726
107 4.5073 3.206440e-02 0.9711
108 4.5073 3.147588e-02 0.9816
109 4.5073 3.095063e-02 0.9833
110 4.5073 3.051699e-02 0.9860
111 4.5073 3.003388e-02 0.9842
112 4.5073 2.918968e-02 0.9719
113 4.5073 2.837800e-02 0.9722
114 4.5073 2.792080e-02 0.9839
115 4.5073 2.745213e-02 0.9832
116 4.5073 2.715536e-02 0.9892
117 4.5073 2.684906e-02 0.9887
118 4.5073 2.618969e-02 0.9754
119 4.5073 2.561850e-02 0.9782
120 4.5073 2.506056e-02 0.9782
121 4.5073 2.451296e-02 0.9781
122 4.5073 2.423323e-02 0.9886
123 4.5073 2.397209e-02 0.9892
124 4.5073 2.358096e-02 0.9837
125 4.5073 2.317335e-02 0.9827
126 4.5073 2.252357e-02 0.9720
127 4.5073 2.182169e-02 0.9688
128 4.5073 2.142175e-02 0.9817
129 4.5073 2.106657e-02 0.9834
130 4.5073 2.076983e-02 0.9859
131 4.5073 2.043719e-02 0.9840
132 4.5073 1.982875e-02 0.9702
133 4.5073 1.925887e-02 0.9713
134 4.5073 1.892543e-02 0.9827
135 4.5073 1.861500e-02 0.9836
136 4.5073 1.841600e-02 0.9893
137 4.5073 1.822721e-02 0.9897
138 4.5073 1.774987e-02 0.9738
139 4.5073 1.723753e-02 0.9711
140 4.5073 1.675373e-02 0.9719
141 4.5073 1.636719e-02 0.9769
142 4.5073 1.617550e-02 0.9883
143 4.5073 1.600921e-02 0.9897
144 4.5073 1.578544e-02 0.9860
145 4.5073 1.549496e-02 0.9816
146 4.5073 1.508306e-02 0.9734
147 4.5073 1.456481e-02 0.9656
148 4.5073 1.427043e-02 0.9798
149 4.5073 1.402647e-02 0.9829
150 4.5073 1.384524e-02 0.9871
151 4.5073 1.359538e-02 0.9820
152 4.5073 1.315202e-02 0.9674
153 4.5073 1.278672e-02 0.9722
154 4.5073 1.253887e-02 0.9806
155 4.5073 1.235779e-02 0.9856
156 4.5073 1.223345e-02 0.9899
157 4.5073 1.210488e-02 0.9895
158 4.5073 1.180894e-02 0.9756
159 4.5073 1.147988e-02 0.9721
160 4.5073 1.116957e-02 0.9730
161 4.5073 1.092455e-02 0.9781
162 4.5073 1.079219e-02 0.9879
163 4.5073 1.069048e-02 0.9906
164 4.5073 1.054792e-02 0.9867
165 4.5073 1.033906e-02 0.9802
166 4.5073 1.006528e-02 0.9735
167 4.5073 9.688645e-03 0.9626
168 4.5073 9.485966e-03 0.9791
169 4.5073 9.321227e-03 0.9826
170 4.5073 9.195419e-03 0.9865
171 4.5073 9.021929e-03 0.9811
172 4.5073 8.718005e-03 0.9663
173 4.5073 8.466200e-03 0.9711
174 4.5073 8.285718e-03 0.9787
175 4.5073 8.173952e-03 0.9865
176 4.5073 8.095984e-03 0.9905
177 4.5073 8.004101e-03 0.9887
178 4.5073 7.829951e-03 0.9782
179 4.5073 7.643146e-03 0.9761
180 4.5073 7.462735e-03 0.9764
181 4.5073 7.309771e-03 0.9795
182 4.5073 7.223806e-03 0.9882
183 4.5073 7.155744e-03 0.9906
184 4.5073 7.058219e-03 0.9864
185 4.5073 6.910895e-03 0.9791
186 4.5073 6.705845e-03 0.9703
187 4.5073 6.452930e-03 0.9623
188 4.5073 6.324550e-03 0.9801
189 4.5073 6.209451e-03 0.9818
190 4.5073 6.117331e-03 0.9852
191 4.5073 6.002762e-03 0.9813
192 4.5073 5.795177e-03 0.9654
193 4.5073 5.617591e-03 0.9694
194 4.5073 5.492606e-03 0.9778
195 4.5073 5.417069e-03 0.9862
196 4.5073 5.365974e-03 0.9906
197 4.5073 5.306792e-03 0.9890
198 4.5073 5.199231e-03 0.9797
199 4.5073 5.089314e-03 0.9789
200 4.5073 4.984597e-03 0.9794
201 4.5073 4.885234e-03 0.9801
202 4.5073 4.831392e-03 0.9890
203 4.5073 4.785620e-03 0.9905
204 4.5073 4.720878e-03 0.9865
205 4.5073 4.617083e-03 0.9780
206 4.5073 4.473098e-03 0.9688
207 4.5073 4.298896e-03 0.9611
208 4.5073 4.211524e-03 0.9797
209 4.5073 4.126273e-03 0.9798
210 4.5073 4.054650e-03 0.9826
211 4.5073 3.976860e-03 0.9808
212 4.5073 3.832365e-03 0.9637
213 4.5073 3.712143e-03 0.9686
214 4.5073 3.627296e-03 0.9771
215 4.5073 3.576847e-03 0.9861
216 4.5073 3.542947e-03 0.9905
217 4.5073 3.507197e-03 0.9899
218 4.5073 3.438890e-03 0.9805
219 4.5073 3.373210e-03 0.9809
220 4.5073 3.309594e-03 0.9811
221 4.5073 3.246149e-03 0.9808
222 4.5073 3.212992e-03 0.9898
223 4.5073 3.182623e-03 0.9905
224 4.5073 3.138007e-03 0.9860
225 4.5073 3.063484e-03 0.9763
226 4.5073 2.962185e-03 0.9669
227 4.5073 2.842631e-03 0.9596
228 4.5073 2.782643e-03 0.9789
229 4.5073 2.721328e-03 0.9780
230 4.5073 2.669796e-03 0.9811
231 4.5073 2.615614e-03 0.9797
232 4.5073 2.516624e-03 0.9622
233 4.5073 2.433075e-03 0.9668
234 4.5073 2.372982e-03 0.9753
235 4.5073 2.338936e-03 0.9857
236 4.5073 2.316631e-03 0.9905
237 4.5073 2.293694e-03 0.9901
238 4.5073 2.248138e-03 0.9801
239 4.5073 2.206995e-03 0.9817
240 4.5073 2.167269e-03 0.9820
241 4.5073 2.124825e-03 0.9804
242 4.5073 2.103450e-03 0.9899
243 4.5073 2.083674e-03 0.9906
244 4.5073 2.054267e-03 0.9859
245 4.5073 2.001011e-03 0.9741
246 4.5073 1.930307e-03 0.9647
247 4.5073 1.848721e-03 0.9577
248 4.5073 1.808268e-03 0.9781
249 4.5073 1.763726e-03 0.9754
250 4.5073 1.726539e-03 0.9789
251 4.5073 1.689947e-03 0.9788
252 4.5073 1.624000e-03 0.9610
253 4.5073 1.565009e-03 0.9637
254 4.5073 1.522738e-03 0.9730
255 4.5073 1.500731e-03 0.9855
256 4.5073 1.486607e-03 0.9906
257 4.5073 1.472336e-03 0.9904
258 4.5073 1.442995e-03 0.9801
259 4.5073 1.401880e-03 0.9715
260 4.5073 1.363245e-03 0.9724
261 4.5073 1.338839e-03 0.9821
262 4.5073 1.324971e-03 0.9896
263 4.5073 1.314374e-03 0.9920
264 4.5073 1.297219e-03 0.9869
265 4.5073 1.265595e-03 0.9756
266 4.5073 1.219403e-03 0.9635
267 4.5073 1.165827e-03 0.9561
268 4.5073 1.136158e-03 0.9746
269 4.5073 1.106047e-03 0.9735
270 4.5073 1.081060e-03 0.9774
271 4.5073 1.057849e-03 0.9785
272 4.5073 1.011653e-03 0.9563
273 4.5073 9.744879e-04 0.9633
274 4.5073 9.499882e-04 0.9749
275 4.5073 9.369585e-04 0.9863
276 4.5073 9.290940e-04 0.9916
277 4.5073 9.204583e-04 0.9907
278 4.5073 9.048393e-04 0.9830
279 4.5073 8.898536e-04 0.9834
280 4.5073 8.755330e-04 0.9839
281 4.5073 8.611554e-04 0.9836
282 4.5073 8.530492e-04 0.9906
283 4.5073 8.459542e-04 0.9917
284 4.5073 8.339620e-04 0.9858
285 4.5073 8.123608e-04 0.9741
286 4.5073 7.823079e-04 0.9630
287 4.5073 7.475207e-04 0.9555
288 4.5073 7.289601e-04 0.9752
289 4.5073 7.082735e-04 0.9716
290 4.5073 6.906207e-04 0.9751
291 4.5073 6.748595e-04 0.9772
292 4.5073 6.470612e-04 0.9588
293 4.5073 6.235908e-04 0.9637
294 4.5073 6.062682e-04 0.9722
295 4.5073 5.972097e-04 0.9851
296 4.5073 5.916863e-04 0.9908
297 4.5073 5.861331e-04 0.9906
298 4.5073 5.771284e-04 0.9846
299 4.5073 5.698246e-04 0.9873
300 4.5073 5.629160e-04 0.9879
301 4.5073 5.548604e-04 0.9857
302 4.5073 5.496628e-04 0.9906
303 4.5073 5.443566e-04 0.9903
304 4.5073 5.351942e-04 0.9832
305 4.5073 5.197481e-04 0.9711
306 4.5073 5.013518e-04 0.9646
307 4.5073 4.820704e-04 0.9615
308 4.5073 4.703863e-04 0.9758
309 4.5073 4.551120e-04 0.9675
310 4.5073 4.420310e-04 0.9713
311 4.5073 4.321963e-04 0.9778
312 4.5073 4.173260e-04 0.9656
313 4.5073 4.027740e-04 0.9651
314 4.5073 3.902635e-04 0.9689
315 4.5073 3.834007e-04 0.9824
316 4.5073 3.794524e-04 0.9897
317 4.5073 3.759048e-04 0.9907
318 4.5073 3.706544e-04 0.9860
319 4.5073 3.662881e-04 0.9882
320 4.5073 3.621661e-04 0.9887
321 4.5073 3.575503e-04 0.9873
322 4.5073 3.542326e-04 0.9907
323 4.5073 3.505049e-04 0.9895
324 4.5073 3.438591e-04 0.9810
325 4.5073 3.327386e-04 0.9677
326 4.5073 3.209327e-04 0.9645
327 4.5073 3.097232e-04 0.9651
328 4.5073 3.025474e-04 0.9768
329 4.5073 2.915968e-04 0.9638
330 4.5073 2.822276e-04 0.9679
331 4.5073 2.758958e-04 0.9776
332 4.5073 2.670500e-04 0.9679
333 4.5073 2.577056e-04 0.9650
334 4.5073 2.490600e-04 0.9665
335 4.5073 2.442267e-04 0.9806
336 4.5073 2.416747e-04 0.9896
337 4.5073 2.395729e-04 0.9913
338 4.5073 2.366283e-04 0.9877
339 4.5073 2.329755e-04 0.9846
340 4.5073 2.295243e-04 0.9852
341 4.5073 2.271336e-04 0.9896
342 4.5073 2.249478e-04 0.9904
343 4.5073 2.225149e-04 0.9892
344 4.5073 2.183825e-04 0.9814
345 4.5073 2.115974e-04 0.9689
346 4.5073 2.037608e-04 0.9630
347 4.5073 1.961096e-04 0.9624
348 4.5073 1.915623e-04 0.9768
349 4.5073 1.844891e-04 0.9631
350 4.5073 1.789415e-04 0.9699
351 4.5073 1.747135e-04 0.9764
352 4.5073 1.685326e-04 0.9646
353 4.5073 1.625619e-04 0.9646
354 4.5073 1.573789e-04 0.9681
355 4.5073 1.543450e-04 0.9807
356 4.5073 1.527411e-04 0.9896
357 4.5073 1.514816e-04 0.9918
358 4.5073 1.498667e-04 0.9893
359 4.5073 1.482822e-04 0.9894
360 4.5073 1.467254e-04 0.9895
361 4.5073 1.453226e-04 0.9904
362 4.5073 1.440473e-04 0.9912
363 4.5073 1.425693e-04 0.9897
364 4.5073 1.399659e-04 0.9817
365 4.5073 1.354891e-04 0.9680
366 4.5073 1.304395e-04 0.9627
367 4.5073 1.255801e-04 0.9627
368 4.5073 1.224384e-04 0.9750
369 4.5073 1.178715e-04 0.9627
370 4.5073 1.141439e-04 0.9684
371 4.5073 1.114363e-04 0.9763
372 4.5073 1.074004e-04 0.9638
373 4.5073 1.037559e-04 0.9661
374 4.5073 1.005130e-04 0.9687
375 4.5073 9.864467e-05 0.9814
376 4.5073 9.765982e-05 0.9900
377 4.5073 9.693787e-05 0.9926
378 4.5073 9.603671e-05 0.9907
379 4.5073 9.536281e-05 0.9930
380 4.5073 9.470601e-05 0.9931
381 4.5073 9.386835e-05 0.9912
382 4.5073 9.312684e-05 0.9921
383 4.5073 9.220860e-05 0.9901
384 4.5073 9.051789e-05 0.9817
385 4.5073 8.757986e-05 0.9675
386 4.5073 8.441697e-05 0.9639
387 4.5073 8.117176e-05 0.9616
388 4.5073 7.917326e-05 0.9754
389 4.5073 7.650414e-05 0.9663
390 4.5073 7.415814e-05 0.9693
391 4.5073 7.243978e-05 0.9768
392 4.5073 6.992122e-05 0.9652
393 4.5073 6.755464e-05 0.9662
394 4.5073 6.550298e-05 0.9696
395 4.5073 6.442420e-05 0.9835
396 4.5073 6.386114e-05 0.9913
397 4.5073 6.343917e-05 0.9934
398 4.5073 6.291857e-05 0.9918
399 4.5073 6.255101e-05 0.9942
400 4.5073 6.218764e-05 0.9942
401 4.5073 6.169533e-05 0.9921
402 4.5073 6.126233e-05 0.9930
403 4.5073 6.074052e-05 0.9915
404 4.5073 5.976100e-05 0.9839
405 4.5073 5.783132e-05 0.9677
406 4.5073 5.582332e-05 0.9653
407 4.5073 5.381541e-05 0.9640
408 4.5073 5.256528e-05 0.9768
409 4.5073 5.088652e-05 0.9681
410 4.5073 4.942387e-05 0.9713
411 4.5073 4.836018e-05 0.9785
412 4.5073 4.681841e-05 0.9681
413 4.5073 4.530922e-05 0.9678
414 4.5073 4.403902e-05 0.9720
415 4.5073 4.342640e-05 0.9861
416 4.5073 4.311669e-05 0.9929
417 4.5073 4.288333e-05 0.9946
418 4.5073 4.258149e-05 0.9930
419 4.5073 4.234146e-05 0.9944
420 4.5073 4.210422e-05 0.9944
421 4.5073 4.182142e-05 0.9933
422 4.5073 4.157963e-05 0.9942
423 4.5073 4.129297e-05 0.9931
424 4.5073 4.075008e-05 0.9869
425 4.5073 3.955404e-05 0.9706
426 4.5073 3.826284e-05 0.9674
427 4.5073 3.703658e-05 0.9680
428 4.5073 3.627819e-05 0.9795
429 4.5073 3.518842e-05 0.9700
430 4.5073 3.427321e-05 0.9740
431 4.5073 3.360933e-05 0.9806
432 4.5073 3.266358e-05 0.9719
433 4.5073 3.170647e-05 0.9707
434 4.5073 3.092739e-05 0.9754
435 4.5073 3.057961e-05 0.9888
436 4.5073 3.040358e-05 0.9942
437 4.5073 3.027417e-05 0.9957
438 4.5073 3.010033e-05 0.9943
439 4.5073 2.991853e-05 0.9940
440 4.5073 2.973998e-05 0.9940
441 4.5073 2.958330e-05 0.9947
442 4.5073 2.944665e-05 0.9954
443 4.5073 2.928091e-05 0.9944
444 4.5073 2.897939e-05 0.9897
445 4.5073 2.826589e-05 0.9754
446 4.5073 2.743131e-05 0.9705
447 4.5073 2.667133e-05 0.9723
448 4.5073 2.615545e-05 0.9807
449 4.5073 2.540920e-05 0.9715
450 4.5073 2.485503e-05 0.9782
451 4.5073 2.436528e-05 0.9803
452 4.5073 2.373976e-05 0.9743
453 4.5073 2.310008e-05 0.9731
454 4.5073 2.261866e-05 0.9792
455 4.5073 2.240689e-05 0.9906
456 4.5073 2.229318e-05 0.9949
457 4.5073 2.221321e-05 0.9964
458 4.5073 2.210959e-05 0.9953
459 4.5073 2.200217e-05 0.9951
460 4.5073 2.189792e-05 0.9953
461 4.5073 2.180671e-05 0.9958
462 4.5073 2.171675e-05 0.9959
463 4.5073 2.161412e-05 0.9953
464 4.5073 2.143114e-05 0.9915
465 4.5073 2.097284e-05 0.9786
466 4.5073 2.038027e-05 0.9717
467 4.5073 1.986920e-05 0.9749
468 4.5073 1.948462e-05 0.9806
469 4.5073 1.898509e-05 0.9744
470 4.5073 1.857711e-05 0.9785
471 4.5073 1.821628e-05 0.9806
472 4.5073 1.782268e-05 0.9784
473 4.5073 1.736033e-05 0.9741
474 4.5073 1.698073e-05 0.9781
475 4.5073 1.684377e-05 0.9919
476 4.5073 1.676710e-05 0.9954
477 4.5073 1.670770e-05 0.9965
478 4.5073 1.663904e-05 0.9959
479 4.5073 1.657661e-05 0.9962
480 4.5073 1.651758e-05 0.9964
481 4.5073 1.646211e-05 0.9966
482 4.5073 1.639271e-05 0.9958
483 4.5073 1.633027e-05 0.9962
484 4.5073 1.622154e-05 0.9933
485 4.5073 1.587925e-05 0.9789
486 4.5073 1.541874e-05 0.9710
487 4.5073 1.509443e-05 0.9790
488 4.5073 1.477211e-05 0.9786
489 4.5073 1.440650e-05 0.9753
490 4.5073 1.411756e-05 0.9799
491 4.5073 1.379558e-05 0.9772
492 4.5073 1.354723e-05 0.9820
493 4.5073 1.314832e-05 0.9706
----------------------------------------------------------------------
Total Iterations: 494
Avg Convergence Rate: 0.9724
Final Residual: 1.314832e-05
Total Reduction in Residual: 9.711542e-07
Maximum Memory Usage: 4.507 GB
----------------------------------------------------------------------
Total Time: 16.4863
setup: 0.550713 s
solve: 15.9356 s
solve(per iteration): 0.0322583 s

Adding strength threshold = 0.6 i manage to get GMRES_AMD_D2 to get to 7s but still longer than CG without preconditioner.
{
"config_version": 2,
"determinism_flag": 1,
"exception_handling" : 1,
"solver": {
"print_grid_stats": 1,
"store_res_history": 0,
"solver": "GMRES",
"print_solve_stats": 1,
"obtain_timings": 1,
"preconditioner": {
"interpolator": "D2",
"print_grid_stats": 1,
"solver": "AMG",
"smoother": "JACOBI_L1",
"presweeps": 2,
"selector": "PMIS",
"coarsest_sweeps": 2,
"coarse_solver": "NOSOLVER",
"max_iters": 1,
"interp_max_elements": 4,
"min_coarse_rows": 2,
"scope": "amg_solver",
"max_levels": 24,
"cycle": "V",
"postsweeps": 2,
"max_row_sum": 0.9,
"strength_threshold": 0.6
},
"max_iters": 1000,
"monitor_residual": 1,
"gmres_n_restart": 10,
"convergence": "RELATIVE_INI_CORE",
"tolerance": 1e-06,
"norm": "L2"
}
}
Here is the out put:
AMGX version 2.5.0
Built on Apr 25 2024, 13:20:11
Compiled with CUDA Runtime 12.4, using CUDA driver 12.4
AMG Grid:
Number of Levels: 8
LVL ROWS NNZ PARTS SPRSTY Mem (GB)
----------------------------------------------------------------------
0(D) 1507876 113305856 1 4.98e-05 1.31
1(D) 427806 45533312 1 0.000249 1.04
2(D) 137879 22690729 1 0.00119 0.513
3(D) 38257 6494695 1 0.00444 0.147
4(D) 9256 1114344 1 0.013 0.0253
5(D) 1820 124304 1 0.0375 0.00285
6(D) 351 12935 1 0.105 0.000303
7(D) 47 451 1 0.204 1.15e-05
----------------------------------------------------------------------
Grid Complexity: 1.40813
Operator Complexity: 1.67049
Total Memory Usage: 3.03017 GB
----------------------------------------------------------------------
iter Mem Usage (GB) residual rate
----------------------------------------------------------------------
Ini 4.52881 1.353886e+01
0 4.52881 8.768310e+00 0.6476
1 4.5288 5.875893e+00 0.6701
2 4.5288 4.114999e+00 0.7003
3 4.5288 2.921429e+00 0.7099
4 4.5288 2.161990e+00 0.7400
5 4.5288 1.646122e+00 0.7614
6 4.5288 1.277718e+00 0.7762
7 4.5288 1.001020e+00 0.7834
8 4.5288 7.980492e-01 0.7972
9 4.5288 6.559208e-01 0.8219
10 4.5288 5.960254e-01 0.9087
11 4.5288 5.332775e-01 0.8947
12 4.5288 4.738347e-01 0.8885
13 4.5288 4.126436e-01 0.8709
14 4.5288 3.498271e-01 0.8478
15 4.5288 2.893011e-01 0.8270
16 4.5288 2.489482e-01 0.8605
17 4.5288 2.146483e-01 0.8622
18 4.5288 1.837306e-01 0.8560
19 4.5288 1.614325e-01 0.8786
20 4.5288 1.520154e-01 0.9417
21 4.5288 1.407548e-01 0.9259
22 4.5288 1.286014e-01 0.9137
23 4.5288 1.176749e-01 0.9150
24 4.5288 1.072537e-01 0.9114
25 4.5288 9.944967e-02 0.9272
26 4.5288 9.291163e-02 0.9343
27 4.5288 8.549769e-02 0.9202
28 4.5288 7.896385e-02 0.9236
29 4.5288 7.237665e-02 0.9166
30 4.5288 6.818068e-02 0.9420
31 4.5288 6.386154e-02 0.9367
32 4.5288 5.982216e-02 0.9367
33 4.5288 5.638698e-02 0.9426
34 4.5288 5.283324e-02 0.9370
35 4.5288 4.850473e-02 0.9181
36 4.5288 4.445382e-02 0.9165
37 4.5288 4.021743e-02 0.9047
38 4.5288 3.686018e-02 0.9165
39 4.5288 3.463181e-02 0.9395
40 4.5288 3.264420e-02 0.9426
41 4.5288 3.038673e-02 0.9308
42 4.5288 2.781750e-02 0.9154
43 4.5288 2.567715e-02 0.9231
44 4.5288 2.368404e-02 0.9224
45 4.5288 2.225356e-02 0.9396
46 4.5288 2.094205e-02 0.9411
47 4.5288 1.957047e-02 0.9345
48 4.5288 1.835569e-02 0.9379
49 4.5288 1.715982e-02 0.9349
50 4.5288 1.621576e-02 0.9450
51 4.5288 1.527343e-02 0.9419
52 4.5288 1.438522e-02 0.9418
53 4.5288 1.351747e-02 0.9397
54 4.5288 1.264552e-02 0.9355
55 4.5288 1.153628e-02 0.9123
56 4.5288 1.046056e-02 0.9068
57 4.5288 9.460004e-03 0.9043
58 4.5288 8.685592e-03 0.9181
59 4.5288 8.173298e-03 0.9410
60 4.5288 7.713984e-03 0.9438
61 4.5288 7.156656e-03 0.9278
62 4.5288 6.530942e-03 0.9126
63 4.5288 5.991247e-03 0.9174
64 4.5288 5.501422e-03 0.9182
65 4.5288 5.160612e-03 0.9381
66 4.5288 4.843384e-03 0.9385
67 4.5288 4.559390e-03 0.9414
68 4.5288 4.290216e-03 0.9410
69 4.5288 4.035412e-03 0.9406
70 4.5288 3.819968e-03 0.9466
71 4.5288 3.611904e-03 0.9455
72 4.5288 3.419467e-03 0.9467
73 4.5288 3.208093e-03 0.9382
74 4.5288 2.993778e-03 0.9332
75 4.5288 2.726845e-03 0.9108
76 4.5288 2.457223e-03 0.9011
77 4.5288 2.225714e-03 0.9058
78 4.5288 2.033611e-03 0.9137
79 4.5288 1.914184e-03 0.9413
80 4.5288 1.799697e-03 0.9402
81 4.5288 1.670334e-03 0.9281
82 4.5288 1.522858e-03 0.9117
83 4.5288 1.388566e-03 0.9118
84 4.5288 1.275700e-03 0.9187
85 4.5288 1.194108e-03 0.9360
86 4.5288 1.119639e-03 0.9376
87 4.5288 1.061499e-03 0.9481
88 4.5288 1.000335e-03 0.9424
89 4.5288 9.452505e-04 0.9449
90 4.5288 8.985699e-04 0.9506
91 4.5288 8.504361e-04 0.9464
92 4.5288 8.098885e-04 0.9523
93 4.5288 7.613204e-04 0.9400
94 4.5288 7.080914e-04 0.9301
95 4.5288 6.480337e-04 0.9152
96 4.5288 5.812936e-04 0.8970
97 4.5288 5.259771e-04 0.9048
98 4.5288 4.851439e-04 0.9224
99 4.5288 4.556539e-04 0.9392
100 4.5288 4.293960e-04 0.9424
101 4.5288 4.009284e-04 0.9337
102 4.5288 3.650740e-04 0.9106
103 4.5288 3.336130e-04 0.9138
104 4.5288 3.087990e-04 0.9256
105 4.5288 2.887910e-04 0.9352
106 4.5288 2.724659e-04 0.9435
107 4.5288 2.612934e-04 0.9590
108 4.5288 2.473665e-04 0.9467
109 4.5288 2.350677e-04 0.9503
110 4.5288 2.251407e-04 0.9578
111 4.5288 2.138501e-04 0.9499
112 4.5288 2.055829e-04 0.9613
113 4.5288 1.951170e-04 0.9491
114 4.5288 1.826712e-04 0.9362
115 4.5288 1.693517e-04 0.9271
116 4.5288 1.546645e-04 0.9133
117 4.5288 1.414307e-04 0.9144
118 4.5288 1.325003e-04 0.9369
119 4.5288 1.265273e-04 0.9549
120 4.5288 1.208508e-04 0.9551
121 4.5288 1.144998e-04 0.9474
122 4.5288 1.060693e-04 0.9264
123 4.5288 9.925553e-05 0.9358
124 4.5288 9.370971e-05 0.9441
125 4.5288 8.940171e-05 0.9540
126 4.5288 8.594241e-05 0.9613
127 4.5288 8.353282e-05 0.9720
128 4.5288 8.024825e-05 0.9607
129 4.5288 7.731457e-05 0.9634
130 4.5288 7.480387e-05 0.9675
131 4.5288 7.216509e-05 0.9647
132 4.5288 7.037042e-05 0.9751
133 4.5288 6.794323e-05 0.9655
134 4.5288 6.514613e-05 0.9588
135 4.5288 6.205491e-05 0.9525
136 4.5288 5.863003e-05 0.9448
137 4.5288 5.502975e-05 0.9386
138 4.5288 5.254025e-05 0.9548
139 4.5288 5.086614e-05 0.9681
140 4.5288 4.932885e-05 0.9698
141 4.5288 4.746658e-05 0.9622
142 4.5288 4.485311e-05 0.9449
143 4.5288 4.297201e-05 0.9581
144 4.5288 4.136565e-05 0.9626
145 4.5288 4.015194e-05 0.9707
146 4.5288 3.900880e-05 0.9715
147 4.5288 3.820714e-05 0.9794
148 4.5288 3.706657e-05 0.9701
149 4.5288 3.596322e-05 0.9702
150 4.5288 3.497656e-05 0.9726
151 4.5288 3.398798e-05 0.9717
152 4.5288 3.333172e-05 0.9807
153 4.5288 3.242272e-05 0.9727
154 4.5288 3.145855e-05 0.9703
155 4.5288 3.036427e-05 0.9652
156 4.5288 2.914779e-05 0.9599
157 4.5288 2.764016e-05 0.9483
158 4.5288 2.659135e-05 0.9621
159 4.5288 2.590356e-05 0.9741
160 4.5288 2.516156e-05 0.9714
161 4.5288 2.434308e-05 0.9675
162 4.5288 2.309417e-05 0.9487
163 4.5288 2.224793e-05 0.9634
164 4.5288 2.147848e-05 0.9654
165 4.5288 2.090239e-05 0.9732
166 4.5288 2.035908e-05 0.9740
167 4.5288 1.995579e-05 0.9802
168 4.5288 1.940278e-05 0.9723
169 4.5288 1.886533e-05 0.9723
170 4.5288 1.835445e-05 0.9729
171 4.5288 1.786542e-05 0.9734
172 4.5288 1.753188e-05 0.9813
173 4.5288 1.706333e-05 0.9733
174 4.5288 1.659876e-05 0.9728
175 4.5288 1.604605e-05 0.9667
176 4.5288 1.542272e-05 0.9612
177 4.5288 1.468205e-05 0.9520
178 4.5288 1.418284e-05 0.9660
179 4.5288 1.381454e-05 0.9740
180 4.5288 1.343885e-05 0.9728
----------------------------------------------------------------------
Total Iterations: 181
Avg Convergence Rate: 0.9265
Final Residual: 1.343885e-05
Total Reduction in Residual: 9.926137e-07
Maximum Memory Usage: 4.529 GB
----------------------------------------------------------------------
Total Time: 7.59087
setup: 0.583601 s
solve: 7.00727 s
solve(per iteration): 0.0387142 s

3. Thats why I tried with ILU smoother to see if it can improve the performance (Using FSAI as complex smoother in HYPRE helped in my case) and when I found that all the configuration involving ILU did not work for my matrix, so I wonder if there may be additional info needed for Multicolor ILU.
   Here is the output I got when using this configuration (based on GMRES_AMG_D2.json but just change smoother to ILU)
   {
   "config_version": 2,
   "determinism_flag": 1,
   "exception_handling" : 1,
   "solver": {
   "print_grid_stats": 1,
   "store_res_history": 1,
   "solver": "GMRES",
   "print_solve_stats": 1,
   "obtain_timings": 1,
   "preconditioner": {
   "interpolator": "D2",
   "print_grid_stats": 1,
   "solver": "AMG",
   "smoother": "MULTICOLOR_DILU",
   "presweeps": 2,
   "selector": "PMIS",
   "coarsest_sweeps": 2,
   "coarse_solver": "NOSOLVER",
   "max_iters": 1,
   "interp_max_elements": 4,
   "min_coarse_rows": 2,
   "scope": "amg_solver",
   "max_levels": 24,
   "cycle": "V",
   "postsweeps": 2
   },
   "max_iters": 100,
   "monitor_residual": 1,
   "gmres_n_restart": 10,
   "convergence": "RELATIVE_INI_CORE",
   "tolerance": 1e-06,
   "norm": "L2"
   }
   }
   Here is the output:
   AMGX version 2.5.0
   Built on Apr 25 2024, 13:20:11
   Compiled with CUDA Runtime 12.4, using CUDA driver 12.4
   AMG Grid:
   Number of Levels: 9
   LVL ROWS NNZ PARTS SPRSTY Mem (GB)
   ----------------------------------------------------------------------
   0(D) 1507876 113305856 1 4.98e-05 1.3
   1(D) 281692 31103364 1 0.000392 0.706
   2(D) 39859 4883517 1 0.00307 0.111
   3(D) 4793 328295 1 0.0143 0.00753
   4(D) 762 35364 1 0.0609 0.000822
   5(D) 198 7160 1 0.183 0.000168
   6(D) 60 1622 1 0.451 3.88e-05
   7(D) 18 254 1 0.784 6.45e-06
   8(D) 6 36 1 1 9.98e-07
   ----------------------------------------------------------------------
   Grid Complexity: 1.21712
   Operator Complexity: 1.3209
   Total Memory Usage: 2.12977 GB
   ----------------------------------------------------------------------
   iter Mem Usage (GB) residual rate
   ----------------------------------------------------------------------
   Ini 4.52295 1.353886e+01
   0 4.52295 1.353814e+01 0.9999
   1 4.5229 1.353091e+01 0.9995
   2 4.5229 1.353089e+01 1.0000
   3 4.5229 1.353076e+01 1.0000
   4 4.5229 1.353076e+01 1.0000
   5 4.5229 1.351923e+01 0.9991
   6 4.5229 1.351887e+01 1.0000
   7 4.5229 1.350736e+01 0.9991
   8 4.5229 1.350569e+01 0.9999
   9 4.5229 1.350562e+01 1.0000
   10 4.5229 1.350562e+01 1.0000
   11 4.5229 1.350562e+01 1.0000
   12 4.5229 1.350562e+01 1.0000
   13 4.5229 1.350562e+01 1.0000
   14 4.5229 1.350562e+01 1.0000
   15 4.5229 1.350562e+01 1.0000
   16 4.5229 1.350562e+01 1.0000
   17 4.5229 1.350562e+01 1.0000
   18 4.5229 1.350562e+01 1.0000
   19 4.5229 1.350555e+01 1.0000
   20 4.5229 1.350555e+01 1.0000
   21 4.5229 1.350555e+01 1.0000
   22 4.5229 1.350555e+01 1.0000
   23 4.5229 1.350555e+01 1.0000
   24 4.5229 1.350555e+01 1.0000
   25 4.5229 1.350555e+01 1.0000
   26 4.5229 1.350555e+01 1.0000
   27 4.5229 1.350555e+01 1.0000
   28 4.5229 1.350555e+01 1.0000
   29 4.5229 1.350552e+01 1.0000
   30 4.5229 1.350552e+01 1.0000
   31 4.5229 1.350552e+01 1.0000
   32 4.5229 1.350552e+01 1.0000
   33 4.5229 1.350552e+01 1.0000
   34 4.5229 1.350552e+01 1.0000
   35 4.5229 1.350552e+01 1.0000
   36 4.5229 1.350552e+01 1.0000
   37 4.5229 1.350552e+01 1.0000
   38 4.5229 1.350552e+01 1.0000
   39 4.5229 1.350551e+01 1.0000
   40 4.5229 1.350551e+01 1.0000
   41 4.5229 1.350551e+01 1.0000
   42 4.5229 1.350551e+01 1.0000
   43 4.5229 1.350551e+01 1.0000
   44 4.5229 1.350551e+01 1.0000
   45 4.5229 1.350551e+01 1.0000
   46 4.5229 1.350551e+01 1.0000
   47 4.5229 1.350551e+01 1.0000
   48 4.5229 1.350551e+01 1.0000
   49 4.5229 1.350551e+01 1.0000
   50 4.5229 1.350551e+01 1.0000
   51 4.5229 1.350551e+01 1.0000
   52 4.5229 1.350551e+01 1.0000
   53 4.5229 1.350551e+01 1.0000
   54 4.5229 1.350551e+01 1.0000
   55 4.5229 1.350551e+01 1.0000
   56 4.5229 1.350551e+01 1.0000
   57 4.5229 1.350551e+01 1.0000
   58 4.5229 1.350551e+01 1.0000
   59 4.5229 1.350551e+01 1.0000
   60 4.5229 1.350551e+01 1.0000
   61 4.5229 1.350551e+01 1.0000
   62 4.5229 1.350551e+01 1.0000
   63 4.5229 1.350551e+01 1.0000
   64 4.5229 1.350551e+01 1.0000
   65 4.5229 1.350551e+01 1.0000
   66 4.5229 1.350551e+01 1.0000
   67 4.5229 1.350551e+01 1.0000
   68 4.5229 1.350551e+01 1.0000
   69 4.5229 1.350551e+01 1.0000
   70 4.5229 1.350551e+01 1.0000
   71 4.5229 1.350551e+01 1.0000
   72 4.5229 1.350551e+01 1.0000
   73 4.5229 1.350551e+01 1.0000
   74 4.5229 1.350551e+01 1.0000
   75 4.5229 1.350551e+01 1.0000
   76 4.5229 1.350551e+01 1.0000
   77 4.5229 1.350551e+01 1.0000
   78 4.5229 1.350551e+01 1.0000
   79 4.5229 1.350551e+01 1.0000
   80 4.5229 1.350551e+01 1.0000
   81 4.5229 1.350551e+01 1.0000
   82 4.5229 1.350551e+01 1.0000
   83 4.5229 1.350551e+01 1.0000
   84 4.5229 1.350551e+01 1.0000
   85 4.5229 1.350551e+01 1.0000
   86 4.5229 1.350551e+01 1.0000
   87 4.5229 1.350551e+01 1.0000
   88 4.5229 1.350551e+01 1.0000
   89 4.5229 1.350551e+01 1.0000
   90 4.5229 1.350551e+01 1.0000
   91 4.5229 1.350551e+01 1.0000
   92 4.5229 1.350551e+01 1.0000
   93 4.5229 1.350551e+01 1.0000
   94 4.5229 1.350551e+01 1.0000
   95 4.5229 1.350551e+01 1.0000
   96 4.5229 1.350551e+01 1.0000
   97 4.5229 1.350551e+01 1.0000
   98 4.5229 1.350551e+01 1.0000
   99 4.5229 1.350551e+01 1.0000
   ----------------------------------------------------------------------
   Total Iterations: 100
   Avg Convergence Rate: 1.0000
   Final Residual: 1.350551e+01
   Total Reduction in Residual: 9.975368e-01
   Maximum Memory Usage: 4.523 GB
   ----------------------------------------------------------------------
   Total Time: 40.3522
   setup: 1.15667 s
   solve: 39.1955 s

When I tried to used PCG_ILU.json, it return nan.

AMGX version 2.5.0
Built on Apr 25 2024, 13:20:11
Compiled with CUDA Runtime 12.4, using CUDA driver 12.4
iter Mem Usage (GB) residual rate
----------------------------------------------------------------------
Ini 0 1.353886e+01
0 0 nan nan
1 0.0000 -nan(ind) -nan(ind)
2 0.0000 nan nan
3 0.0000 -nan(ind) -nan(ind)
4 0.0000 nan nan
5 0.0000 -nan(ind) -nan(ind)
6 0.0000 nan nan
7 0.0000 -nan(ind) -nan(ind)
8 0.0000 nan nan
9 0.0000 -nan(ind) -nan(ind)
10 0.0000 nan nan
11 0.0000 -nan(ind) -nan(ind)
12 0.0000 nan nan
13 0.0000 -nan(ind) -nan(ind)
14 0.0000 nan nan
15 0.0000 -nan(ind) -nan(ind)
16 0.0000 nan nan
17 0.0000 -nan(ind) -nan(ind)
18 0.0000 nan nan
19 0.0000 -nan(ind) -nan(ind)
----------------------------------------------------------------------
Total Iterations: 20
Avg Convergence Rate: -nan(ind)
Final Residual: -nan(ind)
Total Reduction in Residual: -nan(ind)
Maximum Memory Usage: 0.000 GB
----------------------------------------------------------------------
Total Time: 92.0629
setup: 0.282855 s
solve: 91.78 s

Basically I am still struggling to get the right configuration for the matrix. The best results I got is from CG without any preconditioner. I hope this should clarify my issues.