v4.0.0 release

[prckent]

Proposed changes

Update main with the 4.0.0rc taken from develop after #5297 was merged.

Please verify:
Version updated to 4.0.0 in CMakeLists.txt
Changelog.md topmost description updated. I made this more verbose, added mention of CPU and GPU execution.

( Following https://github.com/QMCPACK/qmcpack/wiki/Release-process ; it needs some updating and
moving to the developer part of the manual. )

What type(s) of changes does this code introduce?

• Bugfix
• New feature
• Code style update (formatting, renaming)
• Refactoring (no functional changes, no api changes)
• Build related changes
• Testing changes (e.g. new unit/integration/performance tests)
• Documentation or build script changes

Does this introduce a breaking change?

• Yes

What systems has this change been tested on?

CI must run, but no consequential changes since e.g. last nightlies and weekend runs.

Checklist

• No. This PR is up to date with current the current state of 'develop'
• Yes. Code added or changed in the PR has been clang-formatted
• Yes. This PR adds tests to cover any new code, or to catch a bug that is being fixed
• Yes. Documentation has been added (if appropriate)

[prckent]

Output of ctest -j 48 -E long- --output-on-failure with PySCF 2.7.0 and Quantum ESPRESSO 7.4 available only shows failures in the "usual suspects", i.e. no obvious new problems. Real build, full precision, GCC 14.2.0, OpenMPI 5.0.0, OpenBLAS 0.3.28.

99% tests passed, 26 tests failed out of 2416

Label Time Summary:
QMCPACK                     = 70861.41 sec*proc (387 tests)
QMCPACK-checking-results    =  92.70 sec*proc (1616 tests)
abort                       = 1224.43 sec*proc (24 tests)
bug                         = 4892.52 sec*proc (58 tests)
cause_unknown               = 12081.42 sec*proc (131 tests)
check_fail                  = 1597.90 sec*proc (19 tests)
converter                   = 4822.23 sec*proc (116 tests)
coverage                    = 196.62 sec*proc (11 tests)
definite_stat_fail          = 1496.29 sec*proc (10 tests)
deterministic               = 1182.02 sec*proc (1330 tests)
hard_fail                   = 580.53 sec*proc (11 tests)
intermittent_stat_fail      = 12081.42 sec*proc (131 tests)
nexus                       = 133.29 sec*proc (70 tests)
performance                 = 28424.18 sec*proc (136 tests)
poor_test                   =  13.79 sec*proc (14 tests)
quality_unknown             = 71240.07 sec*proc (2188 tests)
unit                        = 133.84 sec*proc (54 tests)
unstable                    =  56.15 sec*proc (609 tests)

Total Test time (real) = 2240.43 sec

The following tests FAILED:
        507 - short-LiH_ae-vmc_hf_noj_estimator_energydensity_voronoi-4-4-check (Failed)
        601 - short-LiH_pp-vmc_hf_sdj_estimator_spindensity-4-4 (Failed)
        602 - short-LiH_pp-vmc_hf_sdj_estimator_spindensity-4-4-check (Failed)
        604 - short-LiH_pp-vmc_hf_sdj_estimator_energydensity_voronoi-4-4-check (Failed)
        620 - short-H4-RHF-orb-opt-16-1-totenergy (Failed)
        625 - short-H4-orb-opt-dmc-16-1 (Failed)
        626 - short-H4-orb-opt-dmc-16-1-totenergy (Failed)
        638 - short-H4-target-descent-1-16-totenergy (Failed)
        804 - short-He_orb_rot_param_grad_legacy-1-1-rot-spo-up_orb_rot_0000_0001 (Failed)
        805 - short-He_orb_rot_param_grad_legacy-1-1-rot-spo-down_orb_rot_0000_0001 (Failed)
        807 - short-He_orb_rot_param_grad_legacy-1-16-rot-spo-up_orb_rot_0000_0001 (Failed)
        808 - short-He_orb_rot_param_grad_legacy-1-16-rot-spo-down_orb_rot_0000_0001 (Failed)
        810 - short-He_orb_rot_param_grad_batch-1-1-rot-spo-up_orb_rot_0000_0001 (Failed)
        811 - short-He_orb_rot_param_grad_batch-1-1-rot-spo-down_orb_rot_0000_0001 (Failed)
        813 - short-He_orb_rot_param_grad_batch-1-16-rot-spo-up_orb_rot_0000_0001 (Failed)
        814 - short-He_orb_rot_param_grad_batch-1-16-rot-spo-down_orb_rot_0000_0001 (Failed)
        816 - short-He_orb_rot_param_grad_rhf-1-16-rot-spo-up_orb_rot_0000_0001 (Failed)
        818 - short-He_orb_rot_jastrow_param_grad_legacy-1-16-rot-spo-up_orb_rot_0000_0001 (Failed)
        819 - short-He_orb_rot_jastrow_param_grad_legacy-1-16-rot-spo-down_orb_rot_0000_0001 (Failed)
        822 - short-He_orb_rot_jastrow_param_grad_batch-1-16-rot-spo-up_orb_rot_0000_0001 (Failed)
        823 - short-He_orb_rot_jastrow_param_grad_batch-1-16-rot-spo-down_orb_rot_0000_0001 (Failed)
        1172 - short-diamondC_1x1x1_pp-vmcbatch-estimator-onebodydensitymatrices-4-4-check (Failed)
        1178 - short-diamondC_1x1x1_pp-vmc-noJ-estimator-1rdm-4-4-check (Failed)
        1180 - short-diamondC_1x1x1_pp-vmc-J2-estimator-1rdm-4-4-check (Failed)
        1184 - short-diamondC_1x1x1_pp-vmc-estimator-energydensity-cell-batched-4-4-check (Failed)
        1188 - short-diamondC_1x1x1_pp-dmc-estimator-energydensity-cell-4-4-check (Failed)
Errors while running CTest

[prckent]

Test this please

[ye-luo]

Please merge after #5299

[prckent]

Test this please

[prckent]

Updated test results (from a different machine, missing some python deps and PySCF):

99% tests passed, 7 tests failed out of 2209
...
Total Test time (real) = 2215.14 sec

The following tests FAILED:
        574 - short-H4-RHF-orb-opt-16-1-totenergy (Failed)      QMCPACK-checking-results quality_unknown unstable
        579 - short-H4-orb-opt-dmc-16-1 (Failed)                QMCPACK abort bug quality_unknown
        580 - short-H4-orb-opt-dmc-16-1-totenergy (Failed)      QMCPACK-checking-results abort bug quality_unknown unstable
        590 - short-H4-target-descent-16-1-totenergy (Failed)   QMCPACK-checking-results quality_unknown unstable
        622 - short-H4-optbatch-OneShiftOnly-multistep-1-16-totenergy (Failed) QMCPACK-checking-results quality_unknown unstable
        750 - short-He_orb_rot_param_grad_legacy-1-1-rot-spo-down_orb_rot_0000_0001_r (Failed) QMCPACK-checking-results quality_unknown unstable
        756 - short-He_orb_rot_param_grad_batch-1-1-rot-spo-down_orb_rot_0000_0001_r (Failed) QMCPACK-checking-results quality_unknown unstable
Errors while running CTest

The orb_rot failures might well be statistical. Easy to address in future when we have seen the distribution:

 253/2209 Test  #750: short-He_orb_rot_param_grad_legacy-1-1-rot-spo-down_orb_rot_0000_0001_r ..................................***Failed    0.04 sec
Tests for series 0
  Testing quantity: rot-spo-down_orb_rot_0000_0001_r
    reference mean value     :  -0.21800000
    reference error bar      :   0.01000000
    computed  mean value     :  -0.18226675
    computed  error bar      :   0.01598691
    pass tolerance           :   0.03000000  (  3.00000000 sigma)
    deviation from reference :   0.03573325  (  3.57332464 sigma)
    error bar of deviation   :   0.01885686
    significance probability :   0.99964744  (gaussian statistics)
    status of this test      :   fail

Test status: fail

 255/2209 Test  #756: short-He_orb_rot_param_grad_batch-1-1-rot-spo-down_orb_rot_0000_0001_r ...................................***Failed    0.05 sec
Tests for series 0
  Testing quantity: rot-spo-down_orb_rot_0000_0001_r
    reference mean value     :  -0.21800000
    reference error bar      :   0.01000000
    computed  mean value     :  -0.18226675
    computed  error bar      :   0.01598691
    pass tolerance           :   0.03000000  (  3.00000000 sigma)
    deviation from reference :   0.03573325  (  3.57332464 sigma)
    error bar of deviation   :   0.01885686
    significance probability :   0.99964744  (gaussian statistics)
    status of this test      :   fail

Test status: fail