Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backports for 1.11.3 #56741

Open
wants to merge 22 commits into
base: release-1.11
Choose a base branch
from
Open

Backports for 1.11.3 #56741

wants to merge 22 commits into from

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Dec 3, 2024

Backported PRs:

Need manual backport:

Contains multiple commits, manual intervention needed:

Non-merged PRs with backport label:

@KristofferC KristofferC added the release Release management and versioning. label Dec 3, 2024
@maleadt
Copy link
Member

maleadt commented Dec 10, 2024

This also needs to include LinearAlgebra.jl back-ports: https://github.com/JuliaLang/LinearAlgebra.jl/issues?q=state%3Aclosed%20label%3A%22backport%201.11%22
Or should those be made into commits here against release-1.11 (where the stdlib hasn't been moved out)?

@KristofferC
Copy link
Member Author

Or should those be made into commits here against release-1.11 (where the stdlib hasn't been moved out)?

I've been thinking about it and I think the easiest is just to cherry pick these commits onto the julia repo onto either a backports-linearalgebra-1.11 branch or directly to backports-1.11.

@IanButterworth
Copy link
Member

@nanosoldier runtests(vs=“release-1.10”)

@maleadt
Copy link
Member

maleadt commented Dec 13, 2024

@IanButterworth You typo'd the syntax, branches need to be specified using :: https://github.com/JuliaCI/Nanosoldier.jl?tab=readme-ov-file#trigger-syntax. In any case, it shouldn't be required here, as PkgEval uses the merge-base, which should be release-1.10 already.

@nanosoldier runtests()

@IanButterworth
Copy link
Member

IanButterworth commented Dec 13, 2024

Ah ok, but this is a 1.11 backports PR. I wanted to check it against 1.10 (because stuff in the blacklist hasn't been tested vs 1.10 because of that bug)

IanButterworth and others added 4 commits December 13, 2024 14:39
(cherry picked from commit 32ea18e)
Fixes #56782
Fix `exp(reinterpret(Float64, 0x7ffbb14880000000))` returning non-NaN
value. This should have minimal performance impact since it's already in
the fallback branch.

(cherry picked from commit 19e06d3)
@DilumAluthge
Copy link
Member

I don't see the Nanosoldier commit status here, which leads me to believe that either the webhook was never delivered, or that the Nanosoldier server didn't accept the job.

@DilumAluthge
Copy link
Member

@nanosoldier runtests()

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

@IanButterworth
Copy link
Member

@nanosoldier runtests(vs=":release-1.10")

@IanButterworth
Copy link
Member

IanButterworth commented Dec 14, 2024

@maleadt please cancel the duplicate run invoked here #56741 (comment) (or let me know how to if I can)

The latest request I did is the correct spec. We want to check against 1.10 state

@maleadt
Copy link
Member

maleadt commented Dec 14, 2024

please cancel the duplicate run invoked here #56741 (comment) (or let me know how to if I can)

It is currently not possible to cancel Nanosoldier invocations. Only thing I can to is restart the service at a strategic time. I'll check tomorrow when this job starts.

@maleadt
Copy link
Member

maleadt commented Dec 16, 2024

@nanosoldier runtests(vs=":release-1.10")

@nanosoldier
Copy link
Collaborator

The package evaluation job you requested has completed - possible new issues were detected.
The full report is available.

@jakobnissen
Copy link
Contributor

The ScanByte segfault is a package issue and not a problem with Julia 1.11. I've added it to the PkgEval blacklist.

@IanButterworth
Copy link
Member

IanButterworth commented Dec 17, 2024

I haven't looked into the pkgeval run of this 1.11 branch against 1.10 in detail, but:

593 packages failed only on the current version

  • Package fails to precompile (99 packages):
  • Illegal method overwrites during precompilation (10 packages):
  • Package has test failures (128 packages):
  • Package tests unexpectedly errored (126 packages):
  • Package is missing a package dependency (1 packages):
  • Package is using an unknown package (2 packages):
  • There were unidentified errors (19 packages):
  • Tests became inactive (3 packages):
  • Test duration exceeded the time limit (203 packages):
  • Test log exceeded the size limit (2 packages):

191 packages passed tests only on the current version.

nsajko and others added 6 commits January 2, 2025 13:45
Fixes #56726 added the changes that were suggested. fixing the mistake.

---------

Co-authored-by: Max Horn <[email protected]>
Co-authored-by: Chengyu Han <[email protected]>
(cherry picked from commit fc9e7c4)
…56842)

Fix #56841.

Currently the documentation states that keys(dict) and values(dict)
iterate in the same order. But it is not stated whether this is the same
order as that used by pairs(dict), or when looping, for (k,v) in dict.

This PR makes this guarantee explicit.

(cherry picked from commit 796d823)
#55886 accidentally created a new function
`Base.MathConstants.rationalize` instead of extending
`Base.rationalize`, which is the reason why `Base.rationalize(Int, π)`
isn’t constant-folded in Julia 1.10 and 1.11:

```
julia> @Btime rationalize(Int,π);
  1.837 ns (0 allocations: 0 bytes)      # v1.9: constant-folded
  88.416 μs (412 allocations: 15.00 KiB) # v1.10: not constant-folded
```

This PR fixes that. It should probably be backported to 1.10 and 1.11.

(cherry picked from commit d3c26b7)
Co-authored-by: Ian Butterworth <[email protected]>
(cherry picked from commit e3f90c4)
dgleich and others added 5 commits January 2, 2025 14:06
… causes deprecation warnings (#56306)

The current version of `subtypes` will throw deprecation errors even if
no one is using the deprecated bindings.

A similar bug was fixed in Aqua.jl -
https://github.com/JuliaTesting/Aqua.jl/pull/89/files

See discussion here:

- JuliaIO/ImageMagick.jl#235 (for identifying
the problem)
- simonster/Reexport.jl#42 (for pointing to
the issue in Aqua.jl)
- https://github.com/JuliaTesting/Aqua.jl/pull/89/files (for the fix in
Aqua.jl)

This adds the `isbindingresolved` test to the `subtypes` function to
avoid throwing deprecation warnings. It also adds a test to check that
this doesn't happen.

---

On the current master branch (before the fix), the added test shows:

```
WARNING: using deprecated binding InternalModule.MyOldType in OuterModule.
, use MyType instead.
Subtypes and deprecations: Test Failed at /home/dgleich/devextern/julia/usr/share/julia/stdlib/v1.12/Test/src/Test.jl:932
  Expression: isempty(stderr_content)
   Evaluated: isempty("WARNING: using deprecated binding InternalModule.MyOldType in OuterModule.\n, use MyType instead.\n")
Test Summary:             | Fail  Total  Time
Subtypes and deprecations |    1      1  2.8s
ERROR: LoadError: Some tests did not pass: 0 passed, 1 failed, 0 errored, 0 broken.
in expression starting at /home/dgleich/devextern/julia/stdlib/InteractiveUtils/test/runtests.jl:841
ERROR: Package InteractiveUtils errored during testing
```

---

Using the results of this pull request:

```
@test_nowarn subtypes(Integer);
```

passes without error. The other tests pass too.

(cherry picked from commit 20f933a)
`adce_pass!` can transform a `Union`-type `PhiNode` into a narrower
`PhiNode`, but in such cases, the `IR_FLAG_REFINED` flag isn’t set on
that `PhiNode` statement. By setting this flag, irinterp can perform
statement reprocessing using the narrowed `PhiNode`, enabling type
stability in cases like #56387.

- fixes #56387

(cherry picked from commit 671e1d8)
…5871)

We were accidentally emitting a different pop order for `Expr(:leave,
...)` if you uncomment the `nothing` below:
```julia
let src = Meta.@lower let
    try
        try
            return 1
        catch
        end
    finally
        # nothing # <- uncomment me
    end
end
    println.(filter(stmt->Base.isexpr(stmt, :leave), src.args[1].code))
    nothing
end
```

(cherry picked from commit deac82a)
@KristofferC KristofferC force-pushed the backports-release-1.11 branch from ce7242c to 905e39a Compare January 2, 2025 13:18
IanButterworth and others added 3 commits January 2, 2025 14:13
Prior to this, especially on macOS, the gc-safepoint here would cause
the process to segfault as we had already freed the current_task state.
Rearrange this code so that the GC interactions (except for the atomic
store to current_task) are all handled before entering GC safe, and then
signaling the thread is deleted (via setting current_task = NULL,
published by jl_unlock_profile_wr to other threads) is last.

```
ERROR: Exception handler triggered on unmanaged thread.
Process 53827 stopped
* thread #5, stop reason = EXC_BAD_ACCESS (code=2, address=0x100018008)
    frame #0: 0x0000000100b74344 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_set(ptls=0x000000011f8b3200, state='\x02', old_state=<unavailable>) at julia_threads.h:272:9 [opt]
   269 	    assert(old_state != JL_GC_CONCURRENT_COLLECTOR_THREAD);
   270 	    jl_atomic_store_release(&ptls->gc_state, state);
   271 	    if (state == JL_GC_STATE_UNSAFE || old_state == JL_GC_STATE_UNSAFE)
-> 272 	        jl_gc_safepoint_(ptls);
   273 	    return old_state;
   274 	}
   275 	STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls,
Target 0: (julia) stopped.
(lldb) up
frame #1: 0x0000000100b74320 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_save_and_set(ptls=0x000000011f8b3200, state='\x02') at julia_threads.h:278:12 [opt]
   275 	STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls,
   276 	                                              int8_t state)
   277 	{
-> 278 	    return jl_gc_state_set(ptls, state, jl_atomic_load_relaxed(&ptls->gc_state));
   279 	}
   280 	#ifdef __clang_gcanalyzer__
   281 	// these might not be a safepoint (if they are no-op safe=>safe transitions), but we have to assume it could be (statically)
(lldb)
frame #2: 0x0000000100b7431c libjulia-internal.1.12.0.dylib`jl_delete_thread(value=0x000000011f8b3200) at threading.c:537:11 [opt]
   534 	    ptls->root_task = NULL;
   535 	    jl_free_thread_gc_state(ptls);
   536 	    // then park in safe-region
-> 537 	    (void)jl_gc_safe_enter(ptls);
   538 	}
```

(test incorporated into #55793)

(cherry picked from commit 0d09f3d,
resolving conflicts from not having backported #52198)
We never have a reason to reference this data again since we already
have native code generated for it, so it is simply wasting memory and
download space.

$ du -sh {old,new}/usr/share/julia/compiled
256M old
227M new

(cherry picked from commit dfe6a13)
Comment on lines +306 to +308
uv_mutex_unlock(&safepoint_lock);
jl_set_gc_and_wait(jl_current_task);
uv_mutex_lock(&safepoint_lock);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is incorrect, and shouldn't be adding the lock calls (without also adding #55105 to backports)

Suggested change
uv_mutex_unlock(&safepoint_lock);
jl_set_gc_and_wait(jl_current_task);
uv_mutex_lock(&safepoint_lock);
jl_set_gc_and_wait(jl_current_task);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Release management and versioning.
Projects
None yet
Development

Successfully merging this pull request may close these issues.