From ea825388134b356210d74b13e75326c7d2b636f1 Mon Sep 17 00:00:00 2001 From: "Steven G. Johnson" Date: Tue, 26 Nov 2024 05:27:29 -0500 Subject: [PATCH 01/19] attach binomial(n::Integer, k::Integer) method to docstring (#56679) Slight tweak to #54307: attach the `binomial(n::Integer, k::Integer)` method to the corresponding docstring, rather than the narrower `binomial(n::T, k::T) where {T<:Integer}` method. --- base/intfuncs.jl | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/base/intfuncs.jl b/base/intfuncs.jl index e8d4b65305be7..db3b1fdeeb521 100644 --- a/base/intfuncs.jl +++ b/base/intfuncs.jl @@ -1205,6 +1205,8 @@ julia> binomial(-5, 3) # External links * [Binomial coefficient](https://en.wikipedia.org/wiki/Binomial_coefficient) on Wikipedia. """ +binomial(n::Integer, k::Integer) = binomial(promote(n, k)...) + Base.@assume_effects :terminates_locally function binomial(n::T, k::T) where T<:Integer n0, k0 = n, k k < 0 && return zero(T) @@ -1233,7 +1235,6 @@ Base.@assume_effects :terminates_locally function binomial(n::T, k::T) where T<: end copysign(x, sgn) end -binomial(n::Integer, k::Integer) = binomial(promote(n, k)...) """ binomial(x::Number, k::Integer) From a17db2b138b0e042329d2a3fb2efd53c68e2563f Mon Sep 17 00:00:00 2001 From: Lilith Orion Hafner Date: Tue, 26 Nov 2024 12:38:49 -0600 Subject: [PATCH 02/19] Make DefaultStable and DefaultUnstable dispatchable and display without internals (#56661) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Previously, `DEFAULT_STABLE` was a giant chain of algorithms reflecting the full sorting dispatch system. Now, it's simply `DefaultStable()`. This has a few minor impacts: Previously, the public binding `Base.Sort.DEFAULT_STABLE` documented non-public implementation details of sorting dispatch in its extended help with a caviat that they are internal. Now, `Base.Sort.DEFAULT_STABLE` refers to the non-public binding `Base.Sort.DefaultStable` and implementation details are documented there with a warning that they are non-public. Previously, dispatching on `Base.Sort.DEFAULT_STABLE` required writing `::typeof(Base.Sort.DEFAULT_STABLE)` whereas now one could alternatively dispatch on the (internal) type `Base.Sort.DefaultStable`. Previously `Base.Sort.DEFAULT_STABLE === Base.Sort.DEFAULT_UNSTABLE` so when writing sorting algorithms for custom collections it was impossible to determine if the user asked for a stable algorithm. Now `DEFAULT_STABLE` is `DefaultStable()` and `DEFAULT_UNSTABLE` is `DefaultUnstable()`. Both the algorithms expand to the same large chain of algorithms `_DEFAULT_ALGORITHMS_FOR_VECTORS` but it is possible to intercept them before that happens. `Base.Sort.DEFAULT_STABLE` now prints as `DefaultStable()` instead of ```julia-repl julia> Base.Sort.DEFAULT_STABLE Base.Sort.SubArrayOptimization( Base.Sort.MissingOptimization( Base.Sort.BoolOptimization( Base.Sort.Small{10}( Base.Sort.InsertionSortAlg(), Base.Sort.IEEEFloatOptimization( Base.Sort.IsUIntMappable( Base.Sort.Small{40}( Base.Sort.InsertionSortAlg(), Base.Sort.CheckSorted( Base.Sort.ComputeExtrema( Base.Sort.ConsiderCountingSort( Base.Sort.CountingSort(), Base.Sort.ConsiderRadixSort( Base.Sort.RadixSort(), Base.Sort.Small{80}( Base.Sort.InsertionSortAlg(), Base.Sort.ScratchQuickSort(missing, missing, Base.Sort.InsertionSortAlg()))))))), Base.Sort.StableCheckSorted( Base.Sort.ScratchQuickSort(missing, missing, Base.Sort.InsertionSortAlg())))))))) ``` Factored out of #54494 at Triage's request (the git history reflects this history). --------- Co-authored-by: Lars Göttgens --- base/sort.jl | 61 +++++++++++++++++++++++++++++++++---------------- test/sorting.jl | 15 +++++++----- 2 files changed, 50 insertions(+), 26 deletions(-) diff --git a/base/sort.jl b/base/sort.jl index 6991f12551ab4..29e67a3eb8d8c 100644 --- a/base/sort.jl +++ b/base/sort.jl @@ -1475,21 +1475,15 @@ InitialOptimizations(next) = SubArrayOptimization( Small{10}( IEEEFloatOptimization( next))))) -""" - DEFAULT_STABLE -The default sorting algorithm. - -This algorithm is guaranteed to be stable (i.e. it will not reorder elements that compare -equal). It makes an effort to be fast for most inputs. - -The algorithms used by `DEFAULT_STABLE` are an implementation detail. See extended help -for the current dispatch system. +""" + struct DefaultStable <: Algorithm end -# Extended Help +`DefaultStable` is an algorithm which indicates that a fast, general purpose sorting +algorithm should be used, but does not specify exactly which algorithm. -`DEFAULT_STABLE` is composed of two parts: the [`InitialOptimizations`](@ref) and a hybrid -of Radix, Insertion, Counting, Quick sorts. +Currently, it is composed of two parts: the [`InitialOptimizations`](@ref) and a hybrid of +Radix, Insertion, Counting, Quick sorts. We begin with MissingOptimization because it has no runtime cost when it is not triggered and can enable other optimizations to be applied later. For example, @@ -1549,7 +1543,39 @@ stage. Finally, if the input has length less than 80, we dispatch to [`InsertionSort`](@ref) and otherwise we dispatch to [`ScratchQuickSort`](@ref). """ -const DEFAULT_STABLE = InitialOptimizations( +struct DefaultStable <: Algorithm end + +""" + DEFAULT_STABLE + +The default sorting algorithm. + +This algorithm is guaranteed to be stable (i.e. it will not reorder elements that compare +equal). It makes an effort to be fast for most inputs. + +The algorithms used by `DEFAULT_STABLE` are an implementation detail. See the docstring +of `Base.Sort.DefaultStable` for the current dispatch system. +""" +const DEFAULT_STABLE = DefaultStable() + +""" + DefaultUnstable <: Algorithm + +Like [`DefaultStable`](@ref), but does not guarantee stability. +""" +struct DefaultUnstable <: Algorithm end + +""" + DEFAULT_UNSTABLE + +An efficient sorting algorithm which may or may not be stable. + +The algorithms used by `DEFAULT_UNSTABLE` are an implementation detail. They are currently +the same as those used by [`DEFAULT_STABLE`](@ref), but this is subject to change in future. +""" +const DEFAULT_UNSTABLE = DefaultUnstable() + +const _DEFAULT_ALGORITHMS_FOR_VECTORS = InitialOptimizations( IsUIntMappable( Small{40}( CheckSorted( @@ -1560,15 +1586,10 @@ const DEFAULT_STABLE = InitialOptimizations( ScratchQuickSort())))))), StableCheckSorted( ScratchQuickSort()))) -""" - DEFAULT_UNSTABLE -An efficient sorting algorithm. +_sort!(v::AbstractVector, ::Union{DefaultStable, DefaultUnstable}, o::Ordering, kw) = + _sort!(v, _DEFAULT_ALGORITHMS_FOR_VECTORS, o, kw) -The algorithms used by `DEFAULT_UNSTABLE` are an implementation detail. They are currently -the same as those used by [`DEFAULT_STABLE`](@ref), but this is subject to change in future. -""" -const DEFAULT_UNSTABLE = DEFAULT_STABLE const SMALL_THRESHOLD = 20 function Base.show(io::IO, alg::Algorithm) diff --git a/test/sorting.jl b/test/sorting.jl index f12486b9c9b40..71af50429027a 100644 --- a/test/sorting.jl +++ b/test/sorting.jl @@ -819,9 +819,9 @@ end let requires_uint_mappable = Union{Base.Sort.RadixSort, Base.Sort.ConsiderRadixSort, Base.Sort.CountingSort, Base.Sort.ConsiderCountingSort, - typeof(Base.Sort.DEFAULT_STABLE.next.next.next.big.next.yes), - typeof(Base.Sort.DEFAULT_STABLE.next.next.next.big.next.yes.big), - typeof(Base.Sort.DEFAULT_STABLE.next.next.next.big.next.yes.big.next)} + typeof(Base.Sort._DEFAULT_ALGORITHMS_FOR_VECTORS.next.next.next.big.next.yes), + typeof(Base.Sort._DEFAULT_ALGORITHMS_FOR_VECTORS.next.next.next.big.next.yes.big), + typeof(Base.Sort._DEFAULT_ALGORITHMS_FOR_VECTORS.next.next.next.big.next.yes.big.next)} function test_alg(kw, alg, float=true) for order in [Base.Forward, Base.Reverse, Base.By(x -> x^2)] @@ -861,15 +861,18 @@ end end end - test_alg_rec(Base.DEFAULT_STABLE) + test_alg_rec(Base.Sort._DEFAULT_ALGORITHMS_FOR_VECTORS) end end @testset "show(::Algorithm)" begin - @test eval(Meta.parse(string(Base.DEFAULT_STABLE))) === Base.DEFAULT_STABLE - lines = split(string(Base.DEFAULT_STABLE), '\n') + @test eval(Meta.parse(string(Base.Sort._DEFAULT_ALGORITHMS_FOR_VECTORS))) === Base.Sort._DEFAULT_ALGORITHMS_FOR_VECTORS + lines = split(string(Base.Sort._DEFAULT_ALGORITHMS_FOR_VECTORS), '\n') @test 10 < maximum(length, lines) < 100 @test 1 < length(lines) < 30 + + @test eval(Meta.parse(string(Base.DEFAULT_STABLE))) === Base.DEFAULT_STABLE + @test string(Base.DEFAULT_STABLE) == "Base.Sort.DefaultStable()" end @testset "Extensibility" begin From f6ebc4b2e87db68490a5c756dd4671c92ef8e789 Mon Sep 17 00:00:00 2001 From: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com> Date: Wed, 27 Nov 2024 14:45:16 +0900 Subject: [PATCH 03/19] optimizer: handle `EnterNode` with `catch_dest == 0` (#56686) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In some parts of the optimizer code, such as `cfg_simplify!` and irinterp, it is assumed that `EnterNode` always has `catch_dest ≠ 0`, but this assumption is incorrect. This commit fixes those cases. --- Compiler/src/ssair/irinterp.jl | 6 ++++-- Compiler/src/ssair/passes.jl | 6 ++++-- Compiler/test/irpasses.jl | 29 ++++++++++++++++++++++++++++- 3 files changed, 36 insertions(+), 5 deletions(-) diff --git a/Compiler/src/ssair/irinterp.jl b/Compiler/src/ssair/irinterp.jl index e96d27a85bc37..a4969e81828cc 100644 --- a/Compiler/src/ssair/irinterp.jl +++ b/Compiler/src/ssair/irinterp.jl @@ -249,8 +249,10 @@ function process_terminator!(@nospecialize(stmt), bb::Int, bb_ip::BitSetBoundedM return backedge elseif isa(stmt, EnterNode) dest = stmt.catch_dest - @assert dest > bb - push!(bb_ip, dest) + if dest ≠ 0 + @assert dest > bb + push!(bb_ip, dest) + end push!(bb_ip, bb+1) return false else diff --git a/Compiler/src/ssair/passes.jl b/Compiler/src/ssair/passes.jl index e61f3207fc07a..ff333b9b0a129 100644 --- a/Compiler/src/ssair/passes.jl +++ b/Compiler/src/ssair/passes.jl @@ -2393,8 +2393,10 @@ function cfg_simplify!(ir::IRCode) end elseif isa(terminator, EnterNode) catchbb = terminator.catch_dest - if bb_rename_succ[catchbb] == 0 - push!(worklist, catchbb) + if catchbb ≠ 0 + if bb_rename_succ[catchbb] == 0 + push!(worklist, catchbb) + end end elseif isa(terminator, GotoNode) || isa(terminator, ReturnNode) # No implicit fall through. Schedule from work list. diff --git a/Compiler/test/irpasses.jl b/Compiler/test/irpasses.jl index 412ff3b98cb19..e9d6f57337530 100644 --- a/Compiler/test/irpasses.jl +++ b/Compiler/test/irpasses.jl @@ -1816,7 +1816,34 @@ function f53521() end end end -@test code_typed(f53521)[1][2] === Nothing +let (ir,rt) = only(Base.code_ircode(f53521, ())) + @test rt == Nothing + Compiler.verify_ir(ir) + Compiler.cfg_simplify!(ir) + Compiler.verify_ir(ir) +end + +Base.@assume_effects :foldable Base.@constprop :aggressive function f53521(x::Int, ::Int) + VALUE = ScopedValue(x) + @with VALUE => 2 begin + for i = 1 + @with VALUE => 3 begin + local v + try + v = sin(VALUE[]) + catch + v = nothing + end + return v + end + end + end +end +let (ir,rt) = only(Base.code_ircode((Int,)) do y + f53521(1, y) + end) + @test rt == Union{Nothing,Float64} +end # Test that adce_pass! sets Refined on PhiNode values let code = Any[ From 42807311db112b0116c3e0923845808efec0fed4 Mon Sep 17 00:00:00 2001 From: Gabriel Baraldi Date: Wed, 27 Nov 2024 10:59:32 -0300 Subject: [PATCH 04/19] Pass `JULIA_NUM_THREADS=1` to the sysimage builder (#56695) Having multiple threads isn't suported during the build since the scheduler doesn't exist at this point. Fixes https://github.com/JuliaLang/julia/issues/56533 --- sysimage.mk | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sysimage.mk b/sysimage.mk index 5371fbd975025..a74aace4dd11c 100644 --- a/sysimage.mk +++ b/sysimage.mk @@ -65,14 +65,14 @@ RELDATADIR := $(call rel_path,$(JULIAHOME)/base,$(build_datarootdir))/ # <-- mak $(build_private_libdir)/basecompiler.ji: $(COMPILER_SRCS) @$(call PRINT_JULIA, cd $(JULIAHOME)/base && \ - $(call spawn,$(JULIA_EXECUTABLE)) -C "$(JULIA_CPU_TARGET)" $(HEAPLIM) --output-ji $(call cygpath_w,$@).tmp \ + JULIA_NUM_THREADS=1 $(call spawn,$(JULIA_EXECUTABLE)) -C "$(JULIA_CPU_TARGET)" $(HEAPLIM) --output-ji $(call cygpath_w,$@).tmp \ --startup-file=no --warn-overwrite=yes -g$(BOOTSTRAP_DEBUG_LEVEL) -O1 Base_compiler.jl --buildroot $(RELBUILDROOT) --dataroot $(RELDATADIR)) @mv $@.tmp $@ $(build_private_libdir)/sys.ji: $(build_private_libdir)/basecompiler.ji $(JULIAHOME)/VERSION $(BASE_SRCS) $(STDLIB_SRCS) @$(call PRINT_JULIA, cd $(JULIAHOME)/base && \ if ! JULIA_BINDIR=$(call cygpath_w,$(build_bindir)) WINEPATH="$(call cygpath_w,$(build_bindir));$$WINEPATH" \ - $(call spawn, $(JULIA_EXECUTABLE)) -g1 -O1 -C "$(JULIA_CPU_TARGET)" $(HEAPLIM) --output-ji $(call cygpath_w,$@).tmp $(JULIA_SYSIMG_BUILD_FLAGS) \ + JULIA_NUM_THREADS=1 $(call spawn, $(JULIA_EXECUTABLE)) -g1 -O1 -C "$(JULIA_CPU_TARGET)" $(HEAPLIM) --output-ji $(call cygpath_w,$@).tmp $(JULIA_SYSIMG_BUILD_FLAGS) \ --startup-file=no --warn-overwrite=yes --sysimage $(call cygpath_w,$<) sysimg.jl --buildroot $(RELBUILDROOT) --dataroot $(RELDATADIR); then \ echo '*** This error might be fixed by running `make clean`. If the error persists$(COMMA) try `make cleanall`. ***'; \ false; \ From 68fa4a94421d4cb6ee406d31697f05b2def6aeef Mon Sep 17 00:00:00 2001 From: Erik Schnetter Date: Wed, 27 Nov 2024 19:01:18 -0500 Subject: [PATCH 05/19] deps/cacert: Update to 2024-11-26 (#56697) --- deps/checksums/cacert-2024-03-11.pem/md5 | 1 - deps/checksums/cacert-2024-03-11.pem/sha512 | 1 - deps/checksums/cacert-2024-11-26.pem/md5 | 1 + deps/checksums/cacert-2024-11-26.pem/sha512 | 1 + deps/libgit2.version | 2 +- stdlib/MozillaCACerts_jll/Project.toml | 2 +- 6 files changed, 4 insertions(+), 4 deletions(-) delete mode 100644 deps/checksums/cacert-2024-03-11.pem/md5 delete mode 100644 deps/checksums/cacert-2024-03-11.pem/sha512 create mode 100644 deps/checksums/cacert-2024-11-26.pem/md5 create mode 100644 deps/checksums/cacert-2024-11-26.pem/sha512 diff --git a/deps/checksums/cacert-2024-03-11.pem/md5 b/deps/checksums/cacert-2024-03-11.pem/md5 deleted file mode 100644 index 618b6c74efdd4..0000000000000 --- a/deps/checksums/cacert-2024-03-11.pem/md5 +++ /dev/null @@ -1 +0,0 @@ -594084120d27f482b1dc48f558d12d48 diff --git a/deps/checksums/cacert-2024-03-11.pem/sha512 b/deps/checksums/cacert-2024-03-11.pem/sha512 deleted file mode 100644 index 441b8e84707b0..0000000000000 --- a/deps/checksums/cacert-2024-03-11.pem/sha512 +++ /dev/null @@ -1 +0,0 @@ -31f03cc19566d007c4cffdad2ada71d99b4734ad7b13bc4f30d73d321f40cbe13b87a801aa61d9788207a851cc1f95a8af8ac732a372d45edb932f204bce3744 diff --git a/deps/checksums/cacert-2024-11-26.pem/md5 b/deps/checksums/cacert-2024-11-26.pem/md5 new file mode 100644 index 0000000000000..865c6abf3e77a --- /dev/null +++ b/deps/checksums/cacert-2024-11-26.pem/md5 @@ -0,0 +1 @@ +92c13373d7dbe43bdc167479274a43e2 diff --git a/deps/checksums/cacert-2024-11-26.pem/sha512 b/deps/checksums/cacert-2024-11-26.pem/sha512 new file mode 100644 index 0000000000000..d51605348faf4 --- /dev/null +++ b/deps/checksums/cacert-2024-11-26.pem/sha512 @@ -0,0 +1 @@ +26c6fa1ac7bcfd523f9ab9e6c2d971103ccfc610ad0df504d4e9b064dad74576d77240c052b808f4c37c9240302a7e973a20f79ee39ac7bf3201a6fa9f0dfa96 diff --git a/deps/libgit2.version b/deps/libgit2.version index d51beb34c27f5..ae475f0b3644f 100644 --- a/deps/libgit2.version +++ b/deps/libgit2.version @@ -11,4 +11,4 @@ LIBGIT2_SHA1=d74d491481831ddcd23575d376e56d2197e95910 # The versions of cacert.pem are identified by the date (YYYY-MM-DD) of their changes. # See https://curl.haxx.se/docs/caextract.html for more details. # Keep in sync with `stdlib/MozillaCACerts_jll/Project.toml`. -MOZILLA_CACERT_VERSION := 2024-03-11 +MOZILLA_CACERT_VERSION := 2024-11-26 diff --git a/stdlib/MozillaCACerts_jll/Project.toml b/stdlib/MozillaCACerts_jll/Project.toml index 181171a4c04c1..5df9bd5949972 100644 --- a/stdlib/MozillaCACerts_jll/Project.toml +++ b/stdlib/MozillaCACerts_jll/Project.toml @@ -1,7 +1,7 @@ name = "MozillaCACerts_jll" uuid = "14a3606d-f60d-562e-9121-12d972cd8159" # Keep in sync with `deps/libgit2.version`. -version = "2024.03.11" +version = "2024.11.26" [extras] Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40" From e50f1ce0f12ab4f6e6bbb719251ae678fb1f3665 Mon Sep 17 00:00:00 2001 From: Jakob Nybo Nissen Date: Thu, 28 Nov 2024 04:13:07 +0100 Subject: [PATCH 06/19] Make Threads.Condition public (#56503) Co-authored-by: Dilum Aluthge --- base/threadingconstructs.jl | 2 ++ 1 file changed, 2 insertions(+) diff --git a/base/threadingconstructs.jl b/base/threadingconstructs.jl index a21d708b4a077..07ff814af1570 100644 --- a/base/threadingconstructs.jl +++ b/base/threadingconstructs.jl @@ -3,6 +3,8 @@ export threadid, nthreads, @threads, @spawn, threadpool, nthreadpools +public Condition + """ Threads.threadid([t::Task]) -> Int From 79d8d3f764e7fde6da323d1063293855612691fd Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Wed, 27 Nov 2024 22:13:34 -0500 Subject: [PATCH 07/19] support passing a specific Method to invoke (#56692) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Mosè Giordano <765740+giordano@users.noreply.github.com> --- Compiler/src/abstractinterpretation.jl | 57 +++++++++++++++----------- Compiler/src/abstractlattice.jl | 2 +- Compiler/src/utilities.jl | 4 +- NEWS.md | 1 + base/docs/basedocs.jl | 15 ++++++- src/builtins.c | 30 +++++++++----- test/core.jl | 7 ++++ 7 files changed, 77 insertions(+), 39 deletions(-) diff --git a/Compiler/src/abstractinterpretation.jl b/Compiler/src/abstractinterpretation.jl index a3abbf814165a..5946adf80ad52 100644 --- a/Compiler/src/abstractinterpretation.jl +++ b/Compiler/src/abstractinterpretation.jl @@ -856,8 +856,7 @@ end struct InvokeCall types # ::Type - lookupsig # ::Type - InvokeCall(@nospecialize(types), @nospecialize(lookupsig)) = new(types, lookupsig) + InvokeCall(@nospecialize(types)) = new(types) end struct ConstCallResult @@ -2218,26 +2217,38 @@ function abstract_invoke(interp::AbstractInterpreter, arginfo::ArgInfo, si::Stmt ft′ = argtype_by_index(argtypes, 2) ft = widenconst(ft′) ft === Bottom && return Future(CallMeta(Bottom, Any, EFFECTS_THROWS, NoCallInfo())) - (types, isexact, isconcrete, istype) = instanceof_tfunc(argtype_by_index(argtypes, 3), false) - isexact || return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) - unwrapped = unwrap_unionall(types) - types === Bottom && return Future(CallMeta(Bottom, Any, EFFECTS_THROWS, NoCallInfo())) - if !(unwrapped isa DataType && unwrapped.name === Tuple.name) - return Future(CallMeta(Bottom, TypeError, EFFECTS_THROWS, NoCallInfo())) - end - argtype = argtypes_to_type(argtype_tail(argtypes, 4)) - nargtype = typeintersect(types, argtype) - nargtype === Bottom && return Future(CallMeta(Bottom, TypeError, EFFECTS_THROWS, NoCallInfo())) - nargtype isa DataType || return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) # other cases are not implemented below - isdispatchelem(ft) || return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) # check that we might not have a subtype of `ft` at runtime, before doing supertype lookup below - ft = ft::DataType - lookupsig = rewrap_unionall(Tuple{ft, unwrapped.parameters...}, types)::Type - nargtype = Tuple{ft, nargtype.parameters...} - argtype = Tuple{ft, argtype.parameters...} - matched, valid_worlds = findsup(lookupsig, method_table(interp)) - matched === nothing && return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) - update_valid_age!(sv, valid_worlds) - method = matched.method + types = argtype_by_index(argtypes, 3) + if types isa Const && types.val isa Method + method = types.val::Method + types = method # argument value + lookupsig = method.sig # edge kind + argtype = argtypes_to_type(pushfirst!(argtype_tail(argtypes, 4), ft)) + nargtype = typeintersect(lookupsig, argtype) + nargtype === Bottom && return Future(CallMeta(Bottom, TypeError, EFFECTS_THROWS, NoCallInfo())) + nargtype isa DataType || return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) # other cases are not implemented below + else + widenconst(types) >: Method && return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) + (types, isexact, isconcrete, istype) = instanceof_tfunc(argtype_by_index(argtypes, 3), false) + isexact || return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) + unwrapped = unwrap_unionall(types) + types === Bottom && return Future(CallMeta(Bottom, Any, EFFECTS_THROWS, NoCallInfo())) + if !(unwrapped isa DataType && unwrapped.name === Tuple.name) + return Future(CallMeta(Bottom, TypeError, EFFECTS_THROWS, NoCallInfo())) + end + argtype = argtypes_to_type(argtype_tail(argtypes, 4)) + nargtype = typeintersect(types, argtype) + nargtype === Bottom && return Future(CallMeta(Bottom, TypeError, EFFECTS_THROWS, NoCallInfo())) + nargtype isa DataType || return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) # other cases are not implemented below + isdispatchelem(ft) || return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) # check that we might not have a subtype of `ft` at runtime, before doing supertype lookup below + ft = ft::DataType + lookupsig = rewrap_unionall(Tuple{ft, unwrapped.parameters...}, types)::Type + nargtype = Tuple{ft, nargtype.parameters...} + argtype = Tuple{ft, argtype.parameters...} + matched, valid_worlds = findsup(lookupsig, method_table(interp)) + matched === nothing && return Future(CallMeta(Any, Any, Effects(), NoCallInfo())) + update_valid_age!(sv, valid_worlds) + method = matched.method + end tienv = ccall(:jl_type_intersection_with_env, Any, (Any, Any), nargtype, method.sig)::SimpleVector ti = tienv[1] env = tienv[2]::SimpleVector @@ -2245,7 +2256,7 @@ function abstract_invoke(interp::AbstractInterpreter, arginfo::ArgInfo, si::Stmt match = MethodMatch(ti, env, method, argtype <: method.sig) ft′_box = Core.Box(ft′) lookupsig_box = Core.Box(lookupsig) - invokecall = InvokeCall(types, lookupsig) + invokecall = InvokeCall(types) return Future{CallMeta}(mresult, interp, sv) do result, interp, sv (; rt, exct, effects, edge, volatile_inf_result) = result local ft′ = ft′_box.contents diff --git a/Compiler/src/abstractlattice.jl b/Compiler/src/abstractlattice.jl index 645c865d085b3..c1f3050739170 100644 --- a/Compiler/src/abstractlattice.jl +++ b/Compiler/src/abstractlattice.jl @@ -229,7 +229,7 @@ end if isa(t, Const) # don't consider mutable values useful constants val = t.val - return isa(val, Symbol) || isa(val, Type) || !ismutable(val) + return isa(val, Symbol) || isa(val, Type) || isa(val, Method) || !ismutable(val) end isa(t, PartialTypeVar) && return false # this isn't forwardable return is_const_prop_profitable_arg(widenlattice(𝕃), t) diff --git a/Compiler/src/utilities.jl b/Compiler/src/utilities.jl index 11d926f0c9d4e..29f3dfa4afd4a 100644 --- a/Compiler/src/utilities.jl +++ b/Compiler/src/utilities.jl @@ -54,8 +54,8 @@ function count_const_size(@nospecialize(x), count_self::Bool = true) # No definite size (isa(x, GenericMemory) || isa(x, String) || isa(x, SimpleVector)) && return MAX_INLINE_CONST_SIZE + 1 - if isa(x, Module) - # We allow modules, because we already assume they are externally + if isa(x, Module) || isa(x, Method) + # We allow modules and methods, because we already assume they are externally # rooted, so we count their contents as 0 size. return sizeof(Ptr{Cvoid}) end diff --git a/NEWS.md b/NEWS.md index 535d14208f0b8..61bad831e261c 100644 --- a/NEWS.md +++ b/NEWS.md @@ -119,6 +119,7 @@ New library features * `Base.require_one_based_indexing` and `Base.has_offset_axes` are now public ([#56196]) * New `ltruncate`, `rtruncate` and `ctruncate` functions for truncating strings to text width, accounting for char widths ([#55351]) * `isless` (and thus `cmp`, sorting, etc.) is now supported for zero-dimensional `AbstractArray`s ([#55772]) +* `invoke` now supports passing a Method instead of a type signature making this interface somewhat more flexible for certain uncommon use cases ([#56692]). Standard library changes ------------------------ diff --git a/base/docs/basedocs.jl b/base/docs/basedocs.jl index c872244964160..5119ceaf2164a 100644 --- a/base/docs/basedocs.jl +++ b/base/docs/basedocs.jl @@ -2030,21 +2030,32 @@ applicable """ invoke(f, argtypes::Type, args...; kwargs...) + invoke(f, argtypes::Method, args...; kwargs...) Invoke a method for the given generic function `f` matching the specified types `argtypes` on the specified arguments `args` and passing the keyword arguments `kwargs`. The arguments `args` must conform with the specified types in `argtypes`, i.e. conversion is not automatically performed. This method allows invoking a method other than the most specific matching method, which is useful when the behavior of a more general definition is explicitly needed (often as part of the -implementation of a more specific method of the same function). +implementation of a more specific method of the same function). However, because this means +the runtime must do more work, `invoke` is generally also slower--sometimes significantly +so--than doing normal dispatch with a regular call. -Be careful when using `invoke` for functions that you don't write. What definition is used +Be careful when using `invoke` for functions that you don't write. What definition is used for given `argtypes` is an implementation detail unless the function is explicitly states that calling with certain `argtypes` is a part of public API. For example, the change between `f1` and `f2` in the example below is usually considered compatible because the change is invisible by the caller with a normal (non-`invoke`) call. However, the change is visible if you use `invoke`. +# Passing a `Method` instead of a signature +The `argtypes` argument may be a `Method`, in which case the ordinary method table lookup is +bypassed entirely and the given method is invoked directly. Needing this feature is uncommon. +Note in particular that the specified `Method` may be entirely unreachable from ordinary dispatch +(or ordinary invoke), e.g. because it was replaced or fully covered by more specific methods. +If the method is part of the ordinary method table, this call behaves similar +to `invoke(f, method.sig, args...)`. + # Examples ```jldoctest julia> f(x::Real) = x^2; diff --git a/src/builtins.c b/src/builtins.c index b129cca0ee71d..c6b0bf130550b 100644 --- a/src/builtins.c +++ b/src/builtins.c @@ -931,22 +931,27 @@ JL_CALLABLE(jl_f__call_in_world_total) // tuples --------------------------------------------------------------------- -JL_CALLABLE(jl_f_tuple) +static jl_value_t *arg_tuple(jl_value_t *a1, jl_value_t **args, size_t nargs) { size_t i; - if (nargs == 0) - return (jl_value_t*)jl_emptytuple; - jl_datatype_t *tt = jl_inst_arg_tuple_type(args[0], &args[1], nargs, 0); + jl_datatype_t *tt = jl_inst_arg_tuple_type(a1, args, nargs, 0); JL_GC_PROMISE_ROOTED(tt); // it is a concrete type if (tt->instance != NULL) return tt->instance; jl_task_t *ct = jl_current_task; jl_value_t *jv = jl_gc_alloc(ct->ptls, jl_datatype_size(tt), tt); for (i = 0; i < nargs; i++) - set_nth_field(tt, jv, i, args[i], 0); + set_nth_field(tt, jv, i, i == 0 ? a1 : args[i - 1], 0); return jv; } +JL_CALLABLE(jl_f_tuple) +{ + if (nargs == 0) + return (jl_value_t*)jl_emptytuple; + return arg_tuple(args[0], &args[1], nargs); +} + JL_CALLABLE(jl_f_svec) { size_t i; @@ -1577,14 +1582,17 @@ JL_CALLABLE(jl_f_invoke) { JL_NARGSV(invoke, 2); jl_value_t *argtypes = args[1]; - JL_GC_PUSH1(&argtypes); - if (!jl_is_tuple_type(jl_unwrap_unionall(args[1]))) - jl_type_error("invoke", (jl_value_t*)jl_anytuple_type_type, args[1]); + if (jl_is_method(argtypes)) { + jl_method_t *m = (jl_method_t*)argtypes; + if (!jl_tuple1_isa(args[0], &args[2], nargs - 1, (jl_datatype_t*)m->sig)) + jl_type_error("invoke: argument type error", argtypes, arg_tuple(args[0], &args[2], nargs - 1)); + return jl_gf_invoke_by_method(m, args[0], &args[2], nargs - 1); + } + if (!jl_is_tuple_type(jl_unwrap_unionall(argtypes))) + jl_type_error("invoke", (jl_value_t*)jl_anytuple_type_type, argtypes); if (!jl_tuple_isa(&args[2], nargs - 2, (jl_datatype_t*)argtypes)) jl_type_error("invoke: argument type error", argtypes, jl_f_tuple(NULL, &args[2], nargs - 2)); - jl_value_t *res = jl_gf_invoke(argtypes, args[0], &args[2], nargs - 1); - JL_GC_POP(); - return res; + return jl_gf_invoke(argtypes, args[0], &args[2], nargs - 1); } // Expr constructor for internal use ------------------------------------------ diff --git a/test/core.jl b/test/core.jl index 836532d661638..39d02d5d567c9 100644 --- a/test/core.jl +++ b/test/core.jl @@ -8352,3 +8352,10 @@ macro define_call(sym) end @test eval(Expr(:toplevel, :(@define_call(f_macro_defined1)))) == 1 @test @define_call(f_macro_defined2) == 1 + +let m = which(+, (Int, Int)) + @eval f56692(i) = invoke(+, $m, i, 4) + global g56692() = f56692(5) == 9 ? "true" : false +end +@test @inferred(f56692(3)) == 7 +@test @inferred(g56692()) == "true" From 7e2e0aeca207e865c72c42282fb86db949854839 Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Wed, 27 Nov 2024 22:15:24 -0500 Subject: [PATCH 08/19] Add mechanism to disable creating new worlds (#56639) --- base/experimental.jl | 9 +++ src/gf.c | 152 ++++++++++++++++++++++++++++------------- src/staticdata.c | 32 ++++++--- src/staticdata_utils.c | 142 +++++++++++++++++++++----------------- src/toplevel.c | 2 +- test/misc.jl | 36 +++++++--- 6 files changed, 243 insertions(+), 130 deletions(-) diff --git a/base/experimental.jl b/base/experimental.jl index 31238d4015b3b..411bb2407cdc5 100644 --- a/base/experimental.jl +++ b/base/experimental.jl @@ -494,4 +494,13 @@ function entrypoint(@nospecialize(argt::Type)) nothing end +""" + Base.Experimental.disable_new_worlds() + +Mark that no new worlds (methods additions, deletions, etc) are permitted to be created at +any future time, allowing for lower latencies for some operations and slightly lower memory +usage, by eliminating the tracking of those possible invalidation. +""" +disable_new_worlds() = ccall(:jl_disable_new_worlds, Cvoid, ()) + end diff --git a/src/gf.c b/src/gf.c index 90b874d614b0c..bbf065a4fac0d 100644 --- a/src/gf.c +++ b/src/gf.c @@ -24,6 +24,7 @@ extern "C" { #endif +_Atomic(int) allow_new_worlds = 1; JL_DLLEXPORT _Atomic(size_t) jl_world_counter = 1; // uses atomic acquire/release jl_mutex_t world_counter_lock; JL_DLLEXPORT size_t jl_get_world_counter(void) JL_NOTSAFEPOINT @@ -1819,38 +1820,42 @@ static void invalidate_backedges(jl_method_instance_t *replaced_mi, size_t max_w // add a backedge from callee to caller JL_DLLEXPORT void jl_method_instance_add_backedge(jl_method_instance_t *callee, jl_value_t *invokesig, jl_code_instance_t *caller) { + if (!jl_atomic_load_relaxed(&allow_new_worlds)) + return; if (invokesig == jl_nothing) invokesig = NULL; // julia uses `nothing` but C uses NULL (#undef) assert(jl_is_method_instance(callee)); assert(jl_is_code_instance(caller)); assert(invokesig == NULL || jl_is_type(invokesig)); JL_LOCK(&callee->def.method->writelock); - int found = 0; - // TODO: use jl_cache_type_(invokesig) like cache_method does to save memory - if (!callee->backedges) { - // lazy-init the backedges array - callee->backedges = jl_alloc_vec_any(0); - jl_gc_wb(callee, callee->backedges); - } - else { - size_t i = 0, l = jl_array_nrows(callee->backedges); - for (i = 0; i < l; i++) { - // optimized version of while (i < l) i = get_next_edge(callee->backedges, i, &invokeTypes, &mi); - jl_value_t *mi = jl_array_ptr_ref(callee->backedges, i); - if (mi != (jl_value_t*)caller) - continue; - jl_value_t *invokeTypes = i > 0 ? jl_array_ptr_ref(callee->backedges, i - 1) : NULL; - if (invokeTypes && jl_is_method_instance(invokeTypes)) - invokeTypes = NULL; - if ((invokesig == NULL && invokeTypes == NULL) || - (invokesig && invokeTypes && jl_types_equal(invokesig, invokeTypes))) { - found = 1; - break; + if (jl_atomic_load_relaxed(&allow_new_worlds)) { + int found = 0; + // TODO: use jl_cache_type_(invokesig) like cache_method does to save memory + if (!callee->backedges) { + // lazy-init the backedges array + callee->backedges = jl_alloc_vec_any(0); + jl_gc_wb(callee, callee->backedges); + } + else { + size_t i = 0, l = jl_array_nrows(callee->backedges); + for (i = 0; i < l; i++) { + // optimized version of while (i < l) i = get_next_edge(callee->backedges, i, &invokeTypes, &mi); + jl_value_t *mi = jl_array_ptr_ref(callee->backedges, i); + if (mi != (jl_value_t*)caller) + continue; + jl_value_t *invokeTypes = i > 0 ? jl_array_ptr_ref(callee->backedges, i - 1) : NULL; + if (invokeTypes && jl_is_method_instance(invokeTypes)) + invokeTypes = NULL; + if ((invokesig == NULL && invokeTypes == NULL) || + (invokesig && invokeTypes && jl_types_equal(invokesig, invokeTypes))) { + found = 1; + break; + } } } + if (!found) + push_edge(callee->backedges, invokesig, caller); } - if (!found) - push_edge(callee->backedges, invokesig, caller); JL_UNLOCK(&callee->def.method->writelock); } @@ -1858,37 +1863,41 @@ JL_DLLEXPORT void jl_method_instance_add_backedge(jl_method_instance_t *callee, JL_DLLEXPORT void jl_method_table_add_backedge(jl_methtable_t *mt, jl_value_t *typ, jl_code_instance_t *caller) { assert(jl_is_code_instance(caller)); + if (!jl_atomic_load_relaxed(&allow_new_worlds)) + return; JL_LOCK(&mt->writelock); - if (!mt->backedges) { - // lazy-init the backedges array - mt->backedges = jl_alloc_vec_any(2); - jl_gc_wb(mt, mt->backedges); - jl_array_ptr_set(mt->backedges, 0, typ); - jl_array_ptr_set(mt->backedges, 1, caller); - } - else { - // check if the edge is already present and avoid adding a duplicate - size_t i, l = jl_array_nrows(mt->backedges); - for (i = 1; i < l; i += 2) { - if (jl_array_ptr_ref(mt->backedges, i) == (jl_value_t*)caller) { - if (jl_types_equal(jl_array_ptr_ref(mt->backedges, i - 1), typ)) { - JL_UNLOCK(&mt->writelock); - return; + if (jl_atomic_load_relaxed(&allow_new_worlds)) { + if (!mt->backedges) { + // lazy-init the backedges array + mt->backedges = jl_alloc_vec_any(2); + jl_gc_wb(mt, mt->backedges); + jl_array_ptr_set(mt->backedges, 0, typ); + jl_array_ptr_set(mt->backedges, 1, caller); + } + else { + // check if the edge is already present and avoid adding a duplicate + size_t i, l = jl_array_nrows(mt->backedges); + for (i = 1; i < l; i += 2) { + if (jl_array_ptr_ref(mt->backedges, i) == (jl_value_t*)caller) { + if (jl_types_equal(jl_array_ptr_ref(mt->backedges, i - 1), typ)) { + JL_UNLOCK(&mt->writelock); + return; + } } } - } - // reuse an already cached instance of this type, if possible - // TODO: use jl_cache_type_(tt) like cache_method does, instead of this linear scan? - for (i = 1; i < l; i += 2) { - if (jl_array_ptr_ref(mt->backedges, i) != (jl_value_t*)caller) { - if (jl_types_equal(jl_array_ptr_ref(mt->backedges, i - 1), typ)) { - typ = jl_array_ptr_ref(mt->backedges, i - 1); - break; + // reuse an already cached instance of this type, if possible + // TODO: use jl_cache_type_(tt) like cache_method does, instead of this linear scan? + for (i = 1; i < l; i += 2) { + if (jl_array_ptr_ref(mt->backedges, i) != (jl_value_t*)caller) { + if (jl_types_equal(jl_array_ptr_ref(mt->backedges, i - 1), typ)) { + typ = jl_array_ptr_ref(mt->backedges, i - 1); + break; + } } } + jl_array_ptr_1d_push(mt->backedges, typ); + jl_array_ptr_1d_push(mt->backedges, (jl_value_t*)caller); } - jl_array_ptr_1d_push(mt->backedges, typ); - jl_array_ptr_1d_push(mt->backedges, (jl_value_t*)caller); } JL_UNLOCK(&mt->writelock); } @@ -2024,10 +2033,55 @@ static void jl_method_table_invalidate(jl_methtable_t *mt, jl_method_t *replaced } } +static int erase_method_backedges(jl_typemap_entry_t *def, void *closure) +{ + jl_method_t *method = def->func.method; + JL_LOCK(&method->writelock); + jl_value_t *specializations = jl_atomic_load_relaxed(&method->specializations); + if (jl_is_svec(specializations)) { + size_t i, l = jl_svec_len(specializations); + for (i = 0; i < l; i++) { + jl_method_instance_t *mi = (jl_method_instance_t*)jl_svecref(specializations, i); + if ((jl_value_t*)mi != jl_nothing) { + mi->backedges = NULL; + } + } + } + else { + jl_method_instance_t *mi = (jl_method_instance_t*)specializations; + mi->backedges = NULL; + } + JL_UNLOCK(&method->writelock); + return 1; +} + +static int erase_all_backedges(jl_methtable_t *mt, void *env) +{ + // removes all method caches + // this might not be entirely safe (GC or MT), thus we only do it very early in bootstrapping + JL_LOCK(&mt->writelock); + mt->backedges = NULL; + JL_UNLOCK(&mt->writelock); + jl_typemap_visitor(jl_atomic_load_relaxed(&mt->defs), erase_method_backedges, env); + return 1; +} + +JL_DLLEXPORT void jl_disable_new_worlds(void) +{ + if (jl_generating_output()) + jl_error("Disabling Method changes is not possible when generating output."); + JL_LOCK(&world_counter_lock); + jl_atomic_store_relaxed(&allow_new_worlds, 0); + JL_UNLOCK(&world_counter_lock); + jl_foreach_reachable_mtable(erase_all_backedges, (void*)NULL); +} + JL_DLLEXPORT void jl_method_table_disable(jl_methtable_t *mt, jl_method_t *method) { jl_typemap_entry_t *methodentry = do_typemap_search(mt, method); JL_LOCK(&world_counter_lock); + if (!jl_atomic_load_relaxed(&allow_new_worlds)) + jl_error("Method changes have been disabled via a call to disable_new_worlds."); JL_LOCK(&mt->writelock); // Narrow the world age on the method to make it uncallable size_t world = jl_atomic_load_relaxed(&jl_world_counter); @@ -2341,6 +2395,8 @@ JL_DLLEXPORT void jl_method_table_insert(jl_methtable_t *mt, jl_method_t *method jl_typemap_entry_t *newentry = jl_method_table_add(mt, method, simpletype); JL_GC_PUSH1(&newentry); JL_LOCK(&world_counter_lock); + if (!jl_atomic_load_relaxed(&allow_new_worlds)) + jl_error("Method changes have been disabled via a call to disable_new_worlds."); size_t world = jl_atomic_load_relaxed(&jl_world_counter) + 1; jl_atomic_store_relaxed(&method->primary_world, world); jl_atomic_store_relaxed(&method->deleted_world, ~(size_t)0); diff --git a/src/staticdata.c b/src/staticdata.c index c2e8d7e3956ea..f5fe088ce1c98 100644 --- a/src/staticdata.c +++ b/src/staticdata.c @@ -89,6 +89,7 @@ External links: #include "julia_assert.h" static const size_t WORLD_AGE_REVALIDATION_SENTINEL = 0x1; +size_t jl_require_world = ~(size_t)0; #include "staticdata_utils.c" #include "precompile_utils.c" @@ -2678,7 +2679,6 @@ jl_genericmemory_t *jl_global_roots_list; jl_genericmemory_t *jl_global_roots_keyset; jl_mutex_t global_roots_lock; extern jl_mutex_t world_counter_lock; -extern size_t jl_require_world; jl_mutex_t precompile_field_replace_lock; jl_svec_t *precompile_field_replace JL_GLOBALLY_ROOTED; @@ -4044,16 +4044,30 @@ static jl_value_t *jl_restore_package_image_from_stream(void* pkgimage_handle, i // Add roots to methods jl_copy_roots(method_roots_list, jl_worklist_key((jl_array_t*)restored)); // Insert method extensions and handle edges + int new_methods = jl_array_nrows(extext_methods) > 0; + if (!new_methods) { + size_t i, l = jl_array_nrows(internal_methods); + for (i = 0; i < l; i++) { + jl_value_t *obj = jl_array_ptr_ref(internal_methods, i); + if (jl_is_method(obj)) { + new_methods = 1; + break; + } + } + } JL_LOCK(&world_counter_lock); - // allocate a world for the new methods, and insert them there, invalidating content as needed - size_t world = jl_atomic_load_relaxed(&jl_world_counter) + 1; - jl_activate_methods(extext_methods, internal_methods, world); - // TODO: inject new_ext_cis into caches here, so the system can see them immediately as potential candidates (before validation) - // allow users to start running in this updated world - jl_atomic_store_release(&jl_world_counter, world); - // now permit more methods to be added again + // allocate a world for the new methods, and insert them there, invalidating content as needed + size_t world = jl_atomic_load_relaxed(&jl_world_counter); + if (new_methods) + world += 1; + jl_activate_methods(extext_methods, internal_methods, world, pkgname); + // TODO: inject new_ext_cis into caches here, so the system can see them immediately as potential candidates (before validation) + // allow users to start running in this updated world + if (new_methods) + jl_atomic_store_release(&jl_world_counter, world); + // now permit more methods to be added again JL_UNLOCK(&world_counter_lock); - // but one of those immediate users is going to be our cache insertions + // but one of those immediate users is going to be our cache insertions jl_insert_backedges((jl_array_t*)edges, (jl_array_t*)new_ext_cis); // restore existing caches (needs to be last) // reinit ccallables jl_reinit_ccallable(&ccallable_list, base, pkgimage_handle); diff --git a/src/staticdata_utils.c b/src/staticdata_utils.c index 32e59d7d7c641..ea0b7216155bd 100644 --- a/src/staticdata_utils.c +++ b/src/staticdata_utils.c @@ -704,10 +704,12 @@ static void jl_add_methods(jl_array_t *external) } } -static void jl_activate_methods(jl_array_t *external, jl_array_t *internal, size_t world) +extern _Atomic(int) allow_new_worlds; +static void jl_activate_methods(jl_array_t *external, jl_array_t *internal, size_t world, const char *pkgname) { size_t i, l = jl_array_nrows(internal); for (i = 0; i < l; i++) { + // allow_new_worlds doesn't matter here, since we aren't actually changing anything external jl_value_t *obj = jl_array_ptr_ref(internal, i); if (jl_typetagis(obj, jl_typemap_entry_type)) { jl_typemap_entry_t *entry = (jl_typemap_entry_t*)obj; @@ -735,11 +737,17 @@ static void jl_activate_methods(jl_array_t *external, jl_array_t *internal, size } } l = jl_array_nrows(external); - for (i = 0; i < l; i++) { - jl_typemap_entry_t *entry = (jl_typemap_entry_t*)jl_array_ptr_ref(external, i); - jl_methtable_t *mt = jl_method_get_table(entry->func.method); - assert((jl_value_t*)mt != jl_nothing); - jl_method_table_activate(mt, entry); + if (l) { + if (!jl_atomic_load_relaxed(&allow_new_worlds)) { + jl_printf(JL_STDERR, "WARNING: Method changes for %s have been disabled via a call to disable_new_worlds.\n", pkgname); + return; + } + for (i = 0; i < l; i++) { + jl_typemap_entry_t *entry = (jl_typemap_entry_t*)jl_array_ptr_ref(external, i); + jl_methtable_t *mt = jl_method_get_table(entry->func.method); + assert((jl_value_t*)mt != jl_nothing); + jl_method_table_activate(mt, entry); + } } } @@ -864,71 +872,80 @@ static int jl_verify_method(jl_code_instance_t *codeinst, size_t *minworld, size size_t depth = stack->len; *bp = (char*)HT_NOTFOUND + depth; JL_TIMING(VERIFY_IMAGE, VERIFY_Methods); - jl_value_t *loctag = NULL; - jl_value_t *sig = NULL; - jl_value_t *matches = NULL; - JL_GC_PUSH3(&loctag, &matches, &sig); jl_svec_t *callees = jl_atomic_load_relaxed(&codeinst->edges); assert(jl_is_svec((jl_value_t*)callees)); // verify current edges - for (size_t j = 0; j < jl_svec_len(callees); ) { - jl_value_t *edge = jl_svecref(callees, j); - size_t min_valid2; - size_t max_valid2; - assert(!jl_is_method(edge)); // `Method`-edge isn't allowed for the optimized one-edge format - if (jl_is_code_instance(edge)) - edge = (jl_value_t*)((jl_code_instance_t*)edge)->def; - if (jl_is_method_instance(edge)) { - jl_method_instance_t *mi = (jl_method_instance_t*)edge; - sig = jl_type_intersection(mi->def.method->sig, (jl_value_t*)mi->specTypes); // TODO: ?? - verify_call(sig, callees, j, 1, world, &min_valid2, &max_valid2, &matches); - sig = NULL; - j += 1; - } - else if (jl_is_long(edge)) { - jl_value_t *sig = jl_svecref(callees, j + 1); - size_t nedges = jl_unbox_long(edge); - verify_call(sig, callees, j + 2, nedges, world, &min_valid2, &max_valid2, &matches); - j += 2 + nedges; - edge = sig; - } - else { - jl_method_instance_t *callee = (jl_method_instance_t*)jl_svecref(callees, j + 1); - jl_method_t *meth; - if (jl_is_mtable(callee)) { - // skip the legacy edge (missing backedge) - j += 2; - continue; + if (callees == jl_emptysvec) { + // quick return: no edges to verify (though we probably shouldn't have gotten here from WORLD_AGE_REVALIDATION_SENTINEL) + } + else if (*maxworld == jl_require_world) { + // if no new worlds were allocated since serializing the base module, then no new validation is worth doing right now either + *minworld = *maxworld; + } + else { + jl_value_t *loctag = NULL; + jl_value_t *sig = NULL; + jl_value_t *matches = NULL; + JL_GC_PUSH3(&loctag, &matches, &sig); + for (size_t j = 0; j < jl_svec_len(callees); ) { + jl_value_t *edge = jl_svecref(callees, j); + size_t min_valid2; + size_t max_valid2; + assert(!jl_is_method(edge)); // `Method`-edge isn't allowed for the optimized one-edge format + if (jl_is_code_instance(edge)) + edge = (jl_value_t*)((jl_code_instance_t*)edge)->def; + if (jl_is_method_instance(edge)) { + jl_method_instance_t *mi = (jl_method_instance_t*)edge; + sig = jl_type_intersection(mi->def.method->sig, (jl_value_t*)mi->specTypes); // TODO: ?? + verify_call(sig, callees, j, 1, world, &min_valid2, &max_valid2, &matches); + sig = NULL; + j += 1; } - if (jl_is_code_instance(callee)) - callee = ((jl_code_instance_t*)callee)->def; - if (jl_is_method_instance(callee)) { - meth = callee->def.method; + else if (jl_is_long(edge)) { + jl_value_t *sig = jl_svecref(callees, j + 1); + size_t nedges = jl_unbox_long(edge); + verify_call(sig, callees, j + 2, nedges, world, &min_valid2, &max_valid2, &matches); + j += 2 + nedges; + edge = sig; } else { - assert(jl_is_method(callee)); - meth = (jl_method_t*)callee; + jl_method_instance_t *callee = (jl_method_instance_t*)jl_svecref(callees, j + 1); + jl_method_t *meth; + if (jl_is_mtable(callee)) { + // skip the legacy edge (missing backedge) + j += 2; + continue; + } + if (jl_is_code_instance(callee)) + callee = ((jl_code_instance_t*)callee)->def; + if (jl_is_method_instance(callee)) { + meth = callee->def.method; + } + else { + assert(jl_is_method(callee)); + meth = (jl_method_t*)callee; + } + verify_invokesig(edge, meth, world, &min_valid2, &max_valid2); + j += 2; } - verify_invokesig(edge, meth, world, &min_valid2, &max_valid2); - j += 2; - } - if (*minworld < min_valid2) - *minworld = min_valid2; - if (*maxworld > max_valid2) - *maxworld = max_valid2; - if (max_valid2 != ~(size_t)0 && _jl_debug_method_invalidation) { - jl_array_ptr_1d_push(_jl_debug_method_invalidation, edge); - loctag = jl_cstr_to_string("insert_backedges_callee"); - jl_array_ptr_1d_push(_jl_debug_method_invalidation, loctag); - jl_array_ptr_1d_push(_jl_debug_method_invalidation, (jl_value_t*)codeinst); - jl_array_ptr_1d_push(_jl_debug_method_invalidation, matches); + if (*minworld < min_valid2) + *minworld = min_valid2; + if (*maxworld > max_valid2) + *maxworld = max_valid2; + if (max_valid2 != ~(size_t)0 && _jl_debug_method_invalidation) { + jl_array_ptr_1d_push(_jl_debug_method_invalidation, edge); + loctag = jl_cstr_to_string("insert_backedges_callee"); + jl_array_ptr_1d_push(_jl_debug_method_invalidation, loctag); + jl_array_ptr_1d_push(_jl_debug_method_invalidation, (jl_value_t*)codeinst); + jl_array_ptr_1d_push(_jl_debug_method_invalidation, matches); + } + //jl_static_show((JL_STREAM*)ios_stderr, (jl_value_t*)edge); + //ios_puts(max_valid2 == ~(size_t)0 ? "valid\n" : "INVALID\n", ios_stderr); + if (max_valid2 == 0 && !_jl_debug_method_invalidation) + break; } - //jl_static_show((JL_STREAM*)ios_stderr, (jl_value_t*)edge); - //ios_puts(max_valid2 == ~(size_t)0 ? "valid\n" : "INVALID\n", ios_stderr); - if (max_valid2 == 0 && !_jl_debug_method_invalidation) - break; + JL_GC_POP(); } - JL_GC_POP(); // verify recursive edges (if valid, or debugging) size_t cycle = depth; jl_code_instance_t *cause = codeinst; @@ -979,6 +996,7 @@ static int jl_verify_method(jl_code_instance_t *codeinst, size_t *minworld, size assert(*bp == (char*)HT_NOTFOUND + stack->len + 1); *bp = HT_NOTFOUND; if (_jl_debug_method_invalidation && *maxworld < current_world) { + jl_value_t *loctag; jl_array_ptr_1d_push(_jl_debug_method_invalidation, (jl_value_t*)child); loctag = jl_cstr_to_string("verify_methods"); JL_GC_PUSH1(&loctag); diff --git a/src/toplevel.c b/src/toplevel.c index 56a5f21f43661..e8286a584b119 100644 --- a/src/toplevel.c +++ b/src/toplevel.c @@ -496,7 +496,7 @@ static void body_attributes(jl_array_t *body, int *has_ccall, int *has_defs, int *forced_compile = jl_has_meta(body, jl_force_compile_sym); } -size_t jl_require_world = ~(size_t)0; +extern size_t jl_require_world; static jl_module_t *call_require(jl_module_t *mod, jl_sym_t *var) JL_GLOBALLY_ROOTED { JL_TIMING(LOAD_IMAGE, LOAD_Require); diff --git a/test/misc.jl b/test/misc.jl index 7f9992e22a3d7..5bbc2c3c65fa2 100644 --- a/test/misc.jl +++ b/test/misc.jl @@ -1211,10 +1211,7 @@ include("testenv.jl") let flags = Cmd(filter(a->!occursin("depwarn", a), collect(test_exeflags))) local cmd = `$test_exename $flags --depwarn=yes deprecation_exec.jl` - - if !success(pipeline(cmd; stdout=stdout, stderr=stderr)) - error("Deprecation test failed, cmd : $cmd") - end + run(cmd, devnull) end # PR #23664, make sure names don't get added to the default `Main` workspace @@ -1489,7 +1486,7 @@ end # Test that read fault on a prot-none region does not incorrectly give # ReadOnlyMemoryError, but rather crashes the program const MAP_ANONYMOUS_PRIVATE = Sys.isbsd() ? 0x1002 : 0x22 -let script = :( +let script = """ let ptr = Ptr{Cint}(ccall(:jl_mmap, Ptr{Cvoid}, (Ptr{Cvoid}, Csize_t, Cint, Cint, Cint, Int), C_NULL, 16*1024, 0, $MAP_ANONYMOUS_PRIVATE, -1, 0)) @@ -1499,19 +1496,24 @@ let script = :( println(e) end end - ) + """ cmd = if Sys.isunix() # Set the maximum core dump size to 0 to keep this expected crash from # producing a (and potentially overwriting an existing) core dump file - `sh -c "ulimit -c 0; $(Base.shell_escape(Base.julia_cmd())) -e '$script'"` + `sh -c "ulimit -c 0; $(Base.shell_escape(Base.julia_cmd())) -e $(Base.shell_escape(script))"` else - `$(Base.julia_cmd()) -e '$script'` + `$(Base.julia_cmd()) -e $script` + end + p = run(ignorestatus(cmd), devnull, stdout, devnull) + if p.termsignal == 0 + Sys.isunix() ? @test(p.exitcode ∈ (128+7, 128+10, 128+11)) : @test(p.exitcode != 0) # expect SIGBUS (7 on BSDs or 10 on Linux) or SIGSEGV (11) + else + @test(p.termsignal ∈ (7, 10, 11)) end - @test !success(cmd) end # issue #41656 -@test success(`$(Base.julia_cmd()) -e 'isempty(x) = true'`) +run(`$(Base.julia_cmd()) -e 'isempty(x) = true'`) @testset "Base/timing.jl" begin @test Base.jit_total_bytes() >= 0 @@ -1596,3 +1598,17 @@ end end @test !occursin("loop not unrolled", out_err) end + +let errs = IOBuffer() + run(`$(Base.julia_cmd()) -e ' + using Test + @test isdefined(DataType.name.mt, :backedges) + Base.Experimental.disable_new_worlds() + @test_throws "disable_new_worlds" @eval f() = 1 + @test !isdefined(DataType.name.mt, :backedges) + @test_throws "disable_new_worlds" Base.delete_method(which(+, (Int, Int))) + @test 1+1 == 2 + using Dates + '`, devnull, stdout, errs) + @test occursin("disable_new_worlds", String(take!(errs))) +end From 680803c33b45864c14b4f55a1cbda65a6a52b9e0 Mon Sep 17 00:00:00 2001 From: Jeff Bezanson Date: Wed, 27 Nov 2024 22:15:52 -0500 Subject: [PATCH 09/19] get trimming test working again (#56689) --- contrib/juliac.jl | 19 ++++++++----------- src/staticdata.c | 1 + test/trimming/Makefile | 2 +- 3 files changed, 10 insertions(+), 12 deletions(-) diff --git a/contrib/juliac.jl b/contrib/juliac.jl index 0f008976d2b4f..20d56615c6357 100644 --- a/contrib/juliac.jl +++ b/contrib/juliac.jl @@ -4,7 +4,6 @@ cmd = Base.julia_cmd() cmd = `$cmd --startup-file=no --history-file=no` output_type = nothing # exe, sharedlib, sysimage -trim = nothing outname = nothing file = nothing add_ccallables = false @@ -15,13 +14,16 @@ if help !== nothing println( """ Usage: julia juliac.jl [--output-exe | --output-lib | --output-sysimage] [options] - --trim= Only output code statically determined to be reachable + --experimental --trim= Only output code statically determined to be reachable --compile-ccallable Include all methods marked `@ccallable` in output --verbose Request verbose output """) exit(0) end +# arguments to forward to julia compilation process +julia_args = [] + let i = 1 while i <= length(ARGS) arg = ARGS[i] @@ -31,17 +33,13 @@ let i = 1 i == length(ARGS) && error("Output specifier requires an argument") global outname = ARGS[i+1] i += 1 - elseif startswith(arg, "--trim") - arg = split(arg, '=') - if length(arg) == 1 - global trim = "safe" - else - global trim = arg[2] - end elseif arg == "--compile-ccallable" global add_ccallables = true elseif arg == "--verbose" global verbose = true + elseif startswith(arg, "--trim") || arg == "--experimental" + # forwarded args + push!(julia_args, arg) else if arg[1] == '-' || !isnothing(file) println("Unexpected argument `$arg`") @@ -79,8 +77,7 @@ open(initsrc_path, "w") do io """) end -static_call_graph_arg() = isnothing(trim) ? `` : `--trim=$(trim)` -cmd = addenv(`$cmd --project=$(Base.active_project()) --output-o $img_path --output-incremental=no --strip-ir --strip-metadata $(static_call_graph_arg()) $(joinpath(@__DIR__,"juliac-buildscript.jl")) $absfile $output_type $add_ccallables`, "OPENBLAS_NUM_THREADS" => 1, "JULIA_NUM_THREADS" => 1) +cmd = addenv(`$cmd --project=$(Base.active_project()) --output-o $img_path --output-incremental=no --strip-ir --strip-metadata $julia_args $(joinpath(@__DIR__,"juliac-buildscript.jl")) $absfile $output_type $add_ccallables`, "OPENBLAS_NUM_THREADS" => 1, "JULIA_NUM_THREADS" => 1) verbose && println("Running: $cmd") if !success(pipeline(cmd; stdout, stderr)) println(stderr, "\nFailed to compile $file") diff --git a/src/staticdata.c b/src/staticdata.c index f5fe088ce1c98..78cfa85695076 100644 --- a/src/staticdata.c +++ b/src/staticdata.c @@ -2578,6 +2578,7 @@ static void strip_specializations_(jl_method_instance_t *mi) if (inferred && inferred != jl_nothing) { if (jl_options.strip_ir) { record_field_change((jl_value_t**)&codeinst->inferred, jl_nothing); + record_field_change((jl_value_t**)&codeinst->edges, (jl_value_t*)jl_emptysvec); } else if (jl_options.strip_metadata) { jl_value_t *stripped = strip_codeinfo_meta(mi->def.method, inferred, codeinst); diff --git a/test/trimming/Makefile b/test/trimming/Makefile index c6e105d637013..d2da21eb71a88 100644 --- a/test/trimming/Makefile +++ b/test/trimming/Makefile @@ -33,7 +33,7 @@ LDFLAGS_ADD = -lm $(shell $(JULIA_CONFIG) --ldflags --ldlibs) -ljulia-internal release: hello$(EXE) hello.o: $(SRCDIR)/hello.jl $(BUILDSCRIPT) - $(JULIA) -t 1 -J $(BIN)/../lib/julia/sys.so --startup-file=no --history-file=no --output-o $@ --output-incremental=no --strip-ir --strip-metadata --trim $(BUILDSCRIPT) $(SRCDIR)/hello.jl --output-exe true + $(JULIA) -t 1 -J $(BIN)/../lib/julia/sys.so --startup-file=no --history-file=no --output-o $@ --output-incremental=no --strip-ir --strip-metadata --experimental --trim $(BUILDSCRIPT) $(SRCDIR)/hello.jl --output-exe true init.o: $(SRCDIR)/init.c $(CC) -c -o $@ $< $(CPPFLAGS_ADD) $(CPPFLAGS) $(CFLAGS_ADD) $(CFLAGS) From 60f0057e03dbf5158c29c8fe6b4bc8e59fbde735 Mon Sep 17 00:00:00 2001 From: CY Han Date: Thu, 28 Nov 2024 11:19:15 +0800 Subject: [PATCH 10/19] doc: fix docstring for `Dates.format` (#56682) --- stdlib/Dates/src/io.jl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stdlib/Dates/src/io.jl b/stdlib/Dates/src/io.jl index 388edb693d76f..aa7019566093c 100644 --- a/stdlib/Dates/src/io.jl +++ b/stdlib/Dates/src/io.jl @@ -713,7 +713,7 @@ except that it does not truncate values longer than the width. When creating a `format` you can use any non-code characters as a separator. For example to generate the string "1996-01-15T00:00:00" you could use `format`: "yyyy-mm-ddTHH:MM:SS". Note that if you need to use a code character as a literal you can use the escape character -backslash. The string "1996y01m" can be produced with the format "yyyy\\ymm\\m". +backslash. The string "1996y01m" can be produced with the format raw"yyyy\\ymm\\m". """ function format(dt::TimeType, f::AbstractString; locale::Locale=ENGLISH) format(dt, DateFormat(f, locale)) From f5dc26b9d30ab4dcd52e0ff29e6b469f1f0c007c Mon Sep 17 00:00:00 2001 From: Cody Tapscott <84105208+topolarity@users.noreply.github.com> Date: Wed, 27 Nov 2024 22:24:22 -0500 Subject: [PATCH 11/19] Make a failed extension load throw an error during pre-compilation (#56668) Co-authored-by: Ian Butterworth --- base/loading.jl | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/base/loading.jl b/base/loading.jl index 0a70564077692..8ed43e4539c20 100644 --- a/base/loading.jl +++ b/base/loading.jl @@ -1587,9 +1587,14 @@ function run_extension_callbacks(extid::ExtensionId) true catch # Try to continue loading if loading an extension errors - errs = current_exceptions() - @error "Error during loading of extension $(extid.id.name) of $(extid.parentid.name), \ + if JLOptions().incremental != 0 + # during incremental precompilation, this should be fail-fast + rethrow() + else + errs = current_exceptions() + @error "Error during loading of extension $(extid.id.name) of $(extid.parentid.name), \ use `Base.retry_load_extensions()` to retry." exception=errs + end false finally global loading_extension = false From 9162b141b7638ffe6db577ae4a91761fc33eb0a5 Mon Sep 17 00:00:00 2001 From: Tim Besard Date: Thu, 28 Nov 2024 08:51:16 +0100 Subject: [PATCH 12/19] Remove arraylist_t from external native code APIs. (#56693) This makes them usable for external consumers like GPUCompiler.jl. --- src/aotcompile.cpp | 56 ++++++++++++++++++++++++++++++-------------- src/julia_internal.h | 8 ++++--- src/staticdata.c | 18 ++++++++++---- 3 files changed, 58 insertions(+), 24 deletions(-) diff --git a/src/aotcompile.cpp b/src/aotcompile.cpp index 6af5227aafd92..7b3c771f9dc12 100644 --- a/src/aotcompile.cpp +++ b/src/aotcompile.cpp @@ -95,33 +95,55 @@ void jl_get_function_id_impl(void *native_code, jl_code_instance_t *codeinst, } } -extern "C" JL_DLLEXPORT_CODEGEN -void jl_get_llvm_mis_impl(void *native_code, arraylist_t* MIs) +extern "C" JL_DLLEXPORT_CODEGEN void +jl_get_llvm_mis_impl(void *native_code, size_t *num_elements, jl_method_instance_t **data) { - jl_native_code_desc_t *data = (jl_native_code_desc_t*)native_code; - auto map = data->jl_fvar_map; + jl_native_code_desc_t *desc = (jl_native_code_desc_t *)native_code; + auto &map = desc->jl_fvar_map; + + if (data == NULL) { + *num_elements = map.size(); + return; + } + + assert(*num_elements == map.size()); + size_t i = 0; for (auto &ci : map) { - jl_method_instance_t *mi = ci.first->def; - arraylist_push(MIs, mi); + data[i++] = ci.first->def; } } -extern "C" JL_DLLEXPORT_CODEGEN -void jl_get_llvm_gvs_impl(void *native_code, arraylist_t *gvs) +extern "C" JL_DLLEXPORT_CODEGEN void jl_get_llvm_gvs_impl(void *native_code, + size_t *num_elements, void **data) { // map a memory location (jl_value_t or jl_binding_t) to a GlobalVariable - jl_native_code_desc_t *data = (jl_native_code_desc_t*)native_code; - arraylist_grow(gvs, data->jl_value_to_llvm.size()); - memcpy(gvs->items, data->jl_value_to_llvm.data(), gvs->len * sizeof(void*)); + jl_native_code_desc_t *desc = (jl_native_code_desc_t *)native_code; + auto &value_map = desc->jl_value_to_llvm; + + if (data == NULL) { + *num_elements = value_map.size(); + return; + } + + assert(*num_elements == value_map.size()); + memcpy(data, value_map.data(), *num_elements * sizeof(void *)); } -extern "C" JL_DLLEXPORT_CODEGEN -void jl_get_llvm_external_fns_impl(void *native_code, arraylist_t *external_fns) +extern "C" JL_DLLEXPORT_CODEGEN void jl_get_llvm_external_fns_impl(void *native_code, + size_t *num_elements, + jl_code_instance_t *data) { - jl_native_code_desc_t *data = (jl_native_code_desc_t*)native_code; - arraylist_grow(external_fns, data->jl_external_to_llvm.size()); - memcpy(external_fns->items, data->jl_external_to_llvm.data(), - external_fns->len * sizeof(jl_code_instance_t*)); + jl_native_code_desc_t *desc = (jl_native_code_desc_t *)native_code; + auto &external_map = desc->jl_external_to_llvm; + + if (data == NULL) { + *num_elements = external_map.size(); + return; + } + + assert(*num_elements == external_map.size()); + memcpy((void *)data, (const void *)external_map.data(), + *num_elements * sizeof(jl_code_instance_t *)); } extern "C" JL_DLLEXPORT_CODEGEN diff --git a/src/julia_internal.h b/src/julia_internal.h index ca3f63b274968..e081c94329deb 100644 --- a/src/julia_internal.h +++ b/src/julia_internal.h @@ -1951,13 +1951,15 @@ JL_DLLIMPORT void *jl_create_native(jl_array_t *methods, LLVMOrcThreadSafeModule JL_DLLIMPORT void jl_dump_native(void *native_code, const char *bc_fname, const char *unopt_bc_fname, const char *obj_fname, const char *asm_fname, ios_t *z, ios_t *s, jl_emission_params_t *params); -JL_DLLIMPORT void jl_get_llvm_gvs(void *native_code, arraylist_t *gvs); -JL_DLLIMPORT void jl_get_llvm_external_fns(void *native_code, arraylist_t *gvs); +JL_DLLIMPORT void jl_get_llvm_gvs(void *native_code, size_t *num_els, void **gvs); +JL_DLLIMPORT void jl_get_llvm_external_fns(void *native_code, size_t *num_els, + jl_code_instance_t *gvs); JL_DLLIMPORT void jl_get_function_id(void *native_code, jl_code_instance_t *ncode, int32_t *func_idx, int32_t *specfunc_idx); JL_DLLIMPORT void jl_register_fptrs(uint64_t image_base, const struct _jl_image_fptrs_t *fptrs, jl_method_instance_t **linfos, size_t n); -JL_DLLIMPORT void jl_get_llvm_mis(void *native_code, arraylist_t* MIs); +JL_DLLIMPORT void jl_get_llvm_mis(void *native_code, size_t *num_els, + jl_method_instance_t *MIs); JL_DLLIMPORT void jl_init_codegen(void); JL_DLLIMPORT void jl_teardown_codegen(void) JL_NOTSAFEPOINT; JL_DLLIMPORT int jl_getFunctionInfo(jl_frame_t **frames, uintptr_t pointer, int skipC, int noInline) JL_NOTSAFEPOINT; diff --git a/src/staticdata.c b/src/staticdata.c index 78cfa85695076..65584c015e228 100644 --- a/src/staticdata.c +++ b/src/staticdata.c @@ -2897,10 +2897,20 @@ static void jl_save_system_image_to_stream(ios_t *f, jl_array_t *mod_array, int en = jl_gc_enable(0); if (native_functions) { - jl_get_llvm_gvs(native_functions, &gvars); - jl_get_llvm_external_fns(native_functions, &external_fns); - if (jl_options.trim) - jl_get_llvm_mis(native_functions, &MIs); + size_t num_gvars, num_external_fns; + jl_get_llvm_gvs(native_functions, &num_gvars, NULL); + arraylist_grow(&gvars, num_gvars); + jl_get_llvm_gvs(native_functions, &num_gvars, gvars.items); + jl_get_llvm_external_fns(native_functions, &num_external_fns, NULL); + arraylist_grow(&external_fns, num_external_fns); + jl_get_llvm_external_fns(native_functions, &num_external_fns, + (jl_code_instance_t *)external_fns.items); + if (jl_options.trim) { + size_t num_mis; + jl_get_llvm_mis(native_functions, &num_mis, NULL); + arraylist_grow(&MIs, num_mis); + jl_get_llvm_mis(native_functions, &num_mis, (jl_method_instance_t *)MIs.items); + } } if (jl_options.trim) { jl_rebuild_methtables(&MIs, &new_methtables); From 447dc2d8e9a71669c6a3f62ff8ac71054c811a94 Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Thu, 28 Nov 2024 07:34:29 -0500 Subject: [PATCH 13/19] fix world_age_at_entry in codegen (#56700) --- src/ccall.cpp | 18 ++++-------- src/codegen.cpp | 73 ++++++++++++++++++++++++++----------------------- src/toplevel.c | 5 +--- 3 files changed, 46 insertions(+), 50 deletions(-) diff --git a/src/ccall.cpp b/src/ccall.cpp index 952625a71287b..707203bd13506 100644 --- a/src/ccall.cpp +++ b/src/ccall.cpp @@ -1712,18 +1712,12 @@ static jl_cgval_t emit_ccall(jl_codectx_t &ctx, jl_value_t **args, size_t nargs) return ghostValue(ctx, jl_nothing_type); } else if (is_libjulia_func(jl_get_tls_world_age)) { - bool toplevel = !(ctx.linfo && jl_is_method(ctx.linfo->def.method)); - if (!toplevel) { // top level code does not see a stable world age during execution - ++CCALL_STAT(jl_get_tls_world_age); - assert(lrt == ctx.types().T_size); - assert(!isVa && !llvmcall && nccallargs == 0); - JL_GC_POP(); - Instruction *world_age = cast(ctx.world_age_at_entry); - setName(ctx.emission_context, world_age, "task_world_age"); - jl_aliasinfo_t ai = jl_aliasinfo_t::fromTBAA(ctx, ctx.tbaa().tbaa_gcframe); - ai.decorateInst(world_age); - return mark_or_box_ccall_result(ctx, world_age, retboxed, rt, unionall, static_rt); - } + ++CCALL_STAT(jl_get_tls_world_age); + assert(lrt == ctx.types().T_size); + assert(!isVa && !llvmcall && nccallargs == 0); + JL_GC_POP(); + Value *world_age = get_tls_world_age(ctx); + return mark_or_box_ccall_result(ctx, world_age, retboxed, rt, unionall, static_rt); } else if (is_libjulia_func(jl_get_world_counter)) { ++CCALL_STAT(jl_get_world_counter); diff --git a/src/codegen.cpp b/src/codegen.cpp index 3645a0b25827e..ebb96837f4db5 100644 --- a/src/codegen.cpp +++ b/src/codegen.cpp @@ -1985,7 +1985,7 @@ class jl_codectx_t { Value *pgcstack = NULL; Instruction *topalloca = NULL; - Value *world_age_at_entry = NULL; // Not valid to use in toplevel code + Value *world_age_at_entry = NULL; bool use_cache = false; bool external_linkage = false; @@ -2115,6 +2115,7 @@ static jl_cgval_t emit_sparam(jl_codectx_t &ctx, size_t i); static Value *emit_condition(jl_codectx_t &ctx, const jl_cgval_t &condV, const Twine &msg); static Value *get_current_task(jl_codectx_t &ctx); static Value *get_current_ptls(jl_codectx_t &ctx); +static Value *get_tls_world_age(jl_codectx_t &ctx); static Value *get_scope_field(jl_codectx_t &ctx); static Value *get_tls_world_age_field(jl_codectx_t &ctx); static void CreateTrap(IRBuilder<> &irbuilder, bool create_new_block = true); @@ -7044,11 +7045,7 @@ static jl_cgval_t emit_expr(jl_codectx_t &ctx, jl_value_t *expr, ssize_t ssaidx_ std::tie(F, specF) = get_oc_function(ctx, (jl_method_t*)source.constant, (jl_tupletype_t*)env_t, argt_typ, ub.constant); if (F) { jl_cgval_t jlcall_ptr = mark_julia_type(ctx, F, false, jl_voidpointer_type); - jl_aliasinfo_t ai = jl_aliasinfo_t::fromTBAA(ctx, ctx.tbaa().tbaa_gcframe); - bool not_toplevel = (ctx.linfo && jl_is_method(ctx.linfo->def.method)); - Instruction *I = not_toplevel ? cast(ctx.world_age_at_entry) : - ctx.builder.CreateAlignedLoad(ctx.types().T_size, get_tls_world_age_field(ctx), ctx.types().alignof_ptr); - jl_cgval_t world_age = mark_julia_type(ctx, ai.decorateInst(I), false, jl_long_type); + jl_cgval_t world_age = mark_julia_type(ctx, get_tls_world_age(ctx), false, jl_long_type); jl_cgval_t fptr; if (specF) fptr = mark_julia_type(ctx, specF, false, jl_voidpointer_type); @@ -7207,6 +7204,25 @@ static Value *get_tls_world_age_field(jl_codectx_t &ctx) return emit_ptrgep(ctx, ct, offsetof(jl_task_t, world_age), "world_age"); } +// Get the value of the world age of the current task +static Value *get_tls_world_age(jl_codectx_t &ctx) +{ + if (ctx.world_age_at_entry) + return ctx.world_age_at_entry; + IRBuilderBase::InsertPointGuard IP(ctx.builder); + bool toplevel = !jl_is_method(ctx.linfo->def.method); + if (!toplevel) { + ctx.builder.SetInsertPoint(ctx.topalloca->getParent(), ++ctx.topalloca->getIterator()); + ctx.builder.SetCurrentDebugLocation(ctx.topalloca->getStableDebugLoc()); + } + jl_aliasinfo_t ai = jl_aliasinfo_t::fromTBAA(ctx, ctx.tbaa().tbaa_gcframe); + auto *world = ctx.builder.CreateAlignedLoad(ctx.types().T_size, get_tls_world_age_field(ctx), ctx.types().alignof_ptr); + ai.decorateInst(world); + if (!toplevel) + ctx.world_age_at_entry = world; + return world; +} + static Value *get_scope_field(jl_codectx_t &ctx) { Value *ct = get_current_task(ctx); @@ -7524,9 +7540,8 @@ static Function* gen_cfun_wrapper( auto world_age_field = get_tls_world_age_field(ctx); jl_aliasinfo_t ai = jl_aliasinfo_t::fromTBAA(ctx, ctx.tbaa().tbaa_gcframe); - Value *last_age = ai.decorateInst( + ctx.world_age_at_entry = ai.decorateInst( ctx.builder.CreateAlignedLoad(ctx.types().T_size, world_age_field, ctx.types().alignof_ptr)); - ctx.world_age_at_entry = last_age; Value *world_v = ctx.builder.CreateAlignedLoad(ctx.types().T_size, prepare_global_in(jl_Module, jlgetworld_global), ctx.types().alignof_ptr); cast(world_v)->setOrdering(AtomicOrdering::Acquire); @@ -7808,7 +7823,7 @@ static Function* gen_cfun_wrapper( r = NULL; } - ctx.builder.CreateStore(last_age, world_age_field); + ctx.builder.CreateStore(ctx.world_age_at_entry, world_age_field); ctx.builder.CreateRet(r); ctx.builder.SetCurrentDebugLocation(noDbg); @@ -8418,7 +8433,6 @@ static jl_llvm_functions_t ctx.source = src; std::map labels; - bool toplevel = false; ctx.module = jl_is_method(lam->def.method) ? lam->def.method->module : lam->def.module; ctx.linfo = lam; ctx.name = TSM.getModuleUnlocked()->getModuleIdentifier().data(); @@ -8438,7 +8452,6 @@ static jl_llvm_functions_t if (vn != jl_unused_sym) ctx.vaSlot = ctx.nargs - 1; } - toplevel = !jl_is_method(lam->def.method); ctx.rettype = jlrettype; ctx.funcName = ctx.name; ctx.spvals_ptr = NULL; @@ -8776,12 +8789,12 @@ static jl_llvm_functions_t // step 6. set up GC frame allocate_gc_frame(ctx, b0); Value *last_age = NULL; - auto world_age_field = get_tls_world_age_field(ctx); - { // scope + Value *world_age_field = NULL; + if (ctx.is_opaque_closure) { + world_age_field = get_tls_world_age_field(ctx); jl_aliasinfo_t ai = jl_aliasinfo_t::fromTBAA(ctx, ctx.tbaa().tbaa_gcframe); last_age = ai.decorateInst(ctx.builder.CreateAlignedLoad( ctx.types().T_size, world_age_field, ctx.types().alignof_ptr)); - ctx.world_age_at_entry = last_age; // Load world age for use in get_tls_world_age } // step 7. allocate local variables slots @@ -9005,6 +9018,7 @@ static jl_llvm_functions_t Value *worldaddr = emit_ptrgep(ctx, oc_this, offsetof(jl_opaque_closure_t, world)); jl_cgval_t closure_world = typed_load(ctx, worldaddr, NULL, (jl_value_t*)jl_long_type, nullptr, nullptr, false, AtomicOrdering::NotAtomic, false, ctx.types().alignof_ptr.value()); + assert(ctx.world_age_at_entry == nullptr); ctx.world_age_at_entry = closure_world.V; // The tls world in a OC is the world of the closure emit_unbox_store(ctx, closure_world, world_age_field, ctx.tbaa().tbaa_gcframe, ctx.types().alignof_ptr); @@ -9282,19 +9296,11 @@ static jl_llvm_functions_t Instruction &prologue_end = ctx.builder.GetInsertBlock()->back(); - // step 11a. For top-level code, load the world age - if (toplevel && !ctx.is_opaque_closure) { - LoadInst *world = ctx.builder.CreateAlignedLoad(ctx.types().T_size, - prepare_global_in(jl_Module, jlgetworld_global), ctx.types().alignof_ptr); - world->setOrdering(AtomicOrdering::Acquire); - ctx.builder.CreateAlignedStore(world, world_age_field, ctx.types().alignof_ptr); - } - - // step 11b. Emit the entry safepoint + // step 11a. Emit the entry safepoint if (JL_FEAT_TEST(ctx, safepoint_on_entry)) emit_gc_safepoint(ctx.builder, ctx.types().T_size, get_current_ptls(ctx), ctx.tbaa().tbaa_const); - // step 11c. Do codegen in control flow order + // step 11b. Do codegen in control flow order SmallVector workstack; std::map BB; std::map come_from_bb; @@ -9966,8 +9972,7 @@ static jl_llvm_functions_t Instruction *root = cast_or_null(ctx.slots[ctx.vaSlot].boxroot); if (root) { bool have_real_use = false; - for (Use &U : root->uses()) { - User *RU = U.getUser(); + for (User *RU : root->users()) { if (StoreInst *SRU = dyn_cast(RU)) { assert(isa(SRU->getValueOperand()) || SRU->getValueOperand() == restTuple); (void)SRU; @@ -9986,21 +9991,21 @@ static jl_llvm_functions_t } } if (!have_real_use) { - Instruction *use = NULL; - for (Use &U : root->uses()) { - if (use) // erase after the iterator moves on - use->eraseFromParent(); - User *RU = U.getUser(); - use = cast(RU); + for (User *RU : make_early_inc_range(root->users())) { + // This is safe because it checked above that each User is known and has at most one Use of root + cast(RU)->eraseFromParent(); } - if (use) - use->eraseFromParent(); root->eraseFromParent(); restTuple->eraseFromParent(); } } } + if (ctx.topalloca->use_empty()) { + ctx.topalloca->eraseFromParent(); + ctx.topalloca = nullptr; + } + // link the dependent llvmcall modules, but switch their function's linkage to internal // so that they don't conflict when they show up in the execution engine. Linker L(*jl_Module); diff --git a/src/toplevel.c b/src/toplevel.c index e8286a584b119..21737119af9a6 100644 --- a/src/toplevel.c +++ b/src/toplevel.c @@ -1050,10 +1050,7 @@ JL_DLLEXPORT jl_value_t *jl_toplevel_eval_flex(jl_module_t *JL_NONNULL m, jl_val // use codegen mfunc = jl_method_instance_for_thunk(thk, m); jl_resolve_globals_in_ir((jl_array_t*)thk->code, m, NULL, 0); - // Don't infer blocks containing e.g. method definitions, since it's probably not - // worthwhile and also unsound (see #24316). - // TODO: This is still not correct since an `eval` can happen elsewhere, but it - // helps in common cases. + // Don't infer blocks containing e.g. method definitions, since it's probably not worthwhile. size_t world = jl_atomic_load_acquire(&jl_world_counter); ct->world_age = world; if (!has_defs && jl_get_module_infer(m) != 0) { From 1ed2b98131cb7074cec5f556f222e8ad023285f9 Mon Sep 17 00:00:00 2001 From: Lilith Orion Hafner Date: Thu, 28 Nov 2024 07:43:03 -0600 Subject: [PATCH 14/19] Fix typo in `@cmd` docstring (#56664) I'm not sure what `` `cmd` `` could refer to, but it would make sense to refer to `` `str` `` in this case. I'm assuming it's a typo. --- base/cmd.jl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/base/cmd.jl b/base/cmd.jl index 84ec52f865e98..b46c8293cdf3c 100644 --- a/base/cmd.jl +++ b/base/cmd.jl @@ -491,7 +491,7 @@ end """ @cmd str -Similar to `cmd`, generate a `Cmd` from the `str` string which represents the shell command(s) to be executed. +Similar to ``` `str` ```, generate a `Cmd` from the `str` string which represents the shell command(s) to be executed. The [`Cmd`](@ref) object can be run as a process and can outlive the spawning julia process (see `Cmd` for more). # Examples From 5053a175274fc973e7d0479f4ee4452cb2865299 Mon Sep 17 00:00:00 2001 From: Erik Schnetter Date: Thu, 28 Nov 2024 14:40:10 -0500 Subject: [PATCH 15/19] deps/pcre: Update to version 10.44 (#56704) --- deps/checksums/pcre | 72 +++++++++++++++---------------- deps/pcre.version | 2 +- stdlib/PCRE2_jll/Project.toml | 2 +- stdlib/PCRE2_jll/test/runtests.jl | 2 +- 4 files changed, 39 insertions(+), 39 deletions(-) diff --git a/deps/checksums/pcre b/deps/checksums/pcre index 018ffd5201653..0c2732f8cc2b5 100644 --- a/deps/checksums/pcre +++ b/deps/checksums/pcre @@ -1,36 +1,36 @@ -PCRE2.v10.43.0+1.aarch64-apple-darwin.tar.gz/md5/f1bee27b8d9465c14eaf9362701fb795 -PCRE2.v10.43.0+1.aarch64-apple-darwin.tar.gz/sha512/33b8f6e3703f0a52cd2d57897c28e35fb3c63af459296a2fef4e414dc99239617833b2ab176068d6aab690122a34a9ab9b6042dfff54b5a30ad60429a809818d -PCRE2.v10.43.0+1.aarch64-linux-gnu.tar.gz/md5/c55a569260e302f315f4a1bd185346ab -PCRE2.v10.43.0+1.aarch64-linux-gnu.tar.gz/sha512/be4d2883e69d562898a157424b2baa146fe79545a8c10935cf25b54e498ca2c14fae026fa0d958d175895fe2cb695d0f96ef7f09fecbf54e1cee4a55b81a382b -PCRE2.v10.43.0+1.aarch64-linux-musl.tar.gz/md5/fb041ccace415ccc26263968c6435a47 -PCRE2.v10.43.0+1.aarch64-linux-musl.tar.gz/sha512/06672ebe18e0f6bfa1dd2d5c02e10d9fd67236a73fd38ee2e8f4496d98f297f7866760f0be3b9cebeca348a5d748a3719e416b84cec96a90c71eac55afbbd905 -PCRE2.v10.43.0+1.aarch64-unknown-freebsd.tar.gz/md5/8c73fe6faa94102616cfafcc6cc1bf9d -PCRE2.v10.43.0+1.aarch64-unknown-freebsd.tar.gz/sha512/464a892e646fb5aa028d2e96e6f8beaa0c15f0ef56a6ba3388cba4ce85151448b0dfd51357a3e8dea4505957394ffbab14ceb29b9fc73a67e2b2f54dd28a7aed -PCRE2.v10.43.0+1.armv6l-linux-gnueabihf.tar.gz/md5/4f303a4cbf26abb7bf4ffb8bfe3d636d -PCRE2.v10.43.0+1.armv6l-linux-gnueabihf.tar.gz/sha512/dddb3b227ee48d8329f6c65c5d0fce9f460eccaec98594a05bf28d1d9af01397cf7ef86c96e88b0e96030a7f6d8406461f78dd5fa558db8fc8f7bfb3b522ed54 -PCRE2.v10.43.0+1.armv6l-linux-musleabihf.tar.gz/md5/eade1fff90404bf3584fd15b62be0cfa -PCRE2.v10.43.0+1.armv6l-linux-musleabihf.tar.gz/sha512/351f6fa11c39b90fcc4086bd00b1b1126ed92272595f0b745757ca4e7e360c84d244446a871029245c3bcf838b23f42d908f858e44fae7deb9002a36cb76753c -PCRE2.v10.43.0+1.armv7l-linux-gnueabihf.tar.gz/md5/daa0a34b2cf0b71a6f8e1f9456cd4b06 -PCRE2.v10.43.0+1.armv7l-linux-gnueabihf.tar.gz/sha512/ae72956ae7a9a5f315bfc816fdbb500937a170dfea306a28289ec9eac57d883cf2fa5a467ce9406eea80546b632a272c63bbb48b89ebe6d9f69d30366fd84180 -PCRE2.v10.43.0+1.armv7l-linux-musleabihf.tar.gz/md5/90bfb9e4efd7c92a2bb6a1a48fd88ecb -PCRE2.v10.43.0+1.armv7l-linux-musleabihf.tar.gz/sha512/147ac98d82fec4695de0c43c87d3d9242b9c024bc6df7ad7504d17ef6a12a029ed703c4deade0e2b24faf5283d66309f880d62f8c4834f27b2cc8889587d7abe -PCRE2.v10.43.0+1.i686-linux-gnu.tar.gz/md5/6fde649bf449c4122438fff32c0706ab -PCRE2.v10.43.0+1.i686-linux-gnu.tar.gz/sha512/edfaa15490497723c095eaa5df26194637b0606e9dce7b89b400024ef8ac42e21f010bb31c2cee5c735ce82fc8de0c42bf2b35b095a1e70a9a111d3bfba6da64 -PCRE2.v10.43.0+1.i686-linux-musl.tar.gz/md5/73aa8d13cc48338a5071e30b3a899109 -PCRE2.v10.43.0+1.i686-linux-musl.tar.gz/sha512/200e2d3ffd68f49b76c70a5be80cb0ae9703049214674485a2ab24abaaea7aefd6dec2042a14bd48cc52b04379f57322ec1e1788dc8c00896e1074921725d9cc -PCRE2.v10.43.0+1.i686-w64-mingw32.tar.gz/md5/4ddf0f31c97463e5216ed71afc4fb014 -PCRE2.v10.43.0+1.i686-w64-mingw32.tar.gz/sha512/75903d81668a66a5c4d830e31657391d507883943d86245998f224655406dcc6a95ba4f5fad20dcf608a98d6ccf49abe50107993448669b03c42a878d8466611 -PCRE2.v10.43.0+1.powerpc64le-linux-gnu.tar.gz/md5/64cb71080da1c97eba3a440ff53d298c -PCRE2.v10.43.0+1.powerpc64le-linux-gnu.tar.gz/sha512/16348b96a45c7a7d86775cb1d082b4d1c060e5a8acfb37554885d8da0db87430d8a40f834f008a90f4a7b1c07b8329df96836ba0430ecec506a143b7347bb101 -PCRE2.v10.43.0+1.x86_64-apple-darwin.tar.gz/md5/31bbb2485f5e06c3616fb061ffb2f022 -PCRE2.v10.43.0+1.x86_64-apple-darwin.tar.gz/sha512/3284ee63ed1e5631267efacb354a1d90bd1b7db0bc81d7233c9580eee4a9af06093c1c4f240786c34299df89a36a17ed92598fc302074f5a200c56cc96081bf1 -PCRE2.v10.43.0+1.x86_64-linux-gnu.tar.gz/md5/2fb7e0e9bbc32dddf543f4d395b50d3f -PCRE2.v10.43.0+1.x86_64-linux-gnu.tar.gz/sha512/5a533a3a01f817689077377835dc88edf914459ed0df7323f8f4dba602a47fd6af700075feb1f448221366b1cf7e2d717c615a5c506eb4ca2db9c600fd290fb0 -PCRE2.v10.43.0+1.x86_64-linux-musl.tar.gz/md5/b432063c93aa477dd0883428191041f8 -PCRE2.v10.43.0+1.x86_64-linux-musl.tar.gz/sha512/36475e90e29d7324046fe1da669fb37f667245a680df23f3978394964e14eb9bda3fd56703ad62cd56e27a5af77d8b6b9612516457ae803cef0627bd919e4628 -PCRE2.v10.43.0+1.x86_64-unknown-freebsd.tar.gz/md5/6124870a991e70c2ed8a64d8f3258760 -PCRE2.v10.43.0+1.x86_64-unknown-freebsd.tar.gz/sha512/4645a2d05af149467f2e4ce5e48853b57c585d6a5950c70726d04bc71a5d82f50809af141ad98e99671e764ac74965651ecad1c49a849caa8fd077c7f4911c7c -PCRE2.v10.43.0+1.x86_64-w64-mingw32.tar.gz/md5/cc4e9f45471f538c1fefa657ab99b878 -PCRE2.v10.43.0+1.x86_64-w64-mingw32.tar.gz/sha512/eed45e621263cb307b6e8ab42e2c12cf9e1d61ad523760fd721a85765c359b74d580752ca7c3d222e0cba26a74e872a6d43dbf2dbf08e4733a3e709417e48651 -pcre2-10.43.tar.bz2/md5/c8e2043cbc4abb80e76dba323f7c409f -pcre2-10.43.tar.bz2/sha512/8ac1520c32e9e5672404aaf6104e23c9ee5c3c28ad28ff101435599d813cbb20e0491a3fd34e012b4411b3e0366a4c6dfa3f02d093acaa6ff0ab25478bb7ade9 +PCRE2.v10.44.0+0.aarch64-apple-darwin.tar.gz/md5/14de26cfc0f6ff7635fac39e81e81a27 +PCRE2.v10.44.0+0.aarch64-apple-darwin.tar.gz/sha512/45079ecca5f4966a32895fcc63585f1dd60f306dc1cb5c098d42452fcff67f7f6b405c200a15747af4680151bb6a6374832a0119b8ddd743d2ed13d0beaef7c9 +PCRE2.v10.44.0+0.aarch64-linux-gnu.tar.gz/md5/3cf179ed36d37bff698ab81cf3d5797b +PCRE2.v10.44.0+0.aarch64-linux-gnu.tar.gz/sha512/db93e5a5c0c46b5536ed49515682d9bfe1d23f6ba8ae2468289ec8f2160140f39f5606a3c7095f45251f3663d8ccf2d6d7e5e8b1efb21c39bbf9a13b6ec60ef9 +PCRE2.v10.44.0+0.aarch64-linux-musl.tar.gz/md5/02baa415218f581a5ceeb7bf7fc0a090 +PCRE2.v10.44.0+0.aarch64-linux-musl.tar.gz/sha512/1685f37ed8f465ecc2f738fdf65d20bb1806934ff2c50194882282fb6c3900121c61c39210e4c0b89847493bfc3e15bb7b9136b0d968103b47c8662a78b412fe +PCRE2.v10.44.0+0.aarch64-unknown-freebsd.tar.gz/md5/4de065ea59ab4f622b46079df1d9d941 +PCRE2.v10.44.0+0.aarch64-unknown-freebsd.tar.gz/sha512/aa6df9edfb690d155a8b5a9390db7ca11622ac0020174cf070a33a075801bfe43bd4c80b8e28017989a8b7374d39897cdcf72ab0e1962e3e234239975f7ac0b4 +PCRE2.v10.44.0+0.armv6l-linux-gnueabihf.tar.gz/md5/f8a0907fbb20a06507fce849db098c4f +PCRE2.v10.44.0+0.armv6l-linux-gnueabihf.tar.gz/sha512/3f5bcc1742380a31683a81740d55e198d7ec8d8ea5a13d6d0556d6603e4fadbf0dc648093c44e36dd6d3793c52a5e3dae6f2f459c73e3d3b5a005f3395d26772 +PCRE2.v10.44.0+0.armv6l-linux-musleabihf.tar.gz/md5/8854c24183441aa6fd21989c00888904 +PCRE2.v10.44.0+0.armv6l-linux-musleabihf.tar.gz/sha512/a74d9378f071dc4cb021e5171d66cd4ac5de3b348e993fc90d824ce5d2f554f7c8af7af55ec31d874d302aaba7d542b6505cc5963e53656c28026a06a53ed48b +PCRE2.v10.44.0+0.armv7l-linux-gnueabihf.tar.gz/md5/04960309ee7cf69a53e280878d5880ef +PCRE2.v10.44.0+0.armv7l-linux-gnueabihf.tar.gz/sha512/a1644daf036daa3799368598427c87c23bcfdddac55a0d06adca08a2e9d617c893285855af562101b05129d0ed0d84d22f5a8a1703316ecd09aa1752b8330eef +PCRE2.v10.44.0+0.armv7l-linux-musleabihf.tar.gz/md5/1335defc6090be76c509840633f7cdfb +PCRE2.v10.44.0+0.armv7l-linux-musleabihf.tar.gz/sha512/9595052eeae4da413b930b14d7e89359a29220cd9e908325e0b7788c8f4a2feb2134e78a0d8f56007787f0fefadc9de31750db6104bbdd048fa50e1d785c2a8c +PCRE2.v10.44.0+0.i686-linux-gnu.tar.gz/md5/e2d6be1d19566c965c2afeb995aba52f +PCRE2.v10.44.0+0.i686-linux-gnu.tar.gz/sha512/4a9d981bb6aa9150b670db7c5d4d188c8391fcb2a16bc710ede7a84bf7ec546fc5fd9096a339720579d25b6dcb5674b2b5b28e9664e5ef589b1a5044ce38b6a7 +PCRE2.v10.44.0+0.i686-linux-musl.tar.gz/md5/23cf857bd3daea4f094fcec48a7712dc +PCRE2.v10.44.0+0.i686-linux-musl.tar.gz/sha512/534f0cfab0cd60db9498eff387f7280a8baaf893a98dd2e7a737e68ba6473ed8236e9da85116eefb9812ec5323c705a00fcaff010b1900f752de8bdff65ef3ad +PCRE2.v10.44.0+0.i686-w64-mingw32.tar.gz/md5/3d05764df2305f16e4ffab60031ad40c +PCRE2.v10.44.0+0.i686-w64-mingw32.tar.gz/sha512/3e21cc6b71849c1a361373de30567990dba13dfd8812e7a7b5e2734b572bf1d45aeb730289d329975e76932c4c40e476824be2ab8e80a40fb7a7e2f46159235a +PCRE2.v10.44.0+0.powerpc64le-linux-gnu.tar.gz/md5/596d7c29d1417ed8959ea3ae3b4df453 +PCRE2.v10.44.0+0.powerpc64le-linux-gnu.tar.gz/sha512/89e03bfd6890150e2c8dddc4e7d024f2e09421c25a3d0fef3b5cd7f6bab7d6402ec1e82b02ecb5d26d01dfa2fb6068d050513894c374b7f2244c8fcbf00d69e2 +PCRE2.v10.44.0+0.x86_64-apple-darwin.tar.gz/md5/18f13c78ff6388c601bd36788e526b31 +PCRE2.v10.44.0+0.x86_64-apple-darwin.tar.gz/sha512/7b43a289f54064fc3c292de98173ec91cde2e49402c99c7848cbdc0e6d90a23a86d41f521e3986fcc8d941ee070d09e29ddc89a4e23009b8e9333e577ae4a09c +PCRE2.v10.44.0+0.x86_64-linux-gnu.tar.gz/md5/9f45feca0955f81ceb898208b9c74e15 +PCRE2.v10.44.0+0.x86_64-linux-gnu.tar.gz/sha512/eac215838306f7b5adb2166c3f620a69ed52fbd752ef3673a887507963a826c305d9b078dbb5236dc9a45eaca0d34f77325aab41703745701a077c84822ec0d0 +PCRE2.v10.44.0+0.x86_64-linux-musl.tar.gz/md5/79f092c6e8e971027ac6c1f0987376fb +PCRE2.v10.44.0+0.x86_64-linux-musl.tar.gz/sha512/2c5655b0f719a7d442c89f1040f2973b03f8becd855a0cfd6c0a985a07b25de351a84e3b9daaebd952b62628db0d937de08a8d05ee4bcace7e72d6b5ce6b8435 +PCRE2.v10.44.0+0.x86_64-unknown-freebsd.tar.gz/md5/a0bc32a099a584d453458a76c892fe47 +PCRE2.v10.44.0+0.x86_64-unknown-freebsd.tar.gz/sha512/6649c1b9e9569a9decccf6ebaa61d44acdb9069208ec796777d8e70a908210f775be2142053f6a5762ebaa321e297f6d8b51db99629766bc702c498b5f772492 +PCRE2.v10.44.0+0.x86_64-w64-mingw32.tar.gz/md5/eeffb6164fba08b0d5c7f50afa081475 +PCRE2.v10.44.0+0.x86_64-w64-mingw32.tar.gz/sha512/f06db992a2070a88559c15224972aeb098d4291a4325970fc0fbbb7cdd539f4a2fd4f90c0de90a34fe454da6c38290f9e0c7fdf2fe8c441f687fe4491d652adc +pcre2-10.44.tar.bz2/md5/9d1fe11e2e919c7b395e3e8f0a5c3eec +pcre2-10.44.tar.bz2/sha512/ee91cc10a2962bc7818b03d368df3dd31f42ea9a7260ae51483ea8cd331b7431e36e63256b0adc213cc6d6741e7c90414fd420622308c0ae3fcb5dd878591be2 diff --git a/deps/pcre.version b/deps/pcre.version index e3ea507376105..681e97e197f51 100644 --- a/deps/pcre.version +++ b/deps/pcre.version @@ -2,4 +2,4 @@ PCRE_JLL_NAME := PCRE2 ## source build -PCRE_VER := 10.43 +PCRE_VER := 10.44 diff --git a/stdlib/PCRE2_jll/Project.toml b/stdlib/PCRE2_jll/Project.toml index ae1fb74922d79..fee83c7ce552c 100644 --- a/stdlib/PCRE2_jll/Project.toml +++ b/stdlib/PCRE2_jll/Project.toml @@ -1,6 +1,6 @@ name = "PCRE2_jll" uuid = "efcefdf7-47ab-520b-bdef-62a2eaa19f15" -version = "10.43.0+1" +version = "10.44.0+0" [deps] Libdl = "8f399da3-3557-5675-b5ff-fb832c97cbdb" diff --git a/stdlib/PCRE2_jll/test/runtests.jl b/stdlib/PCRE2_jll/test/runtests.jl index af0ed9434d2b6..21df2ec430e0e 100644 --- a/stdlib/PCRE2_jll/test/runtests.jl +++ b/stdlib/PCRE2_jll/test/runtests.jl @@ -6,5 +6,5 @@ using Test, Libdl, PCRE2_jll vstr = zeros(UInt8, 32) @test ccall((:pcre2_config_8, libpcre2_8), Cint, (UInt32, Ref{UInt8}), 11, vstr) > 0 vn = VersionNumber(split(unsafe_string(pointer(vstr)), " ")[1]) - @test vn == v"10.43.0" + @test vn == v"10.44.0" end From c1f806dfc168fa2a3dd426f0c003a843bd01aaad Mon Sep 17 00:00:00 2001 From: Jameson Nash Date: Thu, 28 Nov 2024 22:20:40 -0500 Subject: [PATCH 16/19] compile: make more efficient by discarding internal names (#56702) These are not user-visible, so this makes the compiler faster and more efficient with no effort on our part, and avoids duplicating the debug_level parameter. --- src/aotcompile.cpp | 7 +++++-- src/ccall.cpp | 10 ++++++++-- src/codegen.cpp | 27 ++++++++++++--------------- src/jitlayers.cpp | 7 ++++--- src/jitlayers.h | 1 - src/llvm-late-gc-lowering.cpp | 21 ++++++--------------- src/llvm-pass-helpers.cpp | 7 ------- 7 files changed, 35 insertions(+), 45 deletions(-) diff --git a/src/aotcompile.cpp b/src/aotcompile.cpp index 7b3c771f9dc12..1d1e48efc8c6c 100644 --- a/src/aotcompile.cpp +++ b/src/aotcompile.cpp @@ -593,9 +593,10 @@ void *jl_create_native_impl(jl_array_t *methods, LLVMOrcThreadSafeModuleRef llvm }); egal_set method_roots; jl_codegen_params_t params(ctxt, std::move(target_info.first), std::move(target_info.second)); + if (!llvmmod) + params.getContext().setDiscardValueNames(true); params.params = cgparams; params.imaging_mode = imaging; - params.debug_level = cgparams->debug_info_level; params.external_linkage = _external_linkage; params.temporary_roots = jl_alloc_array_1d(jl_array_any_type, 0); JL_GC_PUSH3(¶ms.temporary_roots, &method_roots.list, &method_roots.keyset); @@ -1719,6 +1720,7 @@ static SmallVector add_output(Module &M, TargetMachine &TM, Stri for (unsigned i = 0; i < threads; i++) { std::function func = [&, i]() { LLVMContext ctx; + ctx.setDiscardValueNames(true); #if JL_LLVM_VERSION < 170000 SetOpaquePointer(ctx); #endif @@ -1930,6 +1932,7 @@ void jl_dump_native_impl(void *native_code, if (z) { JL_TIMING(NATIVE_AOT, NATIVE_Sysimg); LLVMContext Context; + Context.setDiscardValueNames(true); #if JL_LLVM_VERSION < 170000 SetOpaquePointer(Context); #endif @@ -2077,6 +2080,7 @@ void jl_dump_native_impl(void *native_code, { JL_TIMING(NATIVE_AOT, NATIVE_Metadata); LLVMContext Context; + Context.setDiscardValueNames(true); #if JL_LLVM_VERSION < 170000 SetOpaquePointer(Context); #endif @@ -2278,7 +2282,6 @@ void jl_get_llvmf_defn_impl(jl_llvmf_dump_t* dump, jl_method_instance_t *mi, jl_ // output.imaging = true; // This would also be nice, but it seems to cause OOMs on the windows32 builder // To get correct names in the IR this needs to be at least 2 - output.debug_level = params.debug_info_level; output.temporary_roots = jl_alloc_array_1d(jl_array_any_type, 0); JL_GC_PUSH1(&output.temporary_roots); auto decls = jl_emit_code(m, mi, src, output); diff --git a/src/ccall.cpp b/src/ccall.cpp index 707203bd13506..52f8f807132e5 100644 --- a/src/ccall.cpp +++ b/src/ccall.cpp @@ -851,6 +851,7 @@ static jl_cgval_t emit_llvmcall(jl_codectx_t &ctx, jl_value_t **args, size_t nar // generate a temporary module that contains our IR std::unique_ptr Mod; + bool shouldDiscardValueNames = ctx.builder.getContext().shouldDiscardValueNames(); Function *f; if (entry == NULL) { // we only have function IR, which we should put in a function @@ -878,7 +879,9 @@ static jl_cgval_t emit_llvmcall(jl_codectx_t &ctx, jl_value_t **args, size_t nar << jl_string_data(ir) << "\n}"; SMDiagnostic Err = SMDiagnostic(); + ctx.builder.getContext().setDiscardValueNames(false); Mod = parseAssemblyString(ir_stream.str(), Err, ctx.builder.getContext()); + ctx.builder.getContext().setDiscardValueNames(shouldDiscardValueNames); // backwards compatibility: support for IR with integer pointers if (!Mod) { @@ -911,8 +914,9 @@ static jl_cgval_t emit_llvmcall(jl_codectx_t &ctx, jl_value_t **args, size_t nar << jl_string_data(ir) << "\n}"; SMDiagnostic Err = SMDiagnostic(); - Mod = - parseAssemblyString(compat_ir_stream.str(), Err, ctx.builder.getContext()); + ctx.builder.getContext().setDiscardValueNames(false); + Mod = parseAssemblyString(compat_ir_stream.str(), Err, ctx.builder.getContext()); + ctx.builder.getContext().setDiscardValueNames(shouldDiscardValueNames); } if (!Mod) { @@ -932,7 +936,9 @@ static jl_cgval_t emit_llvmcall(jl_codectx_t &ctx, jl_value_t **args, size_t nar if (jl_is_string(ir)) { SMDiagnostic Err = SMDiagnostic(); + ctx.builder.getContext().setDiscardValueNames(false); Mod = parseAssemblyString(jl_string_data(ir), Err, ctx.builder.getContext()); + ctx.builder.getContext().setDiscardValueNames(shouldDiscardValueNames); if (!Mod) { std::string message = "Failed to parse LLVM assembly: \n"; raw_string_ostream stream(message); diff --git a/src/codegen.cpp b/src/codegen.cpp index ebb96837f4db5..0b483696567ad 100644 --- a/src/codegen.cpp +++ b/src/codegen.cpp @@ -167,7 +167,7 @@ void setName(jl_codegen_params_t ¶ms, Value *V, const Twine &Name) // is not checking that setName is only called for non-folded instructions (e.g. folded bitcasts // and 0-byte geps), which can result in information loss on the renamed instruction. assert((isa(V) || isa(V)) && "Should only set names on instructions!"); - if (params.debug_level >= 2 && !isa(V)) { + if (!isa(V)) { V->setName(Name); } } @@ -175,23 +175,21 @@ void setName(jl_codegen_params_t ¶ms, Value *V, const Twine &Name) void maybeSetName(jl_codegen_params_t ¶ms, Value *V, const Twine &Name) { // To be used when we may get an Instruction or something that is not an instruction i.e Constants/Arguments - if (params.debug_level >= 2 && isa(V)) { + if (isa(V)) V->setName(Name); - } } void setName(jl_codegen_params_t ¶ms, Value *V, std::function GetName) { assert((isa(V) || isa(V)) && "Should only set names on instructions!"); - if (params.debug_level >= 2 && !isa(V)) { + if (!params.getContext().shouldDiscardValueNames() && !isa(V)) V->setName(Twine(GetName())); - } } void setNameWithField(jl_codegen_params_t ¶ms, Value *V, std::function GetObjName, jl_datatype_t *jt, unsigned idx, const Twine &suffix) { assert((isa(V) || isa(V)) && "Should only set names on instructions!"); - if (params.debug_level >= 2 && !isa(V)) { + if (!params.getContext().shouldDiscardValueNames() && !isa(V)) { if (jl_is_tuple_type(jt)){ V->setName(Twine(GetObjName()) + "[" + Twine(idx + 1) + "]"+ suffix); return; @@ -8327,7 +8325,7 @@ static jl_returninfo_t get_specsig_function(jl_codectx_t &ctx, Module *M, Value if (f == NULL) { f = Function::Create(ftype, GlobalVariable::ExternalLinkage, name, M); jl_init_function(f, ctx.emission_context.TargetTriple); - if (ctx.emission_context.debug_level >= 2) { + if (ctx.emission_context.params->debug_info_level >= 2) { ios_t sigbuf; ios_mem(&sigbuf, 0); jl_static_show_func_sig((JL_STREAM*) &sigbuf, sig); @@ -8435,7 +8433,7 @@ static jl_llvm_functions_t std::map labels; ctx.module = jl_is_method(lam->def.method) ? lam->def.method->module : lam->def.module; ctx.linfo = lam; - ctx.name = TSM.getModuleUnlocked()->getModuleIdentifier().data(); + ctx.name = name_from_method_instance(lam); size_t nreq = src->nargs; int va = src->isva; ctx.nargs = nreq; @@ -8488,7 +8486,7 @@ static jl_llvm_functions_t // jl_printf(JL_STDERR, "\n*** compiling %s at %s:%d\n\n", // jl_symbol_name(ctx.name), ctx.file.str().c_str(), toplineno); - bool debug_enabled = ctx.emission_context.debug_level != 0; + bool debug_enabled = ctx.emission_context.params->debug_info_level != 0; if (dbgFuncName.empty()) // Should never happen anymore? debug_enabled = false; @@ -8564,7 +8562,6 @@ static jl_llvm_functions_t // allocate Function declarations and wrapper objects //Safe because params holds ctx lock Module *M = TSM.getModuleUnlocked(); - M->addModuleFlag(Module::Warning, "julia.debug_level", ctx.emission_context.debug_level); jl_debugcache_t debugcache; debugcache.initialize(M); jl_returninfo_t returninfo = {}; @@ -8572,7 +8569,7 @@ static jl_llvm_functions_t bool has_sret = false; if (specsig) { // assumes !va and !needsparams SmallVector ArgNames(0); - if (ctx.emission_context.debug_level >= 2) { + if (!M->getContext().shouldDiscardValueNames()) { ArgNames.resize(ctx.nargs, ""); for (int i = 0; i < ctx.nargs; i++) { jl_sym_t *argname = slot_symbol(ctx, i); @@ -8639,7 +8636,7 @@ static jl_llvm_functions_t declarations.functionObject = needsparams ? "jl_fptr_sparam" : "jl_fptr_args"; } - if (ctx.emission_context.debug_level >= 2 && lam->def.method && jl_is_method(lam->def.method) && lam->specTypes != (jl_value_t*)jl_emptytuple_type) { + if (!params.getContext().shouldDiscardValueNames() && ctx.emission_context.params->debug_info_level >= 2 && lam->def.method && jl_is_method(lam->def.method) && lam->specTypes != (jl_value_t*)jl_emptytuple_type) { ios_t sigbuf; ios_mem(&sigbuf, 0); jl_static_show_func_sig((JL_STREAM*) &sigbuf, (jl_value_t*)lam->specTypes); @@ -8694,7 +8691,7 @@ static jl_llvm_functions_t if (debug_enabled) { topfile = dbuilder.createFile(ctx.file, "."); DISubroutineType *subrty; - if (ctx.emission_context.debug_level <= 1) + if (ctx.emission_context.params->debug_info_level <= 1) subrty = debugcache.jl_di_func_null_sig; else if (!specsig) subrty = debugcache.jl_di_func_sig; @@ -8715,7 +8712,7 @@ static jl_llvm_functions_t ); topdebugloc = DILocation::get(ctx.builder.getContext(), toplineno, 0, SP, NULL); f->setSubprogram(SP); - if (ctx.emission_context.debug_level >= 2) { + if (ctx.emission_context.params->debug_info_level >= 2) { const bool AlwaysPreserve = true; // Go over all arguments and local variables and initialize their debug information for (i = 0; i < nreq; i++) { @@ -10161,7 +10158,7 @@ jl_llvm_functions_t jl_emit_codeinst( if (// keep code when keeping everything !(JL_DELETE_NON_INLINEABLE) || // aggressively keep code when debugging level >= 2 - // note that this uses the global jl_options.debug_level, not the local emission_ctx.debug_level + // note that this uses the global jl_options.debug_level, not the local emission_ctx.debug_info_level jl_options.debug_level > 1) { // update the stored code if (inferred != (jl_value_t*)src) { diff --git a/src/jitlayers.cpp b/src/jitlayers.cpp index d7e8ca4a4850a..42ddfb688af39 100644 --- a/src/jitlayers.cpp +++ b/src/jitlayers.cpp @@ -690,9 +690,9 @@ static void jl_emit_codeinst_to_jit( JL_TIMING(CODEINST_COMPILE, CODEINST_COMPILE); // emit the code in LLVM IR form to the new context jl_codegen_params_t params(std::make_unique(), jl_ExecutionEngine->getDataLayout(), jl_ExecutionEngine->getTargetTriple()); // Locks the context + params.getContext().setDiscardValueNames(true); params.cache = true; params.imaging_mode = imaging_default(); - params.debug_level = jl_options.debug_level; orc::ThreadSafeModule result_m = jl_create_ts_module(name_from_method_instance(codeinst->def), params.tsctx, params.DL, params.TargetTriple); params.temporary_roots = jl_alloc_array_1d(jl_array_any_type, 0); @@ -795,9 +795,10 @@ int jl_compile_extern_c_impl(LLVMOrcThreadSafeModuleRef llvmmod, void *p, void * Module *M = into->getModuleUnlocked(); jl_codegen_params_t params(into->getContext(), M->getDataLayout(), Triple(M->getTargetTriple())); params.imaging_mode = imaging_default(); - params.debug_level = jl_options.debug_level; - if (pparams == NULL) + if (pparams == NULL) { + M->getContext().setDiscardValueNames(true); pparams = ¶ms; + } assert(pparams->tsctx.getContext() == into->getContext().getContext()); const char *name = jl_generate_ccallable(wrap(into), sysimg, declrt, sigt, *pparams); if (!sysimg) { diff --git a/src/jitlayers.h b/src/jitlayers.h index d5fa878211200..baba5412226e3 100644 --- a/src/jitlayers.h +++ b/src/jitlayers.h @@ -259,7 +259,6 @@ struct jl_codegen_params_t { bool cache = false; bool external_linkage = false; bool imaging_mode; - int debug_level; bool use_swiftcc = true; jl_codegen_params_t(orc::ThreadSafeContext ctx, DataLayout DL, Triple triple) : tsctx(std::move(ctx)), diff --git a/src/llvm-late-gc-lowering.cpp b/src/llvm-late-gc-lowering.cpp index 3e372ec9884e7..d95cc9c49b698 100644 --- a/src/llvm-late-gc-lowering.cpp +++ b/src/llvm-late-gc-lowering.cpp @@ -1944,28 +1944,19 @@ void LateLowerGCFrame::CleanupWriteBarriers(Function &F, State *S, const SmallVe if (CFGModified) { *CFGModified = true; } - auto DebugInfoMeta = F.getParent()->getModuleFlag("julia.debug_level"); - int debug_info = 1; - if (DebugInfoMeta != nullptr) { - debug_info = cast(cast(DebugInfoMeta)->getValue())->getZExtValue(); - } IRBuilder<> builder(CI); builder.SetCurrentDebugLocation(CI->getDebugLoc()); - auto parBits = builder.CreateAnd(EmitLoadTag(builder, T_size, parent), GC_OLD_MARKED); - setName(parBits, "parent_bits", debug_info); - auto parOldMarked = builder.CreateICmpEQ(parBits, ConstantInt::get(T_size, GC_OLD_MARKED)); - setName(parOldMarked, "parent_old_marked", debug_info); + auto parBits = builder.CreateAnd(EmitLoadTag(builder, T_size, parent), GC_OLD_MARKED, "parent_bits"); + auto parOldMarked = builder.CreateICmpEQ(parBits, ConstantInt::get(T_size, GC_OLD_MARKED), "parent_old_marked"); auto mayTrigTerm = SplitBlockAndInsertIfThen(parOldMarked, CI, false); builder.SetInsertPoint(mayTrigTerm); - setName(mayTrigTerm->getParent(), "may_trigger_wb", debug_info); + mayTrigTerm->getParent()->setName("may_trigger_wb"); Value *anyChldNotMarked = NULL; for (unsigned i = 1; i < CI->arg_size(); i++) { Value *child = CI->getArgOperand(i); - Value *chldBit = builder.CreateAnd(EmitLoadTag(builder, T_size, child), GC_MARKED); - setName(chldBit, "child_bit", debug_info); - Value *chldNotMarked = builder.CreateICmpEQ(chldBit, ConstantInt::get(T_size, 0),"child_not_marked"); - setName(chldNotMarked, "child_not_marked", debug_info); + Value *chldBit = builder.CreateAnd(EmitLoadTag(builder, T_size, child), GC_MARKED, "child_bit"); + Value *chldNotMarked = builder.CreateICmpEQ(chldBit, ConstantInt::get(T_size, 0), "child_not_marked"); anyChldNotMarked = anyChldNotMarked ? builder.CreateOr(anyChldNotMarked, chldNotMarked) : chldNotMarked; } assert(anyChldNotMarked); // handled by all_of test above @@ -1973,7 +1964,7 @@ void LateLowerGCFrame::CleanupWriteBarriers(Function &F, State *S, const SmallVe SmallVector Weights{1, 9}; auto trigTerm = SplitBlockAndInsertIfThen(anyChldNotMarked, mayTrigTerm, false, MDB.createBranchWeights(Weights)); - setName(trigTerm->getParent(), "trigger_wb", debug_info); + trigTerm->getParent()->setName("trigger_wb"); builder.SetInsertPoint(trigTerm); if (CI->getCalledOperand() == write_barrier_func) { builder.CreateCall(getOrDeclare(jl_intrinsics::queueGCRoot), parent); diff --git a/src/llvm-pass-helpers.cpp b/src/llvm-pass-helpers.cpp index cc6c73161968d..4e9e4826b4f75 100644 --- a/src/llvm-pass-helpers.cpp +++ b/src/llvm-pass-helpers.cpp @@ -335,10 +335,3 @@ namespace jl_well_known { return addGCAllocAttributes(allocTypedFunc); }); } - -void setName(llvm::Value *V, const llvm::Twine &Name, int debug_info) -{ - if (debug_info >= 2 && !llvm::isa(V)) { - V->setName(Name); - } -} From d32843b606bdb59957662d267f258d3dc7bf27b5 Mon Sep 17 00:00:00 2001 From: Valentin Churavy Date: Sat, 30 Nov 2024 16:34:42 +0100 Subject: [PATCH 17/19] Automatically enable JITPROFILING with ITTAPI (#55598) This helps when profiling remotely since VTunes doesn't support setting environment variables on remote systems. Will still respect `ENABLE_JITPROFILING=0`. --- src/codegen.cpp | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/src/codegen.cpp b/src/codegen.cpp index 0b483696567ad..b8bed0793730b 100644 --- a/src/codegen.cpp +++ b/src/codegen.cpp @@ -81,6 +81,10 @@ #include #include +#ifdef USE_ITTAPI +#include "ittapi/ittnotify.h" +#endif + using namespace llvm; static bool jl_fpo_disabled(const Triple &TT) { @@ -10427,8 +10431,16 @@ extern "C" void jl_init_llvm(void) const char *jit_profiling = getenv("ENABLE_JITPROFILING"); #if defined(JL_USE_INTEL_JITEVENTS) - if (jit_profiling && atoi(jit_profiling)) { - jl_using_intel_jitevents = 1; + if (jit_profiling) { + if (atoi(jit_profiling)) { + jl_using_intel_jitevents = 1; + } + } else { +#ifdef USE_ITTAPI + __itt_collection_state state = __itt_get_collection_state(); + jl_using_intel_jitevents = state == __itt_collection_init_successful || + state == __itt_collection_collector_exists; +#endif } #endif From ef328064a96fb143c28e2765f5a84b3ab8c43c87 Mon Sep 17 00:00:00 2001 From: Zentrik Date: Sun, 1 Dec 2024 04:35:39 +0000 Subject: [PATCH 18/19] Fix string handling in jlchecksum (#56720) A `TAGGED_RELEASE_BANNER` with spaces such as `Official https://julialang.org release` produces the error `/cache/build/builder-amdci4-5/julialang/julia-master/deps/tools/jlchecksum: 66: [: Official: unexpected operator`. --- deps/tools/jlchecksum | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deps/tools/jlchecksum b/deps/tools/jlchecksum index 329d3a2a845d4..9945ec89e6bda 100755 --- a/deps/tools/jlchecksum +++ b/deps/tools/jlchecksum @@ -63,7 +63,7 @@ find_checksum() fi done if [ ! -f "$DEPSDIR/checksums/$BASENAME/$CHECKSUM_TYPE" ]; then - if [ ${TAGGED_RELEASE_BANNER:-} ]; then + if [ "${TAGGED_RELEASE_BANNER:-}" ]; then echo "WARNING: $CHECKSUM_TYPE checksum for $BASENAME not found in deps/checksums/, failing release build." >&2 exit 3 fi From ea421126d831b18fa00408794247317b544b18d8 Mon Sep 17 00:00:00 2001 From: Priynsh <119518987+Priynsh@users.noreply.github.com> Date: Mon, 2 Dec 2024 00:29:52 +0530 Subject: [PATCH 19/19] Clarifying ispunct behavior difference between Julia and C in documentation (#56727) Fixes #56680. This PR updates the documentation for the ispunct function in Julia to explicitly note its differing behavior from the similarly named function in C. --------- Co-authored-by: Lilith Orion Hafner --- base/strings/unicode.jl | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/base/strings/unicode.jl b/base/strings/unicode.jl index ad047514c85a6..fcb4a371e9898 100644 --- a/base/strings/unicode.jl +++ b/base/strings/unicode.jl @@ -534,11 +534,17 @@ iscntrl(c::AbstractChar) = c <= '\x1f' || '\x7f' <= c <= '\u9f' Tests whether a character belongs to the Unicode general category Punctuation, i.e. a character whose category code begins with 'P'. +!!! note + This behavior is different from the `ispunct` function in C. + # Examples ```jldoctest julia> ispunct('α') false +julia> ispunct('=') +false + julia> ispunct('/') true