Skip to content
This repository has been archived by the owner on May 14, 2024. It is now read-only.

[pull] master from NixOS:master #12

Open
wants to merge 2,189 commits into
base: master
Choose a base branch
from
Open

[pull] master from NixOS:master #12

wants to merge 2,189 commits into from

Conversation

pull[bot]
Copy link

@pull pull bot commented May 13, 2024

See Commits and Changes for more details.


Created by pull[bot]

Can you help keep this open source service alive? 💖 Please sponsor : )

@pull pull bot added ⤵️ pull merge-conflict Resolve conflicts manually labels May 13, 2024
edolstra and others added 27 commits January 9, 2025 16:38
derivation-goal: unlock output lock to avoid deadlock
…uery-without-fragment

parsePathFlakeRefWithFragment(): Handle 'path?query' without a fragment
Relative, local git repo used to work (for submodules), but it
fails after 3e0129c.

This commit adds a test to prevent such failure in the future.
Plus, switched CURLOPT_PROGRESSFUNCTION to CURLOPT_XFERINFOFUNCTION since docs say it's deprecated
This allows writing lock files with dirty inputs, so long as they have
a NAR hash. (Currently they always have a NAR hash, but with lazy
trees that may not always be the case.)

Generally dirty locks are bad for reproducibility (we can detect if
the dirty input has changed, but we have no way to fetch it except
substitution). Hence we don't allow them by default.

Fixes #11181.
nix upgrade-nix: Give a better error message if the profile is using 'nix profile'
…ocker

nix flake: clarify error message when file is an unknown type
scripts/install-darwin-multi-user: workaround dscl failing sometimes
This fixes segfaults with nix copy when there was an error processing
addMultipleToStore.

Running with ASAN/TSAN pointed at an use-after-free with threads from
the pool accessing the graph declared in processGraph after the function
was exiting and destructing the variables.

It turns out that if there is an error before pool.process() is called,
for example while we are still enqueuing tasks, then pool.process()
isn't called and threads are still left to run.

By creating the pool last we ensure that it is stopped first before
running other destructors even if an exception happens early.

[ lix porting note: nix does not name threads so the patch has been
adapted to not pass thread name ]

Link: https://git.lix.systems/lix-project/lix/issues/618
Link: https://gerrit.lix.systems/c/lix/+/2355
They're usually found by other checks, but docs would remain
susceptible.
During garbage collection we cache several things -- a set of known-dead
paths, a set of known-alive paths, and a map of paths to their derivers.
Currently they use STL maps and sets, which are ordered structures that
typically are backed by binary trees. Since we are putting pseudorandom
paths into these and looking them up by exact key, we don't need the
ordering, and we're paying a nontrivial cost per insertion.

The existing maps require O(n log n) memory and have O(log n) insertion
and lookup time.

We could instead use unordered maps, which are typically backed by
hashmaps. These require O(n) memory and have O(1) insertion and lookup
time.

On my system this appears to result in a dramatic speedup -- prior to
this patch I was able to delete 400k paths out of 9.5 million over the
course of 34.5 hours. After this patch the same result took 89 minutes.

This result should NOT be taken at face value because the two runs
aren't really comparable; in particular the first started when I had 9.5
million store paths and the seconcd started with 7.8 million, so we are
deleting a different set of paths starting from a much cleaner
filesystem. But I do think it's indicative.

Related: #9581
libutil: thread-pool: ensure threads finished on error
gc: replace ordered sets with unordered sets for in-memory caches
edolstra and others added 30 commits February 10, 2025 15:19
Eventually this should probably become a struct of options.
Parser: Respect the accessor of the source file for relative paths
Add `inputs.self.submodules` flake attribute
This makes `nix flake prefetch` more useful for scripting and prevents
the result from being GC'ed prematurely.
E.g. in a derivation attribute `foo = ./bar`, if ./bar is a symlink,
we should copy the symlink to the store, not its target. This restores
the behaviour of Nix <= 2.19.
…-attrs

copyPathToStore(): Preserve symlinks
This is a workaround to avoid differing evaluation results between Nix
2.19 and >= 2.20 (#12449).
SinkToSource: avoid heap allocation
I do not believe there is any problem with computing
`hashDerivationModulo` the normal way with impure derivations.

Conversely, the way this used to work is very suspicious because two
almost-equal derivations that only differ in depending on different
impure derivations could have the same drv hash modulo. That is very
suspicious because there is no reason to think those two different
impure derivations will end up producing the same content-addressed
data!

Co-authored-by: Alain Zscheile <zseri.devel@ytrizja.de>
packaging/components.nix: Don't import a symlink
Set FD_CLOEXEC on sockets created by curl
Otherwise we may accidentally update a lock when we shouldn't.

Fixes #12445.
lockFlake(): When refetching a locked flake, use the locked ref
Get rid of `impureOutputHash`; fix possible bug
nix flake prefetch: Add --out-link option
It's best we teach users that the "foo" derivation is less than pure in the sense that it cannot be built just on any system, in particular that builders cannot be selected arbitrarily but based on their system-features. The `"recursive-nix"` system-feature is automatically defined by `--extra-experimental-features recursive-nix`
doc: recursive-nix: advertise requiredSystemFeatures
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
⤵️ pull merge-conflict Resolve conflicts manually
Projects
None yet
Development

Successfully merging this pull request may close these issues.