adding autodiff tests

[ZuseZ4]

I'd like to get started with upstreaming some tests, even though I'm still waiting for an answer on how to best integrate the enzyme pass. Can we therefore temporarily support the -Z llvm-plugins here without too much effort? And in that case, how would that work? I saw you can do remapping, e.g. rust-src-base, but I don't think that will give me the path to libEnzyme.so. Do you have another suggestion?

Other than that this test simply checks that the derivative of x*x is 2.0 * x, which in this case is computed as
%0 = fadd fast double %x.0.val, %x.0.val
(I'll add a few more tests and move it to an autodiff folder if we can use the -Z flag)

r? @jieyouxu

Locally at least -Zllvm-plugins=${PWD}/build/x86_64-unknown-linux-gnu/enzyme/build/Enzyme/libEnzyme-19.so seems to work if I copy the command I get from x.py test and run it manually. However, running x.py test itself fails.

Tracking:

• Tracking Issue for autodiff #124509

Zulip discussion: https://rust-lang.zulipchat.com/#narrow/channel/326414-t-infra.2Fbootstrap/topic/Enzyme.20build.20changes

[jieyouxu]

We can add another remapping variable or do something like //@ add-core-stubs, I'd like #136542 cleanup to land first though to un-confuse the build directory paths in compiletest.
@rustbot blocked (#136542)

[ZuseZ4]

@jieyouxu You can remove the blocked message, I don't need a path anymore.
I don't think it will work with another gh repo, but let's try:

@rustbot blocked(EnzymeAD/Enzyme#2238)

[jieyouxu]

You can remove the blocked message, I don't need a path anymore. I don't think it will work with another gh repo, but let's try

The message doesn't matter at all, it's just @_rustbot blocked that matters lol

[ZuseZ4]

This works locally, so I'll probably make registerEnzyme a weak symbol and gate the pass loading on enzyme being enabled, then this should be ready to merge.

I will leave fighting CI and getting this to work on nightly for the other open PR in which I already started testing to enable this by default.

[rustbot]

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

[ZuseZ4]

Nope, not changing how LLVM is build, just Enzyme. So no change should be needed, I'd think.

[ZuseZ4]

Ok that should be it. Summary of the changes.

1. A small fix autodiff.rs to our parser. We were not handling forward-mode autodiff correctly, breaking one testcase.
2. Improve the Enzyme build. Enzyme consists of two passes, one which applies optimizations and one which does autodiff. In the past I was only building the autodiff pass, now I adjusted build_steps/llvm.rs to also build (and run) the opt pass by defining "ENZYME" during the build.
3. Automatically load the now combined Enzyme opt+autodiff pass. This allows the test to pass without extra flags.
4. Based on discussion with some LLVM devs and having better control thanks to 3) we now have a better llvm opt scheduling. We first run individual pre-autodiff opt passes on individual CGU. Then we enter the (fat)-lto opt stage (since autodiff atm enforces fat-lto). When rust schedules the fat-lto optimizations we append Enzyme's opt/autodiff pass.
   Finally, we run the whole fat-lto pipeline a second time, to make sure that all the code now is fully optimized.
5. The last commit removes ForwardFirst and ReverseFirst. Those were temporarily needed for higher-order derivatives (e.g. you generate df by differentiating f, and then you differentiate df). By now using a pass (instead of differentiating functions one at a time) the pass takes care of differentiating this in the right order.

To make 4) a bit clearer I introduced an enum AutodiffStage.
The enzyme submodule updated marked registerEnzyme as extern "C", so I'm able to link against it from Rust.

Those seem to be enough things for one PR, so I don't consider how this impacts compile times (and runtime perf) if Enzyme is enabled (there are no impacts if it's disabled). There is a second PR where I make sure that we can ship this on nightly without regressions.

@onur-ozkan Any other requests?

[rustbot]

Some changes occurred in compiler/rustc_codegen_ssa/src/codegen_attrs.rs

cc @jdonszelmann

[ZuseZ4]

r? @onur-ozkan (lmk if you want someone else to review it).

[rustbot]

Could not assign reviewer from: onur-ozkan.
User(s) onur-ozkan are either the PR author, already assigned, or on vacation. Please use r? to specify someone else to assign.

[ZuseZ4]

Interesting, Onur is assigned, but not a reviewer, so I couldn't request a review. Requested one from jieyouxu, just s.t. the PR status is correct.

[onur-ozkan]

Thanks!

[onur-ozkan]

@bors r+

[bors]

📌 Commit 8578aae has been approved by onur-ozkan

It is now in the queue for this repository.

[ZuseZ4]

(I didn't directly expect an r+ and didn't want to force-push while people were potentially reviewing, so I just cleaned it up now.)

[jieyouxu]

I can never remember what the force-push behavior was, so...
@bors r=onur-ozkan,jieyouxu

[bors]

📌 Commit 061abbc has been approved by onur-ozkan,jieyouxu

It is now in the queue for this repository.

[onur-ozkan]

(I didn't directly expect an r+ and didn't want to force-push while people were potentially reviewing, so I just cleaned it up now.)

As you r? me I thought you didn't want to re-work on commit history (most people avoid that). It would be better to have cleaner commit history for sure.