Bump llvm to llvm/llvm-project@c06d0ff806b7

[hanhanW]

Add local fixes to torch-mlir because of llvm/llvm-project@956c070

Local revert commits:

• llvm/llvm-project@327d627
• llvm/llvm-project@de7438e
• llvm/llvm-project@3a33775

Additional changes:

• Switch DumpExecutableBenchmarks to use eliminateCommonSubExpressions.
• Remove cleanups (dynamic pass manager) from IREEBufferizeOp::apply

Because the dynamic pass pipeline triggers the below error. The dialects either need to be loaded or we need to initialize the objects before running the pass. The transform op is not a pass, which makes it much tricker. We follow what upstream does, which separate the cleanups, i.e., users need to clean the IR themselves. Upstream example: https://github.com/llvm/llvm-project/blob/cf9806eb4da23b42702aa88784969520702dae00/mlir/test/Integration/Dialect/Linalg/CPU/unpack-dynamic-inner-tile.mlir#L96-L104

Assertion failed: (impl->multiThreadedExecutionContext == 0 && "appending to the MLIRContext dialect registry while in a " "multi-threaded execution context"), function ap
pendDialectRegistry, file MLIRContext.cpp, line 403.
Please report issues to https://github.com/iree-org/iree/issues and include the crash backtrace.
Stack dump:
0.      Program arguments: build/tools/iree-opt --iree-transform-dialect-interpreter --transform-dialect-drop-schedule compiler/src/iree/compiler/Codegen/LLVMCPU/test/tran
sform_dialect_bufferize.mlir --mlir-disable-threading

[hanhanW]

Here is the dump from the new CUDA smoketest: https://gist.github.com/hanhanW/b5399262d73b05d119fa2a5550bba00e

It looks like .entry add_dispatch becomes .visible .func add_dispatch. The .maxntid 32, 1, 1 disappears. @MaheshRavishankar @benvanik @kuhar What is the best fix for the test? It is the only failure.

iree/compiler/plugins/target/CUDA/test/smoketest.mlir

Lines 64 to 70 in dc4e900

	// PTX: .entry add_dispatch
	// PTX: .maxntid 32, 1, 1
	// PTX: add.rn.f32
	// PTX: .entry mul_dispatch
	// PTX: .maxntid 32, 1, 1
	// PTX: mul.rn.f32

[hanhanW]

This is the PTX from the current main branch: https://gist.github.com/hanhanW/baa52eb162b7370a1dcf9bb943b71739

Confirmed that entry becomes function, and only maxntid disappears. The mlir dumps are the same, and the generated data from SerializeTargetExecutablesPass is different.

[hanhanW]

Okay, I think llvm/llvm-project@de7438e is the root cause. Let's revert it local for now, and we can revisit later.

[MaheshRavishankar]

Good catch! Report it to the original author?

[hanhanW]

Hitting the below failure, I'm still figuring why

Assertion failed: (impl->multiThreadedExecutionContext == 0 && "appending to the MLIRContext dialect registry while in a " "multi-threaded execution context"), function ap
pendDialectRegistry, file MLIRContext.cpp, line 403.
Please report issues to https://github.com/iree-org/iree/issues and include the crash backtrace.
Stack dump:
0.      Program arguments: build/tools/iree-opt --iree-transform-dialect-interpreter --transform-dialect-drop-schedule compiler/src/iree/compiler/Codegen/LLVMCPU/test/tran
sform_dialect_bufferize.mlir --mlir-disable-threading

[hanhanW]

It seems to be dynamic pipeline issue, I need to learn more about it.

[hanhanW]

I'm not looking at this now, will get back to it later

[hanhanW]

yeah, I think the issue happens in dynamic pipeline.. I dont see the issue for a failure if I disable the canonicalization and cse in the pass. I'm looking at how to do it properly now. (I can't find suspicious commit in upstream.)

iree/compiler/src/iree/compiler/Dialect/HAL/Transforms/DumpExecutableBenchmarks.cpp

Lines 450 to 457 in 56bb652

	// Run CSE and the canonicalizer to pretty up the output.
	PassManager passManager(moduleOp->getContext());
	passManager.addPass(mlir::createCanonicalizerPass());
	passManager.addPass(mlir::createCSEPass());
	if (failed(passManager.run(*moduleOp))) {
	moduleOp->emitError("failed to run canonicalizer; malformed output");
	return {};
	}

[benvanik]

oo yeah, that looks wrong (running the pass manager inline like that) - given that it's canonicalization + CSE we could avoid using passes (applyPatternsAndFoldGreedily + eliminateCommonSubExpressions)

back when I wrote that eliminateCommonSubExpressions didn't exist - nice that it does now!

[hanhanW]

oo yeah, that looks wrong (running the pass manager inline like that) - given that it's canonicalization + CSE we could avoid using passes (applyPatternsAndFoldGreedily + eliminateCommonSubExpressions)

back when I wrote that eliminateCommonSubExpressions didn't exist - nice that it does now!

The transform interpreter one is also tricky, I'll take a look after this.

[benvanik]

oh yeah, no clue about transform dialect >_>

[hanhanW]

eliminateCommonSubExpressions works! I think we do not need applyPatternsAndFoldGreedily because it is not a pattern-based pass. Looking at other failures now!

[benvanik]

the apply patterns would be the replacement for canonicalization:

 RewritePatternSet patterns(context);

    for (auto *dialect : context->getLoadedDialects()) {
      dialect->getCanonicalizationPatterns(patterns);
    }
    for (auto op : context->getRegisteredOperations()) {
      op.getCanonicalizationPatterns(patterns, context);
    }
    IREE::Util::populateCommonPatterns(context, patterns);

    FrozenRewritePatternSet frozenPatterns(std::move(patterns));
    if (failed(applyPatternsGreedily(getOperation(), frozenPatterns))) {
      getOperation()->emitError()
          << "failed to apply patterns, likely due to a bad pattern that "
             "causes an infinite fixed point iteration";
      return signalPassFailure();
    }

[hanhanW]

It only fails in IREEBufferizeOp op, looking >__>

iree/compiler/src/iree/compiler/Codegen/Common/TransformExtensions/CommonExtensions.cpp

Lines 907 to 977 in 0a2862c

	DiagnosedSilenceableFailure transform_dialect::IREEBufferizeOp::apply(
	transform::TransformRewriter &rewriter,
	transform::TransformResults &results, transform::TransformState &state) {
	auto payload = state.getPayloadOps(getTarget());
	// Bufferize the dispatch.
	using mlir::bufferization::BufferizationOptions;
	BufferizationOptions::AllocationFn allocationFn =
	cpuComprehensiveBufferizeAllocationFn;
	BufferizationOptions::MemCpyFn memCpyFn = cpuComprehensiveBufferizeCopyFn;
	if (getTargetGpu()) {
	allocationFn = gpuComprehensiveBufferizeAllocationFn;
	memCpyFn = gpuComprehensiveBufferizeCopyFn;
	}
	Operation *target = *payload.begin();
	ErrorCheckingTrackingListener listener(state, *this);
	// 1. Rewrite tensor.empty to tensor.alloc, without the pass baggage.
	{
	RewritePatternSet patterns(getContext());
	patterns.add<EmptyTensorLoweringPattern>(patterns.getContext());
	GreedyRewriteConfig config;
	config.listener = &listener;
	// Manually gather list of ops because the other GreedyPatternRewriteDriver
	// overloads only accepts ops that are isolated from above.
	LogicalResult result =
	applyOpPatternsGreedily(target, std::move(patterns), config);
	if (failed(result)) {
	return mlir::emitDefiniteFailure(target,
	"greedy pattern application failed");
	}
	if (listener.failed())
	return listener.checkAndResetError();
	}
	// 2. Run one-shot-bufferize, without the pass baggage.
	IREEOneShotBufferizationOptions options = getBufferizationOptions();
	options.allocationFn = allocationFn;
	options.memCpyFn = memCpyFn;
	options.testAnalysisOnly = getTestAnalysisOnly();
	options.printConflicts = getPrintConflicts();
	if (getTargetGpu()) {
	options.defaultMemorySpaceFn =
	[&](TensorType t) -> std::optional<Attribute> {
	Attribute addressSpaceAttr = gpu::AddressSpaceAttr::get(
	t.getContext(), gpu::GPUDialect::getWorkgroupAddressSpace());
	return addressSpaceAttr;
	};
	}
	if (failed(runIREEOneShotBufferize(target, options))) {
	return mlir::emitDefiniteFailure(target, "bufferization failed");
	}
	// Early exit if test_analysis_only is set.
	if (getTestAnalysisOnly()) {
	results.set(getOperation()->getOpResult(0), {*payload.begin()});
	return listener.checkAndResetError();
	}
	// 3. Post-bufferization passes are fine.
	PassManager pm(getContext());
	addIREEPostBufferizationPasses(pm);
	if (failed(pm.run(target))) {
	return mlir::emitDefiniteFailure(target)
	<< "post-bufferization passes failed";
	}
	results.set(getOperation()->getOpResult(0), {target});
	return listener.checkAndResetError();
	}

[hanhanW]

I think it is dynamic pipeline issue again.. It does not crash if I comment out it, looking.

   //   3. Post-bufferization passes are fine. 
   PassManager pm(getContext()); 
   addIREEPostBufferizationPasses(pm); 
   if (failed(pm.run(target))) { 
     return mlir::emitDefiniteFailure(target) 
            << "post-bufferization passes failed"; 
   } 

[benvanik]

oh yeah that's tricky - the benchmarks pass was running nested because it was producing a module to output to a file (the newly produced module never exists in the main module IR so there's no way to setup dynamic passes for it) - addIREEPostBufferizationPasses is added in several places in normal pipelines, though, so maybe it's not needed there

[hanhanW]

That part is just adding some cleanup passes. I think we should follow what upstream does, drop the dynamic passes, and users should run canonicalization themselves. The transform dialect tests are not default, so I'd just drop the cleanup support from the op. Just tested, it works.

[hanhanW]

oh no, there are numerical issues in Sharktank.. I thought that my fix is very reasonable... It follows what the upstream does: llvm/llvm-project@956c070 :/

iree-org/torch-mlir@eefc553...f8eaf8d

[hanhanW]

Let's try if llvm/torch-mlir#4001 helps or not

[ScottTodd]

oh no, there are numerical issues in Sharktank.. I thought that my fix is very reasonable... It follows what the upstream does: llvm/llvm-project@956c070 :/

iree-org/torch-mlir@eefc553...f8eaf8d

I'm pretty sure TOSA changes shouldn't affect sharktank. The sharktank models go through a PyTorch -> Torch MLIR -> Linalg path. We only use TOSA for TFLite/LiteRT -> TOSA -> Linalg.

[hanhanW]

oh no, there are numerical issues in Sharktank.. I thought that my fix is very reasonable... It follows what the upstream does: llvm/llvm-project@956c070 :/
iree-org/torch-mlir@eefc553...f8eaf8d

I'm pretty sure TOSA changes shouldn't affect sharktank. The sharktank models go through a PyTorch -> Torch MLIR -> Linalg path. We only use TOSA for TFLite/LiteRT -> TOSA -> Linalg.

Then it looks like an upstream issue... The changes that I made in IREE do not impact any lowering, unless they use transform dialect. Even though, it should not impact numerics.. I'll take a look later

[hanhanW]

@ScottTodd is there a quick way to get the MLIR file and input data? Otherwise, I think I need to follow https://github.com/iree-org/iree-test-suites/tree/main/sharktank_models/llama3.1 to reproduce the issue.

[hanhanW]

Inlining the error, it produces NAN:

  comparison failed
  Obtained: nan
  Expected: 0.589 ± 5.9e-03

[ScottTodd]

@ScottTodd is there a quick way to get the MLIR file and input data? Otherwise, I think I need to follow https://github.com/iree-org/iree-test-suites/tree/main/sharktank_models/llama3.1 to reproduce the issue.

MLIR file yes, download from https://github.com/iree-org/iree-test-suites/tree/main/sharktank_models/llama3.1/assets.

Input data and run commands... not as easy as you might like. Instructions are at https://github.com/iree-org/iree-test-suites/tree/main/sharktank_models. You'd need to build the IREE Python bindings and run pytest.

[hanhanW]

@ScottTodd is there a quick way to get the MLIR file and input data? Otherwise, I think I need to follow https://github.com/iree-org/iree-test-suites/tree/main/sharktank_models/llama3.1 to reproduce the issue.

MLIR file yes, download from https://github.com/iree-org/iree-test-suites/tree/main/sharktank_models/llama3.1/assets.

Input data and run commands... not as easy as you might like. Instructions are at https://github.com/iree-org/iree-test-suites/tree/main/sharktank_models. You'd need to build the IREE Python bindings and run pytest.

Thanks for the pointers, I'll give it a shot!

[hanhanW]

Confirm that the root cause of numerics is llvm/llvm-project@3a33775. I haven't got a small repro yet.

cc @ita9naiwa

[pashu123]

LGTM!

[ScottTodd]

Something in here broke the MSVC/Windows build: https://github.com/iree-org/iree/actions/runs/13196900742/job/36840094404#step:9:7647

FAILED: compiler/plugins/input/Torch/torch-mlir/CMakeFiles/iree_compiler_plugins_input_Torch_torch-mlir_ConversionPasses.objects.dir/__/__/__/__/__/third_party/torch-mlir/lib/Conversion/TorchToTosa/TosaLegalizeCommon.cpp.obj 
C:\ProgramData\chocolatey\bin\ccache "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\bin\Hostx64\x64\cl.exe"  /nologo /TP  -IC:\home\runner\_work\iree\iree -IC:\mnt\azure\b\092805 -IC:\home\runner\_work\iree\iree\third_party\torch-mlir\include -IC:\home\runner\_work\iree\iree\compiler\plugins\input\Torch -IC:\mnt\azure\b\092805\compiler\plugins\input\Torch -IC:\home\runner\_work\iree\iree\third_party\llvm-project\llvm\include -IC:\mnt\azure\b\092805\llvm-project\include -IC:\home\runner\_work\iree\iree\third_party\llvm-project\mlir\include -IC:\mnt\azure\b\092805\llvm-project\tools\mlir\include -IC:\home\runner\_work\iree\iree\third_party\llvm-project\lld\include -IC:\mnt\azure\b\092805\llvm-project\tools\lld\include /DWIN32 /D_WINDOWS /EHsc /Z7 /O2 /Ob1  -std:c++17 -MD /wd4996 /Zc:preprocessor /DWIN32_LEAN_AND_MEAN /DNOMINMAX /D_USE_MATH_DEFINES /D_CRT_SECURE_NO_WARNINGS /D_CRT_SECURE_CPP_OVERLOAD_STANDARD_NAMES /D_SILENCE_NONFLOATING_COMPLEX_DEPRECATION_WARNING /GR- /bigobj /W3 /wd4200 /wd4018 /wd4146 /wd4244 /wd4267 /wd4005 /wd4065 /wd4141 /wd4624 /wd4576 /wd5105 /showIncludes /Focompiler\plugins\input\Torch\torch-mlir\CMakeFiles\iree_compiler_plugins_input_Torch_torch-mlir_ConversionPasses.objects.dir\__\__\__\__\__\third_party\torch-mlir\lib\Conversion\TorchToTosa\TosaLegalizeCommon.cpp.obj /Fdcompiler\plugins\input\Torch\torch-mlir\CMakeFiles\iree_compiler_plugins_input_Torch_torch-mlir_ConversionPasses.objects.dir\ /FS -c C:\home\runner\_work\iree\iree\third_party\torch-mlir\lib\Conversion\TorchToTosa\TosaLegalizeCommon.cpp
C:\home\runner\_work\iree\iree\third_party\llvm-project\mlir\include\mlir/Dialect/Tosa/Utils/ConversionUtils.h(148): error C2872: 'OpTrait': ambiguous symbol
C:\home\runner\_work\iree\iree\third_party\llvm-project\mlir\include\mlir/Dialect/Tosa/IR/TosaOps.h(49): note: could be 'mlir::OpTrait'
C:\home\runner\_work\iree\iree\third_party\torch-mlir\include\torch-mlir/Dialect/Torch/IR/TorchTraits.h(23): note: or       'mlir::torch::Torch::OpTrait'
C:\home\runner\_work\iree\iree\third_party\llvm-project\mlir\include\mlir/Dialect/Tosa/Utils/ConversionUtils.h(148): note: the template instantiation context (the oldest one first) is
C:\home\runner\_work\iree\iree\third_party\torch-mlir\lib\Conversion\TorchToTosa\TosaLegalizeCommon.cpp(126): note: see reference to function template instantiation 'TosaOp mlir::tosa::CreateOpAndInfer<mlir::tosa::MulOp,mlir::Value&,mlir::Value&,mlir::Value&>(mlir::PatternRewriter &,mlir::Location,mlir::Type,mlir::Value &,mlir::Value &,mlir::Value &)' being compiled
        with
        [
            TosaOp=mlir::tosa::MulOp
        ]
C:\home\runner\_work\iree\iree\third_party\torch-mlir\include\torch-mlir/Conversion/TorchToTosa/TosaLegalizeUtils.h(83): note: see reference to function template instantiation 'TosaOp mlir::tosa::CreateOpAndInfer<TosaOp,mlir::Value&,mlir::Value&,mlir::Value&>(mlir::ImplicitLocOpBuilder &,mlir::Type,mlir::Value &,mlir::Value &,mlir::Value &)' being compiled
        with
        [
            TosaOp=mlir::tosa::MulOp
        ]
C:\home\runner\_work\iree\iree\third_party\torch-mlir\include\torch-mlir/Conversion/TorchToTosa/TosaLegalizeUtils.h(76): note: see reference to function template instantiation 'TosaOp mlir::tosa::CreateOpAndInferShape<TosaOp,mlir::Value&,mlir::Value&,mlir::Value&>(mlir::ImplicitLocOpBuilder &,mlir::Type,mlir::Value &,mlir::Value &,mlir::Value &)' being compiled
        with
        [
            TosaOp=mlir::tosa::MulOp
        ]

I saw some recent changes to https://github.com/llvm/torch-mlir/blame/main/lib/Conversion/TorchToTosa/TosaLegalizeCommon.cpp and https://github.com/llvm/llvm-project/tree/main/mlir/include/mlir/Dialect/Tosa/Utils but I'm not sure what specifically

I'm also not sure why we build TorchToTosa at all in IREE, since we don't use it...?

[ScottTodd]

This patch fixes for me locally... still not sure what triggered that failure though

D:\dev\projects\iree\third_party\llvm-project (HEAD detached at a1984ec)
λ git diff
diff --git a/mlir/include/mlir/Dialect/Tosa/Utils/ConversionUtils.h b/mlir/include/mlir/Dialect/Tosa/Utils/ConversionUtils.h
index 78a882885543..baee63308ee2 100644
--- a/mlir/include/mlir/Dialect/Tosa/Utils/ConversionUtils.h
+++ b/mlir/include/mlir/Dialect/Tosa/Utils/ConversionUtils.h
@@ -145,7 +145,7 @@ TosaOp createOpAndInferShape(ImplicitLocOpBuilder &builder, Type resultTy,
 template <typename TosaOp, typename... Args>
 TosaOp CreateOpAndInferShape(ImplicitLocOpBuilder &builder, Type resultTy,
                              Args &&...args) {
-  if (TosaOp::template hasTrait<OpTrait::SameOperandsAndResultRank>()) {
+  if (TosaOp::template hasTrait<::mlir::OpTrait::SameOperandsAndResultRank>()) {
     // op requires same ranks for tensor operands
     if constexpr (sizeof...(Args) == 2) {
       auto argX = std::get<0>(std::tie(args...));

The namespace code in torch-mlir is sketchy: https://github.com/llvm/torch-mlir/blob/52299e6a659e68879b2c656ffd82ae2a9d28afd8/include/torch-mlir/Dialect/Torch/IR/TorchTraits.h#L20-L23

namespace mlir {
namespace torch {
namespace Torch {
namespace OpTrait {

[ScottTodd]

I'm also not sure why we build TorchToTosa at all in IREE, since we don't use it...?

Ah, TOSA is a non-optional dep of torch-mlir. We could carve it out with a TORCH_MLIR_ENABLE_TOSA like there already is for TORCH_MLIR_ENABLE_STABLEHLO 🤔

• https://github.com/llvm/torch-mlir/blob/main/CMakeLists.txt
• https://github.com/llvm/torch-mlir/blob/2063ec71a68f6f08a89d28233a95a3f933b8f037/include/torch-mlir/Conversion/Passes.td#L117-L148

[ScottTodd]

Sent my fix/workaround upstream: llvm/llvm-project#126286