Skip to content
On PR

Rewrote tests for scalars and dtypes. Wrote tests for ranks. #394

Sign in to view logs

Rewrote tests for scalars and dtypes. Wrote tests for ranks.

Rewrote tests for scalars and dtypes. Wrote tests for ranks. #394

Triggered via pull request January 31, 2025 12:35
@kmitrovicTTkmitrovicTT
synchronize #168
kmitrovic/fix_tests_for_dtypes_and_scalars
Status Failure
Total duration 1h 38m 1s
Artifacts 2

on-pr.yml

on: pull_request
pre-commit  /  pre-commit
32s
pre-commit / pre-commit
spdx  /  check-spdx-headers
3s
spdx / check-spdx-headers
build-and-test  /  build-image
6m 17s
build-and-test / build-image
Matrix: build-and-test / build-and-run-tests
Fit to window
Zoom out
Zoom in

Annotations

16 errors and 1 warning
test_example_model_mixed_args_and_kwargs.test_example_model_inference: tests/jax/models/example_model/mixed_args_and_kwargs/test_example_model_mixed_args_and_kwargs.py#L84
RuntimeError: TT_THROW @ /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/tensor/tensor.hpp:317: tt::exception info: Cannot get the device from a tensor with host storage backtrace: --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x6a8bf8) [0x7f772fabebf8] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885eb4) [0x7f772fc9beb4] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885bbe) [0x7f772fc9bbbe] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885a26) [0x7f772fc9ba26] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883b13) [0x7f772fc99b13] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883601) [0x7f772fc99601] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x870081) [0x7f772fc86081] --- tt::runtime::ttnn::LayoutConverter::handleHostInputNoLayoutTypecast(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertHostTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::toLayout(tt::runtime::Tensor, tt::runtime::Device, tt::runtime::Layout) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::pjrt::LoadedExecutableInstance::Execute(PJRT_LoadedExecutable_Execute_Args*) --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1ada) [0x7f773c2dfada] --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1b3b) [0x7f773c2dfb3b] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x8a99cc) [0x7f7741a949cc] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7743f53) [0x7f774892ef53] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691d719) [0x7f7747b08719] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691ecd0) [0x7f7747b09cd0] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767f55) [0x7f7741952f55] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767d9d) [0x7f7741952d9d] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x77153f8) [0x7f77489003f8] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0xc4) [0x557e2dabb574] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0xc1) [0x557e2dad0691] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x557e2dad410b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bcf61) [0x7f77419a7f61] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bb4ff) [0x7f77419a64ff] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x557e2dabb61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x557e2dad062c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_MakeTpCall+0x25b) [0x557e2dabc39b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x68ce) [0x557e2dab699e] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x557e2dabb61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x557e2dad062c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x557e2dad410b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x557e2dab0790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x557e2dab19ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x557e2dab0790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x557e2dab19ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1]
test_example_model_only_args.test_example_model_inference: tests/jax/models/example_model/only_args/test_example_model_only_args.py#L77
RuntimeError: TT_THROW @ /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/tensor/tensor.hpp:317: tt::exception info: Cannot get the device from a tensor with host storage backtrace: --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x6a8bf8) [0x7f772fabebf8] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885eb4) [0x7f772fc9beb4] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885bbe) [0x7f772fc9bbbe] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885a26) [0x7f772fc9ba26] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883b13) [0x7f772fc99b13] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883601) [0x7f772fc99601] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x870081) [0x7f772fc86081] --- tt::runtime::ttnn::LayoutConverter::handleHostInputNoLayoutTypecast(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertHostTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::toLayout(tt::runtime::Tensor, tt::runtime::Device, tt::runtime::Layout) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::pjrt::LoadedExecutableInstance::Execute(PJRT_LoadedExecutable_Execute_Args*) --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1ada) [0x7f773c2dfada] --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1b3b) [0x7f773c2dfb3b] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x8a99cc) [0x7f7741a949cc] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7743f53) [0x7f774892ef53] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691d719) [0x7f7747b08719] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691ecd0) [0x7f7747b09cd0] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767f55) [0x7f7741952f55] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767d9d) [0x7f7741952d9d] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x77153f8) [0x7f77489003f8] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0xc4) [0x557e2dabb574] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0xc1) [0x557e2dad0691] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x557e2dad410b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bcf61) [0x7f77419a7f61] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bb4ff) [0x7f77419a64ff] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x557e2dabb61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x557e2dad062c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_MakeTpCall+0x25b) [0x557e2dabc39b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x68ce) [0x557e2dab699e] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x557e2dabb61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x557e2dad062c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x557e2dad410b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x557e2dab0790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x557e2dab19ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x557e2dab0790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x557e2dab19ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x557e2dabb61d]
test_example_model_only_kwargs.test_example_model_inference: tests/jax/models/example_model/only_kwargs/test_example_model_only_kwargs.py#L77
RuntimeError: TT_THROW @ /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/tensor/tensor.hpp:317: tt::exception info: Cannot get the device from a tensor with host storage backtrace: --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x6a8bf8) [0x7f772fabebf8] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885eb4) [0x7f772fc9beb4] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885bbe) [0x7f772fc9bbbe] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885a26) [0x7f772fc9ba26] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883b13) [0x7f772fc99b13] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883601) [0x7f772fc99601] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x870081) [0x7f772fc86081] --- tt::runtime::ttnn::LayoutConverter::handleHostInputNoLayoutTypecast(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertHostTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::toLayout(tt::runtime::Tensor, tt::runtime::Device, tt::runtime::Layout) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::pjrt::LoadedExecutableInstance::Execute(PJRT_LoadedExecutable_Execute_Args*) --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1ada) [0x7f773c2dfada] --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1b3b) [0x7f773c2dfb3b] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x8a99cc) [0x7f7741a949cc] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7743f53) [0x7f774892ef53] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691d719) [0x7f7747b08719] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691ecd0) [0x7f7747b09cd0] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767f55) [0x7f7741952f55] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767d9d) [0x7f7741952d9d] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x77153f8) [0x7f77489003f8] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0xc4) [0x557e2dabb574] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0xc1) [0x557e2dad0691] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x557e2dad410b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bcf61) [0x7f77419a7f61] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bb4ff) [0x7f77419a64ff] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x557e2dad4172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x557e2dabb61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x557e2dad062c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_MakeTpCall+0x25b) [0x557e2dabc39b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x68ce) [0x557e2dab699e] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x557e2dab097f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x557e2dabb61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x557e2dad062c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x557e2dbd8464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x557e2dad410b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x557e2dab0790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x557e2dab19ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x557e2dab0790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x557e2dab19ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x557e2dab2c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x557e2dac638c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x557e2dab5702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x557e2dad34d1]
test_constant.test_constant_zeros[shape0]: tests/jax/ops/test_constant.py#L15
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Executable expected parameter 0 of size 8192 but got buffer with incompatible size 4096
test_constant.test_constant_zeros[shape1]: tests/jax/ops/test_constant.py#L15
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Executable expected parameter 0 of size 8 but got buffer with incompatible size 4
test_constant.test_constant_ones[shape0]: tests/jax/ops/test_constant.py#L23
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Executable expected parameter 0 of size 8192 but got buffer with incompatible size 4096
test_constant.test_constant_ones[shape1]: tests/jax/ops/test_constant.py#L23
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Executable expected parameter 0 of size 8 but got buffer with incompatible size 4
test_example_model_mixed_args_and_kwargs.test_example_model_inference: tests/jax/models/example_model/mixed_args_and_kwargs/test_example_model_mixed_args_and_kwargs.py#L84
RuntimeError: TT_THROW @ /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/tensor/tensor.hpp:317: tt::exception info: Cannot get the device from a tensor with host storage backtrace: --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x6a8bf8) [0x7fbb8cc61bf8] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885eb4) [0x7fbb8ce3eeb4] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885bbe) [0x7fbb8ce3ebbe] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885a26) [0x7fbb8ce3ea26] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883b13) [0x7fbb8ce3cb13] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883601) [0x7fbb8ce3c601] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x870081) [0x7fbb8ce29081] --- tt::runtime::ttnn::LayoutConverter::handleHostInputNoLayoutTypecast(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertHostTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::toLayout(tt::runtime::Tensor, tt::runtime::Device, tt::runtime::Layout) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::pjrt::LoadedExecutableInstance::Execute(PJRT_LoadedExecutable_Execute_Args*) --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1ada) [0x7fbb99482ada] --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1b3b) [0x7fbb99482b3b] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x8a99cc) [0x7fbb9ea9a9cc] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7743f53) [0x7fbba5934f53] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691d719) [0x7fbba4b0e719] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691ecd0) [0x7fbba4b0fcd0] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767f55) [0x7fbb9e958f55] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767d9d) [0x7fbb9e958d9d] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x77153f8) [0x7fbba59063f8] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0xc4) [0x558e3541d574] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0xc1) [0x558e35432691] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x558e3543610b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bcf61) [0x7fbb9e9adf61] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bb4ff) [0x7fbb9e9ac4ff] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x558e3541d61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x558e3543262c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_MakeTpCall+0x25b) [0x558e3541e39b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x68ce) [0x558e3541899e] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x558e3541d61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x558e3543262c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x558e3543610b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x558e35412790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x558e354139ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x558e35412790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x558e354139ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1]
test_example_model_only_args.test_example_model_inference: tests/jax/models/example_model/only_args/test_example_model_only_args.py#L77
RuntimeError: TT_THROW @ /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/tensor/tensor.hpp:317: tt::exception info: Cannot get the device from a tensor with host storage backtrace: --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x6a8bf8) [0x7fbb8cc61bf8] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885eb4) [0x7fbb8ce3eeb4] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885bbe) [0x7fbb8ce3ebbe] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885a26) [0x7fbb8ce3ea26] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883b13) [0x7fbb8ce3cb13] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883601) [0x7fbb8ce3c601] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x870081) [0x7fbb8ce29081] --- tt::runtime::ttnn::LayoutConverter::handleHostInputNoLayoutTypecast(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertHostTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::toLayout(tt::runtime::Tensor, tt::runtime::Device, tt::runtime::Layout) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::pjrt::LoadedExecutableInstance::Execute(PJRT_LoadedExecutable_Execute_Args*) --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1ada) [0x7fbb99482ada] --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1b3b) [0x7fbb99482b3b] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x8a99cc) [0x7fbb9ea9a9cc] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7743f53) [0x7fbba5934f53] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691d719) [0x7fbba4b0e719] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691ecd0) [0x7fbba4b0fcd0] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767f55) [0x7fbb9e958f55] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767d9d) [0x7fbb9e958d9d] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x77153f8) [0x7fbba59063f8] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0xc4) [0x558e3541d574] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0xc1) [0x558e35432691] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x558e3543610b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bcf61) [0x7fbb9e9adf61] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bb4ff) [0x7fbb9e9ac4ff] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x558e3541d61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x558e3543262c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_MakeTpCall+0x25b) [0x558e3541e39b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x68ce) [0x558e3541899e] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x558e3541d61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x558e3543262c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x558e3543610b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x558e35412790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x558e354139ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x558e35412790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x558e354139ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x558e3541d61d]
test_example_model_only_kwargs.test_example_model_inference: tests/jax/models/example_model/only_kwargs/test_example_model_only_kwargs.py#L77
RuntimeError: TT_THROW @ /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir/third_party/tt-metal/src/tt-metal/ttnn/cpp/ttnn/tensor/tensor.hpp:317: tt::exception info: Cannot get the device from a tensor with host storage backtrace: --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x6a8bf8) [0x7fbb8cc61bf8] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885eb4) [0x7fbb8ce3eeb4] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885bbe) [0x7fbb8ce3ebbe] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x885a26) [0x7fbb8ce3ea26] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883b13) [0x7fbb8ce3cb13] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x883601) [0x7fbb8ce3c601] --- /__w/tt-xla/tt-xla/third_party/tt-mlir/src/tt-mlir-build/lib/SharedLib/libTTMLIR.so(+0x870081) [0x7fbb8ce29081] --- tt::runtime::ttnn::LayoutConverter::handleHostInputNoLayoutTypecast(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertHostTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::LayoutConverter::convertTensorLayout(tt::tt_metal::Tensor const&, std::optional<std::variant<std::reference_wrapper<tt::tt_metal::v0::IDevice>, std::reference_wrapper<tt::tt_metal::distributed::MeshDevice> > >) --- tt::runtime::ttnn::toLayout(tt::runtime::Tensor, tt::runtime::Device, tt::runtime::Layout) --- tt::runtime::ttnn::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::runtime::submit(tt::runtime::Device, tt::runtime::Binary, unsigned int, std::vector<tt::runtime::Tensor, std::allocator<tt::runtime::Tensor> > const&) --- tt::pjrt::LoadedExecutableInstance::Execute(PJRT_LoadedExecutable_Execute_Args*) --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1ada) [0x7fbb99482ada] --- /__w/tt-xla/tt-xla/build/src/tt/pjrt_plugin_tt.so(+0x2f1b3b) [0x7fbb99482b3b] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x8a99cc) [0x7fbb9ea9a9cc] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7743f53) [0x7fbba5934f53] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691d719) [0x7fbba4b0e719] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x691ecd0) [0x7fbba4b0fcd0] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767f55) [0x7fbb9e958f55] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x767d9d) [0x7fbb9e958d9d] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x77153f8) [0x7fbba59063f8] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0xc4) [0x558e3541d574] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0xc1) [0x558e35432691] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x558e3543610b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bcf61) [0x7fbb9e9adf61] --- /__w/tt-xla/tt-xla/venv/lib/python3.10/site-packages/jaxlib/xla_extension.so(+0x7bb4ff) [0x7fbb9e9ac4ff] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0x122) [0x558e35436172] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x558e3541d61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x558e3543262c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_MakeTpCall+0x25b) [0x558e3541e39b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x68ce) [0x558e3541899e] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x8af) [0x558e3541297f] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_FastCallDictTstate+0x16d) [0x558e3541d61d] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyObject_Call_Prepend+0x5c) [0x558e3543262c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x29d464) [0x558e3553a464] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(PyObject_Call+0xbb) [0x558e3543610b] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x558e35412790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x558e354139ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x6c0) [0x558e35412790] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x18db) [0x558e354139ab] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x2b60) [0x558e35414c30] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyFunction_Vectorcall+0x7c) [0x558e3542838c] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(_PyEval_EvalFrameDefault+0x5632) [0x558e35417702] --- /__w/tt-xla/tt-xla/venv/bin/python3.10(+0x1984d1) [0x558e354354d1]
test_constant.test_constant_zeros[shape0]: tests/jax/ops/test_constant.py#L15
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Executable expected parameter 0 of size 8192 but got buffer with incompatible size 4096
test_constant.test_constant_zeros[shape1]: tests/jax/ops/test_constant.py#L15
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Executable expected parameter 0 of size 8 but got buffer with incompatible size 4
test_constant.test_constant_ones[shape0]: tests/jax/ops/test_constant.py#L23
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Executable expected parameter 0 of size 8192 but got buffer with incompatible size 4096
test_constant.test_constant_ones[shape1]: tests/jax/ops/test_constant.py#L23
jaxlib.xla_extension.XlaRuntimeError: INVALID_ARGUMENT: Executable expected parameter 0 of size 8 but got buffer with incompatible size 4
build-and-test / build-and-run-tests (n150, run)
Process completed with exit code 1.
build-and-test / build-and-run-tests (n300, run)
Process completed with exit code 1.
pre-commit / pre-commit
The `python-version` input is not set. The version of Python currently in `PATH` will be used.

Artifacts

Produced during runtime
Name Size
test-reports-n150-
9.34 KB
test-reports-n300-
9.49 KB