test_nlp_create_q_and_kv_heads_separate_test[2-4096-96-8-8-64-2-8-True-BFLOAT8_B] RuntimeError: TT_THROW @ /work/tt_metal/impl/program/program.cpp:49: tt::exception info: Failed to generate binaries for transpose_wh_sharded TT_THROW @ /work/tt_metal/jit_build/build.cpp:598: tt::exception info: trisc0 build failed backtrace: --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/_ttnn.cpython-310-x86_64-linux-gnu.so(+0x10790c0) [0x7f95215d50c0] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x31b9f9) [0x7f951fcc49f9] --- tt::tt_metal::JitBuildState::compile_one(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, tt::tt_metal::JitBuildSettings const*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) const --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x31fa43) [0x7f951fcc8a43] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x323a8a) [0x7f951fccca8a] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x32262d) [0x7f951fccb62d] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x321b39) [0x7f951fccab39] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x3219ca) [0x7f951fcca9ca] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9591bc6ac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9591c57a04] backtrace: --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x21bcb9) [0x7f951fbc4cb9] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x20bb5d) [0x7f951fbb4b5d] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x31fa43) [0x7f951fcc8a43] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x323a8a) [0x7f951fccca8a] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x32262d) [0x7f951fccb62d] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x321b39) [0x7f951fccab39] --- /home/ubuntu/actions-runner/_work/tt-metal/tt-metal/.local/lib/python3.10/site-packages/ttnn/build/lib/libtt_metal.so(+0x3219ca) [0x7f951fcca9ca] --- /lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f9591bc6ac3] --- /lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f9591c57a04]

No files were found with the provided path: ~/run-log/20250226115702_sys_logs.tar. No artifacts will be uploaded.

Hugepages service not found. Using old rc.local method

[DEPRECATION] This action is deprecated. Please migrate to reading the Docker image from the pipeline.

Disk usage is 55 %

Touching and printing out SMI info

Attempting to reset card(s). Sleeping first

tt-smi reset was successful