Unable to reprogram FPGA and run NEW kernels in combination with XRT, affects both non-DFX and DFX base platforms #74

vmayoral · 2021-05-10T11:51:40Z

Context

This ticket's part of a debugging effort and connected to past issues including #69 #70 #71 and #72.

While developing kernels it's common to test (also in hw target) incremental variants of it. So far, unfortunately, I haven't been able to do so. Only the first kernel produced (the first time the sd card is introduced is the hardware) remains. I haven't been able to successfully change this. The current issues has been validated in the following hardware and base platform combinations:

Hardware	Base platform
ZCU102	xilinx_zcu102_base_202020_1
ZCU102	xilinx_zcu102_base_dfx_202020_1
ZCU104	xilinx_zcu104_base_202020_1

A simple way to reproduce this is to build and test variants (silly ones, simple modifications of the kernel) of https://github.com/Xilinx/Vitis-Tutorials/tree/master/Getting_Started/Vitis. An exemplary flow to reproduce this issue is as follows:

Follow https://github.com/Xilinx/Vitis-Tutorials/blob/master/Getting_Started/Vitis/Part4.md and build the sd_card.img
Flash raw image into SD card and boot
Run app.exe obtaining results coherent to the vadd.xclbin kernel (whatever they are)
Modify vadd.cpp (source code) introduce a small (or big, up to you) change
Rebuild kernel with Vitis v++ compiler
Scp kernel to hardware as vadd.xclbin.new for DFX platforms (or just build the whole image complete again with v++ -p for non-DFX platforms)
Run app.exe vadd.xclbin.new

The expected observation is that result remains fixed to 3. and not related to the changes introduced in 4., 5. and 6.. A past issue in XRT identified a similar behavior Xilinx/XRT#2747.

Results so far

NOTE: I'm using Vitis suite 2020.2 on top of Ubuntu 20.04.
NOTE 2: Ignore paths, they're specific to my development machines.

xilinx_zcu102_base_202020_1

Regardless of what I do the result's always the same which corresponds with the first kernel I flashed which modified https://github.com/Xilinx/Vitis-Tutorials/blob/master/Getting_Started/Vitis/example/src/vadd.cpp#L15 to:

$$out = in1 + 2 * in2$$

cd /home/xilinx/Vitis-Tutorials/Getting_Started/Vitis/example/zcu102/hw; rm -r package/ _x/ vadd.x* app.exe a.xclbin v++*

# configure env.
source /tools/Xilinx/Vitis/2020.2/settings64.sh
export PLATFORM_REPO_PATHS=/home/xilinx/xilinx_zcu102_base_202020_1
export ROOTFS=/home/xilinx/rootfs
source /home/xilinx/rootfs/sdk/environment-setup-aarch64-xilinx-linux

# build
${CXX} -Wall -g -std=c++11 ../../src/host.cpp -o app.exe -I/usr/include/xrt -lOpenCL -lpthread -lrt -lstdc++
v++ -c -t hw --config ../../src/zcu102.cfg -k vadd -I../../src ../../src/vadd.cpp -o vadd.xo
v++ -l -t hw --config ../../src/zcu102.cfg ./vadd.xo -o vadd.xclbin
v++ -p -t hw --config ../../src/zcu102.cfg ./vadd.xclbin --package.out_dir package --package.rootfs ${ROOTFS}/rootfs.ext4 --package.sd_file ${ROOTFS}/Image --package.sd_file xrt.ini --package.sd_file app.exe --package.sd_file vadd.xclbin --package.sd_file run_app.sh

# test acceleration
cd /media/sd-mmcblk0p1
export XILINX_XRT=/usr
./app.exe vadd.xclbin

root@zynqmp-common-2020_2:/media/sd-mmcblk0p1# ./app.exe vadd.xclbin
[  265.853107] [drm] Pid 1221 opened device
[  265.857065] [drm] Pid 1221 closed device
INFO: Found Xilinx Platform
[  265.863503] [drm] Pid 1221 opened device
[  265.869667] [drm] Pid 1221 closed device
[  265.873637] [drm] Pid 1221 opened device
INFO: Loading 'vadd.xclbin'
[  265.962660] [drm] zocl_xclbin_read_axlf The XCLBIN already loaded
[  265.962676] [drm] zocl_xclbin_read_axlf c82ababc-a138-4cb2-bfd8-595f8a4cd9b3 ret: 0
[  265.971245] [drm] bitstream c82ababc-a138-4cb2-bfd8-595f8a4cd9b3 locked, ref=1
[  265.978917] [drm] Reconfiguration not supported
[  265.990677] [drm] bitstream c82ababc-a138-4cb2-bfd8-595f8a4cd9b3 unlocked, ref=0
[  265.991808] [drm] bitstream c82ababc-a138-4cb2-bfd8-595f8a4cd9b3 locked, ref=1
[  266.005109] [drm] User buffer is not physical contiguous
Error: Result mismatch
i = 0 CPU result = 2349 Device result = 4281
TEST FAILED
... (continues)

xilinx_zcu102_base_dfx_202020_1

cd /home/xilinx/Vitis-Tutorials/Getting_Started/Vitis/example/zcu102/hw; rm -r package/ _x/ vadd.x* app.exe a.xclbin v++*

# configure env.
source /tools/Xilinx/Vitis/2020.2/settings64.sh
export PLATFORM_REPO_PATHS=/home/xilinx/xilinx_zcu102_base_dfx_202020_1
export ROOTFS=/home/xilinx/rootfs
source /home/xilinx/rootfs/sdk/environment-setup-aarch64-xilinx-linux

# build
${CXX} -Wall -g -std=c++11 ../../src/host.cpp -o app.exe -I/usr/include/xrt -lOpenCL -lpthread -lrt -lstdc++
v++ -c -t hw --config ../../src/zcu102_dfx.cfg -k vadd -I../../src ../../src/vadd.cpp -o vadd.xo
v++ -l -t hw --config ../../src/zcu102_dfx.cfg ./vadd.xo -o vadd.xclbin
v++ -p -t hw --config ../../src/zcu102_dfx.cfg ./vadd.xclbin --package.out_dir package --package.rootfs ${ROOTFS}/rootfs.ext4 --package.sd_file ${ROOTFS}/Image --package.sd_file xrt.ini --package.sd_file app.exe --package.sd_file vadd.xclbin --package.sd_file run_app.sh

# test acceleration
cd /media/sd-mmcblk0p1
export XILINX_XRT=/usr
./app.exe vadd.xclbin

root@zynqmp-common-2020_2:/media/sd-mmcblk0p1# ./app.exe vadd.xclbin
[  265.853107] [drm] Pid 1221 opened device
[  265.857065] [drm] Pid 1221 closed device
INFO: Found Xilinx Platform
[  265.863503] [drm] Pid 1221 opened device
[  265.869667] [drm] Pid 1221 closed device
[  265.873637] [drm] Pid 1221 opened device
INFO: Loading 'vadd.xclbin'
[  265.962660] [drm] zocl_xclbin_read_axlf The XCLBIN already loaded
[  265.962676] [drm] zocl_xclbin_read_axlf c82ababc-a138-4cb2-bfd8-595f8a4cd9b3 ret: 0
[  265.971245] [drm] bitstream c82ababc-a138-4cb2-bfd8-595f8a4cd9b3 locked, ref=1
[  265.978917] [drm] Reconfiguration not supported
[  265.990677] [drm] bitstream c82ababc-a138-4cb2-bfd8-595f8a4cd9b3 unlocked, ref=0
[  265.991808] [drm] bitstream c82ababc-a138-4cb2-bfd8-595f8a4cd9b3 locked, ref=1
[  266.005109] [drm] User buffer is not physical contiguous
Error: Result mismatch
i = 0 CPU result = 2349 Device result = 4281
TEST FAILED

(gets stucked in here)

xilinx_zcu104_base_202020_1

cd /home/xilinx/Vitis-Tutorials/Getting_Started/Vitis/example/zcu104/hw
# configure env.
source /tools/Xilinx/Vitis/2020.2/settings64.sh
export PLATFORM_REPO_PATHS=/home/xilinx/xilinx_zcu104_base_202020_1
# export ROOTFS=/home/xilinx/rootfs
export ROOTFS=/home/xilinx/petalinux-xilinx-zcu104-2020.2/images/linux
source /home/xilinx/rootfs/sdk/environment-setup-aarch64-xilinx-linux

# build
${CXX} -Wall -g -std=c++11 ../../src/host.cpp -o app.exe -I/usr/include/xrt -lOpenCL -lpthread -lrt -lstdc++
v++ -c -t hw --config ../../src/zcu104.cfg -k vadd -I../../src ../../src/vadd.cpp -o vadd.xo
v++ -l -t hw --config ../../src/zcu104.cfg ./vadd.xo -o vadd.xclbin
v++ -p -t hw --config ../../src/zcu104.cfg ./vadd.xclbin --package.out_dir package --package.rootfs ${ROOTFS}/rootfs.ext4 --package.sd_file ${ROOTFS}/Image --package.sd_file xrt.ini --package.sd_file app.exe --package.sd_file vadd.xclbin --package.sd_file run_app.sh

# run
...
TEST PASSED

I haven't been able to get tests to fail after this. Regardless of the modifications in the kernels.

Additional tests performed

Reinstalled Vitis suite and PetaLinux completely in a newly formatted machine with Ubuntu 20.04. Same result.

Temporary fix

A user proposed a temporary fix flashing manually the FPGA which I have not been able to reproduce Xilinx/XRT#2747 (comment). Here're my results when attempting this:

xilinx_zcu104_base_202020_1

# in the workstation
xclbinutil --dump-section BITSTREAM:RAW:vadd.bit -i vadd.xclbin
# then scp-it to embedded

# in the embedded platform
cd /media/sd-mmcblk0p1
export XILINX_XRT=/usr
fpgautil -b ./vadd.bit
root@xilinx-zcu104-2020_2:/media/sd-mmcblk0p1# ./app.exe
[  158.257308] [drm] Pid 1028 opened device
[  158.261267] [drm] Pid 1028 closed device
INFO: Found Xilinx Platform
[  158.267677] [drm] Pid 1028 opened device
[  158.273834] [drm] Pid 1028 closed device
[  158.277794] [drm] Pid 1028 opened device
INFO: Loading 'vadd.xclbin'
[  158.430057] [drm] zocl_xclbin_read_axlf The XCLBIN already loaded
[  158.430073] [drm] zocl_xclbin_read_axlf 030235fc-c6ab-4195-83b3-00e2e0860b11 ret: 0
[  158.438503] [drm] bitstream 030235fc-c6ab-4195-83b3-00e2e0860b11 locked, ref=1
[  158.446209] [drm] Reconfiguration not supported
[  158.457973] [drm] bitstream 030235fc-c6ab-4195-83b3-00e2e0860b11 unlocked, ref=0

(gets stucked in here)

Further debugging this issue led to a kernel panic:

root@xilinx-zcu104-2020_2:~# cd /media/sd-mmcblk0p1
root@xilinx-zcu104-2020_2:/media/sd-mmcblk0p1# export XILINX_XRT=/usr
root@xilinx-zcu104-2020_2:/media/sd-mmcblk0p1# fpgautil -b ./vadd.bit
[   42.264337] fpga_manager fpga0: writing vadd.bit to Xilinx ZynqMP FPGA Manager
Time taken to load BIN is 9302.000000 Milli Seconds
BIN FILE loaded through FPGA manager successfully
root@xilinx-zcu104-2020_2:/media/sd-mmcblk0p1#
root@xilinx-zcu104-2020_2:/media/sd-mmcblk0p1#
root@xilinx-zcu104-2020_2:/media/sd-mmcblk0p1# ls
BOOT.BIN  boot.scr	     profile_summary.csv  timeline_trace.csv  xrt.ini
Image	  init.sh	     run_app.sh		  vadd.bit
app.exe   platform_desc.txt  system.dtb		  vadd.xclbin
root@xilinx-zcu104-2020_2:/media/sd-mmcblk0p1# gdb ./app.exe
GNU gdb (GDB) 8.3.1
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "aarch64-xilinx-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./app.exe...
(gdb) r
Starting program: /media/sd-mmcblk0p1/app.exe
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
[   81.754011] Unable to handle kernel NULL pointer dereference at virtual address 00000000000007f3
[   81.762787] Mem abort info:
[   81.765563]   ESR = 0x96000004
[   81.768601]   EC = 0x25: DABT (current EL), IL = 32 bits
[   81.773895]   SET = 0, FnV = 0
[   81.776933]   EA = 0, S1PTW = 0
[   81.780057] Data abort info:
[   81.782922]   ISV = 0, ISS = 0x00000004
[   81.786741]   CM = 0, WnR = 0
[   81.789695] user pgtable: 4k pages, 48-bit VAs, pgdp=0000000877e81000
[   81.796116] [00000000000007f3] pgd=0000000000000000
[   81.800978] Internal error: Oops: 96000004 [#1] SMP
[   81.805838] Modules linked in: dmaproxy(O) mali(O) zocl(O)
[   81.811319] CPU: 0 PID: 977 Comm: gdb Tainted: G           O      5.4.0-xilinx-v2020.2 #1
[   81.819483] Hardware name: ZynqMP ZCU104 RevC (DT)
[   81.824258] pstate: a0000085 (NzCv daIf -PAN -UAO)
[   81.829039] pc : xilinx_dpdma_chan_err_task+0xe0/0x1b8
[   81.834162] lr : xilinx_dpdma_chan_err_task+0x1c/0x1b8
[   81.839281] sp : ffff800010003e40
[   81.842580] x29: ffff800010003e40 x28: 0000000000000000
[   81.847875] x27: 0000000000000038 x26: 0000000000000060
[   81.853169] x25: 0000000000000040 x24: 00000000000000e0
[   81.858464] x23: ffff00087a37f140 x22: ffff8000110c1b00
[   81.863759] x21: 0000000000000000 x20: ffff000879678c80
[   81.869054] x19: ffff0008796eae80 x18: 0000000000000000
[   81.874348] x17: 0000000000000000 x16: 0000000000000000
[   81.879643] x15: 0000000000000000 x14: 5f3874534e497373
[   81.884938] x13: 656363615f4d5f39 x12: 617461645f796e41
[   81.890233] x11: 5f3974534b4e5a5f x10: 0000000000000040
[   81.895528] x9 : ffff8000110f5368 x8 : ffff8000110f5360
[   81.900822] x7 : ffff00087f777858 x6 : ffff800010003dc0
[   81.906117] x5 : ffff80086e6bf000 x4 : 0000000000000000
[   81.911412] x3 : 0000000000000001 x2 : ffff00087682dcc8
[   81.916707] x1 : ffff00087682dc80 x0 : 03000000000006f3
[   81.922002] Call trace:
[   81.924435]  xilinx_dpdma_chan_err_task+0xe0/0x1b8
[   81.929210]  tasklet_action_common.isra.0+0xcc/0x178
[   81.934164]  tasklet_action+0x24/0x30
[   81.937811]  __do_softirq+0x118/0x22c
[   81.941456]  irq_exit+0x98/0xc0
[   81.944581]  __handle_domain_irq+0x64/0xb8
[   81.948659]  gic_handle_irq+0x5c/0xb8
[   81.952305]  el1_irq+0xb8/0x140
[   81.955433]  __arch_copy_to_user+0x1a0/0x218
[   81.959694]  copy_page_to_iter+0xe8/0x338
[   81.963687]  generic_file_read_iter+0x384/0xb28
[   81.968210]  ext4_file_read_iter+0x38/0x50
[   81.972297]  new_sync_read+0xe4/0x170
[   81.975941]  __vfs_read+0x2c/0x40
[   81.979239]  vfs_read+0xb8/0x190
[   81.982451]  ksys_read+0x68/0xf0
[   81.985663]  __arm64_sys_read+0x18/0x20
[   81.989484]  el0_svc_common.constprop.0+0x68/0x160
[   81.994265]  el0_svc_handler+0x6c/0x88
[   81.997997]  el0_svc+0x8/0xc
[   82.000865] Code: f8448c40 eb00005f d1040000 540000a0 (f9408000)
[   82.006952] ---[ end trace 1b932a3dbbd74e07 ]---
[   82.011555] Kernel panic - not syncing: Fatal exception in interrupt
[   82.017891] SMP: stopping secondary CPUs
[   82.021799] Kernel Offset: disabled
[   82.025269] CPU features: 0x0002,20002004
[   82.029260] Memory Limit: none
[   82.032302] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---

xilinx_zcu102_base_dfx_202020_1

# in the workstation
xclbinutil --dump-section BITSTREAM:RAW:vadd.bit -i vadd.xclbin
# then scp-it to embedded

# in the embedded platform
cd /media/sd-mmcblk0p1
export XILINX_XRT=/usr
fpgautil -b ./vadd.bit

root@zynqmp-common-2020_2:/media/sd-mmcblk0p1# fpgautil -b ./vadd.bit
[  430.136806] fpga_manager fpga0: writing vadd.bit to Xilinx ZynqMP FPGA Manager
[  445.466900] mmc0: Timeout waiting for hardware interrupt.
[  445.472292] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[  445.478714] mmc0: sdhci: Sys addr:  0x00000000 | Version:  0x00001002
[  445.485137] mmc0: sdhci: Blk size:  0x00007200 | Blk cnt:  0x00000000
[  445.491560] mmc0: sdhci: Argument:  0x0042c800 | Trn mode: 0x00000027
[  445.497983] mmc0: sdhci: Present:   0x01ff0000 | Host ctl: 0x0000001f
[  445.504406] mmc0: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
[  445.510830] mmc0: sdhci: Wake-up:   0x00000000 | Clock:    0x00000207
[  445.517253] mmc0: sdhci: Timeout:   0x00000009 | Int stat: 0x00000002
[  445.523676] mmc0: sdhci: Int enab:  0x03ff008b | Sig enab: 0x03ff008b
[  445.530099] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000001
[  445.536522] mmc0: sdhci: Caps:      0x35ecc881 | Caps_1:   0x00002007
[  445.542945] mmc0: sdhci: Cmd:       0x0000193a | Max curr: 0x00000000
[  445.549369] mmc0: sdhci: Resp[0]:   0x00000900 | Resp[1]:  0x003b377f
[  445.555792] mmc0: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00000c00
[  445.562214] mmc0: sdhci: Host ctl2: 0x00000001
[  445.566643] mmc0: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x000000087926d2c0
[  445.573759] mmc0: sdhci: ============================================
[  456.306892] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[  456.312799] rcu: 	0-...0: (1 GPs behind) idle=926/0/0x1 softirq=4120/4121 fqs=2596
[  456.320444] rcu: 	1-...0: (1 GPs behind) idle=e82/1/0x4000000000000000 softirq=2437/2438 fqs=2596
[  456.329391] 	(detected by 3, t=5257 jiffies, g=8457, q=24)
[  456.334858] Task dump for CPU 0:
[  456.338071] swapper/0       R  running task        0     0      0 0x0000000a
[  456.345109] Call trace:
[  456.347548]  __switch_to+0x1c4/0x288
[  456.351108]  0xffff8000110d98a8
[  456.354232] Task dump for CPU 1:
[  456.357443] sh              R  running task        0  1257   1253 0x00000022
[  456.364482] Call trace:
[  456.366915]  __switch_to+0x1c4/0x288
[  456.370472]  0xfffffe0021cd95c0

(board hangs in here)

Questions

I was surprised that this (by "this" I mean the development flows explained above in Context) couldn't work in non-DFX platforms. Is this expected to work in non-DFX platforms? If not, what's the proposal for reprogramming the FPGA with XRT flows? addressed in Unable to reprogram FPGA and run NEW kernels in combination with XRT, affects both non-DFX and DFX base platforms #74 (comment)
I was very surprised that I couldn't get this flow working even with DFX base platform. Can you confirm this should work with DFX platforms?
~~Since none of the above worked and the proposed fix in a past issue only led to more issues, how can I manually reprogram FPGAs for using them with XRT?~~ addressed in Unable to reprogram FPGA and run NEW kernels in combination with XRT, affects both non-DFX and DFX base platforms #74 (comment)

I'm currently blocked with this (and have been blocked by it for more than a week already) so It'd be great if someone could help with it or suggest and alternative route. Anything for ZCU102 or ZCU104 would do.
Thanks!

The text was updated successfully, but these errors were encountered:

vmayoral · 2021-05-10T12:01:45Z

It'd be awesome if I could get your thoughts on this @rwarmstr, @randyh62 or someone else.

vmayoral · 2021-05-10T20:57:07Z

All right, after a few good hints (thanks @randyh62) and insisting on power-booting hardware platforms I conclude the following for non-DFX platforms:

Rebuilding an image completely (including rebuilting the kernel, etc) as described in the Vitis-Tutorials DOES get the kernel to reprogram the FPGA successfully. This requires:
- modify kernel
- v++ -c -t hw --config ../../src/zcu102.cfg -k vadd -I../../src ../../src/vadd.cpp -o vadd.xo
- v++ -l -t hw --config ../../src/zcu102.cfg ./vadd.xo -o vadd.xclbin
- v++ -p -t hw --config ../../src/zcu102.cfg ./vadd.xclbin --package.out_dir package --package.rootfs ${ROOTFS}/rootfs.ext4 --package.sd_file ${ROOTFS}/Image --package.sd_file xrt.ini --package.sd_file app.exe --package.sd_file vadd.xclbin --package.sd_file run_app.sh
- copy image into into sd, boot
- launch kernel
Rebuilding the kernel, copying it, rebooting and launching again doesn't work, picks up older kernel:
- modify kernel
- v++ -c -t hw --config ../../src/zcu102.cfg -k vadd -I../../src ../../src/vadd.cpp -o vadd.xo
- v++ -l -t hw --config ../../src/zcu102.cfg ./vadd.xo -o vadd.xclbin
- scp kernel file vadd.xclbin into running image
- reboot
- launch kernel
Rebuilding the kernel, copying it, launching it, rebooting and launching again doesn't work, picks up older kernel:
- modify kernel
- v++ -c -t hw --config ../../src/zcu102.cfg -k vadd -I../../src ../../src/vadd.cpp -o vadd.xo
- v++ -l -t hw --config ../../src/zcu102.cfg ./vadd.xo -o vadd.xclbin
- scp kernel file vadd.xclbin into running image
- launch kernel
- reboot
- launch kernel
Rebuilding the kernel, copying it, shuting down, booting up and launching again doesn't work, picks up older kernel:
- modify kernel
- v++ -c -t hw --config ../../src/zcu102.cfg -k vadd -I../../src ../../src/vadd.cpp -o vadd.xo
- v++ -l -t hw --config ../../src/zcu102.cfg ./vadd.xo -o vadd.xclbin
- scp kernel file vadd.xclbin into running image
- launch kernel
- reboot
- launch kernel

Summarizing:

Approach	Description	Result
1	recreating whole sd image	works
2	scp-ing new kernel, rebooting, launch	doesn't
3	scp-ing new kernel, launching, rebooting, launch	doesn't
4	scp-ing new kernel, power-off, booting, launch	doesn't

Why's only option 1. working? Can someone explain what are the processes behind reprogramming the FPGA in non-DFX platforms? Do I need to recreate the whole raw image (for the sd card) with v++ -p every single time? where's this being reflected files-wise (i.e. what's changing in the file system for the FPGA to be reprogrammed?).

vmayoral · 2021-05-11T09:27:56Z

All right, further testing things out (and through good feedback provided by Randy, Wes, Terry and others) I fount out that for non-DFX base platforms what matters isn't really the *.xclbin file since this seems to be ignored completely. Instead, the corresponding FPGA is programmed by the bitstream contained into the BOOT.BIN file.

This then leads to the following fifth approach:
5. Rebuilding the kernel, packaging it for generation of BOOT.BIN, copying BOOT.BIN to boot partition,rebooting up and launching again does work:

modify kernel
v++ -c -t hw --config ../../src/zcu102.cfg -k vadd -I../../src ../../src/vadd.cpp -o vadd.xo
v++ -l -t hw --config ../../src/zcu102.cfg ./vadd.xo -o vadd.xclbin
v++ -p -t hw --config ../../src/zcu102.cfg ./vadd.xclbin --package.out_dir package --package.no_image
scp BOOT.BIN into boot partition
reboot
launch kernel

Summarizing:

Approach	Description	Result
1	recreating whole sd image	works
2	scp-ing new kernel, rebooting, launch	doesn't
3	scp-ing new kernel, launching, rebooting, launch	doesn't
4	scp-ing new kernel, power-off, booting, launch	doesn't
5	scp-ing BOOT.bin, reboot, launch	works

This addresses questions above 1 and 3, but 2 remains:

I was surprised that this (by "this" I mean the development flows explained above in Context) couldn't work in non-DFX platforms. Is this expected to work in non-DFX platforms? If not, what's the proposal for reprogramming the FPGA with XRT flows?
I was very surprised that I couldn't get this flow working even with DFX base platform. Can you confirm this should work with DFX platforms?
~~Since none of the above worked and the proposed fix in a past issue only led to more issues, how can I manually reprogram FPGAs for using them with XRT?~~

Premduth · 2021-08-18T16:18:32Z

@imrickysu can you look at this?

dj-park · 2022-04-29T19:07:49Z

Hi @vmayoral,
I know this is an year ago, but what was your conclusion on the 2nd point?(did you get it work?) I think in the recent version of user guide(https://docs.xilinx.com/r/en-US/ug1393-vitis-application-acceleration/Configuring-the-PL-Kernel-in-DFX-Platforms-and-Non-DFX-Platform), your question regarding non-DFX platform is addressed, but isn't reloading vadd_new.xclbin in DFX platfrom supposed to work? Isn't that the point of DFX platform?

dj-park · 2022-04-29T22:05:33Z

In fact, I am experiencing the exact same issue.
For instance,
app_0.exe vadd_0.xclbin => test passed
reboot
app_1.exe vadd_1.xclbin => test passed

But if I don't reboot, for example,
app_0.exe vadd_0.xclbin => test passed
app_1.exe vadd_1.xclbin => hangs

mruizglz · 2022-06-09T04:57:22Z

I have exactly the same behaviour than @dj-park. I am using a ZCU102 (DFX) and 2021.1 tools. Is there any solution to this problem?

imrickysu · 2022-06-09T06:39:21Z

Hi @dj-park , @mruizglz , thanks for your feedback. I'll try to reproduce my end and follow up.

xerpi · 2023-04-08T00:19:53Z

If I'm not mistaken, when you target the real hardware (-t hw), a boot.bin file that contains the bitstream is created, which then you can copy to the boot partition on the SD to replace the bitstream loaded at boot.
Which would be equivalent when targeting the hardware emulator (-t hw_emu)?

shantirao · 2024-02-19T19:13:41Z

Still the case in 2023.2. Is there a way to put the PL in the xclbin file instead of the boot.bin file? Or maybe the FPGA has to boot before petalinux does, so there's no way to hot-swap the PL?

Premduth assigned imrickysu Aug 18, 2021

imrickysu added the Working on it label Aug 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to reprogram FPGA and run NEW kernels in combination with XRT, affects both non-DFX and DFX base platforms #74

Unable to reprogram FPGA and run NEW kernels in combination with XRT, affects both non-DFX and DFX base platforms #74

vmayoral commented May 10, 2021 •

edited

Loading

vmayoral commented May 10, 2021

vmayoral commented May 10, 2021 •

edited

Loading

vmayoral commented May 11, 2021

Premduth commented Aug 18, 2021

dj-park commented Apr 29, 2022 •

edited

Loading

dj-park commented Apr 29, 2022

mruizglz commented Jun 9, 2022

imrickysu commented Jun 9, 2022

xerpi commented Apr 8, 2023 •

edited

Loading

shantirao commented Feb 19, 2024

Unable to reprogram FPGA and run NEW kernels in combination with XRT, affects both non-DFX and DFX base platforms #74

Unable to reprogram FPGA and run NEW kernels in combination with XRT, affects both non-DFX and DFX base platforms #74

Comments

vmayoral commented May 10, 2021 • edited Loading

Context

Results so far

Additional tests performed

Temporary fix

Questions

vmayoral commented May 10, 2021

vmayoral commented May 10, 2021 • edited Loading

vmayoral commented May 11, 2021

Premduth commented Aug 18, 2021

dj-park commented Apr 29, 2022 • edited Loading

dj-park commented Apr 29, 2022

mruizglz commented Jun 9, 2022

imrickysu commented Jun 9, 2022

xerpi commented Apr 8, 2023 • edited Loading

shantirao commented Feb 19, 2024

vmayoral commented May 10, 2021 •

edited

Loading

vmayoral commented May 10, 2021 •

edited

Loading

dj-park commented Apr 29, 2022 •

edited

Loading

xerpi commented Apr 8, 2023 •

edited

Loading