Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flashing of GAP8 sometimes stops at 3-4% #105

Open
krichardsson opened this issue Dec 7, 2022 · 14 comments
Open

Flashing of GAP8 sometimes stops at 3-4% #105

krichardsson opened this issue Dec 7, 2022 · 14 comments
Labels
bug Something isn't working

Comments

@krichardsson
Copy link
Contributor

Sometimes the flashing process is stopping around 3-4% when flashing a binary to the GAP8 from the cfloader.
I have not investigated this deeply, but it seems as if the GAP8 is blocking for some reason. From a STM fw point of view, buffers are pushed to CPX, and the expected behavior is that the ESP passes them on to the GAP8 where they should be written to flash. The first incoming buffer will also trigger an erase of the first flash page.

3-4% is most likely the amount of data that fits into the CPX buffers between the STM and GAP8, that is the GAP8 is not consuming (writing data to flash) as expected. When the CPX buffers are full, the STM can not write any more which blocks the memory mapping task. When the memory mapping task stops sending back ACK packets to the python lib, the memory mapping write in the python lib stops. This is finally what stops the cfloader.

We recently added asserts in the STM fw that are triggered if a write to the GAP8 takes too long, one of them should reboot the Crazyflie when this happens.
It can be a bit tricky to actually get the assert information out in this scenario, but there are two options

  1. Connect to the CF with the python client and click the "Assert Info" button in the console log
  2. Set enable_console_lo to True in the deck flashing code
@whoenig
Copy link
Contributor

whoenig commented Dec 7, 2022

I have one AI-deck where this is very consistently the issue. This AI deck has the latest ESP firmware (flashed via cfclient update) and gap8-bootloader (flashed via JTAG). When flashing, cfloader hangs for a while and the Crazyflie reboots. The assert information (via cfclient Assert info) is surprisingly an unrelated assert (note that the firmware does not crash using the 2022.09 release):

SYS: Assert failed at ../src/hal/src/radiolink.c:164

@krichardsson
Copy link
Contributor Author

Ah, the magic radiolink.c:164 assert!
We have seen this in relation to all sorts of problems and I think the general interpretation should be: "something in the STM is using too much CPU power". We get this assert when NRF is pushing radio packets faster than the STM is consuming them. I don't think the problem actually is radio link related, it is just a symptom.

Interesting that you get this assert though, I would have expected it to be one of the new asserts in the aideck code.

@krichardsson
Copy link
Contributor Author

We just got this problem in the stability lab, but the assert was at line 136 in aideck.c

@krichardsson
Copy link
Contributor Author

@whoenig do you still see this problem?

@whoenig
Copy link
Contributor

whoenig commented Nov 8, 2023

No - we have only used the AI deck with the JTAG adapter lately.

@whoenig
Copy link
Contributor

whoenig commented Feb 6, 2024

Had this again today with an assert in stream_buffer.c (unfortunately didn't write down the exact line). There was no way out, except of flashing the GAP8 directly via JTAG.

@knmcguire
Copy link
Member

hmm this is strange... We need to find time at one point to stabilize the aideck again but it is not really on our priority list right now. We now say for everybody that a jtag programmer is crucial.

@gemenerik
Copy link
Member

gap8 flashing stops at ~6% when having a flow deck attached

@williamleong
Copy link
Contributor

williamleong commented Mar 20, 2024

I find that the success rate seems to be higher if we remove the multiranger and flow deck.

@gemenerik
Copy link
Member

gemenerik commented May 30, 2024

bitcraze/crazyflie-firmware#1381 fixes OTA flashing for aideck+flowdeck+multiranger

@gemenerik
Copy link
Member

Still happens for aideck+lighthouse. Assert aideck.c ln 136

@gemenerik
Copy link
Member

The lighthouse deck seems to add capacitance on VCOM, causing the GAP8 to not reset correctly

@gemenerik
Copy link
Member

As a workaround, we tried a 4.5 kOhm resistor between VCOM and ground. This configuration drops the voltage quickly enough to successfully flash with the lighthouse attached. Note that we are using long wires, which might influence the outcome. Users may experiment with resistances around this range to achieve similar results.

@gemenerik
Copy link
Member

gemenerik commented Jul 10, 2024

One neat way of doing this would be to solder the resistor between the relevant pads on a battery holder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants