west: Introduce run once commands (e.g. erase, recover) and deferred reset

[nordicjm]

Introduces additional fields to soc.yml or board.yml which can be used to specify west flash commands that should be grouped for multiple boards when using sysbuild, this allows usage of e.g. --erase to allow the flash to be erased once when flashing a set of images rather than once per image which would erase what had just been programmed. The format of this is the command then a list of boards which are part of the group whereby it will only be executed one for. Also supported is grouping boards to prevent resetting of the CPUs after programming when multiple images are being programmed until all images for the grouped cores have been programmed, at which point the CPU can be reset, this is ideal for multiple cores or things like TF-M.

TODO list:

• documentation

[carlescufi]

Why would you have an entry with a single board target like this? what's the point of saying "run once" for a single target?

[nordicjm]

You could be flashing multiple images to the network core, for example mcuboot, slot0 application and slot1 application, with this if you do west flash --recover then you get one recover and 3 flashes, without it you get 3 recovers and 3 flashes (thus a broken image). The run one actions only apply if there is an entry

[carlescufi]

OK, I see!

[gmarull]

a top-level "run_conce" key feels weird here, I'd suggest something like:

flashing:
  - singlerun:
  ...
  - anyfuturestuff:  

[gmarull]

(or flash.yml...)

[nordicjm]

This was a separate file initially but moved into these files because "of too many yaml files" concerns raised by others

[gmarull]

This was a separate file initially but moved into these files because "of too many yaml files" concerns raised by others

IMHO mixing files purpose is not a good idea either.

[tejlmand]

IMHO mixing files purpose is not a good idea either.

feel free to join the discussion: #50305 (comment)
I feel the same, multiple files can be easier to read understand if the are well named and organized, quote:

The fact that both files are in yaml doesn't mean there should be one huge yaml file containing everything.

But there was request to unify and have a single source for various meta data:
#50305 (comment)

[tejlmand]

Main issues I see with current approach relates to yaml file / schema:

• Multiple boards: entries seems to be creating a group, but it's not clear what a group is, and whether such a group should have a name. Should be made clearer.
• Misuse of yaml keys, so that instead of being a key, it actually is a value.
  Please use a key: value pair for such cases.
• board.yml seems to be able to reference board names which have no relation to the boards defined by the board.yml. Please ensure that only boards defined by the board.yml itself can be referenced in the commands sections. Similar for the soc.yml.

[tejlmand]

it has been mentioned in offline discussion that a reason for placing value as the key is simpler Python parsing.
I assume that comment referred to this code, but why is this code simpler than:

for command in t['run_once']:
    used_cmds.append(UsedFlashCommand(command.get('command'), command.get('boards))

To me, removing two for loops seems even simpler.

[tejlmand]

this makes the command to become a key in the yaml file, not a value.

As I see it, there are two main implecations of such approach

• The free form .* means there is effectively no schema check, as anything is allowed for a value.
• A : must suddenly follow the value.
  In yaml, a value is either after the colon, like:
  <key>: <value>
  or a list of values, like:

  my_list:
  - value1
  - value2
  - value3
  - ...
  

but you're creating a list of maps, but where the key in that list is actually the value.

I don't see why regular key: value mapping cannot be used, and thereby avoid tricks like this.

[nordicjm]

Unification with snippets

[tejlmand]

but not unified with boards.yml.

For example, the board.yml for nrf9160dk does:

board:
  name: nrf9160dk
  vendor: nordic
  socs:
  - name: nrf9160
    variants:
    - name: 'ns'
  - name: nrf52840

and not:

board:
  name: nrf9160dk
  vendor: nordic
  socs:
  - nrf9160:
      variants:
      - name: 'ns'
  - nrf52840:

The fact that snippet does this does not justify this principle in board or soc yaml files.

Especially not when this proposal introduces inconsistency on how things are done within the same yaml file.

[tejlmand]

See also comment here: #69748 (comment)

[tejlmand]

something to handle ?

[nordicjm]

No, hwmv1 so can be ignored

[tejlmand]

then please add a comment in code, for example:

Suggested change

	except FileNotFoundError:
	continue
	except FileNotFoundError:
	# Legacy boards don't have a board.yml or soc.yml.
	continue

[tejlmand]

IMHO mixing files purpose is not a good idea either.

feel free to join the discussion: #50305 (comment)
I feel the same, multiple files can be easier to read understand if the are well named and organized, quote:

The fact that both files are in yaml doesn't mean there should be one huge yaml file containing everything.

But there was request to unify and have a single source for various meta data:
#50305 (comment)

[tejlmand]

A soc.yml should really not know anything about board variants.

A board variant is defined by the board and could be named anything.

[nordicjm]

There needs to be a uniform naming scheme. And ns is going to be mandated by TFM anyhow

[tejlmand]

For ns, but there can be many other variant names.

Variant names, just like other board names, should not be defined by the soc.yml.

[nordicjm]

That's up to vendors if they want to change, and if they do they can add this into their board.yml file, it is not an issue

[57300]

I agree that the groups in soc.yml should only account for socs and cpuclusters, not whole board targets.

I can't imagine a use case where a board variant would be in a separate group from its parent soc/cpucluster anyway.
But if there is one, then further refinement should be possible in board.yml.

[carlescufi]

Architecture WG:

• @nordicjm explains the feature
• @henrikbrixandersen asks if this can be overridden from a board. The answer is yes, you can override whatever the default is for the SoC
• @nashif asks for other vendors to take a look so that we can ensure that the soc.yml doesn't get out of hand if we keep inserting
• @henrikbrixandersen suggests having this information in a snippet's yml file
• @nashif wants to generalize this so that it is applicable to other vendors' use cases

[nordicjm]

@fabiobaltieri @JordanYates is the PR acceptable? If so, can you approve please?

[fabiobaltieri]

@fabiobaltieri @JordanYates is the PR acceptable? If so, can you approve please?

To be totally honest I don't love it, it seems fairly complex, but I also don't really have any alternative to suggest so fine by me.

[carlescufi]

@nordicjm please rebase

[carlescufi]

As described here and here I am not a huge fan of the yaml format, but I see no reason not to move forward with this at this point.

[carlescufi]

Removing assignee since @tejlmand is opting out of this particular review.