Hardware model v2

[mbolivar-nordic]

Parent issue

This issue is part of a larger issue: #51833

Proposed introduction plan and timeline

IMPORTANT NOTE: During the Architecture WG meetings where this topic was discussed it was agreed to avoid the complex transition plan below and instead adopt a direct approach where the switch between v1 and v2 happens in a single Pull Request. A collaboration branch is being used to collaboratively prepare the state of the tree once we switch to v2.
This implies that, starting with Zephyr 3.7.0, only the new hardware model will be supported. Any out-of-tree boards will need to transition to the v2 model as soon as they adopt Zephyr 3.7.0.
This decision was later ratified during a TSC call of January 3rd, 2024.

1. Before Zephyr 3.6.0 is released, PR #50305 is merged into the main tree
- This does not mean that the current hardware model is deprecated, since only a small subset of SoCs and boards (enough to have a set representative examples) will have been ported from v1 to v2
2. Starting right after the merge:
- New SoCs in will only be accepted in v2 format
- New boards based on SoCs that are in v2 format will need to be submitted in v2 format as well
- Boards that use existing SoCs may be accepted in the current (v1) format. This will give users a chance to get boards in that were in development before the hardware model v2 PR was merged
3. Starting right after 3.6.0
- New boards will only be accepted in v2 format. Note that converting most boards from v1 to v2 only requires creating a .yaml file, shuffling files around a bit and making small changes to the Kconfig options
- Existing SoCs and boards will start transitioning in the tree from the current model to v2 progressively
- Important note: whenever an SoC transitions from v1 to v2 in the upstream project, out-of-tree boards based on that SoC that are not updated to v2 will not work anymore. However, users of the out-of-tree board(s) have an alternative path forward if they decide not to port their board to v2: they can copy the existing v1 SoC files to their out-of-tree repo and continue to work as before (they will of course need to backport any SoC fixes themselves)
4. Zephyr 3.7.0 LTS may ship with a combination of v1 and v2 SoCs and boards, since it is possible that not all of them will have transitioned before its release. Much of the SoC and board code and files are very similar between the two revisions, so this should not be a big issue for backporting fixes, even with SoCs and boards that transition to v2 after LTS release
5. Hardware model v1 support will be deprecated with the release of Zephyr 3.8.0 (4.0.0?)
6. Hardware model v1 support will be removed right after the release of Zephyr 3.10.0 (4.2.0?)
- Nordic commits to maintaining any infrastructure related to hardware model v2 until this point

Is your enhancement proposal related to a problem?

There is no way to define a single board directory or SoC directory for AMP SoCs that have CPUs with different architectures.

This is because of the artificial file system hierarchy restriction that requires SoC and board porters to provide the architecture via the file system, e.g.:

• <SOC_ROOT>/soc/<ARCH>/<SOC_FAMILY>/...
• <BOARD_ROOT>/boards/<ARCH>/<BOARD_NAME>/...

This hardware model has served us well for years when SoCs with Zephyr support were simple microcontrollers with a single CPU or single SMP cluster. The industry is increasingly moving towards heterogeneous AMP SoCs, however, and Zephyr needs to keep up if we wish to continue to serve our users well.

Describe the solution you'd like

A backwards-compatible "hardware model v2". That is, a file system hierarchy standard supported by Zephyr's build system that lets you:

• support an SoC family etc. in a single directory regardless of the number of architectures (1, 2, ...) involved
• support a board in a single directory for such SoCs

This would look something like this (details in the ... TBD):

• <SOC_ROOT>/soc/v2/<VENDOR>/<SOC_FAMILY>/...
• <BOARD_ROOT>/boards/v2/<VENDOR>/<BOARD_NAME>/...

That is, we allow both in-tree and out-of-tree users to define v2 subdirectories of both soc and boards. These respectively contain SoC and board definitions which support as many simultaneous architectures as are required.

Hardware model v2 will support both "multi-architecture" and "single-architecture" SoCs to allow for an eventual migration path if the project ever wishes to deprecate the current hardware model. However, deprecation and migration is a non-goal for this initial effort: as stated above, the current model has served us well for years, and there seems to be no reason to force satisfied users away from it.

The <VENDOR> path components shown will be namespace controlled using the existing vendor-prefixes.txt file format already supported by Zephyr's devicetree tooling. Users may already provide their own vendor-prefixes.txt files in individual modules, so this mechanism gives us full control over what we support in the zephyr repository itself and allows for extensibility.

Describe alternatives you've considered

None exist (prove me wrong 😄 ).

System devicetree support postponed

We originally expected that this would allow use of system devicetree, an extension to the devicetree specification described in #51830.

However, integrating system devicetree support is not a blocker for implementing this issue (note that the below issue description never mentioned system devicetree specifically). Support for system DT has been postponed until after the initial hardware model v2 implementation.

Open issues

Those issue must be resolved before merging collab-hwm branch to zephyr/main

• Oot arch for HWMv2
  ref: Introduction of board and SoC scheme v2. #50305 (comment)
• TF-M does not work (merge conflict at address 0x0 for nrf5340dk)
  ref: Introduction of board and SoC scheme v2. #50305 (review)
• Board / board identifier reconfiguration in project
  Details: Configuring for a board which has a soc without the soc will give an error (expected), if you then re-configure the project with the soc without cleaning, an oddity happens: 1) a warning is emitted that the old board name will be used instead as you need to clear the build directory, 2) the identifier from the new value is actually used, 3) the "identifier" cmake message is not output
  Fixed: cmake: cache BOARD_IDENTIFIER to preserve it between CMake invocations #67426
• Support multiple boards in single folder, use-cases intel_adsp , arduino_nano_33
• Building for a v1 build with a variant gives strange results, best tried with samples/subsys/mgmt/mcumgr/smp_svr with a non-converted board e.g. nrf52840dk_nrf52840/blah
  Fixed: cmake: fail when board identifier is applied in legacy hw model #67433
• West board completion handler, the completion handler uses board_list script, ref: https://discord.com/channels/720317445772017664/1194671118422913114/1201082341842505858
• Have twister sanitize board names before folder creation, twister: normalize platform name when storing files/data #69338
• Snippets do not work in hwmv2 snippets: Initial HWMv2 support #69355
• Issue with base board name overlay files now being used on boards which are variants that in hwmv1 were not applied to, causing issues and failing tests hwmv2: Fix rpi_pico w variant #69025 (comment)
• Files are referenced in ported boards that do not exist hwmv2: Board documentation that references files that do not exist #69065
• (issue does not originate with hwmv2 but becomes apparent) boards that have revisions that do not have a .yaml file for each revision need to have one added so that tests run against these boards and issues are found rather than having boards that might not even build in-tree. Fix: hwmv2: Add and use board identifiers for boards with revisions #69568

In order to merge back to main, a checklist has been created:
#69575

Enhancement issues which exists in zephyr/main but has been identified during porting.
Not required to be solved before merging collab-hwm branch to zephyr/main

See #69546

Board porting where porting is in progress / completed

See issue here for board / SoC porting progress: #67682

Slides presented at arch wg meeting

Zephyr_arch_wg_HWMv2.pptx

Note, some adjustments has been made later in the HW model v2 progress based on community feedback.

Change history

• 2023-12: Added section with known issues for collab-hwm branch
• 2023-11: add transition proposal
• 2023-05-05: note that system DT will not be a requirement
• 2022-10-31: initial proposal describing top level hierarchies and namespacing. Many details TBD.

[tejlmand]

Slides presented at arch wg meeting
Zephyr_arch_wg_HWMv2.pptx

Note, some adjustments has been made later in the HW model v2 progress based on community feedback.

[carlescufi]

Merged here: #69607