Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[stable-v2.8] clean cherry-pick of xtensa-build-zephyr --deployable #8858

Merged
merged 11 commits into from
Feb 13, 2024

Conversation

marc-hb
Copy link
Collaborator

@marc-hb marc-hb commented Feb 12, 2024

I had to pick a few more commits than I expected but it all makes sense in the end.

The .toml changes may seem surprising but they are here because they are IPC4-related changes and --deployable is 100% about IPC4 too. The git conflicts surprised me at first but they were actually telling me to do the right thing and pick those .toml changes for a good IPC4 reason; not just because of some accidental textual conflicts.

The original commit messages were pretty good, please read them before asking any question.

Thanks to those, absolutely zero git conflicts in this PR. 100% pure cherry-pick.

I could have omitted the last 3 xtensa-build-zephyr.py commits but they are very well tested in the main branch, zero-risk and could really catch last-minute release issues - it's always been their purpose.

marc-hb and others added 11 commits February 12, 2024 19:20
The main branch hasn't supported IPC3 for tgl for a long time.

Follow-up to commit a17c282 ("xtensa-build-zephyr: remove ipc
option")

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
(cherry picked from commit 79e5080)
Rename:
- tgl-cavs.toml   to tgl.toml
- tgl-h-cavs.toml to tgl-h.toml

Remove the IPC3/IPC4 switch added by commit 6f71808
("xtensa-build-zephyr.py: add ipc4 build support for tgl")

This brings back consistency which is required for the .toml
split (thesofproject#8490)

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
(cherry picked from commit ded019b)
Let west/sign.py find the appropriate .toml file.

This prepares for .toml modularization thesofproject#8490

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
(cherry picked from commit d661aad)
The IPC4_CONFIG_OVERLAY is no longer used, remove it.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
(cherry picked from commit 4d0da98)
The following aliases are missing:

tgl: adl-n, rpl
tgl-h: rpl-s
mtl: arl

Update the aliases list accordingly.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
(cherry picked from commit 38d447f)
Print platform alias information to reduce the need for guessing when
trying to figure out what platform supports what platform.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
(cherry picked from commit 7e42c43)
The default firmware file path is standardized among vendors as follows:
IPC3
    /lib/firmware/VENDOR/sof/
    ├── community
    │   └── sof-PLAT.ri
    ├── dbgkey
    │   └── sof-PLAT.ri
    └── sof-PLAT.ri
IPC4
    /lib/firmware/VENDOR/sof-ipc4/
    └── PLAT
        ├── community
        │   └── sof-PLAT.ri
        ├── dbgkey
        │   └── sof-PLAT.ri
        └── sof-PLAT.ri\n

Currently the binaries created by the build can only be used for direct
deployment on IPC3 platforms but if one builds different vendor firmwares
the files have to be manually picked and sorted out.
We have two flags: --fw-naming and --use-platform-subdir which can be
played with but still not going to produce deployable build.

Introduce a new flag: --deployable-build
With the flag specified all other modificators are going to be ignored and
the build will do the 'right thing' to create a directory structure which
can be deployed as it is to the target's firmware directory.

To achieve this several changes needed:
PlatformConfig:
- drop the name member and replace it with a vendor string
- add a flag to indicate IPC4 platforms
  Later a new option can be added if needed for platforms which can be
  IPC3 or IPC4

Ignore fw-naming and use-platform-subdir in case of deployable build. The
options will be reset to their default in case they are changed.

symlink_or_copy extended to handle relative symlinks when the target and
link is not under the same directory.

The --deployable-build is disabled by default, it has to be enabled to
create deployable build for now.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
(cherry picked from commit 1adb1e4)
ARL-S is using the same debugkey/community key binary as MTL.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@linux.intel.com>
(cherry picked from commit 98cc1d3)
Not sure why I thought it would be complicated; I should have done this
much sooner.

Before:

```
build-sof-staging
|-- sof
|   +-- intel
|       +-- sof-ipc4
|           |-- adl
|           |   +-- community
|           |       +-- sof-adl.ri
|           |-- adl-n
|           |   +-- community
|           |       +-- sof-adl-n.ri
|           |-- arl
|           |   +-- community
|           |       +-- sof-arl.ri

```

After:

```
build-sof-staging
|-- sof
|   +-- intel
|       +-- sof-ipc4
|           |-- adl
|           |   +-- community
|           |       +-- sof-adl.ri  -> ../../tgl/community/sof-tgl.ri
|           |-- adl-n
|           |   +-- community
|           |       +-- sof-adl-n.ri  -> ../../tgl/community/sof-tgl.ri
|           |-- arl
|           |   +-- community
|           |       +-- sof-arl.ri  -> ../../mtl/community/sof-mtl.ri

```

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
(cherry picked from commit c8eb216)
Better late than never... zero functional change.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
(cherry picked from commit e24ccd2)
From experience, no one scrolls up and looks at warnings that are not in
the current screen.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
(cherry picked from commit f4ce8b5)
@marc-hb
Copy link
Collaborator Author

marc-hb commented Feb 12, 2024

Abusing github labels to get expedited review, sorry about that.

@marc-hb marc-hb added bug Something isn't working as expected P1 Blocker bugs or important features labels Feb 12, 2024
@marc-hb marc-hb marked this pull request as ready for review February 12, 2024 19:50
@kv2019i kv2019i requested a review from abonislawski February 12, 2024 19:52
@marc-hb marc-hb mentioned this pull request Feb 12, 2024
@marc-hb
Copy link
Collaborator Author

marc-hb commented Feb 12, 2024

Compared to #8855, the lack of --deployable flag and the firmware build is now fixed in https://sof-ci.01.org/sofpr/PR8858/build2667/build

However topology2 deployment fails later for both ACE and CAVS, something like this (jenkins job /testflow_new/17543/)

11:52:37  [2024-02-12 19:52:37 UTC][SOF-SH] Deploy tplg https://sof-storage.ostc.intel.com/build/sof/79/e7d0344149f94d21529e84882647b66836a536e979e6140ec2d23505e543f3/12189//topology2/target/development/ to /lib/firmware/intel/development.
11:52:44  rsync: [sender] change_dir "/home/ubuntu/build/sof/79/e7d0344149f94d21529e84882647b66836a536e979e6140ec2d23505e543f3/12189//topology2/target/development/" failed: No such file or directory (2)
11:52:44  rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1338) [sender=3.2.7]
11:52:44  [2024-02-12 19:52:43 UTC][SOF-SH] Failed to download tplg folder from:[https://sof-storage.ostc.intel.com/build/sof/79/e7d0344149f94d21529e84882647b66836a536e979e6140ec2d23505e543f3/12189//topology2/target/development/.](https://sof-storage.ostc.intel.com/build/sof/79/e7d0344149f94d21529e84882647b66836a536e979e6140ec2d23505e543f3/12189//topology2/target/development/.%1B[0m)

@marc-hb
Copy link
Collaborator Author

marc-hb commented Feb 12, 2024

Quoting @kv2019i

... so we'd need also the tplg move PRs, i.e. 65e4c1f and friends

@marc-hb
Copy link
Collaborator Author

marc-hb commented Feb 12, 2024

Adding a partial backport of 65e4c1f "merge sof-avs-tplg and sof-ace-tplg" (partially backported in commit 1fd55c3) , topology2 deployment is fixed and tests are running.

Topologies are building fine in https://github.com/thesofproject/sof/actions/runs/7878724943/job/21497551245?pr=8858 and in https://sof-ci.01.org/sofpr/PR8858/build2670/build

HOWEVER there are many test failures in https://sof-ci.01.org/sofpr/PR8858/build2671/devicetest/index.html and https://sof-ci.01.org/sofpr/PR8858/build2672/devicetest/index.html (internal runs 37855 and 37856) most likely because I didn't backport the other parts of 65e4c1f. That commit did way too many things in a single commit: renaming and splitting files + making code changes at the same time. When you add the usual "stable vs main" divergence on top, it's practically impossible to backport. It must be re-implemented again on stable-v2.8 and that's not something I'm capable of doing. This requires @ujfalusi .

@marc-hb marc-hb changed the title [stable-v2.8] clean cherry-pick of xtensa-build-zephyr --deployable [stable-v2.8] clean cherry-pick of xtensa-build-zephyr --deployable + dirty topology2 backport Feb 12, 2024
@marc-hb marc-hb force-pushed the deployable-stable2.8 branch from 1fd55c3 to c33329d Compare February 12, 2024 23:22
@marc-hb
Copy link
Collaborator Author

marc-hb commented Feb 12, 2024

Adding a partial backport of 65e4c1f "merge sof-avs-tplg and sof-ace-tplg" (partially backported in commit 1fd55c3) , CI deployment is fixed and tests are running.
...
HOWEVER there are many test failures in ...

I just removed that partial topology2 backport of mine so this PR can be merged as is tomorrow when Europe wakes up. This PR is not enough to make the tests pass but it's very clean and makes CI go FURTHER and BETTER. Once merged, it will provide a great base for a proper backport of 65e4c1f. One problem at a time.

@marc-hb marc-hb changed the title [stable-v2.8] clean cherry-pick of xtensa-build-zephyr --deployable + dirty topology2 backport [stable-v2.8] clean cherry-pick of xtensa-build-zephyr --deployable Feb 12, 2024
@marc-hb
Copy link
Collaborator Author

marc-hb commented Feb 12, 2024

https://sof-ci.01.org/sof-pr-viewer/#/build/PR8858/build13558346 failed because QB is also unable to test older branches (in this particular case it's trying and failing to use the --no-deployable flag). Ignore QB results in stable branches.

In June 2023 I requested in internal ticket PTS-45980 that QB stops wasting very precious bandwidth trying and failing to test older branches. @lrudyX , @wszypelt could you help withPTS-45980?

@kv2019i kv2019i merged commit 87e9055 into thesofproject:stable-v2.8 Feb 13, 2024
75 of 84 checks passed
@marc-hb marc-hb deleted the deployable-stable2.8 branch February 13, 2024 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working as expected P1 Blocker bugs or important features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants