Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rosdep: Add missing entries for OpenEmbedded #43942

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

robwoolley
Copy link

This adds missing entries for OpenEmbedded to the rosdep database.

This was generated by automatically querying the OpenEmbedded Layer Index (https://layers.openembedded.org/)

In cases where a layer was not specified, this is because it exists in multiple ROS 2 specific layers (eg meta-ros2-humble, iron, jazzy, rolling, etc). A future change may be required if meta-ros consolidates these recipes into the meta-ros2 sublayer.

Package name:

  • ace
  • alsa-utils
  • apache2
  • apparmor
  • aravis
  • at-spi2-core
  • autossh
  • babeltrace
  • backward-cpp
  • can-utils
  • castxml
  • catch2
  • ccid
  • chrony
  • clang
  • cli11
  • cpprest
  • cunit
  • dbus
  • devilspie2
  • dfu-util
  • docker-compose
  • dpkg
  • espeak
  • ethtool
  • fcgi
  • fltk
  • fluidsynth
  • fping
  • freetype
  • fsharp
  • ftgl
  • gcc-arm-none-eabi
  • gdal
  • geos
  • gnome-terminal
  • gperftools
  • gphoto2
  • gpsd
  • graphviz
  • gtkmm
  • gts
  • hidapi
  • htop
  • imagemagick
  • intltool
  • iw
  • jack
  • jasper
  • json-glib
  • leptonica
  • libao
  • libb64
  • libbsd
  • libcereal
  • libconfig
  • libedit
  • libepoxy
  • libevdev
  • libevent
  • libev
  • libexif
  • libmnl
  • libnl
  • libnotify
  • libopus
  • librdkafka
  • libsodium
  • libspatialite
  • libssh2
  • libssh
  • libunwind
  • libvorbis
  • libxft
  • libxinerama
  • libxtst
  • libzip
  • lighttpd
  • lttng-modules
  • lua
  • lz4
  • mercurial
  • mpfr
  • mpg123
  • mpich
  • msr-tools
  • muparser
  • nanopb
  • nasm
  • octave
  • openexr
  • openni2
  • openni
  • openscenegraph
  • php
  • pigz
  • popt
  • postfix
  • powertop
  • procps
  • psutils
  • qtwebkit
  • rapidjson
  • rt-tests
  • setserial
  • shaderc
  • simde
  • smartmontools
  • sound-theme-freedesktop
  • sox
  • speex
  • sysbench
  • systemd
  • tclap
  • tcsh
  • tesseract
  • tiff
  • tmux
  • tree
  • vim
  • vorbis-tools
  • wireshark
  • x264
  • xdg-utils
  • yasm
  • zenity
  • zip
  • zsh
  • zziplib

Purpose of using this:

This will allow the Superflore build tool to correctly resolve these dependencies when generating BitBake recipes.

Links to Distribution Packages

@robwoolley robwoolley requested a review from a team as a code owner December 18, 2024 00:22
@github-actions github-actions bot added the rosdep Issue/PR is for a rosdep key label Dec 18, 2024
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sending a pull request to ROS distro!

This is an automated tool that helps check your pull request for correctness.
This tool checks a number of attributes associated with your ROS package and generates a report that helps our reviewers merge your pull request in a timely fashion. Here are a few things to consider when sending adding or updating a package to ROS Distro.
ROS Distro includes a very helpful CONTRIBUTING.md file that we recommend reading if it is your first time submitting a package.
Please also read the ROS Distro review guidelines which summarizes this release process.

ROS Distro Considerations

Package Considerations

Having your package included in a ROS Distro is a badge of quality, and we recommend that package developers strive to create packages of the highest quality. We recommend package developers review the following resources before submitting their package.

Need Help?

Please post your questions to Robotics Stack Exchange or refer to the #infra-help channel on our Discord server.


For changes related to rosdep:

  • ✅ Platforms for new rosdep rules are valid

For changes related to yamllint:

  • ✅ All new lines of YAML pass linter checks

@robwoolley
Copy link
Author

robwoolley commented Dec 19, 2024

Based on the CI tests there are 3 cases:

  1. xcursor should be libxcursor and acptiool should be acpitool

  2. Having recipes without a layer will be necessary if any recipes are specific to a distro sublayer (as stated in the commit message). That is true today, but I will need to check whether this is actually necessary or not.

  3. OE has virtual targets for cases where a dependency can be satisfied by multiple options. The CI test script will need to be fixed to allow this.

I will investigate and submit amendments.

@@ -39,6 +40,7 @@ acpitool:
fedora: [acpitool]
gentoo: [sys-power/acpitool]
nixos: [acpitool]
openembedded: [acptiool@meta-oe]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

acpitool does exist. Going to layers.openembedded.org and searching for "pn:acpitool layer:meta-oe" (without quotes) finds it. Perhaps a glitch in the script or network.

While acpitool does exist, acptiool does not 🙃

Suggested change
openembedded: [acptiool@meta-oe]
openembedded: [acpitool@meta-oe]

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @cottsay . I was staring right at that and couldn't see ti. ;)

Also thanks for writing the CI check, I just figured out how to run it on the command-line and am extending it for virtual support.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it supports virtual packages already, but definitely not @layer omission. Can you describe how that should behave?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The virtual targets are supposed to be a bit of a wildcard which seems opposed to the purpose of the check. The OE layerindex is meant to be inclusive and has obscure and unmaintained layers that I can't reasonably support. In another thread you had suggested that the documentation be updated to a list of "supported" OpenEmbedded layers.

What if I were to provide a list for the config.yaml of supported OE layers? Then we were to add something like this pseudocode to enumerate_layer_index_packages():

        provides = recipe.get('provides', '')

        for prov in provides.split():
            if not prov:
                continue
            if not layer in allowed_layers(): ## NEW
                continue ## NEW
            yield PackageEntry(f'{prov}', pv, recipe_url, pn, pn)  ## Modified to remove @{layer}

Where the intent is to permit virtual/foobar globally provided that a supported layer has a recipe that lists it in the provides?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have two initial thoughts:

  1. Can you still specify a virtual provision with @layer as well? Instead of modifying the PackageEntry, would it be more appropriate to yield an additional PackageEntry without the @layer?
  2. Should we allow entries from layers other than the "blessed" layers if the rule specifies the @layer specifically?

It seems like you want to treat these virtual provisions differently from the actual package names, which we don't do for any other platforms. I'd just like to make sure we're properly reflecting the semantics of the package manager here. If they work the same way there, then we should follow that behavior.

We should consider whether curating a list of blessed layers is really what we want. Other platforms draw the line of "official" and "supported" packages on a per-repository basis. If the official OE layerindex is a mixed bag, it would make sense to re-draw the line in this context by declaring a list of blessed layers. If that's not really the case, maybe we should view the check as a measure of validity, and allow the person performing rosdep reviews to decide whether the rule (the layer in particular) is appropriate.

In terms of how this would look in config.yaml, I think this would need to be a modification of the !layer_index_url to pass additional data. Most of the other platforms take only a URL for specifying the repository, but !deb_base_url takes an additional string. We could do something similar here, too. Beyond that, I'd like to avoid adding any OE-specific configurations to the rosdep_repo_check subsystem as a whole. The other platforms color inside the lines and it would be a shame if we needed to break the abstraction to support this.

Copy link
Author

@robwoolley robwoolley Dec 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I spent some time digging into the original intent behind the OpenEmbedded support for rosdep. I verified that from the very beginning the intent wasn't to track packages, but to "map the ROS package names to the bitbake recipes". To your point, this means that we are already treating OpenEmbedded differently from any other platform as we aren't talking about package management.

This is important because the metadata in base.yaml gets used to generate recipes that specify dependencies on other recipes. For a variety of reasons the package names may not be the same. This means that we may encounter problems down the road if we start to try to use rosdep to install system dependencies (either on the host or target).

The document they wrote for the generation scheme mentions that " is only a hint: a recipe can be in different layers in different OpenEmbedded release series". For this reason, I think it is okay if we specify a recipe in one layer (eg. ffmpeg@openembedded-core) that may get overridden by another layer.

I prefer your suggestion of letting the reviewer do the check instead of trying to make a list of blessed layers. However, I do think that this puts extra responsibility on the reviewer to ensure that the layer in question is in good standing. It seems to imply that the reviewer must be familiar with OpenEmbedded. This is also something that will need to be monitored as recipes are removed and layers become unmaintained (but remain in the layerindex).

Regarding the virtual targets, it is the "right thing to do" but it may be unnecessary just for ocl-icd:

virtual/opencl-icd:
- opencl-icd-loader_2024.05.08.bb@meta-oe
- ocl-icd_2.3.2.bb@meta-oe
- imx-gpu-viv_5.0.11.p8.4-hfp@meta-fsl-arm
- imx-gpu-viv_6.4.11.p2.8-aarch32.bb@meta-freescale
- imx-gpu-viv_6.4.11.p2.8-aarch64.bb@meta-freescale
- ocl-icd_2.2.11.bb@meta-refkit-computervision
- rockchip-libmali.bb@meta-orangepi-r1plus

The alternatives are specific to a board that we aren't actively supporting or testing with. Since oci-icd is in meta-oe and matches the one used in Ubuntu, I can resubmit using ocl-icd@meta-oe instead of virtual/opencl-icd.

Adding support for virtual targets becomes more important when I try to get Gazebo / Ignition listed. I will have to revisit this discussion then when I have more context to work with.

Thanks for the help! I will work to revise the commit and update the PR once I have moved box2d, range-v3, xsimd, xtensor from meta-ros2-* into meta-ros2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rosdep Issue/PR is for a rosdep key
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants