Support for filtering extension catalog returned from a registry by offerings #34516

aloubyansky · 2023-07-04T11:46:40Z

This is based on the assumption that platform member extension catalogs may include offerings metadata in their extension catalogs, that would be a list of offering names they are a part of.

For example, a member extension catalog could look like:

{
  "id" : "org.acme.platform:acme-baz-bom-quarkus-platform-descriptor:1.0.1:json:1.0.1",
  "platform" : true,
  "bom" : "org.acme.platform:acme-baz-bom::pom:1.0.1",
  "quarkus-core-version" : "1.1.2",
  "extensions" : [ {
    "name" : "acme-baz",
    "artifact" : "org.acme.platform:acme-baz::jar:1.0.1",
    "origins" : [ "org.acme.platform:acme-baz-bom-quarkus-platform-descriptor:1.0.1:json:1.0.1" ]
  } ],
  "metadata" : {
    "offerings" : [ "baz", "core" ],

and the .quarkus/config.yaml could be configured as

registries:
- registry.acme.org:
    offerings:
    - "core"
    - "foo"

offering names configured in platform member extension catalogs

aloubyansky · 2023-07-04T11:47:19Z

FYI @maxandersen @tqvarnst

gastaldi · 2023-07-04T14:20:55Z

Although I don't not very fond of the term offerings being used as a filter, the implementation makes sense. How about calling it keywords or something similar?

aloubyansky · 2023-07-04T14:40:16Z

keywords would be too generic in this case. We already have keywords in the extension metadata.
This one is about enabling products/offerings in which to look for extensions with possibly provided keywords by the user, such as orm, jdbc, etc

maxandersen · 2023-07-04T16:35:05Z

This way we can add any kind of metadata matching?

aloubyansky · 2023-07-04T16:38:13Z

Not sure what you mean @maxandersen. Extension matching isn't touched by this PR. Only the "origins" are, i.e. here we are allowing users to control the BOMs in which to look for extensions.

maxandersen · 2023-07-04T16:42:22Z

I'm just not fond of coupling it that hard to boms.

Would have been nice to express things like redhat-support: "supported" or other extension level criterias.

If we don't have that we 100% are tied to BOM granularity.

aloubyansky · 2023-07-04T17:04:38Z

There are two things that end up in projects: BOMs and extension artifacts as dependencies.

Which means whatever way we choose we will be looking for both types of things to add to projects.

We could let "offerings" in the extension metadata drive BOM imports. Although, from the BOM perspective, it'd be the same, we might even consider collecting the "offering" metadata from all the extensions at the BOM generation time and recording the summary in the catalog to be able to check quickly whether a given BOM is a part of a specific offering. In which case it will look exactly as in this PR.

aloubyansky · 2023-07-04T17:07:58Z

Also we do have "redhat-support: supported" already at extension level. Could you elaborate further what you are think of?

maxandersen · 2023-07-04T17:09:24Z

Btw should it be "redhat-offerings" to be consistent in approach to "*-support" ?

aloubyansky · 2023-07-04T17:32:46Z

I see it as a general purpose feature not specific to RH.

maxandersen · 2023-07-04T17:43:41Z

independent-projects/tools/registry-client/src/main/java/io/quarkus/registry/Constants.java

@@ -20,4 +20,6 @@ public interface Constants {
    String JSON = "json";

    String LAST_UPDATED = "last-updated";
+
+    String OFFERINGS = "offerings";


Why do we care about the hardcoded key? Could it not be any key in metadata?

For the same reason we have other "keys" for other values.

How do you imagine it working otherwise?

That we don't hard code the key and make it wherever users put in their config?

It would still have to be under some key, which one?

correct - a recent case for example with microsft and azure extensions - they want to be able to express they support the extension but we would not want that to be mingled in with redhat's own notion of support.

Would Azure re-publish RHBQ?

The example you gave is exactly hardcoding of a support provider-specific keys in both: json model and implementation of the tools handling those keys.
I don't see it as a good but a bad design decision.
It makes it impossible to implement non-vendor-specific tools that could display all support options.

They know that *-support is for support and they be grouped that way.

If they were all in one key called support you would instead have to prefix every value with I.e. redhat.

I actually didn't realize we used property name expressions to represent categories. It would have been cleaner to have e.g. support as a parent and then nested support provider key.

Yes but we Also wanted it to possible to capture in a hashmap and not have to repeatedly change the typed apis across different versions.

quarkus-bot · 2023-07-04T18:49:52Z

Failing Jobs - Building `3e66f64`

Status	Name	Step	Failures	Logs	Raw logs
⌛	JVM Tests - JDK 11	`Build`	⚠️ Check →	Logs	Raw logs
✔️	JVM Tests - JDK 17
⌛	JVM Tests - JDK 17 Windows	`Build`	⚠️ Check →	Logs	Raw logs
✖	JVM Tests - JDK 19	`Build`	Failures	Logs	Raw logs
✔️	Maven Tests - JDK 11
⌛	Maven Tests - JDK 11 Windows	`Build`	⚠️ Check →	Logs	Raw logs

Failures

⚙️ JVM Tests - JDK 19 #

- Failing: extensions/reactive-pg-client/deployment 
! Skipped: extensions/hibernate-reactive/deployment extensions/panache/hibernate-reactive-panache-common/deployment extensions/panache/hibernate-reactive-panache-kotlin/deployment and 14 more

📦 extensions/reactive-pg-client/deployment

✖ Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:exec (docker-prune) on project quarkus-reactive-pg-client-deployment: Command execution failed.

gastaldi · 2023-07-04T22:00:51Z

...ools/devtools-testing/src/test/java/io/quarkus/devtools/project/create/NoOfferringsTest.java

+    private static Map<ArtifactKey, Set<ArtifactCoords>> toExtensionMap(ExtensionCatalog catalog) {
+        final Map<ArtifactKey, Set<ArtifactCoords>> extensionMap = new HashMap<>(catalog.getExtensions().size());
+        for (Extension e : catalog.getExtensions()) {
+            final Set<ArtifactCoords> boms = new HashSet<>(e.getOrigins().size());
+            for (ExtensionOrigin o : e.getOrigins()) {
+                boms.add(o.getBom());
+            }
+            extensionMap.put(e.getArtifact().getKey(), boms);
+        }
+        return extensionMap;
+    }


This would be useful as an utility method inside ExtensionCatalog

Also, the test class name has an extra r, just noticed 😉

There is also the possibility of writing it using Streams, but the actual is okay:

return catalog.getExtensions().stream() .collect(Collectors.toMap( e -> e.getArtifact().getKey(), e -> e.getOrigins().stream().map(ExtensionOrigin::getBom).collect(Collectors.toUnmodifiableSet())) );

maxandersen · 2023-07-11T20:48:11Z

I see it as a general purpose feature not specific to RH.

Yes. Thus why limit it and not allow for specialization?

aloubyansky · 2023-07-11T21:45:40Z

I see it as a general purpose feature not specific to RH.

Yes. Thus why limit it and not allow for specialization?

offering or whatever we call it at the end isn't RH-specific. Otherwise, which requirement specifically are you trying to satisfy?

Support for filtering extension catalog returned from a registry by

3e66f64

offering names configured in platform member extension catalogs

quarkus-bot bot added area/devtools Issues/PR related to maven, gradle, platform and cli tooling/plugins area/platform Issues related to definition and interaction with Quarkus Platform labels Jul 4, 2023

aloubyansky added the triage/backport? label Jul 4, 2023

gastaldi approved these changes Jul 4, 2023

View reviewed changes

maxandersen requested changes Jul 4, 2023

View reviewed changes

gastaldi reviewed Jul 4, 2023

View reviewed changes

gsmet added the triage/backport-3.2 label Aug 9, 2023

gsmet added the triage/backport-3.3? label Sep 6, 2023

gsmet removed the triage/backport-3.3? label Oct 3, 2023

gsmet added triage/backport? and removed triage/backport-3.4? labels Oct 11, 2023

gsmet added the triage/backport-3.5? label Nov 15, 2023

gsmet added the triage/backport-3.8 label Jan 17, 2024

gsmet removed triage/backport-3.5 labels Apr 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for filtering extension catalog returned from a registry by offerings #34516

Support for filtering extension catalog returned from a registry by offerings #34516

aloubyansky commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

gastaldi commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

maxandersen commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

maxandersen commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

maxandersen commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

maxandersen Jul 4, 2023

aloubyansky Jul 4, 2023

aloubyansky Jul 4, 2023

maxandersen Jul 4, 2023

aloubyansky Jul 4, 2023

maxandersen Jul 13, 2023

aloubyansky Jul 13, 2023

maxandersen Jul 13, 2023

aloubyansky Jul 13, 2023

maxandersen Jul 14, 2023

quarkus-bot bot commented Jul 4, 2023

gastaldi Jul 4, 2023

gastaldi Jul 4, 2023

gastaldi Jul 4, 2023 •

edited

Loading

maxandersen commented Jul 11, 2023

aloubyansky commented Jul 11, 2023

Support for filtering extension catalog returned from a registry by offerings #34516

Are you sure you want to change the base?

Support for filtering extension catalog returned from a registry by offerings #34516

Conversation

aloubyansky commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

gastaldi commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

maxandersen commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

maxandersen commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

maxandersen commented Jul 4, 2023

aloubyansky commented Jul 4, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

quarkus-bot bot commented Jul 4, 2023

Failing Jobs - Building 3e66f64

Failures

⚙️ JVM Tests - JDK 19 #

📦 extensions/reactive-pg-client/deployment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gastaldi Jul 4, 2023 • edited Loading

Choose a reason for hiding this comment

maxandersen commented Jul 11, 2023

aloubyansky commented Jul 11, 2023

Failing Jobs - Building `3e66f64`

gastaldi Jul 4, 2023 •

edited

Loading