Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Questions on @internal, @tags, and enum values #1737

Closed
david-perez opened this issue Apr 18, 2023 · 7 comments
Closed

Questions on @internal, @tags, and enum values #1737

david-perez opened this issue Apr 18, 2023 · 7 comments

Comments

@david-perez
Copy link
Contributor

I think the following two PRs introduce behavior that is not documented well:

These prompted the following questions.

1. Should @internal shapes automatically be filtered out by all tooling?

The documentation for the @internal trait reads:

Shapes marked with the internal trait are meant only for internal use. Tooling can use the internal trait to filter out shapes from models that are not intended for external customers.

As an example, a service team may wish to use a version of a model that includes features that are only available to internal customers within the same company, whereas clients for external customers could be built from a filtered version of the model.

(emphasis mine)

#1658 adds a protocol test to exclude returning enum values that have been marked with the @internal trait.

However, my reading of the documentation is that @internal is just a marker trait that carries no semantics — it is up to tooling to decide whether to filter out @internal-tagged shapes or not. Therefore, I would have expected that the user first needs to define a projection that explicitly filtered out @internal shapes for a server SDK to be built that would pass that protocol test; if the @internal shapes are not filtered out, then a server SDK should still include them in the validation error message.

2. Does @tags["internal"] imply @internal?

Both PRs above add tests such that enum values tagged (i.e. using the @tags trait or the tags property on the @enum trait) with the string "internal" are not included in the validation error message.

This seems to imply that @tags["internal"] implies @internal. However, I don't see this documented. I was under the impression that tags for a shape lived in a completely separate data plane that would not interact with that of Smithy traits, and that they would just boil down to a bag of string values whose meaning is entirely for user-defined tooling to decide.

3. Does the tags property on the @enum trait imply @tags?

The answer to this is obviously yes, but I think that the docs for the property:

Attaches a list of tags that allow the enum value to be categorized and grouped.

could be clearer, perhaps by having the word tags link to the trait @tags. As written, one could interpret the tags for an enum value to be completely independent of those defined using @tags.

@mtdowling
Copy link
Member

  1. You’re right, and I don’t understand the rationale for those tests either.
  2. No, the internal tag is an old convention used only inside Amazon. You don’t need to special case it. The related tests should also be reevaluated.
  3. The tags property of the enum trait is independent of the tags trait, since the enum trait doesn’t introduce members. You need to run model transforms to convert this trait to an enum shape. We could say it’s similar to that tags trait.

@david-perez
Copy link
Contributor Author

Thanks for the prompt reply!

The tags property of the enum trait is independent of the tags trait, since the enum trait doesn’t introduce members. You need to run model transforms to convert this trait to an enum shape. We could say it’s similar to that tags trait.

This is technically correct, but said model transform is part of the core Smithy library, which all code generators should apply, provided as changeStringEnumsToEnumShapes in ModelTransformer.java. This eventually delegates to ChangeShapeType.java which calls EnumShape.fromStringShape, where we see that the tags property on @enum gets converted to the @tags trait on enum shape:

https://github.com/awslabs/smithy/blob/f02f5ddcc7c585f8335683688dcbe9d5a2f7ebb9/smithy-model/src/main/java/software/amazon/smithy/model/shapes/EnumShape.java#L230-L232

So the end-user reading the documentation will just see them as equivalent when synthesizing their IDL v1 model.

@gosar
Copy link
Contributor

gosar commented Apr 20, 2023

We have revisited the changes made related to this. Answering your questions below with related changes we plan to make.

  1. The motivation for those changes: Services may have in-progress features (operations, members, etc.) in development/private-beta, that they do not want available in their public client SDKs. They can mark these as "internal" and have a projection that filters it out and client SDKs are built from that. The server needs to implement those features, so would use the full model (i.e., including "internal" parts of the model). If the feature is adding a new enum value (marked "internal", for now) and a customer makes a request with existing public operation where this enum is an input member, and uses an invalid value, we don't want the validation message to "leak" the "internal" enum value. We will update the docs to mention this use.

  2. Does @tags["internal"] imply @internal?

In general, no. But since we can't use @internal trait on the enum trait, using internal tag is the only way to indicate the same on enum trait. So as a special case, when converting string enum to enum shape, we will add @internal trait, so where needed (like SSDK validation message use case above) logic can rely on just @internal trait and not on internal tag.

  1. Does the tags property on the @enum trait imply @tags?

So the end-user reading the documentation will just see them as equivalent when synthesizing their IDL v1 model.

Kind of. Depends on the context of the end-user, like if you are in the context of transforming the string enum to enum shape. We will update the javadocs on the related transform/codegen classes. We are also missing docs on this transform here which we will fix. We will add some clarification to the enum trait docs for this.

Based on these, in regards to the protocol tests referenced, we will change them so that:

  • avoid leaking @internal enum members from enum shape - general case for internal enum in validation message
  • avoid leaking internal tagged enum values from enum trait - special case for enum trait
  • remove logic related to internal tagged members on enum shape

@gosar
Copy link
Contributor

gosar commented Apr 20, 2023

These changes are being made in
#1739
#1740
#1746

@david-perez
Copy link
Contributor Author

david-perez commented Apr 21, 2023

Thanks! That makes sense.

One last question. We have a rather old PR open in smithy-rs to migrate entirely to EnumShape (i.e., never use EnumTrait in the codebase). At the moment we have to handle both separately, which causes branching and deprecation warnings (the EnumTrait class is marked as deprecated). That PR opts to always internally apply the changeStringEnumsToEnumShapes transform as initially suggested by the Smithy team and just handle EnumShape thereafter. Would you suggest then that we don't do that and have the user add the transform to their smithy-build.json explicitly? If we do unconditionally apply the transform, then the answer to the last two questions:

  • Does @tags["internal"] imply @internal?
  • Does the tags property on the @enum trait imply @tags?

would always be "yes" to the eyes of a smithy-rs user by virtue of how the transform works.

@gosar
Copy link
Contributor

gosar commented Apr 22, 2023

I don't think this discussion changes the recommendation to apply the changeStringEnumsToEnumShapes in the codegen and just handle EnumShape. It does mean yes to the questions in the eyes of smithy-rs user, and that sounds fine.

@kstich
Copy link
Contributor

kstich commented May 1, 2023

Closing as this discussion seems resolved and 1.31.0 contains the latest updates to enum upgrades.

@kstich kstich closed this as completed May 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants