Skip to content

Commit

Permalink
Improve IDL 1 and 2 interoperability
Browse files Browse the repository at this point in the history
This commit makes several significant updates to Smithy IDL 2 that
improve lossless interoperability with Smithy IDL 1:

* Default traits can now coexist with required. This indicates that
  a member should be serialized, but it is a protocol-specific
  decision if and how this is enforced. This was a pattern that occurred
  in Smithy 1.0 models when a member was required and targeted a shape
  with a zero value.
* Default traits can be added to root-level shapes. Any structure member
  that targets a shape marked with the default trait must repeat the
  default on the member. This removes the action at a distance problem
  observed in Smithy IDL 1 where a root level shape implicitly introduced
  a default zero value, and to know if that's the case for any member, you
  had to look through from the member to the target shape. This change
  allows us to know if a root level shape was boxed in IDL 1.0 too -- if
  it is a shape that had a zero value in IDL 1 and either doesn't have
  a default or the default is not the zero value, then it was boxed in v1.
* When loading v2 models, we will now patch synthetic box traits onto shapes
  that would have been considered boxed by existing smithy-model consumers.
  This improves further interop with tooling that has not yet adopted
  Smithy IDL 2 or that hasn't yet migrated to use the NullableIndex
  abstraction.
* Added an optional nullability report to smithy-build that shows the
  computed nullability semantics of each member in a model. This can be
  used to better understand nullability semantics.
* Updated smithy-diff to not emit events when diffing a 1.0 model against
  the 2.0 serialized version of the model. This means that changes to the
  box trait are ignored unless the change impacts the nullability of the
  shape. Special handling was added to detect breaking changes with the
  default trait too (you can't change a default value of a root-level shape
  for example, you can't change a default value of a shape to or from the
  zero value of a type as this might break code generators, etc).
* Add new NullableIndex modes for testing if a member is nullable based on
  the supported features of the generator. For example, some generators only
  make members non-optional when the member is set to the zero value of a
  type, so there is a NullableIndex check mode for that and other use cases.
* Added the @addedDefault trait which is used to indicate that a `@default`
  trait was added to a member after it was initially released. This can be
  used by tooling to make an appropriate determination if generating a
  non-nullable type for the member is a backward compatible change. For
  example, if a generator only uses default zero values to generate
  non-nullable types, then the removal of the required trait and addition
  of a default trait would be a breaking change for them, so they can use
  addedDefault to ignore the default trait.
* When loading 1.0 models, rather than dropping the default trait from a
  member when the range trait of a shape is invalid for its zero value, we
  now instead emit only a warning for this specific case. This prevents
  changing the type and also doesn't lose the range constraint in case
  implementations are relying on this validation.
* Un-deprecated Primitive* shapes in the prelude and add the `@default`
  trait to them. Any member that targets one of these primitive prelude
  shapes must now also repeat the zero value of the target shape.
* Fixed an issue where synthetic box traits were being dropped when the
  same model is loaded multiple times.
  • Loading branch information
mtdowling committed Sep 10, 2022
1 parent 2bc09aa commit 72e4b5a
Show file tree
Hide file tree
Showing 100 changed files with 2,508 additions and 639 deletions.
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.24.0
1.25.0
27 changes: 17 additions & 10 deletions docs/source-2.0/spec/aggregate-types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -168,10 +168,9 @@ Structure member optionality

Whether a structure member is optional is determined by evaluating the
:ref:`required-trait`, :ref:`default-trait`, :ref:`clientOptional-trait`,
and :ref:`input-trait`. Authoritative model consumers like servers MAY choose
to determine optionality using more restrictive rules by ignoring the
``@input`` and ``@clientOptional`` traits.

:ref:`input-trait`, and :ref:`addedDefault-trait`. Authoritative model
consumers like servers MAY choose to determine optionality using more
restrictive rules by ignoring the ``@input`` and ``@clientOptional`` traits.

.. list-table::
:header-rows: 1
Expand Down Expand Up @@ -262,20 +261,28 @@ need to be provided without breaking previously generated code.
.. rubric:: Migrating ``@required`` to ``@default``

If a ``required`` member no longer needs to be be required, the ``required``
trait MAY be removed and replaced with the :ref:`default-trait`. The member
is still considered always present to tools like code generators, but instead
of requiring the value to be provided by an end-user, a default value is
automatically provided if missing. For example, the previous ``TimeSpan``
model can be backward compatibly changed to:
trait MAY be removed and replaced with the :ref:`default-trait`. Alternatively,
a ``default`` trait MAY be added to a member marked as ``required`` to
provide a default value for the member but require that it is serialized.
Either way, the member is still considered always present to tools like code
generators, but instead of requiring the value to be provided by an end-user,
a default value is automatically provided if missing. For example, the previous
``TimeSpan`` model can be backward compatibly changed to:

.. code-block:: smithy
structure TimeSpan {
// @required is replaced with @default
// @required is replaced with @default and @addedDefault
@addedDefault
years: Integer = 0
days: Integer = 0
}
The :ref:`addeddefault-trait` trait SHOULD be used any time a ``default`` trait is
added to a previously published member. Some tooling does not treat the
``required`` trait as non-nullable but does treat the ``default`` trait as
non-nullable.

.. rubric:: Requiring members to be optional

The :ref:`clientOptional-trait` is used to indicate that a member that is
Expand Down
35 changes: 7 additions & 28 deletions docs/source-2.0/spec/model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1288,46 +1288,25 @@ referenced from within any namespace using a relative shape ID.
@unitType
structure Unit {}
@deprecated(
message: "Use Boolean instead, and add the @default trait to structure members that targets this shape",
since: "2.0"
)
@default(false)
boolean PrimitiveBoolean
@deprecated(
message: "Use Byte instead, and add the @default trait to structure members that targets this shape",
since: "2.0"
)
@default(0)
byte PrimitiveByte
@deprecated(
message: "Use Short instead, and add the @default trait to structure members that targets this shape",
since: "2.0"
)
@default(0)
short PrimitiveShort
@deprecated(
message: "Use Integer instead, and add the @default trait to structure members that targets this shape",
since: "2.0"
)
@default(0)
integer PrimitiveInteger
@deprecated(
message: "Use Long instead, and add the @default trait to structure members that targets this shape",
since: "2.0"
)
@default(0)
long PrimitiveLong
@deprecated(
message: "Use Float instead, and add the @default trait to structure members that targets this shape",
since: "2.0"
)
@default(0)
float PrimitiveFloat
@deprecated(
message: "Use Double instead, and add the @default trait to structure members that targets this shape",
since: "2.0"
)
@default(0)
double PrimitiveDouble
Expand Down
95 changes: 84 additions & 11 deletions docs/source-2.0/spec/type-refinement-traits.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,22 +15,20 @@ type of a shape.
Summary
Provides a structure member with a default value.
Trait selector
``structure > member :test(> :is(simpleType, list, map))``
``:is(simpleType, list, map, structure > member :test(> :is(simpleType, list, map)))``

A member of a structure that targets a simple type, list, or map.
A simple type, list, map, or a member of a structure that targets a simple type, list, or map.
Value type
Document type.
Conflicts with
* :ref:`required-trait`
See also
* :ref:`structure-optionality`
* :ref:`required-trait`
* :ref:`clientOptional-trait`
* :ref:`input-trait`

The ``@default`` trait assigns a default value to a structure member using
a document type. The following example defines a structure with a "language"
member that has a default value:
The ``@default`` trait assigns a default value to a structure member. The
following example defines a structure with a "language" member that has a
default value:

.. code-block:: smithy
Expand Down Expand Up @@ -58,6 +56,40 @@ The above example uses syntactic sugar to apply the ``@default`` trait. The
language: Language
}
The ``@default`` trait can be added to root-level simple types, lists, or
maps. This can serve as a kind of template to enforce default values across
structure members in a model. Any structure member that targets a shape
marked with ``@default`` MUST also add a matching ``@default`` trait to the
member.

.. code-block:: smithy
@default(0)
integer ZeroValueInteger
structure Message {
zeroValueInteger: ZeroValueInteger = 0 // must be repeated and match the target.
}
The ``@default`` trait on a structure member can be set to ``null`` to
explicitly indicate that the member is optional or to override the default
value requirement of a targeted shape.

.. code-block:: smithy
@default(0)
integer ZeroValueInteger
structure Message {
zeroValueInteger: ZeroValueInteger = null // forces the member to be optional
}
.. note::

* The ``@default`` trait on root-level shapes has no impact when targeted by
any other shape than a structure member.
* The ``@default`` trait on root-level shapes cannot be set to ``null``.


Default value constraints
-------------------------
Expand Down Expand Up @@ -94,6 +126,12 @@ property was omitted.
Updating default values
-----------------------

The default value of a root-level shape MUST NOT be changed nor can the default
trait be added or removed from an existing root-level shape. Changing the
default value of a root-level shape would cause any member reference to the
shape to break and could inadvertently impact code generated types for the
shape.

The default value of a member SHOULD NOT be changed. However, it MAY be
necessary in extreme cases to change a default value if changing the default
value addresses a customer-impacting issue or availability issue for a service.
Expand All @@ -111,8 +149,45 @@ default values if the member is marked with the :ref:`internal-trait`.

To allow servers to change default values if necessary, clients SHOULD NOT
serialize default values unless the member is explicitly set to the default
value. This implies that clients SHOULD implement a kind of "presence tracking"
of defaulted members.
value or marked with the :ref:`default-trait`. This implies that clients
SHOULD implement a kind of "presence tracking" of defaulted members so that
the member is only serialized if it is explicitly set to the default value.


Default and required
--------------------

A member that is both ``@default`` and ``@required`` SHOULD always be
serialized, and implementations SHOULD NOT use any form of presence tracking
to omit a member if the member is not explicitly set to the default value.
It is a protocol-specific decision as to whether this is enforced in
serialized messages; some protocols follow this strictly whereas others may
not.


.. smithy-trait:: smithy.api#addedDefault
.. _addedDefault-trait:

``addedDefault`` trait
======================

Summary
Indicates that the :ref:`default-trait` was added to a structure member
after initially publishing the member. This allows tooling to decide
whether to ignore the ``@default`` trait if it will break backward
compatibility in the tool.
Trait selector
``structure > member [trait|default]``

*Member of a structure marked with the default trait*
Value type
Annotation trait.
See also
* :ref:`structure-optionality`
* :ref:`default-trait`
* :ref:`clientOptional-trait`
* :ref:`input-trait`
* :ref:`recommended-trait`


.. smithy-trait:: smithy.api#required
Expand All @@ -130,8 +205,6 @@ Trait selector
*Member of a structure*
Value type
Annotation trait.
Conflicts with
* :ref:`default-trait`
See also
* :ref:`structure-optionality`
* :ref:`default-trait`
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
/*
* Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License").
* You may not use this file except in compliance with the License.
* A copy of the License is located at
*
* http://aws.amazon.com/apache2.0
*
* or in the "license" file accompanying this file. This file is distributed
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
* express or implied. See the License for the specific language governing
* permissions and limitations under the License.
*/

package software.amazon.smithy.build.plugins;

import software.amazon.smithy.build.PluginContext;
import software.amazon.smithy.build.SmithyBuildPlugin;
import software.amazon.smithy.model.Model;
import software.amazon.smithy.model.knowledge.NullableIndex;
import software.amazon.smithy.model.node.Node;
import software.amazon.smithy.model.node.ObjectNode;
import software.amazon.smithy.model.shapes.MemberShape;
import software.amazon.smithy.model.shapes.StructureShape;
import software.amazon.smithy.model.traits.BoxTrait;

/**
* Generates a JSON report that contains a mapping of every structure member to
* whether the member is considered nullable in v1 and v2 implementations.
*/
public final class NullabilityReportPlugin implements SmithyBuildPlugin {

static final String NULLABILITY_REPORT_PATH = "nullability-report.json";
private static final String NAME = "nullabilityReport";

@Override
public String getName() {
return NAME;
}

@Override
public boolean requiresValidModel() {
return false;
}

@Override
public void execute(PluginContext context) {
if (context.getOriginalModel().isPresent() && context.getProjection().isPresent()) {
context.getFileManifest().writeJson(NULLABILITY_REPORT_PATH, serializeReport(context));
}
}

private static Node serializeReport(PluginContext context) {
ObjectNode.Builder root = Node.objectNodeBuilder();
Model model = context.getModel();
NullableIndex index = NullableIndex.of(model);

for (StructureShape structure : model.getStructureShapes()) {
// Only generate for structures in "sources" that have members.
if (!structure.getAllMembers().isEmpty() && context.isSourceShape(structure)) {
ObjectNode.Builder struct = Node.objectNodeBuilder();
for (MemberShape member : structure.getAllMembers().values()) {
ObjectNode.Builder entry = Node.objectNodeBuilder();
entry.withMember("v1", index.isNullable(member));
entry.withMember("v1-via-box", member.getMemberTrait(model, BoxTrait.class).isPresent());
entry.withMember("v2-client",
index.isMemberNullable(member, NullableIndex.CheckMode.CLIENT));
entry.withMember("v2-client-careful",
index.isMemberNullable(member, NullableIndex.CheckMode.CLIENT_CAREFUL));
entry.withMember("v2-client-zero-value",
index.isMemberNullable(member, NullableIndex.CheckMode.CLIENT_ZERO_VALUE_V1));
entry.withMember("v2-client-zero-value-no-input",
index.isMemberNullable(member,
NullableIndex.CheckMode.CLIENT_ZERO_VALUE_V1_NO_INPUT));
entry.withMember("v2-server",
index.isMemberNullable(member, NullableIndex.CheckMode.SERVER));
struct.withMember(member.getMemberName(), entry.build());
}
root.withMember(structure.getId().toString(), struct.build());
}
}

return root.build();
}
}
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
software.amazon.smithy.build.plugins.BuildInfoPlugin
software.amazon.smithy.build.plugins.ModelPlugin
software.amazon.smithy.build.plugins.SourcesPlugin
software.amazon.smithy.build.plugins.NullabilityReportPlugin
Loading

0 comments on commit 72e4b5a

Please sign in to comment.