[avro] Add documentation for the Avro format. #2452
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gz
approved these changes
Sep 9, 2024
| /// Custom headers that will be added to every call to the schema registry. | ||
| /// | ||
| /// This property is only applicable to output connectors. | ||
| /// |
| /// nullability of columns. | ||
| /// When this property is set, the connector uses the provided schema instead of | ||
| /// retrieving the schema from the schema registry. This setting is mutually exclusive | ||
| /// with `registry_urls`. | ||
| pub schema: Option<String>, |
There was a problem hiding this comment.
don't we usually use Object for json stuff?
There was a problem hiding this comment.
Not sure what's an object. We could try parsing this into serde_json::Value, but this is an escaped string, not a regular piece of JSON, so not sure how to do this exactly, plus the library function for deserializing schemas works with strings.
|
|
||
| * **Raw** - every message contains a single Avro-encoded record that represents a row in a SQL | ||
| table or view. This format does not carry any additional metadata that can be used to distinguish | ||
| inserts and deletes. It is therefore only suitable for representing inserts and upserts, but not |
There was a problem hiding this comment.
add link to where we explain how upserts work in feldera (if we have one?)
| The Avro encoder generates an Avro schema, which the consumer can use to decode messages produced | ||
| by the encoder. For message formats that include key and value components with different schemas, | ||
| e.g., the Confluent JDBC connector format, the encoder generates both key and value schemas. | ||
| If the connector configuration specifies a schema registry, the encoder publishes both |
There was a problem hiding this comment.
maybe add link to the relevant example
| If the connector configuration specifies a schema registry, the encoder publishes both | ||
| key and value schemas in the registry. | ||
|
|
||
| The encoder supports an alternative workflow where users provide the schema as a JSON string |
Signed-off-by: Leonid Ryzhyk <leonid@feldera.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.