Improve block loader fallback to source when source mode is synthetic. #115394
Description
Sometimes MappedFieldType#blockLoader(...)
implementations fallback to an implementation that uses source. For example when a field has doc values or stored fields disabled, when ignore above or ignore above have been configured. Meaning it would read the _source field and then extract the relevant field out of it and use that as value to be returned by the block loader.
When synthetic source is enabled then instead the source gets computed from many doc value or stored fields, and then the relevant field gets extracted. This is very slow and this should be improved. The interesting part with synthetic source is that we don't need to compute the source in order to provided a fallback values as part of block loaders returned by MappedFieldType#blockLoader(...)
.
Synthetic source details relevant to block loader fallback logic:
- A field value exceeds the configured ignore above, then the value is stored in a separate stored field with the suffix
_original
. - A field value is malformed, then the value gets stored in a stored field with the same name as is defined in the mapping. Regardless of whether stored fields are enabled.
- A field is has no stored or doc values fields. Then it gets stored in
_ignored_source
stored field. - Ignored source is the fallback for synthetic source to avoid content in source getting lost. So for example if a an object field is disabled or number of allowed mapped fields is exceeded, the field values / content should end up in ignored source.
In case of synthetic source the block loaders returned by MappedFieldType#blockLoader(...)
can be made aware if these details and instead of returning a BlockSourceReader
based implementation, return an implementation that uses the right stored field or uses ignored source.
Tasks:
- Handle ignore above more efficiently.
- Handle ignore malformed more efficiently.
- Handle the reading field values that are stored in ignored source.
Activity