change: Consider disabling catalog caching by default and enabling it on a case-by-case basis in Meltano Hub #8970
Open
Description
Feature scope
Configuration (settings parsing, validation, etc.)
Description
Rationale
Catalog caching address the time and stability aspect of syncing dynamic sources, but causes issues when the catalog becomes stale and breaks users' expectations.
After the implementation of on-demand cache invalidation via --refresh-catalog
(#8580), and the fortunate detail of it adding a _use_cached_catalog
extra setting to extractors, we can consider setting it to false
by default and going through the Hub updating adding use_cached_catalog: true
to those extractors that most definitely need it: all SQL sources, Salesforce and any other dynamic sources.
This could be a breaking change, so we wanna be careful.
Metadata
Assignees
Type
Projects
Status
Discussed
Activity