Skip to content

[PROPOSAL] Relocate Job Scheduler into core as a native pluginΒ #5310

Open
@joshpalis

Description

Based on this discussion, Job Scheduler will be relocated to core as a native plugin. This issue shall be used to track the associated work items and discuss any questions / concerns.

Draft PR to relocate Job Scheduler to native plugins : #5453

Update :

Previously, this change has been slated for the 2.6.0 release now that we have ensured that rolling/restart upgrades are not detrimentally affected by this relocation. However, upon consolidating the necessary build.gradle modifications for dependent plugins [3], the move to relocate Job Scheduler to native plugins has been determined to be a breaking change. In our efforts to adhere to semantic versioning, we shall relocate Job Scheduler to OpenSearch core as a native plugin on the 3.x branch of OpenSearch, and in order to backport this change to 2.x, we shall forgo any dependent plugin
build.gradle modifications for 2.x and instead publish Job Scheduler artifacts to two separate Maven coordinates.

[1] Native Job Scheduler Maven Coordinates :

  • org.opensearch.plugin:opensearch-job-scheduler-(3 digit version number)
  • org.opensearch.plugin:opensearch-job-scheduler-spi-(3 digit version number)

[2] Original Job Scheduler Maven Coordinates :

  • org.opensearch:opensearch-job-scheduler-(4 digit version number)
  • org.orpensearch:opensearch-job-scheduler-spi-(4 digit version number)

The original Job Scheduler Repository shall consume JS Native plugin artifacts[1] and repackage and publish these artifacts to the original Job Scheduler Maven coordinates[2]. By doing so, we can achieve both the relocation of Job Scheduler into native plugins, whilst adhering to semantic versioning by circumnavigating any modifications to how dependent plugins consume Job Scheduler for 2.x.

Build and Assemble Workflow Modifications

Update Now that the Job Scheduler relocation to native plugins will target the 3.0.0 release, modifications for the input manifest yml schema will now only be necessary for 3.0.0 and onwards.

Since Job Scheduler acts as a dependency for other plugins (AD, IM, Reporting), relocating this project within core as a native plugin will necessitate changes to the yml schema of the input manifest used to assemble the full distribution of OpenSearch.

Discussions for the input manifest changes has been started here .

Dependent Plugin Modifications
Now that the modifications for dependent plugin build.gradles[3] has been determined to be a breaking change, moving forward, dependent plugins will have no modifications for 2.x in order to adhere to semantic versioning, however this change will be raised for 3.x.

Simply put, for 2.x, dependent plugins will still consume Job Scheduler artifacts from the original maven coordinates[2], for 3.x, dependent plugins will consume Job Scheduler artifacts from the native JS maven coordinates[1].

  • [3] Job Scheduler SPI artifacts are currently uploaded with and consumed from the following maven coordinates org.opensearch:opensearch-job-scheduler-spi:${job_scheduler_version}. Upon relocation, artifacts will be uploaded as part of the OpenSearch component with the maven coordinates org.opensearch.plugin:opensearch-job-scheduler-spi:${job_scheduler_version}

Plans to support previous versions of Job Scheduler
In order to support security patches to previous versions of Job Scheduler, upon relocation, the original Job Scheduler repository will not be archived.

Metadata

Assignees

No one assigned

    Labels

    MetaMeta issue, not directly linked to a PRPluginsdiscussIssues intended to help drive brainstorming and decision makingdistributed frameworkenhancementEnhancement or improvement to existing feature or requestfeatureNew feature or requestv3.0.0Issues and PRs related to version 3.0.0

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions