-
Notifications
You must be signed in to change notification settings - Fork 40k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ScheduledJob controller proposal #11980
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,225 @@ | ||
<!-- BEGIN MUNGE: UNVERSIONED_WARNING --> | ||
|
||
<!-- BEGIN STRIP_FOR_RELEASE --> | ||
|
||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING" | ||
width="25" height="25"> | ||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING" | ||
width="25" height="25"> | ||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING" | ||
width="25" height="25"> | ||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING" | ||
width="25" height="25"> | ||
<img src="http://kubernetes.io/img/warning.png" alt="WARNING" | ||
width="25" height="25"> | ||
|
||
<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2> | ||
|
||
If you are using a released version of Kubernetes, you should | ||
refer to the docs that go with that version. | ||
|
||
Documentation for other releases can be found at | ||
[releases.k8s.io](http://releases.k8s.io). | ||
</strong> | ||
-- | ||
|
||
<!-- END STRIP_FOR_RELEASE --> | ||
|
||
<!-- END MUNGE: UNVERSIONED_WARNING --> | ||
|
||
# ScheduledJob Controller | ||
|
||
## Abstract | ||
|
||
A proposal for implementing a new controller - ScheduledJob controller - which | ||
will be responsible for managing time based jobs, namely: | ||
* once at a specified point in time, | ||
* repeatedly at a specified point in time. | ||
|
||
There is already a discussion regarding this subject: | ||
* Distributed CRON jobs [#2156](https://issues.k8s.io/2156) | ||
|
||
There are also similar solutions available, already: | ||
* [Mesos Chronos](https://github.com/mesos/chronos) | ||
* [Quartz](http://quartz-scheduler.org/) | ||
|
||
|
||
## Use Cases | ||
|
||
1. Be able to schedule a job execution at a given point in time. | ||
1. Be able to create a periodic job, eg. database backup, sending emails. | ||
|
||
|
||
## Motivation | ||
|
||
ScheduledJobs are needed for performing all time-related actions, namely backups, | ||
report generation and the like. Each of these tasks should be allowed to run | ||
repeatedly (once a day/month, etc.) or once at a given point in time. | ||
|
||
|
||
## Implementation | ||
|
||
### ScheduledJob resource | ||
|
||
The ScheduledJob controller relies heavily on the [Job API](job.md) | ||
for running actual jobs, on top of which it adds information regarding the date | ||
and time part according to [Cron](https://en.wikipedia.org/wiki/Cron) format. | ||
|
||
The new `ScheduledJob` object will have the following contents: | ||
|
||
```go | ||
// ScheduledJob represents the configuration of a single scheduled job. | ||
type ScheduledJob struct { | ||
TypeMeta | ||
ObjectMeta | ||
|
||
// Spec is a structure defining the expected behavior of a job, including the schedule. | ||
Spec ScheduledJobSpec | ||
|
||
// Status is a structure describing current status of a job. | ||
Status ScheduledJobStatus | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would go ahead and add json tags into this struct, there's no reason not to fully define API objects in the proposal (especially ideally they would be the exact same as what went into the code). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I didn't do it up until now not to clutter the proposal with too much of implementation details. The comments are exactly like the ones in the actual code. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe except for the places where I'm just showing the fact of existence of a structure, eg. |
||
|
||
// ScheduledJobList is a collection of scheduled jobs. | ||
type ScheduledJobList struct { | ||
TypeMeta | ||
ListMeta | ||
|
||
Items []ScheduledJob | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same comment applies here: json tags / comments. |
||
``` | ||
|
||
The `ScheduledJobSpec` structure is defined to contain all the information how the actual | ||
job execution will look like, including the `JobSpec` from [Job API](job.md) | ||
and the schedule in [Cron](https://en.wikipedia.org/wiki/Cron) format. This implies | ||
that each ScheduledJob execution will be created from the JobSpec actual at a point | ||
in time when the execution will be started. This also implies that any changes | ||
to ScheduledJobSpec will be applied upon subsequent execution of a job. | ||
|
||
```go | ||
// ScheduledJobSpec describes how the job execution will look like and when it will actually run. | ||
type ScheduledJobSpec struct { | ||
|
||
// Schedule contains the schedule in Cron format, see https://en.wikipedia.org/wiki/Cron. | ||
Schedule string | ||
|
||
// Optional deadline in seconds for starting the job if it misses scheduled | ||
// time for any reason. Missed jobs executions will be counted as failed ones. | ||
StartingDeadlineSeconds *int64 | ||
|
||
// ConcurrencyPolicy specifies how to treat concurrent executions of a Job. | ||
ConcurrencyPolicy ConcurrencyPolicy | ||
|
||
// Suspend flag tells the controller to suspend subsequent executions, it does | ||
// not apply to already started executions. Defaults to false. | ||
Suspend bool | ||
|
||
// JobTemplate is the object that describes the job that will be created when | ||
// executing a ScheduledJob. | ||
JobTemplate *JobTemplateSpec | ||
} | ||
|
||
// JobTemplateSpec describes of the Job that will be created when executing | ||
// a ScheduledJob, including its standard metadata. | ||
type JobTemplateSpec struct { | ||
ObjectMeta | ||
|
||
// Specification of the desired behavior of the job. | ||
Spec JobSpec | ||
} | ||
|
||
// ConcurrencyPolicy describes how the job will be handled. | ||
// Only one of the following concurrent policies may be specified. | ||
// If none of the following policies is specified, the default one | ||
// is AllowConcurrent. | ||
type ConcurrencyPolicy string | ||
|
||
const ( | ||
// AllowConcurrent allows ScheduledJobs to run concurrently. | ||
AllowConcurrent ConcurrencyPolicy = "Allow" | ||
|
||
// ForbidConcurrent forbids concurrent runs, skipping next run if previous | ||
// hasn't finished yet. | ||
ForbidConcurrent ConcurrencyPolicy = "Forbid" | ||
|
||
// ReplaceConcurrent cancels currently running job and replaces it with a new one. | ||
ReplaceConcurrent ConcurrencyPolicy = "Replace" | ||
) | ||
``` | ||
|
||
`ScheduledJobStatus` structure is defined to contain information about scheduled | ||
job executions. The structure holds a list of currently running job instances | ||
and additional information about overall successful and unsuccessful job executions. | ||
|
||
```go | ||
// ScheduledJobStatus represents the current state of a Job. | ||
type ScheduledJobStatus struct { | ||
// Active holds pointers to currently running jobs. | ||
Active []ObjectReference | ||
|
||
// Successful tracks the overall amount of successful completions of this job. | ||
Successful int64 | ||
|
||
// Failed tracks the overall amount of failures of this job. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to distinguish between Failure Condition of the Job, and failure to start the job entirely (name collision, lack of permissions to create job, apiserver not available, etc) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think so, at least for now I don't see the need for that distinction. |
||
Failed int64 | ||
|
||
// LastScheduleTime keeps information of when was the last time the job was successfully scheduled. | ||
LastScheduleTime Time | ||
} | ||
``` | ||
|
||
### Modifications to Job resource | ||
|
||
In order to distinguish Job runs, we need to add `UniqueLabelKey` field to `JobSpec`. | ||
This field will be used for creating unique label selectors. | ||
|
||
```go | ||
type JobSpec { | ||
|
||
//... | ||
|
||
// Key of the selector that is added to prevent concurrently running Jobs | ||
// selecting their pods. | ||
// Users can set this to an empty string to indicate that the system should | ||
// not add any selector and label. If unspecified, system uses | ||
// "scheduledjob.kubernetes.io/podTemplateHash". | ||
// Value of this key is hash of ScheduledJobSpec.PodTemplateSpec. | ||
// No label is added if this is set to an empty string. | ||
UniqueLabelKey *string | ||
} | ||
``` | ||
|
||
Although at Job level empty string is perfectly valid, `ScheduledJob` cannot have | ||
empty selector, it needs to be defined, either by user or generated automatically. | ||
For this to happen, validation will be tightened at ScheduledJob level for this | ||
field to be either nil or non-empty string. | ||
|
||
### Running ScheduledJobs using kubectl | ||
|
||
A user should be able to easily start a Scheduled Job using `kubectl` (similarly | ||
to running regular jobs). For example to run a job with a specified schedule, | ||
a user should be able to type something simple like: | ||
|
||
``` | ||
kubectl run pi --image=perl --restart=OnFailure --runAt="0 14 21 7 *" -- perl -Mbignum=bpi -wle 'print bpi(2000)' | ||
``` | ||
|
||
In the above example: | ||
|
||
* `--restart=OnFailure` implies creating a job instead of replicationController. | ||
* `--runAt="0 14 21 7 *"` implies the schedule with which the job should be run, here | ||
July 7th, 2pm. This value will be validated according to the same rules which | ||
apply to `.spec.schedule`. | ||
|
||
|
||
## Future evolution | ||
|
||
Below are the possible future extensions to the Job controller: | ||
* Be able to specify workflow template in `.spec` field. This relates to the work | ||
happening in [#18827](https://issues.k8s.io/18827). | ||
* Be able to specify more general template in `.spec` field, to create arbitrary | ||
types of resources. This relates to the work happening in [#18215](https://issues.k8s.io/18215). | ||
|
||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS --> | ||
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/proposals/scheduledjob.md?pixel)]() | ||
<!-- END MUNGE: GENERATED_ANALYTICS --> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comment still holds on name
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem with "Cron" is that is technically focused (a particular linux
tool) but is opaque to higher level users who come from different
backgrounds.
Even the Chronos GitHub page describes it with "It allows you to schedule
your jobs using ISO8601 repeating interval notation, which enables more
flexibility in job scheduling". RepeatingJob or CalendarJob or
RecurringJob all seem less clear to an outside user (someone who is not an
expert at these systems) than ScheduledJob.
On Fri, Jul 31, 2015 at 12:41 PM, Timothy St. Clair <
notifications@github.com> wrote:
Clayton Coleman | Lead Engineer, OpenShift
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Proposal was around (diurnal, periodic, etc.) There may be a couple more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Periodic suffers some of the same usability concerns. I do think API
objects should be precise in their meaning, but designed for end users, not
systems researchers.
On Fri, Jul 31, 2015 at 1:02 PM, Timothy St. Clair <notifications@github.com
Clayton Coleman | Lead Engineer, OpenShift
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with ScheduledJob. It's similar to ScheduledExecutorService in Java.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could also call it CalendaredJob