I've created a ScheduledJobs as follows:

apiVersion: batch/v2alpha1
kind: ScheduledJob
metadata:
  name: test
spec:
  schedule: "*/1 * * * ?"
  concurrencyPolicy: Replace
  jobTemplate:
    spec:
      template:
        metadata:
          name: test
        spec:
          containers:
          - name: test
            image: gcr.io/google_containers/busybox:1.24
            command: ["sleep", "300"]
          restartPolicy: Never

This should replace an existing job with a new one upon next run, so it does the old job is being removed but not its pod. So after a while I have only one job (correct) and two pods (incorrect), one running the old job, the other the new one. I assume this is due to lack of server side deletion (#19054), since currently the job deletion logic is in JobReaper.

@janetkuo depending on timing either you or me need to fix this.
@erictune fyi