-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Copy v3 data dir when performing backup #3860
Conversation
@@ -13,6 +12,8 @@ | |||
role: etcd | |||
local_facts: {} | |||
when: "'etcd' not in openshift" | |||
- set_fact: | |||
timestamp: "{{ lookup('pipe', 'date +%Y%m%d%H%M%S') }}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did this because without this it was actually re-evaluating the lookup each time {{ timestamp }}
was used below which meant that the backup location it told you existed usually didn't unless the set of tasks happened within the same second, on my test vm at least that never happened but maybe that's only because i've added additional backup steps.
path: "{{ openshift.etcd.etcd_data_dir }}/member/snap/db" | ||
register: v3_db | ||
|
||
- name: Copy etcd v3 data store |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is mentioned in [1] as well: "Recovering a cluster first needs a snapshot of the keyspace from an etcd member. A snapshot may either be taken from a live member with the etcdctl snapshot save command or by copying the member/snap/db file from an etcd data directory."
aos-ci-test |
Fixes TASK [Copy etcd v3 data store] ************************************************* fatal: [host.redhat.com]: FAILED! => { "changed": true, "cmd": [ "cp", "-a", "/var/lib/etcd//member/snap", "/var/lib/origin/etcd-backup-pre-upgrade-20170407055413/member/" ], "delta": "0:00:00.003152", "end": "2017-04-07 01:54:17.584685", "failed": true, "rc": 1, "start": "2017-04-07 01:54:17.581533", "warnings": [] } STDERR: cp: cannot create directory ?/var/lib/origin/etcd-backup-pre-upgrade-20170407055413/member/?: No such file or directory
aos-ci-test |
[merge] |
Because containerized installs don't mount /var/lib/origin and we switched to running the backup inside the container that meant that we were backing up the etcd data into a directory inside the container filesystem. Since we have no other volume mounted we need to backup into /var/lib/etcd.
[test]ing while waiting on the merge queue |
Evaluated for openshift ansible test up to 974f01c |
continuous-integration/openshift-jenkins/test FAILURE (https://ci.openshift.redhat.com/jenkins/job/test_pull_request_openshift_ansible/10/) (Base Commit: 5fe1609) |
aos-ci-test |
[merge] |
flake on openshift/origin#13067 |
Evaluated for openshift ansible merge up to 974f01c |
continuous-integration/openshift-jenkins/merge SUCCESS (https://ci.openshift.redhat.com/jenkins/job/merge_pull_request_openshift_ansible/291/) (Base Commit: 4805e68) |
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1433272