.github: Enable SSH to linux runners #62280

seemethere · 2021-07-27T18:19:08Z

Stack from ghstack:

-> .github: Enable SSH to linux runners #62280

Enables SSH to linux GHA runners for FB employees while on the FB VPN

SSH keys will be added to runners when the label "with-ssh" is applied to
your pull request.

Depends on https://github.com/fairinternal/pytorch-gha-infra/pull/8

Signed-off-by: Eli Uriegas eliuriegas@fb.com

Closes https://github.com/fairinternal/pytorch-gha-infra/issues/5

Differential Revision: D29941681

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> [ghstack-poisoned]

facebook-github-bot · 2021-07-27T18:19:13Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/62280
📄 Preview docs built from this PR

💊 CI failures summary and remediations

As of commit e70b4c1 (more details on the Dr. CI page):

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

seemethere · 2021-07-27T18:26:36Z

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depnds on pytorch-labs/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: 2ae885698390d48850ff8e1529ab6e27154e8f56 Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depnds on pytorch-labs/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: f3c2f147c0009d596072fe2ee12d7dfa4abdfdab Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depnds on pytorch-labs/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: 6cc87aeac10469430453bfba7ef288630d9debcf Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

seemethere · 2021-08-03T17:00:19Z

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

janeyx99 · 2021-08-03T18:34:08Z

.github/templates/linux_ci_workflow.yml.j2

+        # Always hold for active ssh sessions
+        if: always()
+        run: |
+          echo "Holding runner for 2 hours until all ssh sessions have logged out"


This means that even if the ssh session is on, the runner will still shut down after 2 hours regardless, right?

Yes, this is consistent with CircleCI's current re-run with ssh functionality, see: https://circleci.com/docs/2.0/ssh-access-jobs/

samestep · 2021-08-03T18:48:44Z

.github/templates/linux_ci_workflow.yml.j2

+      - name: Hold runner for 2 hours or until ssh sessions have drained
+        # Always hold for active ssh sessions
+        if: always()
+        run: |
+          echo "Holding runner for 2 hours until all ssh sessions have logged out"
+          for _ in $(seq 1440); do
+            # Break if no ssh session exists anymore
+            if [[ "$(who)" = "" ]]; then
+              break
+            fi
+            echo "."
+            sleep 5
+          done


looks like this same run script is used in a few different places; would it make sense to put it into .github/scripts and just call it from here?

samestep

lgtm! I guess most of the logic is in https://github.com/seemethere/add-github-ssh-key?

seemethere · 2021-08-03T19:49:04Z

lgtm! I guess most of the logic is in seemethere/add-github-ssh-key?

Yup most of the logic for this particular feature is in the terraform for the security groups as well as the the github actions to add the ssh keys.

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depends on pytorch-labs/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Closes pytorch-labs/pytorch-gha-infra#5 Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: 203f3f618eae7897ec37491073a49f70c8e34a81 Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

seemethere · 2021-08-03T20:00:12Z

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Depends on pytorch-labs/pytorch-gha-infra#8 Signed-off-by: Eli Uriegas <eliuriegas@fb.com> Closes pytorch-labs/pytorch-gha-infra#5 Differential Revision: [D29941681](https://our.internmc.facebook.com/intern/diff/D29941681) [ghstack-poisoned]

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> ghstack-source-id: 6ee7575b239737570e264f3a5d16c3cde7f2432e Pull Request resolved: #62280 Signed-off-by: Eli Uriegas <eliuriegas@fb.com>

seemethere · 2021-08-03T21:06:08Z

@seemethere has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

facebook-github-bot · 2021-08-03T22:17:10Z

@seemethere merged this pull request in c48dfe0.

.github: Enable SSH to linux runners

5812a53

Enables SSH to linux GHA runners for FB employees while on the FB VPN SSH keys will be added to runners when the label "with-ssh" is applied to your pull request. Signed-off-by: Eli Uriegas <eliuriegas@fb.com> [ghstack-poisoned]

seemethere requested review from driazati, janeyx99 and zhouzhuojie as code owners July 27, 2021 18:19

facebook-github-bot added the cla signed label Jul 27, 2021

seemethere marked this pull request as draft July 27, 2021 18:27

seemethere added the module: ci Related to continuous integration label Jul 27, 2021

seemethere added the with-ssh label Jul 29, 2021

seemethere marked this pull request as ready for review August 3, 2021 16:59

seemethere requested a review from a team August 3, 2021 17:02

janeyx99 reviewed Aug 3, 2021

View reviewed changes

samestep reviewed Aug 3, 2021

View reviewed changes

samestep approved these changes Aug 3, 2021

View reviewed changes

janeyx99 approved these changes Aug 3, 2021

View reviewed changes

facebook-github-bot closed this in c48dfe0 Aug 3, 2021

facebook-github-bot added the Merged label Aug 3, 2021

facebook-github-bot deleted the gh/seemethere/121/head branch August 7, 2021 14:17

seemethere mentioned this pull request Aug 24, 2021

[RFC] Using with-ssh for Github Actions CI (Feedback / Meta issue) #63889

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

.github: Enable SSH to linux runners #62280

.github: Enable SSH to linux runners #62280

seemethere commented Jul 27, 2021 •

edited

Loading

facebook-github-bot commented Jul 27, 2021 •

edited

Loading

seemethere commented Jul 27, 2021

seemethere commented Aug 3, 2021

janeyx99 Aug 3, 2021

seemethere Aug 3, 2021

samestep Aug 3, 2021

seemethere Aug 3, 2021

samestep left a comment

seemethere commented Aug 3, 2021

seemethere commented Aug 3, 2021

seemethere commented Aug 3, 2021

facebook-github-bot commented Aug 3, 2021

.github: Enable SSH to linux runners #62280

.github: Enable SSH to linux runners #62280

Conversation

seemethere commented Jul 27, 2021 • edited Loading

facebook-github-bot commented Jul 27, 2021 • edited Loading

🔗 Helpful links

💊 CI failures summary and remediations

seemethere commented Jul 27, 2021

seemethere commented Aug 3, 2021

janeyx99 Aug 3, 2021

Choose a reason for hiding this comment

seemethere Aug 3, 2021

Choose a reason for hiding this comment

samestep Aug 3, 2021

Choose a reason for hiding this comment

seemethere Aug 3, 2021

Choose a reason for hiding this comment

samestep left a comment

Choose a reason for hiding this comment

seemethere commented Aug 3, 2021

seemethere commented Aug 3, 2021

seemethere commented Aug 3, 2021

facebook-github-bot commented Aug 3, 2021

seemethere commented Jul 27, 2021 •

edited

Loading

facebook-github-bot commented Jul 27, 2021 •

edited

Loading