Doc: add a "Cloud Administration" page. #1888
Conversation
concretevitamin
changed the title
|
We can also call this "Cloud-specific Features" or "Cloud-specific Guides". |
There was a problem hiding this comment.
Thanks for adding the doc page @concretevitamin! Left a question for it.
| $ aws --version | ||
|
|
||
|
|
||
| Then, after the usual ``aws configure sso`` and ``aws sso login --profile <profile_name>`` commands, SkyPilot will work as usual. |
There was a problem hiding this comment.
Are we sure that SkyPilot will work with v2 as the default aws command?
Currently, what I did for SSO was to create an awsv2 command that is only used for awsv2 configure sso and awsv2 sso login, while let the SkyPilot uses the v1.
There was a problem hiding this comment.
Working like a champ for me. I admit I'm using a subset of features though. Is there a list of CLI-based commands that Sky runs?
There was a problem hiding this comment.
Here are all the places we invoke AWS CLI directly:
» ag "'aws " sky -s
sky/cloud_stores.py
43: 'aws --version >/dev/null 2>&1 || pip3 install awscli',
73: download_via_awscli = ('aws s3 sync --no-follow-symlinks '
82: download_via_awscli = f'aws s3 cp {source} {destination}'
154: 'aws --version >/dev/null 2>&1 || pip3 install awscli',
187: download_via_awscli = ('aws s3 sync --no-follow-symlinks '
199: download_via_awscli = (f'aws s3 cp s3://{source} {destination} '
sky/backends/backend_utils.py
1589: query_cmd = ('aws ec2 describe-instances --filters '
sky/backends/cloud_vm_ray_backend.py
3055: f'aws ec2 describe-instances --region {region} --filters '
3060: f'aws ec2 terminate-instances --region {region} '
sky/clouds/aws.py
404: proc = subprocess.run('aws --version',
464: proc = subprocess.run('aws configure list',
sky/data/storage.py
1030: sync_command = ('aws s3 sync --no-follow-symlinks --exclude="*" '
1038: 'aws s3 sync --no-follow-symlinks --exclude ".git/*" '
1095: command = f'aws s3 ls {self.name}'
1182: remove_command = f'aws s3 rb s3://{bucket_name} --force'
1700: sync_command = ('aws s3 sync --no-follow-symlinks --exclude="*" '
1711: 'aws s3 sync --no-follow-symlinks --exclude ".git/*" '
1770: command = (f'aws s3 ls s3://{self.name} '
1866: remove_command = (f'aws s3 rb s3://{bucket_name} --force '
These are invoked by the following Sky call paths, which we need to test
sky status -rwhere we have an AWS spot clustersky down <an AWS cluster with status STOPPED>- Using file_mounts
- where we have
<dst>: s3://<a "dir" or a "file">, covered bypytest tests/test_smoke.py::test_file_mounts - where we have a Sky Storage with
source: s3://<a "dir" or a "file"> - I think these can also be independently tested by runing
aws s3 cp/syncetc.
- where we have
We could get to testing this soon. @Taytay if you've already run some operations here and it works, please let us know.
There was a problem hiding this comment.
Tested all of the above with aws aliased to v2.
Prompted by #1883.
Tested (run the relevant ones):