Skip to content

Commit

Permalink
Clean up broken links to repair aggregate report pipeline (#29289)
Browse files Browse the repository at this point in the history
  • Loading branch information
scbedd authored Mar 14, 2023
1 parent 48f8205 commit 962f930
Show file tree
Hide file tree
Showing 8 changed files with 49 additions and 42 deletions.
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ Mypy install and run.
```

#### `sphinx` environment
Generate shpinx doc for this package.
Generate sphinx doc for this package.

```
\> tox -e sphinx -c <path to tox.ini>
Expand All @@ -117,7 +117,7 @@ Generate shpinx doc for this package.

`tox` supports custom arguments, and the defined pytest environments within the common `tox.ini` also allow these. Essentially, separate the arguments you want passed to `pytest` by a `--` in your tox invocation.

[Tox Documentation on Positional Arguments](https://tox.readthedocs.io/en/latest/example/general.html#interactively-passing-positional-arguments)
[Tox Documentation on Positional Arguments](https://tox.wiki/en/latest/config.html#substitutions-for-positional-arguments-in-commands)

**Example: Invoke tox, breaking into the debugger on failure**
`tox -e whl -c ../../../eng/tox/tox.ini -- --pdb`
Expand Down
10 changes: 4 additions & 6 deletions doc/dev/mgmt/multiapi.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Multi-api packaging and ARM SDK

Several SDKs on this repository are able to support multi-api in a single package. If you're a SDK maintener, this documentation explains the rational of it, and the technical details.
Several SDKs on this repository are able to support multi-api in a single package. If you're a SDK maintainer, this documentation explains the rational of it, and the technical details.

## Overview and rationale

Expand All @@ -10,7 +10,7 @@ It means a given specific package is able to do calls to several different API v

### Why would I need to call anything else than the latest API version?

Because there is different flavors of Azure that are not necessarly provided with the same set of API versions. For example Azure Governement, Azure Stack usually needs a few months to get the latest available API version.
Because there is different flavors of Azure that are not necessarily provided with the same set of API versions. For example Azure Government, Azure Stack usually needs a few months to get the latest available API version.

### Why a multi-api package?

Expand Down Expand Up @@ -66,9 +66,9 @@ Main design guidelines:
- For people that want a specific API version for a specific need, specifying API version should be possible
e.g. `client = ComputeManagementClient(credentials, sub_id, api_version='2018-06-01')`
- For people who target a single Azure Profile, specifying it should be be possible
e.g. `client = ComputeManagementClient(credentials, sub_id, profile=KnownProfile.v2018_06_01_bybrid)`
e.g. `client = ComputeManagementClient(credentials, sub_id, profile=KnownProfile.v2018_06_01_hybrid)`

The first condition has impact on models loading, by default they should load the latest API vesion transparently:
The first condition has impact on models loading, by default they should load the latest API version transparently:
```python
# Loads the latest version of the model
from azure.mgmt.compute.models import VirtualMachineCreateParameter
Expand Down Expand Up @@ -118,5 +118,3 @@ If this is not the same API version, then we need to bend the rules a little: we
## Possible improvements

Current implementation assumes operation group are unique, and as discussed it's not always the case. Also, this limitation has impact on intellisense right now. Example, if a user types `compute_client.virtual_machines.` and hit the intellisense shortcut, users won't see any suggestions. It's because the `virtual_machines` property is dynamic and can change depending of dynamic configuration.

To improve intellisense and allow operation level profile, the concept would be to make the [operation mixin multi-api concept](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/network/azure-mgmt-network/azure/mgmt/network/_operations_mixin.py) applicable to operation groups as well. I estimate this work to a week of dev-ish.
26 changes: 15 additions & 11 deletions doc/dev/test_proxy_troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,18 @@ Documentation of the motivations and goals of the test proxy can be found [here]
GitHub repository, and documentation of how to set up and use the proxy can be found [here][detailed_docs].

## Table of contents
- [General troubleshooting tip](#general-troubleshooting-tip)
- [Test collection failure](#test-collection-failure)
- [Errors in tests using resource preparers](#errors-in-tests-using-resource-preparers)
- [Playback failures from body matching errors](#playback-failures-from-body-matching-errors)
- [Recordings not being produced](#recordings-not-being-produced)
- [KeyError during container startup](#keyerror-during-container-startup)
- [ConnectionError during test startup](#connectionerror-during-test-startup)
- [Different error than expected when using proxy](#different-error-than-expected-when-using-proxy)
- [Test setup failure in test pipeline](#test-setup-failure-in-test-pipeline)
- [Fixture not found error](#fixture-not-found-error)
- [Guide for test proxy troubleshooting](#guide-for-test-proxy-troubleshooting)
- [Table of contents](#table-of-contents)
- [General troubleshooting tip](#general-troubleshooting-tip)
- [Test collection failure](#test-collection-failure)
- [Errors in tests using resource preparers](#errors-in-tests-using-resource-preparers)
- [Playback failures from body matching errors](#playback-failures-from-body-matching-errors)
- [Recordings not being produced](#recordings-not-being-produced)
- [KeyError during container startup](#keyerror-during-container-startup)
- [ConnectionError during test startup](#connectionerror-during-test-startup)
- [Different error than expected when using proxy](#different-error-than-expected-when-using-proxy)
- [Test setup failure in test pipeline](#test-setup-failure-in-test-pipeline)
- [Fixture not found error](#fixture-not-found-error)

## General troubleshooting tip

Expand Down Expand Up @@ -107,6 +109,7 @@ Tests that aren't recorded should omit the `recorded_by_proxy` decorator. Howeve
parameters that are provided by a preparer like the `devtools_testutils` [EnvironmentVariableLoader][env_var_loader],
you may see a new test setup error after migrating to the test proxy. For example, imagine a test is decorated with a
preparer that provides a Key Vault URL as a `azure_keyvault_url` parameter:

```python
class TestExample(AzureRecordedTestCase):

Expand All @@ -115,6 +118,7 @@ class TestExample(AzureRecordedTestCase):
```

The above would work in the old test setup, but with the test proxy, running the test will yield

```text
_______ ERROR at setup of TestExample.test_example _______
...
Expand All @@ -134,7 +138,7 @@ expected in either case.
[detailed_docs]: https://github.com/Azure/azure-sdk-tools/tree/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md
[env_var_loader]: https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/envvariable_loader.py
[env_var_section]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/test_proxy_migration_guide.md#fetch-environment-variables
[general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/README.md
[general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/documentation/test-proxy/initial-investigation.md
[mgmt_recorded_test_case]: https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/mgmt_recorded_testcase.py
[migration_guide]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/test_proxy_migration_guide.md
[proxy_pipelines]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/test_proxy_migration_guide.md#enable-the-test-proxy-in-pipelines
Expand Down
21 changes: 11 additions & 10 deletions doc/dev/tests-advanced.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
# Setup Python Development Environment - Advanced
In this document we will provide additional information about the test environments:

- [Test Mixin Classes](#test-mixin-classes)
- [Resource Preparers](#preparers)
- [Examples with Preparers](#examples-with-preparers)
- [mgmt_settings_real.py](#mgmt_settings_real-file)
- [Setup Python Development Environment - Advanced](#setup-python-development-environment---advanced)
- [Test Mixin Classes](#test-mixin-classes)
- [Preparers](#preparers)
- [Examples with Preparers](#examples-with-preparers)
- [Example 2: Basic Preparer Usage with Storage](#example-2-basic-preparer-usage-with-storage)
- [Example 3: Cached Preparer Usage](#example-3-cached-preparer-usage)
- [mgmt\_settings\_real file](#mgmt_settings_real-file)

## Test Mixin Classes
Many of our test suites use a mixin class to reduce re-writing code in multiple test files. For example, in the Tables test suite there is a `_shared` directory containing two of these mixin classes, a [sync one](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/tables/azure-data-tables/tests/_shared/testcase.py) and an [async version](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/tables/azure-data-tables/tests/_shared/asynctestcase.py). These classes will often have ways to create connection strings from an account name and key, formulate the account url, configure logging, or validate service responses. In order for these mixin classes to be used by both the functional and unit tests they should inherit from `object`. For example:
Expand Down Expand Up @@ -74,8 +77,8 @@ class TestTablesUnit(TablesTestMixin):
assert client.account_url == "https://{}.tables.core.windows.net/".format(account)
```


## Preparers

The Azure SDK team has created some in house tools to help with easier testing. These additional tools are located in the `devtools_testutils` package that was installed with your `dev_requirements.txt`. In this package are the preparers that will be commonly used throughout the repository to test various resources. A preparer is a way to programmatically create fresh resources to run our tests against and then deleting them after running a test suite. These help guarantee standardized behavior by starting each test group from a fresh resource and account.

If this situation is a requirement for your tests, you can opt to create a new preparer for your service from the management plane library for a service. There are already a few preparers built in the [devtools_testutils](https://github.com/Azure/azure-sdk-for-python/tree/main/tools/azure-sdk-tools/devtools_testutils). Most prepares will start with the [`ResourceGroupPreparer`](https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/resource_testcase.py#L29-L99) to first create a resource group for your service.
Expand All @@ -86,8 +89,6 @@ To build your own preparer you will need to use the management plane library to
|-|-|-|
| Resource Group | [link](https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/resource_testcase.py#L57-L85) | [link](https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/resource_testcase.py#L87-L99) |
| Storage Account | [link](https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/storage_testcase.py#L53-L102) | [link](https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/storage_testcase.py#L104-L107) |
| KeyVault | [link](https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/keyvault_preparer.py#L84-L131) | [link](https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/keyvault_preparer.py#L133-L138) |


## Examples with Preparers

Expand All @@ -107,7 +108,7 @@ from devtools_testutils import (
class ExampleStorageTestCase(AzureTestCase):

@ResourceGroupPreparer()
@StorageAcountPreparer()
@StorageAccountPreparer()
def test_create_table(self, resource_group, location, storage_account, storage_account_key):
account_url = self.account_url(storage_account, "table")
client = self.create_client_from_credential(TableServiceClient, storage_account_key, account_url=account_url)
Expand Down Expand Up @@ -141,7 +142,7 @@ from devtools_testutils import (
class ExampleStorageTestCase(AzureTestCase):

@CachedResourceGroupPreparer(name_prefix="storagetest")
@CachedStorageAcountPreparer(name_prefix="storagetest")
@CachedStorageAccountPreparer(name_prefix="storagetest")
def test_create_table(self, resource_group, location, storage_account, storage_account_key):
account_url = self.account_url(storage_account, "table")
client = self.create_client_from_credential(TableServiceClient, storage_account_key, account_url=account_url)
Expand All @@ -152,7 +153,7 @@ class ExampleStorageTestCase(AzureTestCase):
assert valid_table_name == table.table_name

@CachedResourceGroupPreparer(name_prefix="storagetest")
@CachedStorageAcountPreparer(name_prefix="storagetest")
@CachedStorageAccountPreparer(name_prefix="storagetest")
def test_create_table_if_exists (self, resource_group, location, storage_account, storage_account_key):
account_url = self.account_url(storage_account, "table")
client = self.create_client_from_credential(TableServiceClient, storage_account_key, account_url=account_url)
Expand Down
11 changes: 6 additions & 5 deletions doc/dev/tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,12 +264,13 @@ ServicePreparer = functools.partial(
```

The parameters for the `functools.partial` method are:
* The EnvironmentVariableLoader class
* The service folder that holds your code (in this example, `sdk/testservice`). This value is used to search your

- The EnvironmentVariableLoader class
- The service folder that holds your code (in this example, `sdk/testservice`). This value is used to search your
environment variables for the appropriate values.
* The remaining arguments are key-value kwargs, with the keys being the environment variables needed for the tests, and
- The remaining arguments are key-value kwargs, with the keys being the environment variables needed for the tests, and
the value being a fake value to use in recordings.
* These values should have the same formatting as the real values because they are used in playback mode and will need
- These values should have the same formatting as the real values because they are used in playback mode and will need
to pass any client side validation. The fake value should also be a unique value to the other key-value pairs.

A method that's decorated by the ServicePreparer from the example would be called with `testservice_endpoint` and
Expand Down Expand Up @@ -733,7 +734,7 @@ Tests that use the Shared Access Signature (SAS) to authenticate a client should
[packaging]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/packaging.md
[podman]: https://podman.io/
[proxy_cert_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/documentation/test-proxy/trusting-cert-per-language.md
[proxy_general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/README.md
[proxy_general_docs]: https://github.com/Azure/azure-sdk-tools/blob/main/tools/test-proxy/Azure.Sdk.Tools.TestProxy/README.md
[proxy_migration_guide]: https://github.com/Azure/azure-sdk-for-python/blob/main/doc/dev/test_proxy_migration_guide.md
[py_sanitizers]: https://github.com/Azure/azure-sdk-for-python/blob/main/tools/azure-sdk-tools/devtools_testutils/sanitizers.py
[pytest_invocation]: https://pytest.org/latest/how-to/usage.html
Expand Down
9 changes: 6 additions & 3 deletions scripts/cadl_refresh_sdk/README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
# Overview

The script is to refresh SDK code

# Prerequisites
1. Install [Powershell 7+](https://learn.microsoft.com/en-us/shows/it-ops-talk/how-to-install-powershell-7)
## Prerequisites

1. Install [Powershell 7+](https://learn.microsoft.com/shows/it-ops-talk/how-to-install-powershell-7)
2. Install [Python 3.7+](https://www.python.org/downloads/release/python-370/)
3. Install [Nodejs](https://nodejs.org/en/download/)

# Usage
## Usage

Step into root folder of this repo and run cmd with sdk folder:
```powershell
D:\azure-sdk-for-python> python .\scripts\cadl_refresh_sdk\main.py .\sdk\contosowidgetmanager\azure-contosowidgetmanager
Expand Down
8 changes: 4 additions & 4 deletions sdk/videoanalyzer/azure-media-videoanalyzer-edge/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Use the client library for Video Analyzer Edge to:
- Simplify interactions with the [Microsoft Azure IoT SDKs](https://github.com/azure/azure-iot-sdks)
- Programmatically construct pipeline topologies and live pipelines

[Package (PyPI)][package] | [Product documentation][doc_product] | [Direct methods][doc_direct_methods] | [Pipelines][doc_pipelines] | [Source code][source] | [Samples][samples]
[Package (PyPI)][package] | [Product documentation][doc_product] | [Direct methods][doc_direct_methods] | [Archived MS Docs][doc_pipelines] | [Source code][source] | [Samples][samples]

## Getting started

Expand Down Expand Up @@ -142,12 +142,12 @@ additional questions or comments.
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
[coc_contact]: mailto:opencode@microsoft.com
[doc_pipelines]: https://go.microsoft.com/fwlink/?linkid=2162396
[doc_pipelines]: https://github.com/MicrosoftDocs/azure-docs/blob/4ba87bedc7b17a32903c99afb3ca4163be0dcc90/articles/azure-video-analyzer/video-analyzer-docs/overview.md
[package]: https://aka.ms/ava/sdk/client/python
[source]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/videoanalyzer
[samples]: https://go.microsoft.com/fwlink/?linkid=2162278
[doc_product]: https://go.microsoft.com/fwlink/?linkid=2162396
[doc_direct_methods]: https://go.microsoft.com/fwlink/?linkid=2162396
[doc_product]: https://github.com/MicrosoftDocs/azure-docs/blob/4ba87bedc7b17a32903c99afb3ca4163be0dcc90/articles/azure-video-analyzer/video-analyzer-docs/overview.md
[doc_direct_methods]: https://github.com/MicrosoftDocs/azure-docs/blob/4ba87bedc7b17a32903c99afb3ca4163be0dcc90/articles/azure-video-analyzer/video-analyzer-docs/overview.md
[iot-device-sdk]: https://pypi.org/project/azure-iot-device/
[iot-hub-sdk]: https://pypi.org/project/azure-iot-hub/
[github-page-issues]: https://github.com/Azure/azure-sdk-for-python/issues
Expand Down
2 changes: 1 addition & 1 deletion sdk/videoanalyzer/azure-mgmt-videoanalyzer/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Microsoft Azure SDK for Python

We’re retiring the Azure Video Analyzer preview service; you're advised to transition your applications off of Video Analyzer by 01 December 2022. This SDK is no longer maintained and won’t work after the service is retired. To learn how to transition off, please refer to: [Transition from Azure Video Analyzer - Azure Video Analyzer | Microsoft Docs](https://docs.microsoft.com/azure/azure-video-analyzer/video-analyzer-docs/transition-from-video-analyzer)
We’re retiring the Azure Video Analyzer preview service; you're advised to transition your applications off of Video Analyzer by 01 December 2022. This SDK is no longer maintained and won’t work after the service is retired. To learn how to transition off, please refer to: [Transition from Azure Video Analyzer - Azure Video Analyzer | Archived Microsoft Docs](https://github.com/MicrosoftDocs/azure-docs/blob/4ba87bedc7b17a32903c99afb3ca4163be0dcc90/articles/azure-video-analyzer/video-analyzer-docs/transition-from-video-analyzer.md)

The complete list of available packages can be found at: https://aka.ms/azsdk/python/all

0 comments on commit 962f930

Please sign in to comment.