Merge pull request dotnet#1186 from zhenlan/devdocs

Update dev doc
joshfree · May 21, 2016 · 198488c · 198488c
2 parents 9076bba + a69e494
commit 198488c
Show file tree

Hide file tree

Showing 3 changed files with 87 additions and 220 deletions.
diff --git a/Documentation/cross-machine-test-guide.md b/Documentation/cross-machine-test-guide.md
diff --git a/Documentation/developer-guide.md b/Documentation/developer-guide.md
@@ -23,58 +23,64 @@ Please read [Contributing](https://github.com/dotnet/wcf/blob/master/Documentati
 Building the repository
 =======================
 
-The WCF repository can be built from a regular, non-admin command prompt using build.cmd. This build produces a single System.Private.ServiceModel assembly that implements the individual client libraries.
+The WCF repository can be built from a regular, non-admin command prompt using build.cmd on Windows or build.sh on other platforms.
+This build produces assets including a single System.Private.ServiceModel assembly that implements the individual client libraries.
 
 Microsoft uses this repository to create and publish separate NuGet packages for each library. 
 
 The repository is a work in progress, and not all of the libraries are complete yet.
 
-Running the unit tests
+Running unit tests
 ======================
 Unit tests are those that test aspects of the WCF libraries that don't require network interactions.
-To build the product and run the unit tests, simply execute this CMD from the root of the repository:
+To build the product and run the unit tests, from the root of the repository simply execute this command on Windows:
 
 ```
     build.cmd
 ```
+or this command on Linux or OS X:
+```
+    ./build.sh
+```
 
-Running the scenario tests
+Running scenario tests
 ==========================
-Scenario tests are those tests that involve network activity between the tests and one or
-more running WCF services.  By default, they are not run with the normal build.cmd.
-To run the scenario tests, execute this CMD from the root of the repository:
+Scenario tests are those tests that involve network activity between the client tests and WCF test services.
+By default, they are not run with normal build.cmd or build.sh.
 
-```
-    build.cmd /p:WithCategories=OuterLoop
-```
+To run the scenario tests on Windows, execute this CMD from the root of the repository. It will automatically start a
+self-hosted WCF test service on the same Windows machine and shut it down at the end of execution of all tests.
 
-Obtaining code coverage
-=======================
-You can also obtain detailed code coverage information by including an additional property
-when you run the build script.  For example, this CMD runs the scenario tests and collects
-code coverage numbers:
 ```
-    build.cmd /p:WithCategories=OuterLoop /p:Coverage=true
+    build.cmd /p:WithCategories=OuterLoop
 ```
+To run scenario tests against an already running WCF test service, you will need to specify the base address 
+of the WCF test service by using the `/p:ServiceUri` parameter. This is also necessary when running
+scenario tests on Linux or OS X as the WCF service can only run on Windows.
 
-Running tests with latest dependency versions
-=============================================
-Running tests against each dependency's latest version can catch functional issues. The
-`FloatingTestRuntimeDependencies` property enables this. To run unit tests against the
-latest dependencies:
+From the root of the repository, run this command to run scenario tests on Windows:
 ```
-    build.cmd /p:FloatingTestRuntimeDependencies=true
+    build.cmd /p:WithCategories=OuterLoop /p:ServiceUri=[WCFTestServiceBaseAddress]
 ```
-To run the scenario tests with latest dependencies:
+or run this command to run scenario tests on Linux or OS X.
+Note some of security related tests require extra manual setup to work, 
+see [Linux manual test guide](https://github.com/dotnet/wcf/blob/master/Documentation/manualtest-guide.md) for details.
 ```
-    build.cmd /p:FloatingTestRuntimeDependencies=true /p:WithCategories=OuterLoop
+    ./build.sh /p:WithCategories=OuterLoop /p:ServiceUri=[WCFTestServiceBaseAddress]
 ```
 
-Running scenario tests across multiple machines
-===============================================
-By default scenario tests run against WCF services that are locally self-hosted.
-To run these tests against WCF services running on a separate machine, see:
-[Cross-machine test guide](https://github.com/dotnet/wcf/blob/master/Documentation/cross-machine-test-guide.md).
-
+You will need to replace `[WCFTestServiceBaseAddress]` in above commands with an actual value.
+See [Scenario test guide](https://github.com/dotnet/wcf/blob/master/Documentation/scenario-test-guide.md)
+for further details including how to start a WCF test service.
 
+Note: in case you need to run scenario tests against a different `ServiceUri`, you can either use environment variable to override the old value or clean up `bin`
 
+Obtaining code coverage
+=======================
+You can also obtain detailed code coverage information by including an additional property
+when you run the build script. For example, this CMD runs the scenario tests and collects
+code coverage numbers on Windows:
+```
+    build.cmd /p:WithCategories=OuterLoop /p:Coverage=true
+```
+Once it is done, the result can be found at `bin\tests\coverage\index.html`.
diff --git a/Documentation/scenario-test-guide.md b/Documentation/scenario-test-guide.md
@@ -0,0 +1,51 @@
+Scenario Testing Guide
+======================
+
+Overview
+--------
+WCF scenario tests are also referred to as outerloop tests. These are tests involving communication between WCF client tests and WCF test services. The WCF client tests running on .NET Core support cross platform, which means they can run on all supported platforms such as Windows, various Linux distros, Mac OS X as well as Universal Windows Platform (UWP). The WCF test services are built on full .NET Framework that is supported on Windows only.
+
+Regardless which platform where you run WCF client tests, you will need a WCF test service for scenario tests to work. WCF Test services can be hosted in two ways like WCF users would usually do, ie., self-hosted or IIS-hosted on a Windows machine, for example, Windows Server 2012 R2 or Windows 10. The following sections provide instructions of how to host WCF test services and how to run WCF scenario tests.
+
+Self-Hosted WCF Test Service
+----------------------------
+To start a self-hosted WCF test service on a Windows machine, execute this script that is located at directory `src\System.Private.ServiceModel\tools\scripts`.
+```
+    StartWCFSelfHostedSvc.cmd
+```
+This script will add necessary firewall rules, generate and install self-signed test certificates needed by tests, build and kick off a self-hosted WCF test service process from project at `src\System.Private.ServiceModel\tools\SelfHostedWcfService`. When you are done with this WCF test service, type `exit` and press `enter` in the window where the service was started. Upon shutting down the service, this script will also clean up firewall rules and remove test certificates it installed.
+
+This script requires elevated privileges. If it is run from a non-elevated window, you may be prompted for elevated permission.
+
+IIS-Hosted WCF Test Service
+---------------------------
+To host WCF services on IIS, the Windows machine has these prerequisites
+
+1. Web Server (IIS) and Application Server roles are installed. This includes ASP.NET and WebSocket protocol.
+2. .NET Framework 4.x features including WCF service and activation features are enabled.
+3. Ports 80 (HTTP), 443 (HTTPS) and 808 (TCP) are opened.
+
+Copy script `src\System.Private.ServiceModel\tools\scripts\SetupWcfIISHostedService.cmd` to the Windows machine and run it in an elevated command window.
+```
+    SetupWcfIISHostedService.cmd {id}
+```
+
+where `id` is an identifier that will be appended to the name of the IIS application to be created, for example, command `SetupWcfIISHostedService.cmd Test` will create an IIS aplication named "WCFService*Test*". This script will create a clone of WCF GitHub repository, an IIS application pool and an IIS application from `src\System.Private.ServiceModel\tools\IISHostedWcfService`. The script will also install necessary test certificates and add IIS HTTPS binding for testing.
+
+This is one time setup. Once you set it up, you do not need to build or start the service manually. IIS will build and activate the service automatically upon the first request from tests. 
+
+Running WCF Scenario Tests
+-------------------------
+As instructed in [Developer Guide](developer-guide.md#running-scenario-tests), to run scenario tests, you will need to specify the WCF service base address via `/p:ServiceUri` parameter to the build script.
+
+The `ServiceUri` for self-hosted WCF service will be the name of the machine. For example, if you start a self-hosted WCF test service on a Windows machine named *myserver*, you will be able to run scenario tests on a Linux or a Mac machine using this command
+```
+    ./build.sh /p:WithCategories=OuterLoop /p:ServiceUri=myserver
+```
+
+The `ServiceUri` for IIS hosted WCF service will be `{MachineName}/WCFService{id}`. For example, if you set up an IIS hosted WCF service with id *Test* on a Windows machine named *myserver*, you will be able to run scenario tests on a Linux or a Mac machine using this command
+
+```
+    ./build.sh /p:WithCategories=OuterLoop /p:ServiceUri=myserver/WcfServiceTest
+```
+To run scenario tests from a Windows machine, replace `./build.sh` with `build.cmd` in above commands.