Merge pull request grpc#6652 from jtattermusch/master

Add very much needed documentation for tools/run_tests/ directory
gillesz · May 19, 2016 · 19fa14b · 19fa14b
2 parents 54140f9 + bf682f4
commit 19fa14b
Show file tree

Hide file tree

Showing 3 changed files with 56 additions and 2 deletions.
diff --git a/README.md b/README.md
@@ -11,10 +11,12 @@ Copyright 2015 Google Inc.
 
 You can find more detailed documentation and examples in the [doc](doc) and [examples](examples) directories respectively.
 
-#Installation
+#Installation & Testing
 
 See [INSTALL](INSTALL.md) for installation instructions for various platforms.
 
+See [tools/run_tests](tools/run_tests) for more guidance on how to run various test suites (e.g. unit tests, interop tests, benchmarks)
+
 #Repository Structure & Status
 
 This repository contains source code for gRPC libraries for multiple languages written on top of shared C core library [src/core] (src/core).

diff --git a/tools/README.md b/tools/README.md
@@ -1,6 +1,6 @@
 buildgen: Template renderer for our build system.
 
-distrib: Scripts to distribute language-specific packages.
+distrib: Scripts to distribute language-specific packages and other distribution-related helper scripts.
 
 dockerfile: Docker files to test gRPC.
 

diff --git a/tools/run_tests/README.md b/tools/run_tests/README.md
@@ -0,0 +1,52 @@
+#Overview
+
+This directory contains scripts that facilitate building and running tests. We are using python scripts as entrypoint for our
+tests because that gives us the opportunity to run tests using the same commandline regardless of the platform you are using.
+
+#Unit tests (run_tests.py)
+
+Builds gRPC in given language and runs unit tests. Use `tools/run_tests/run_tests.py --help` for more help.
+
+######Example
+`tools/run_tests/run_tests.py -l csharp -c dbg`
+
+######Useful options (among many others)
+- `--use_docker` Builds a docker container containing all the prerequisites for given language and runs the tests under that container.
+- `--build_only` Only build, do not run the tests.
+
+#Interop tests (run_interop_tests.py)
+
+Runs tests for cross-platform/cross-language interoperability. For more details, see [Interop tests descriptions](/doc/interop-test-descriptions.md)
+The script is also capable of running interop tests for grpc-java and grpc-go, using sources checked out alongside the ones of the grpc repository.
+
+######Example
+`tools/run_tests/run_interop_tests.py -l csharp -s c++ --use_docker` (run interop tests with C# client and C++ server)
+
+#Performance benchmarks (run_performance_tests.py)
+
+Runs predefined benchmark scenarios for given languages. Besides the simple configuration of running all the scenarios locally,
+the script also supports orchestrating test runs with client and server running on different machines and uploading the results
+to BigQuery.
+
+######Example
+`tools/run_tests/run_peformance_tests.py -l c++ node`
+
+######Useful options
+- `--regex` use regex to select particular scenarios to run.
+
+#Stress tests (run_stress_tests.py)
+
+Runs modified interop tests clients and servers under heavy load for an extended period of time to discover potential stability issues.
+The tests are internally using Kubernetes to run the client and server on GKE and upload statistics to BigQuery.
+
+`tools/run_tests/stress_test/run_on_gke.py --gcp_project_id=<google-cloud-platform-project-id> --config_file=<path-to-config-file>` 
+
+The directory `tools/run_tests/stress_test/configs/` contains the config files for several scenarios
+
+#Artifacts & Packages (task_runner.py)
+
+A generalized framework for running predefined tasks based on their labels. We use this to building binary artifacts & distrib packages and testing them)
+
+######Example
+`tools/run_tests/task_runner.py -f python artifact linux x64` (build tasks with labels `python`, `artifact`, `linux`, and `x64`)
+