Skip to content

Commit

Permalink
refactor: Separate harness run from regular run (stoplightio#367)
Browse files Browse the repository at this point in the history 
* fix: do not require source code

* fix: transfer dist and root on build options

* fix: trnasfer dist and root in build files

* fix: i hate oclif

* feat: move process management to circle

* refactor: put out specs in processes

* chore: ignore host

* chore: set deps

* docs: partial updates
  • Loading branch information
@XVincentX
XVincentX authored Jun 19, 2019
1 parent 6001407 commit 05948e9
Show file tree
Hide file tree
Showing 26 changed files with 133 additions and 266 deletions.
54 changes: 45 additions & 9 deletions .circleci/config.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,23 +29,53 @@ jobs:
- run:
name: Run tests
command: yarn test.coverage --verbose
- run:
name: cc-after
command: |
./cc-test-reporter after-build --coverage-input-type lcov --exit-code $?
- run:
name: Run build
command: yarn build
harness:
docker:
- image: circleci/node:10
steps:
- checkout
- restore_cache:
name: Restore Yarn Package Cache
keys:
- yarn-packages-{{ .Branch }}-{{ checksum "yarn.lock" }}
- run:
name: Install dockerize
command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && sudo tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
environment:
DOCKERIZE_VERSION: v0.6.1
- run:
name: Create Linux CLI binary
command: HOST=host yarn build.binary
- run:
name: Download prism v2
command: curl -L https://github.com/stoplightio/prism/releases/download/v2.0.17/prism_linux_amd64 -o prism && chmod +x ./prism
name: Start Prism server for OAS2 document
command: ./cli-binaries/prism-cli-linux mock ./examples/petstore.oas2.json -p 4010
background: true
- run:
name: Test Harness
command: |
PRISM_PORT=4010 SPEC=./examples/petstore.oas2.json,./examples/petstore.oas3.json BINARY=./cli-binaries/prism-cli-linux yarn test.binary
name: Start Prism server for OAS3 document
command: ./cli-binaries/prism-cli-linux mock ./examples/petstore.oas3.json -p 4011
background: true
- run:
name: cc-after
name: Wait Prism OAS2
command: dockerize -wait tcp://localhost:4010 -timeout 10s
- run:
name: Run OAS2 Harness
command: |
./cc-test-reporter after-build --coverage-input-type lcov --exit-code $?
PRISM_PORT=4010 SPEC=./examples/petstore.oas2.json yarn test.binary
- run:
name: Run build
command: yarn build
name: Wait Prism OAS3
command: dockerize -wait tcp://localhost:4011 -timeout 10s
- run:
name: Run OAS3 Harness
command: |
PRISM_PORT=4011 SPEC=./examples/petstore.oas3.json yarn test.binary
publish:
docker:
- image: circleci/node:10
Expand Down Expand Up @@ -91,6 +121,10 @@ workflows:
filters:
tags:
only: /.*/
- harness:
filters:
tags:
only: /.*/
- publish:
filters:
branches:
Expand All @@ -99,6 +133,7 @@ workflows:
only: /^v.*/
requires:
- build
- harness
- upload_artifacts:
filters:
branches:
Expand All @@ -107,4 +142,5 @@ workflows:
only: /^v.*/
requires:
- build
- harness
- publish
4 changes: 4 additions & 0 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,10 @@
"*.ts": [
"tslint -p packages/tsconfig.test.json --fix",
"git add"
],
"*.json": [
"prettier --write",
"git add"
]
},
"husky": {
Expand Down
22 changes: 5 additions & 17 deletions test-harness/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,35 +17,23 @@ If the dependencies were not installed, you might experience `Error: command moc

## To run tests, run any of the following:

_There is no need to manually start prism binary to run tests._
Start Prism with the target file you'd like and then run the command

1. `yarn test.binary`
2. `SPEC=./examples/petstore.oas2.json BINARY=./cli-binaries/prism-cli-linux yarn test.binary`
3. `SPEC=./examples/petstore.oas2.json,./examples/petstore.oas3.json BINARY=./cli-binaries/prism-cli-linux yarn test.binary`
4. `RUN_V2_TESTS=1 SPEC=./examples/petstore.oas2.json,./examples/petstore.oas3.json yarn test.binary`
2. `SPEC=./examples/petstore.oas2.json yarn test.binary`
3. `SPEC=./examples/petstore.oas2.json PRISM_PORT=4011 yarn test.binary`

> `SPEC` can take comma delimited paths to specs
Adding `RUN_V2_TESTS=1` will additionally run the tests against `prism` version two binary.
Please make sure that `prism` version 2 is available in the project root directory beforehand.
If you have `prism` version 2 installed already, you can symlink the executable.
Also, you can just run `curl -L https://github.com/stoplightio/prism/releases/download/v2.0.17/prism_linux_amd64 -o prism && chmod +x ./prism`
to have `prism` version 2 downloaded to the project root directory.

When doing `yarn test.binary`, the envs are optional, they have defaults.

1. `SPEC` defaults to using `petstore.oas2.json`
2. `BINARY` defaults to using `prism-cli-linux`
2. `PRISM_PORT` defaults to using `4010`

## To record gold master files:

1. `SPEC=./examples/petstore.oas2.json BINARY=./cli-binaries/prism-cli-linux yarn run.binary` - this will start up prism binary. When doing `yarn run.binary`, both `SPEC` and `BINARY` have to be defined.
1. Run the Prism binary with the target file you want to record
2. in another terminal window: `node test-harness/createMasterFiles.js` - this will use requests definitions from `requests.js` and save master files under `/gold-master-files`

Gold master files contain data about both request and response.

## Specifying a port for prism

When doing `yarn test.binary`, `yarn run.binary` or `yarn run.binary.v2`, you can specify `PRISM_PORT` environment variable.
So, for example, you can run `PRISM_PORT=8090 SPEC=./examples/petstore.oas2.json,./examples/petstore.oas3.json BINARY=./cli-binaries/prism-cli-linux yarn test.binary`
to have prism running on port `8090`. If not specified, prism will run on port `4010`.
5 changes: 3 additions & 2 deletions test-harness/createMasterFiles.js
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
const fs = require('fs');
const { join } = require('path');
const { omit } = require('lodash')
const requests = require('./requests');

const { makeRequest, constructMasterFileName } = require('./helpers');
Expand All @@ -19,13 +20,13 @@ async function recordMasterFile({ path, method, headers, body }) {
body,
}))
}.json`,
`${JSON.stringify(reqRes, null, 2)}\n`
`${JSON.stringify(omit(reqRes, 'request.host'), null, 2)}\n`
)
} catch (err) {
console.error(err);
}
}

(async function() {
(async function () {
await Promise.all(requests.map(recordMasterFile));
})();
15 changes: 4 additions & 11 deletions test-harness/gold-master-files/__path____no_auth_pets?name=abc___method___GET__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,15 @@
"request": {
"method": "GET",
"headers": {},
"path": "/no_auth/pets?name=abc",
"host": "http://localhost:4010"
"path": "/no_auth/pets?name=abc"
},
"response": {
"status": 200,
"statusText": "OK",
"headers": {
"content-type": [
"application/json; charset=utf-8"
],
"content-length": [
"34"
],
"connection": [
"close"
]
"content-type": ["application/json; charset=utf-8"],
"content-length": ["34"],
"connection": ["close"]
},
"body": [
{
Expand Down
15 changes: 4 additions & 11 deletions ...h____no_auth_pets_10___method___GET___headers____accept___application_idonotexist___.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,22 +4,15 @@
"headers": {
"accept": "application/idonotexist"
},
"path": "/no_auth/pets/10",
"host": "http://localhost:4010"
"path": "/no_auth/pets/10"
},
"response": {
"status": 406,
"statusText": "Not Acceptable",
"headers": {
"content-type": [
"application/json; charset=utf-8"
],
"content-length": [
"100"
],
"connection": [
"close"
]
"content-type": ["application/json; charset=utf-8"],
"content-length": ["100"],
"connection": ["close"]
},
"body": {
"statusCode": 406,
Expand Down
15 changes: 4 additions & 11 deletions test-harness/gold-master-files/__path____no_auth_pets_123?__code=404___method___GET__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,15 @@
"request": {
"method": "GET",
"headers": {},
"path": "/no_auth/pets/123?__code=404",
"host": "http://localhost:4010"
"path": "/no_auth/pets/123?__code=404"
},
"response": {
"status": 404,
"statusText": "Not Found",
"headers": {
"content-type": [
"application/json; charset=utf-8"
],
"content-length": [
"21"
],
"connection": [
"close"
]
"content-type": ["application/json; charset=utf-8"],
"content-length": ["21"],
"connection": ["close"]
},
"body": {
"msg": "no_resource"
Expand Down
3 changes: 1 addition & 2 deletions test-harness/gold-master-files/__path____no_auth_pets_123?__code=418___method___GET__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
"request": {
"method": "GET",
"headers": {},
"path": "/no_auth/pets/123?__code=418",
"host": "http://localhost:4010"
"path": "/no_auth/pets/123?__code=418"
},
"response": {
"status": 418,
Expand Down
3 changes: 1 addition & 2 deletions test-harness/gold-master-files/__path____no_auth_pets_123?__code=499___method___GET__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
"request": {
"method": "GET",
"headers": {},
"path": "/no_auth/pets/123?__code=499",
"host": "http://localhost:4010"
"path": "/no_auth/pets/123?__code=499"
},
"response": {
"status": 499,
Expand Down
3 changes: 1 addition & 2 deletions test-harness/gold-master-files/__path____no_auth_pets_123___method___GET__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
"request": {
"method": "GET",
"headers": {},
"path": "/no_auth/pets/123",
"host": "http://localhost:4010"
"path": "/no_auth/pets/123"
},
"response": {
"status": 200,
Expand Down
15 changes: 4 additions & 11 deletions ...ers____Content-Type___application_x-www-form-urlencoded____body___name=just_a_name__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,15 @@
"headers": {
"Content-Type": "application/x-www-form-urlencoded"
},
"path": "/no_auth/pets/123",
"host": "http://localhost:4010"
"path": "/no_auth/pets/123"
},
"response": {
"status": 415,
"statusText": "Unsupported Media Type",
"headers": {
"content-type": [
"application/json; charset=utf-8"
],
"content-length": [
"87"
],
"connection": [
"close"
]
"content-type": ["application/json; charset=utf-8"],
"content-length": ["87"],
"connection": ["close"]
},
"body": {
"statusCode": 415,
Expand Down
15 changes: 4 additions & 11 deletions test-harness/gold-master-files/__path____no_auth_pets___method___GET__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,15 @@
"request": {
"method": "GET",
"headers": {},
"path": "/no_auth/pets",
"host": "http://localhost:4010"
"path": "/no_auth/pets"
},
"response": {
"status": 422,
"statusText": "Unprocessable Entity",
"headers": {
"content-type": [
"application/problem+json"
],
"content-length": [
"275"
],
"connection": [
"close"
]
"content-type": ["application/problem+json"],
"content-length": ["275"],
"connection": ["close"]
},
"body": {
"type": "https://stoplight.io/prism/errors#UNPROCESSABLE_ENTITY",
Expand Down
15 changes: 4 additions & 11 deletions ...antity__3__shipDate___2002-10-02T10_00_00-05_00___status___placed___complete__true__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,22 +5,15 @@
"headers": {
"Content-Type": "application/json"
},
"path": "/no_auth/pets",
"host": "http://localhost:4010"
"path": "/no_auth/pets"
},
"response": {
"status": 422,
"statusText": "Unprocessable Entity",
"headers": {
"content-type": [
"application/problem+json"
],
"content-length": [
"518"
],
"connection": [
"close"
]
"content-type": ["application/problem+json"],
"content-length": ["518"],
"connection": ["close"]
},
"body": {
"type": "https://stoplight.io/prism/errors#UNPROCESSABLE_ENTITY",
Expand Down
15 changes: 4 additions & 11 deletions ...les/__path____no_auth_pets_findByStatus?status=available&__code=499___method___GET__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,15 @@
"request": {
"method": "GET",
"headers": {},
"path": "/no_auth/pets/findByStatus?status=available&__code=499",
"host": "http://localhost:4010"
"path": "/no_auth/pets/findByStatus?status=available&__code=499"
},
"response": {
"status": 500,
"statusText": "Internal Server Error",
"headers": {
"content-type": [
"application/problem+json"
],
"content-length": [
"139"
],
"connection": [
"close"
]
"content-type": ["application/problem+json"],
"content-length": ["139"],
"connection": ["close"]
},
"body": {
"type": "https://stoplight.io/prism/errors#UNKNOWN",
Expand Down
3 changes: 1 addition & 2 deletions ...es/__path____no_auth_pets_findByStatus?status=available&status=sold___method___GET__.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
"request": {
"method": "GET",
"headers": {},
"path": "/no_auth/pets/findByStatus?status=available&status=sold",
"host": "http://localhost:4010"
"path": "/no_auth/pets/findByStatus?status=available&status=sold"
},
"response": {
"status": 200,
Expand Down
Loading

0 comments on commit 05948e9

Please sign in to comment.