feat: contract testing: add missing output checks (stoplightio#650)

XVincentX · lag-of-death · web-flow · commit 7a424bb838fd · 2019-09-27T15:24:53.000+02:00
* feat: add range status code checks

* feat: add media type check

* test: update

* fix: no error if contet is not defined

* test: add media type mismatch tests

* feat: log validation errors

* docs: add

* docs

* Apply suggestions from code review

Co-Authored-By: Phil Sturgeon &lt;phil@stoplight.io&gt;

* docs: it's pull not pulls

* Apply suggestions from code review

Co-Authored-By: lag-of-death &lt;mateuszwit21@gmail.com&gt;

* refactor: prefer inRange

* fix: handle info and hint

* Apply suggestions from code review

Co-Authored-By: lag-of-death &lt;mateuszwit21@gmail.com&gt;
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -10,6 +10,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 - Support for encoding > allowReserved flag when validating application/x-www-form-urlencoded body [#630](https://github.com/stoplightio/prism/pull/630)
 - Validating output status code against available response specs [#648](https://github.com/stoplightio/prism/pull/648)
+- Support for Contract Testing [#650](https://github.com/stoplightio/prism/pull/650)
 
 ## Fixed
 
@@ -19,42 +20,42 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ## Fixed
 
-- Prism is now giving precedence to `application/json` instead of using it as a "fallback" serializer, fixing some conditions where it wouldn't get triggered correctly. [#604](https://github.com/stoplightio/prism/pulls/604)
-- Prism is now taking in consideration the `required` properties for combined schemas (`oneOf, allOf`). This is coming through an update to the Json Schema Faker Library [#623](https://github.com/stoplightio/prism/pulls/623)
-- Prism will never have enough information to return a `403` status code; all these occurences have been now replaced with a `401` status code which is more appropriate [#625](https://github.com/stoplightio/prism/pulls/625)
-- Prism is now negotiating the error response dynamically based on the validation result (security or schema validation) instead of always returning a static order of responses [#628](https://github.com/stoplightio/prism/pulls/628)
+- Prism is now giving precedence to `application/json` instead of using it as a "fallback" serializer, fixing some conditions where it wouldn't get triggered correctly. [#604](https://github.com/stoplightio/prism/pull/604)
+- Prism is now taking in consideration the `required` properties for combined schemas (`oneOf, allOf`). This is coming through an update to the Json Schema Faker Library [#623](https://github.com/stoplightio/prism/pull/623)
+- Prism will never have enough information to return a `403` status code; all these occurences have been now replaced with a `401` status code which is more appropriate [#625](https://github.com/stoplightio/prism/pull/625)
+- Prism is now negotiating the error response dynamically based on the validation result (security or schema validation) instead of always returning a static order of responses [#628](https://github.com/stoplightio/prism/pull/628)
 - Prism is now selecting proper serializer when Accept header contains content type which is missing in spec. This is a result of simplifying serializer selection approach. [#620](https://github.com/stoplightio/prism/pull/620)
 - HEAD requests no longer fail with 406 Not Acceptable [#603](https://github.com/stoplightio/prism/pull/603)
 
 # 3.1.0 (2019-09-03)
 
 ## Added
 
-- Prism is now able to validate the security specification of the loaded document [#484](https://github.com/stoplightio/prism/pulls/484)
+- Prism is now able to validate the security specification of the loaded document [#484](https://github.com/stoplightio/prism/pull/484)
 
 ## Fixed
 
-- Prism is not crashing anymore when referencing the same model multiple times in the specification document [#552](https://github.com/stoplightio/prism/pulls/552)
-- Prism will now correctly use the `example` keyword for a Schema Object in OpenAPI 3.0 documents [#560](https://github.com/stoplightio/prism/pulls/560)
-- Prism won't return 406 when users request a `text/plain` response whose content is a primitive (string, number) [#560](https://github.com/stoplightio/prism/pulls/560)
-- Prism's router is now able to correctly handle a path ending with a parameter, such as `/test.{format}`, while it would previously not match with anything. [#561](https://github.com/stoplightio/prism/pulls/561)
-- Prism is correctly handling the `allowEmptyValue` property in OAS2 documents [#569](https://github.com/stoplightio/prism/pulls/569)
-- Prism is correctly handling the `csv` collection format argument property in OAS2 documents [#577](https://github.com/stoplightio/prism/pulls/577)
-- Prism is correctly returning the response when the request has `*/*` as Accept header [#578](https://github.com/stoplightio/prism/pulls/578)
-- Prism is correctly returning a single root node with the payload for XML data [#578](https://github.com/stoplightio/prism/pulls/578)
+- Prism is not crashing anymore when referencing the same model multiple times in the specification document [#552](https://github.com/stoplightio/prism/pull/552)
+- Prism will now correctly use the `example` keyword for a Schema Object in OpenAPI 3.0 documents [#560](https://github.com/stoplightio/prism/pull/560)
+- Prism won't return 406 when users request a `text/plain` response whose content is a primitive (string, number) [#560](https://github.com/stoplightio/prism/pull/560)
+- Prism's router is now able to correctly handle a path ending with a parameter, such as `/test.{format}`, while it would previously not match with anything. [#561](https://github.com/stoplightio/prism/pull/561)
+- Prism is correctly handling the `allowEmptyValue` property in OAS2 documents [#569](https://github.com/stoplightio/prism/pull/569)
+- Prism is correctly handling the `csv` collection format argument property in OAS2 documents [#577](https://github.com/stoplightio/prism/pull/577)
+- Prism is correctly returning the response when the request has `*/*` as Accept header [#578](https://github.com/stoplightio/prism/pull/578)
+- Prism is correctly returning a single root node with the payload for XML data [#578](https://github.com/stoplightio/prism/pull/578)
 - Prism is correctly returning payload-less responses #606
 
 # 3.0.4 (2019-08-20)
 
 ## Added
 
-- Prism is now returning CORS headers by default and responding to all the preflights requests. You can disable this behaviour by running Prism with the `--cors` flag set to false [#525](https://github.com/stoplightio/prism/pulls/525)
+- Prism is now returning CORS headers by default and responding to all the preflights requests. You can disable this behaviour by running Prism with the `--cors` flag set to false [#525](https://github.com/stoplightio/prism/pull/525)
 
 ## Fixed
 
-- Prism now respects the `nullable` value for OpenAPI 3.x documents when generating examples [#506](https://github.com/stoplightio/prism/pulls/506)
-- Prism now loads correctly OpenAPI 3.x documents with `encodings` with non specified `style` property [#507](https://github.com/stoplightio/prism/pulls/507)
-- Prism got rid of some big internal dependencies that now aren't required anymore, making it faster and lighter. [#490](https://github.com/stoplightio/prism/pulls/490)
+- Prism now respects the `nullable` value for OpenAPI 3.x documents when generating examples [#506](https://github.com/stoplightio/prism/pull/506)
+- Prism now loads correctly OpenAPI 3.x documents with `encodings` with non specified `style` property [#507](https://github.com/stoplightio/prism/pull/507)
+- Prism got rid of some big internal dependencies that now aren't required anymore, making it faster and lighter. [#490](https://github.com/stoplightio/prism/pull/490)
 - Prism now correctly validates OAS2 `application/x-www-urlencoded` (form data) params (#483)
 
 # 3.0.3 (2019-07-25)
diff --git a/README.md b/README.md
@@ -42,7 +42,7 @@ After [installation](https://stoplight.io/p/docs/gh/stoplightio/prism/docs/getti
 **Cannot access mock server when using docker?**
 
 Prism uses localhost by default, which usually means 127.0.0.1. When using docker the mock server will
-be unreachable outside of the container unless you run the mock command with `-h 0.0.0.0`. 
+be unreachable outside of the container unless you run the mock command with `-h 0.0.0.0`.
 
 **Why am I getting 404 errors when I include my basePath?**
 
diff --git a/docs/guides/client.md b/docs/guides/client.md
@@ -0,0 +1,83 @@
+# Prism Client
+
+Prism includes a fully-featured HTTP Client that you can use to seamlessly perform requests to both a real server and a mocked document. The client is modeled after Axios so it may feel familiar.
+
+### Create from a filename or http resource
+
+```ts
+const client = await createClientFromResource('examples/petstore.oas2.yaml', {
+  mock: true,
+  validateRequest: true,
+  validateResponse: true,
+});
+```
+
+### Create from OpenAPI string
+
+```ts
+const descriptionDoc = `
+openapi: 3.0.2
+paths:
+  /hello:
+    get:
+      responses:
+        200:
+          description: hello
+`;
+
+const client = await createClientFromString(descriptionDoc, {
+  mock: true,
+  validateRequest: true,
+  validateResponse: true,
+});
+```
+
+### Create From Manual Http Operations
+
+```ts
+const client = createClientFromOperations(
+  [
+    {
+      method: 'get',
+      path: '/hello',
+      id: 'n1',
+      responses: [{ code: '200' }],
+    },
+  ],
+  { mock: true, validateRequest: true, validateResponse: true }
+);
+```
+
+---
+
+Once you've got a client instance:
+
+1. You can perform the request using the generic method:
+
+```ts
+client.request('https://google.it', { method: 'get' }).then(response => console.log(response));
+```
+
+The response object has all the information you need, including the used configuration object.
+
+2. You can override the configuration object on the request level if you prefer
+
+```ts
+client
+  .request('https://google.it', { method: 'get' }, { validateResponse: false })
+  .then(response => console.log(response));
+```
+
+This disables response validation _only for the current request_
+
+3. You can do the same thing using the shortcut methods
+
+```ts
+client.get('https://google.it', { mock: false }).then(response => console.log(response));
+```
+
+For the shortcut methods (since the only mandatory option is intrinsic in the function name) the option parameter can be omitted
+
+```ts
+client.get('https://google.it', { validateRequest: false }).then(response => console.log(response));
+```
diff --git a/package.json b/package.json
@@ -68,7 +68,7 @@
       "tslint -p packages/tsconfig.test.json --fix",
       "git add"
     ],
-    "*.json, *.md": [
+    "*.{json,md}": [
       "prettier --write",
       "git add"
     ]
diff --git a/packages/http-server/src/server.ts b/packages/http-server/src/server.ts
@@ -1,6 +1,6 @@
 import { createLogger } from '@stoplight/prism-core';
 import { createInstance, IHttpConfig, IHttpMethod, PrismHttpInstance, ProblemJsonError } from '@stoplight/prism-http';
-import { IHttpOperation } from '@stoplight/types';
+import { DiagnosticSeverity, IHttpOperation } from '@stoplight/types';
 import * as fastify from 'fastify';
 import * as fastifyCors from 'fastify-cors';
 import { IncomingMessage, ServerResponse } from 'http';
@@ -108,6 +108,16 @@ export const createServer = (operations: IHttpOperation[], opts: IPrismHttpServe
             reply.headers(output.headers);
           }
 
+          response.validations.output.forEach(validation => {
+            if (validation.severity === DiagnosticSeverity.Error) {
+              request.log.error(validation.message);
+            } else if (validation.severity === DiagnosticSeverity.Warning) {
+              request.log.warn(validation.message);
+            } else {
+              request.log.info(validation.message);
+            }
+          });
+
           reply.serializer((payload: unknown) => serialize(payload, reply.getHeader('content-type'))).send(output.body);
         } else {
           throw new Error('Unable to find any decent response for the current request.');
diff --git a/packages/http/README.md b/packages/http/README.md
@@ -10,6 +10,10 @@ In essence it's an HTTP client (similar to axios - in fact, we use axios to make
 
 The goal of this document is to provide you with some basic code examples to get you started and to cover some of the advanced scenarios.
 
+## Important
+
+If you're a regular user and not a PRO, you might to want to use the [User facing client](../../docs/guides/client.md) which provides a better an higher lever API
+
 # Table of Contents
 
 - [Installation](#installation)
diff --git a/packages/http/src/validator/__tests__/__snapshots__/functional.spec.ts.snap b/packages/http/src/validator/__tests__/__snapshots__/functional.spec.ts.snap
@@ -33,6 +33,10 @@ Array [
 
 exports[`HttpValidator validateOutput() all validations are turned on returns validation errors for whole request structure 1`] = `
 Array [
+  Object {
+    "message": "The received media type does not match the one specified in the document",
+    "severity": 0,
+  },
   Object {
     "code": "type",
     "message": "should be boolean",
diff --git a/packages/http/src/validator/__tests__/index.spec.ts b/packages/http/src/validator/__tests__/index.spec.ts
diff --git a/packages/http/src/validator/index.ts b/packages/http/src/validator/index.ts

Original file line number	Diff line number	Diff line change
`@@ -68,7 +68,7 @@`
`68`	`68`	`"tslint -p packages/tsconfig.test.json --fix",`
`69`	`69`	`"git add"`
`70`	`70`	`],`
`71`		`- ".json, .md": [`
	`71`	`+ "*.{json,md}": [`
`72`	`72`	`"prettier --write",`
`73`	`73`	`"git add"`
`74`	`74`	`]`