A case for provider driven contract tests

8th October 2015

Consumer driven contract testing is growing in popularity, and rightly so. It works particularly well in mature organisations where service maintainers and service consumers can communicate openly and freely. They provide a rapid source of feedback, and allows service interfaces to evolve rapidly with confidence.

However, many organisations are either sufficiently large that such communication comes with a large overhead, or organisational boundaries exist that inhibit communication between maintainers and consumers. Indeed, in many contexts, a maintainer may not even be aware of all of the consumers of the service in question.

Alternatively, the infrastructure requirements for consumer driven contract testing may be too onerous. For a contract testing tool to be used across maintainer and consumer teams, a degree of shared infrastructure and technology is required. For example, the use of (the wonderful) pact requires users to introduce ruby into their pipelines.

In these scenarios, consumer driven contract testing is not always feasible.

Provider Driven Contract Tests

These are contract tests that are written by the maintainer of the dependency. They run as part of the maintainers pipeline and serve as executable documentation regarding how the maintainer believes their library or service should be used. Also, by omission, they indicate which parts of the library or service should not be consumed and are liable to change. Consumers should only bind to behaviour dictated by the contract test.

This style of testing works well when the maintainer has no direct influence over the consumers of the dependency in question. Indeed, they may not even be aware of that the consumers exist. This is often the case with public APIs and various open source libraries. Typically, these contract tests should change very rarely. These tests serve to give the maintainer confidence that changes to the library or service do not break the contract that they have published.

Consumers may use these tests simply as documentation. They can get further confidence that their applications are consuming services correctly by running the contract test suite against any stubs that they have built.

Organisational Evolution

Using provider driven contract tests, large organisations foster collaboration between disparate departments and teams. They can be incorporated into delivery pipelines of both service maintainers and consumers.

When a provider driven contract test is forced to change in the maintainer’s pipeline, it drives communication to potentially affected teams. Similarly, when a consumer stub fails a provider driven contract test, it forces the consumer to reevaluate their understanding of the provider’s service.

Ultimately, the behaviours and infrastructure driven out by these tests pave the way for a framework of more rigorous consumer driven contract tests. Provider driven contract tests can provide a great deal of value, with a very low barrier to introduction. However, they do not provide the level of confidence that a structured set of consumer driven contract tests can.

tags: draft, post
blog comments powered by Disqus