Skip to content

Instantly share code, notes, and snippets.

@jiffle
Last active July 9, 2024 16:20
Show Gist options
  • Save jiffle/a1951268cb1d07e0d98d469cbab458db to your computer and use it in GitHub Desktop.
Save jiffle/a1951268cb1d07e0d98d469cbab458db to your computer and use it in GitHub Desktop.
Evolving REST Services and Backward Compatibility

Evolving Rest Services and Backward Compatibility

My aim is to define a set of rules that can be used to decide whether an incremental data change is backwardly compatible or not, list things to beware of for a particular change, and mitigations that can be put in place.

Clearly it is assumed that the services will be using API versioning, and that a major API version number change would allow any required change to that API. The scope of this document is to explore what changes can be made within a single API version.

Main Release Management models

There are four main release management models, each of which has its own requirements and challenges in versioning the APIs.

  • Lock-step: Releases done together: manifest of services deployed and tested together. No backward compatibility required between releases and no discussion is included here. Development-time backward compatibility may however well still be needed (and that evolution model determined). Advantages of microservice decoupling lost.
  • Server-first: The service provider is always upgraded before the clients. May become difficult to enforce if there are circular dependencies between services.
  • Client-first: The client is updated ahead of the service implementation. Unlikely to be a system-wide release policy, however included for scenarios where external requirements may require this approach on an individual-case basis.
  • Uncontrolled: Both clients and servers can be upgraded independently. Included mostly to highlight how difficult/impossible it is to perform incremental versioning in this environment.

In the examples below the words ‘mandatory’ and ‘optional’ relate to whether the field is required to exist, and be non-null for the recipient to process.

Server-First Versioning:

As the server is always upgraded ahead of the clients, the pattern can be used is similar to the Covariance/Contravariance model used in object subclasses, with the restrictions on evolving request data being different from those on evolving response data.

Evolving Request Data

Modification Allowed Details Notes / Mitigation
None → Optional Server must implement default processing if field not present
None → Mandatory Legacy clients will cause request processing to fail -
Optional → Mandatory Legacy clients will cause request processing to fail -
Mandatory → Optional New server default processing for missing optional only invoked for updated clients -
Mandatory → None ✅❓ Client data will be ignored Aggressive request validation will cause extraneous fields to fail the request
Optional → None ✅❓ Client data will be ignored Aggressive request validation will cause extraneous fields to fail the request
Type change: More specialised 1 Legacy clients will send data that cause the requests to fail Add new, optional, field that updated clients can use to process the more specific data. Remove old data-type on major version bump3
Type change: More general 2 Server will process legacy client data in a more general way -
Enumeration addition Legacy Clients will not send -
Enumeration removal Only if field is optional Legacy Clients will be supported if server is tolerantly implemented - removed field mapped to null. Review Impact
Enumeration change ❌❓ Conditions for removal apply as above Safer to Add new, then remove legacy on version bump

Evolving Response Data

Modification Allowed Details Notes / Mitigation
None → Optional Legacy clients unaffected by new field -
None → Mandatory Legacy clients unaffected by new field -
Optional → Mandatory Legacy clients unaffected by new field behaviour -
Mandatory → Optional Clients that depend on field may break Audit Clients
Mandatory → None Clients that depend on field may break Audit Clients
Optional → None As long as previous client processing scenarios do not require field Audit Clients
Type change: More specialised 1 Legacy clients can process data as more general type
Type change: More general 2 Legacy clients will receive unexpected data Add new, optional, field that updated servers can use to return more general data. Remove old data-type on major version bump3
Enumeration addition Only if field is optional Legacy Clients will work only if tolerantly implemented - new field mapped to null
Enumeration removal No effect -
Enumeration change Conditions for addition apply as above Add, then remove legacy on version bump

Client-First Versioning:

Where the clients are always upgraded ahead of the server. It is effectively the reverse of the server-first model above (the evolution model for the request becomes that for the response and vice versa).

Evolving Request Data

Modification Allowed Details Notes / Mitigation
None → Optional Legacy server unaffected by new field -
None → Mandatory Legacy server unaffected by new field -
Optional → Mandatory Legacy server unaffected by new field behaviour -
Mandatory → Optional Server that depends on field may break Audit Server
Mandatory → None Server that depends on field may break Audit Server
Optional → None As long as previous server processing scenarios do not require field Audit Server
Type change: More specialised 1 Legacy server can process data as more general type
Type change: More general 2 Legacy server will receive unexpected data Add new, optional, field that updated clients can use to send more general data. Remove old data-type on major version bump3
Enumeration addition Only if field is optional Legacy server will work only if tolerantly implemented - new field mapped to null
Enumeration removal No effect -
Enumeration change Conditions for addition apply as above Add, then remove legacy on version bump

Evolving Response Data

Modification Allowed Details Notes / Mitigation
None → Optional Client must implement default processing if field not present
None → Mandatory Legacy server will cause response processing to fail -
Optional → Mandatory Legacy server will cause response processing to fail -
Mandatory → Optional New client default processing for missing optional only invoked when server updated -
Mandatory → None ✅❓ Server data will be ignored Aggressive response validation will cause extraneous fields to fail processing
Optional → None ✅❓ Server data will be ignored Aggressive response validation will cause extraneous fields to fail processing
Type change: More specialised 1 Legacy server will send data that cause the response processing to fail Add new, optional, field that updated server can use to send the more specific data. Remove old data-type on major version bump3
Type change: More general 2 Client will process legacy server data in a more general way -
Enumeration addition Legacy Server will not send -
Enumeration removal Only if field is optional Legacy Server OK if server is tolerantly implemented - e.g. removed field mapped to null. Review Impact
Enumeration change ❌❓ Conditions for removal apply as above Safer to Add new, then remove legacy on version bump

Uncontrolled: Client or server first

Both Request and Response Data

Modification Allowed Details Notes / Mitigation
None → Optional Both sides can support when needed -
None → Mandatory Will break if recipient-first -
Optional → Mandatory Will break if recipient-first -
Mandatory → Optional Will break if sender-first Audit Recipient Impact
Mandatory → None Will break if sender-first Audit Recipient Impact
Optional → None As long as not required by particular processing scenarios and recipient does not aggressively validate Audit Recipient Impact
Type change: More specialised 1 Will break if recipient-first Add new, optional, field that sender can use to send more specific data. Remove old data-type on major version bump3
Type change: More general 2 Will break if sender-first Add new, optional, field that sender can use to send more general data. Remove old data-type on major version bump3
Enumeration addition Possible if field is optional Recipient can accept only if tolerantly implemented - new field processed as null
Enumeration removal Possible if field is optional Recipient can accept only if tolerantly implemented - old field processed as null
Enumeration change No Mitigation: Create completely new (optional) enumeration field3

Footnotes:

1: Change of the types used in the data field to be more specific. For example, String --> Int

2: Change of the types used in the data field to be more general. For example, Int --> String

3: The Expand/Contract pattern, also documented by Martin Fowler as Parallel Change

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment