Adding a new optional field to the json schema flagged as backwards incompatible

[kuhema]

@blacktooth
@hhkkxxx133

The diffType enum 'OBJECT_TYPE_PROPERTY_SCHEMAS_MEMBER_ADDED' is marked as backwards incompatible in the library. This diff type corresponds to an addition of a new optional field in the Json schema. Example:

Old schema:

{
	"definitions": {},
	"$schema": "http://json-schema.org/draft-07/schema#",
	"$id": "https://example.com/object1631809280.json",
	"title": "Root",
	"type": "object",
	"required": [
		"quantity",
		"status",
		"tenantId",
		"createdDate",
		"updatedDate",
		"version"
	],
	"properties": {
		"quantity": {
			"$id": "#root/productQuantity/quantity",
			"type": "integer",
			"examples": [
				10
			],
			"default": 1,
			"minimum": 0,
			"maximum": 1000000
		},
		"status": {
			"$id": "#root/status",
			"type": "string",
			"default": "",
			"examples": [
				"CANCELLED"
			],
			"minLength": 0,
			"maxLength": 32
		},
		"tenantId": {
			"$id": "#root/tenantId",
			"type": "string",
			"default": "",
			"examples": [
				"d808de29-4685-4718-9fe8-0934034593bd"
			],
			"minLength": 0,
			"maxLength": 64
		},
		"createdDate": {
			"$id": "#root/createdDate",
			"type": "string",
			"examples": [
				"2018-11-13T20:20:39Z"
			],
			"default": "1970-01-01T00:00:00Z",
			"minLength": 0,
			"maxLength": 64
		},
		"updatedDate": {
			"$id": "#root/updatedDate",
			"type": "string",
			"examples": [
				"2018-11-13T20:20:39Z"
			],
			"default": "1970-01-01T00:00:00Z",
			"minLength": 0,
			"maxLength": 64
		},
		"version": {
			"$id": "#root/version",
			"type": "integer",
			"examples": [
				1
			],
			"default": 0,
			"minimum": 0,
			"maximum": 10000000000000
		}
	}
}

Added a new optional field 'reasonCode' in the New schema:

{
	"definitions": {},
	"$schema": "http://json-schema.org/draft-07/schema#",
	"$id": "https://example.com/object1631809280.json",
	"title": "Root",
	"type": "object",
	"required": [
		"quantity",
		"status",
		"tenantId",
		"createdDate",
		"updatedDate",
		"version"
	],
	"properties": {
		"quantity": {
			"$id": "#root/productQuantity/quantity",
			"type": "integer",
			"examples": [
				10
			],
			"default": 1,
			"minimum": 0,
			"maximum": 1000000
		},
		"reasonCode": {
			"$id": "#root/reasonCode",
			"type": "string",
			"default": "",
			"examples": [
				"INCOMPLETE"
			],
			"minLength": 0,
			"maxLength": 32
		},
		"status": {
			"$id": "#root/status",
			"type": "string",
			"default": "",
			"examples": [
				"CANCELLED"
			],
			"minLength": 0,
			"maxLength": 32
		},
		"tenantId": {
			"$id": "#root/tenantId",
			"type": "string",
			"default": "",
			"examples": [
				"d808de29-4685-4718-9fe8-0934034593bd"
			],
			"minLength": 0,
			"maxLength": 64
		},
		"createdDate": {
			"$id": "#root/createdDate",
			"type": "string",
			"examples": [
				"2018-11-13T20:20:39Z"
			],
			"default": "1970-01-01T00:00:00Z",
			"minLength": 0,
			"maxLength": 64
		},
		"updatedDate": {
			"$id": "#root/updatedDate",
			"type": "string",
			"examples": [
				"2018-11-13T20:20:39Z"
			],
			"default": "1970-01-01T00:00:00Z",
			"minLength": 0,
			"maxLength": 64
		},
		"version": {
			"$id": "#root/version",
			"type": "integer",
			"examples": [
				1
			],
			"default": 0,
			"minimum": 0,
			"maximum": 10000000000000
		}
	}
}

Old sample data is compatible with both old and new schema which should make the schema update backwards compatible by definition.

{
  "quantity": 10,
  "status": "COMPLETE",
  "tenantId": "SomeTenant",
  "createdDate": "2022-01-26T14:41:14.603Z",
  "updatedDate": "2022-01-26T17:51:29.173Z",
  "version": 3
}

Wanted to understand the reason behind marking 'OBJECT_TYPE_PROPERTY_SCHEMAS_MEMBER_ADDED' as backwards incompatible. Is it a bug or is it expected behaviour?

References:

• Used jsonschema.dev for validating data against schema
• Code reference:

  apicurio-registry/schema-util/json/src/main/java/io/apicurio/registry/rules/compatibility/jsonschema/diff/DiffType.java

  Line 76 in eab9faf

  OBJECT_TYPE_PROPERTY_SCHEMAS_MEMBER_ADDED(false),

[hhkkxxx133]

Hello,

By default, any additional properties are allowed in JsonSchema according to the documentation. Backward compatibility requires data written in the old schema can still be read by the new schema. In this case, the data written by your old schema can potentially have a field with the same name reasonCode yet in a different type, for example an integer. Therefore, the new schema cannot read the data due to the type mismatch, which results in backward incompatibility.

I would suggest setting additionalProperties as false in the old schema and backward compatibility should holds.

Let me know if this helps or you have any other questions. Thanks!

[EricWittmann]

This is the way.