Change max encrypted document and max chunk size

[DRK3]

closes #22

• Max encrypted document size is now 10MiB (used to be 16MB).
• Updated some text to indicate that the limit is on Encrypted Documents, not Structured Documents.

[DRK3]

Feedback from call:

Add normative language

Explain how that limit is calculated (e.g. based on UTF-8 JSON)

[DRK3]

@msporny To make this normative, should the 10MiB limit be moved to https://identity.foundation/edv-spec/#encrypteddocument?

[msporny]

@msporny To make this normative, should the 10MiB limit be moved to https://identity.foundation/edv-spec/#encrypteddocument?

Yes, it might be better to move the limit there.

As an aside, the thing that makes a statement normative or not is the use of the words (all caps): MUST, SHOULD, MAY, RECOMMENDED... and the rest of the IETF RFC normative statement language -- https://www.rfc-editor.org/rfc/rfc2119.html

ReSpec (the editing tool) is usually good about only letting you use those statements in normative sections of the specification. If you try to use those words in "informative" sections, like the introduction, it'll complain loudly. :)

[DRK3]

@msporny To make this normative, should the 10MiB limit be moved to https://identity.foundation/edv-spec/#encrypteddocument?

Yes, it might be better to move the limit there.

As an aside, the thing that makes a statement normative or not is the use of the words (all caps): MUST, SHOULD, MAY, RECOMMENDED... and the rest of the IETF RFC normative statement language -- https://www.rfc-editor.org/rfc/rfc2119.html

ReSpec (the editing tool) is usually good about only letting you use those statements in normative sections of the specification. If you try to use those words in "informative" sections, like the introduction, it'll complain loudly. :)

Good to know! I've updated my PR to include normative text under the Encrypted Document section.

An encrypted document's size MUST NOT exceed 10MiB.

[DRK3]

Regarding how the limit is calculated - does this standard mandate what character set to use? Should be pick one now? Or maybe we can create an issue to explore this later.

[msporny]

Regarding how the limit is calculated - does this standard mandate what character set to use? Should be pick one now? Or maybe we can create an issue to explore this later.

Yes, a good question. I expect that we may want to pick some place where it's easy to measure... like "on the wire". So, if your HTTP body is above 10MiB, the request is rejected. How the data is stored behind the API is implementation specific, and so will vary depending on optimizations implementations may do.

If it were me, I'd limit it to sent/received HTTP body size... then the question becomes, is that uncompressed or compressed HTTP body size? I expect that we might want application front-ends to be able to abort when you go above 10MiB without concern for uncompressed size... that is, we want the application servers to be able to abort using a simple rule... "is the HTTP body I'm receiving going above 10MiB? If so, abort... we don't care if it's compressed or uncompressed via gzip." I'll also note that because we're dealing w/ encrypted blobs, gzip compression isn't expected to provide huge savings.

[DRK3]

@msporny HTTP body size (regardless of compression) sounds like a good idea. To me. Anyone else have any comments?

[msporny]

Anyone else have any comments?

@dlongley any thoughts here?

[dlongley]

@DRK3, @msporny,

HTTP body size (regardless of compression) sounds like a good idea. To me. Anyone else have any comments?

+1 to HTTP body size. I can't remember if there will be any frustrating edge cases in implementations with transfer-encoding: chunked or that sort of thing, but we should make the spec say HTTP body size and then get implementation experience.

[DRK3]

@msporny @dlongley I've updated my PR to now specify that the size is based on the HTTP body size. Let me know if you can think of any improvements to the wording.

[DRK3]

Discussed on the July 22, 2021 WG call:

• Chunk size should be smaller (1 MiB) in order to mitigate DoS attacks (find out more quickly if a chunk is bad) (no objections)

[DRK3]

Updated this PR to specify the max chunk size.

[dmitrizagidulin]

👍