Instill end-user confidence in data integrity

[gobengo]

Challenge Description

• users want to store their data on someone else's computer and fetch it later
• but it could be tampered with or otherwise corrupted while on the other person's computer
• when the end-user retrieves the data from storage, how do they know some data wasn't lost, or malicious data wasn't added?

Impact and Importance

• Without communicating data integrity, end-users may not be able to ensure the Data Space is, in fact, storing the entirety of data
• Without data integrity checks, there is no way to check the Data Space server did not inject malware or other junk data into the served data

Desired Solution

• individual data objects in LWS should have something like one or more checksums. The end-user should be able to deterministically create a checksum from a representation of the Data Space Data Object. The Data Space Server should be able to advertise the checksum of the objects it stores.
• If Data Space allows organizing several data objects into a collection, the collection itself should also have a checksum that changes when collections members are added, removed, and updated
• If I have data in Data Space A with checksum X, and i migrate the data to Data Space B. The checksum should still be X.

Acceptance Criteria

References and Resources

Additional Notes

• this could be necessary but not sufficient https://www.w3.org/TR/vc-data-integrity/

[pietercolpaert]

I take it LWS refers to the on-going Linked Web Storage working group from W3C? https://www.w3.org/groups/wg/lws/

I believe an additional complexity here is how to refer to a “package” of data on which the checksum should apply. Will the checksum only be based on an HTTP response or file? Or should we also consider the fact that we can refer to a set of triples/quads as a package?

[JohannesLipp]

+1 for the challenge and general approach.

Just two things that came into my mind:

1. The term "the Data Space server" should be improved, because there is no single dataspace server. Dataspaces are distributed systems per definition / design.
2. Dataspaces distinguish between data discovery and data access. Thus, is might make sense to not only apply this to the data but also to its metadata to enable the same benefits for it.