-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[uss_qualifier] rid: dss0130 - Cell ID synchronization #837
Conversation
940c198
to
7a698d1
Compare
) as check: | ||
searched_subs = dss.search_subs(check, VERTICES) | ||
if not searched_subs.success: | ||
check.record_failed( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This check fails DSS0130,A2-6-1,6
, but this requirement talks about 'consistent results'. Here only the success of the query is checked. Sounds like this should validate instead DSS0130.A2-3-1,3-e
?
A check for DSS0130,A2-6-1,6
may be added by validating and comparing the consistency of the actual content of the response.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't have documentation for DSS0130,A2-3-1,3-e
(or DSS0130,A2-3-1,3e
, nor DSS0130,A2-3-1
), or at least I cannot find anything in the repo's markdown files?
There are other checks in this scenario that validate DSS0130,A2-6-1,6
without checking for consistency when checking Subscriptions queried successfully
, I mainly chose to imitate what was done up to now: if this is incorrect we should address it in a separate PR?
However, given the formulation of the requirement:
- GET Subscriptions: (DSS0130,A2-6-1,6) Tests must demonstrate that the complete set of subscriptions in an area for a Net-RID Display Provider can be retrieved from any DSS instance with consistent results.
If the request failed, then the DSS instance clearly failed to return the complete set of subscriptions, so it is fair to fail that requirement.
Some of the DSS0130,A2-6-1,6
checks do verify consistency, ie, the Can query all Subscriptions in area from all DSSs
does so: a DSS deployment that consistently fails to deliver consistent (hah!) results would likely still fail the requirement.
Unless there is a rule or convention that, when the qualifier runs a block of code in self.check()
, that any referenced requirement must exhaustively be checked (ie, if the check is not marked as failed by that single block we can interpret that the requirement is fulfilled), I don't think we need to obligatorily check for consistency in here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't have documentation for DSS0130,A2-3-1,3-e (or DSS0130,A2-3-1,3e, nor DSS0130,A2-3-1 ), or at least I cannot find anything in the repo's markdown files?
My bad I got mistaken, I meant DSS0030,e
here (this one does exist).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the request failed, then the DSS instance clearly failed to return the complete set of subscriptions, so it is fair to fail that requirement.
I'd argue that in that case the DSS fails 'first' the DSS0030,e
requirement because an error response would contradict the OpenAPI specs, but indeed at a second order DSS0130,A2-6-1,6
fails too. We did have several cases where several requirements may be failed and I don't believe we converged on a convention to treat those cases, but maybe what I would suggest here is to fail both requirements in that check?
Unless there is a rule or convention that, when the qualifier runs a block of code in self.check(), that any referenced requirement must exhaustively be checked (ie, if the check is not marked as failed by that single block we can interpret that the requirement is fulfilled), I don't think we need to obligatorily check for consistency in here?
Agreed. We may want to track in an issue the missing coverage of that requirement though if not already tracked elsewhere.
0c41f3b
to
02d1ada
Compare
58dcb13
to
0fe999e
Compare
0fe999e
to
da66b49
Compare
da66b49
to
8324327
Compare
Adds a check that a subscription's area (represented by the cell identifiers that overlap with it) is properly synchronized across all DSS's participating in the deployment.
Part of the effort for #754