Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Telefonica test cases proposal #70

Merged
merged 9 commits into from
Aug 21, 2024
Merged

Telefonica test cases proposal #70

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
c2b2c47
initial proposal
fernandopradocabrillo Nov 8, 2023
4b64570
include 403 invalid token context test cases
fernandopradocabrillo Nov 16, 2023
16a686c
remove version date temporary
fernandopradocabrillo Nov 16, 2023
e1f7dbb
fix scenario
fernandopradocabrillo Nov 20, 2023
d463d55
Merge branch 'main' of https://github.com/fernandopradocabrillo/SimSw…
fernandopradocabrillo Jun 26, 2024
fca89f2
include test cases with commonalities format
fernandopradocabrillo Jun 26, 2024
c720872
update test cases
fernandopradocabrillo Jul 24, 2024
ab5cccd
update scenarios
fernandopradocabrillo Jul 31, 2024
afb2a03
rephrase GIven conditions to improve readability
fernandopradocabrillo Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 176 additions & 0 deletions code/Test_definitions/checkSimSwap.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
Feature: CAMARA SIM Swap API, 0.5.0 - Operation checkSimSwap

# Input to be provided by the implementation to the tester
#
# Testing assets:
#
# References to OAS spec schemas refer to schemas specifies in sim_swap.yaml, version 0.5.0

check if SIM swap has been performed during a past period

Background: Common checkSimSwap setup
Given the resource "sim-swap/v0/check"
And the header "Content-Type" is set to "application/json"
And the header "Authorization" is set to a valid access token
And the header "x-correlator" is set to a UUID value
And the request body is set by default to a request body compliant with the schema


# This first scenario serves as a minimum, not testing any specific verificationResult
@check_sim_swap_1_generic_success_scenario
Scenario: Common validations for any sucess scenario
Given the request body property "$.phoneNumber" is set to a valid testing phoneNumber
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

success test cases (all of them) only cover scenario where $.phoneNumber is provided. However, the recommended way to use the API is without phoneNumber in the payload. And, it looks like there is no single test case which covers this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I would keep the specification about the phone number tho. For example:

And the header "Authorization" is set to a valid access token from which a valid testing phoneNumber can be deducted

wdyt? @gregory1g

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still keeps the main usage scenario outside of the tests coverage. With this change the only valid shown success invocation includes both access token and phoneNumber, meanwhile the recommended way to do not send phoneNumber. Probably we should focus successful tests on access token identifcation only and only keep error tests when phoneNumber does not match access token?

Copy link
Collaborator Author

@fernandopradocabrillo fernandopradocabrillo Jul 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think I haven't expressed it correctly. I agree with removing the phoneNumber from the body in the success scenarios, but I think we should keep the phone number characteristics for each case.

It is important for example to state that the phone number associated to the token didn't have a swap in the last X months or that there has been a swap but outside of the maxAge, etc

This way the tests can be set properly for the certification.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fully agree. phoneNumber is still needed for the service logic. But for the main success cases, it is better to move a source of this information from the request body ("$.phoneNumber" reference request body) to access token.
No sure what is the best way to express this. Probably like:

@check_sim_swap_1_generic_success_scenario
  Scenario: Common validations for any sucess scenario
    Given the request header "Authorization" is set to a valid access token from which a valid testing phoneNumber can be deducted
. . .

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that sounds like what I had in mind, I'll update it

When the request "checkSimSwap" is sent
Then the response status code is 200
And the response header "Content-Type" is "application/json"
And the response header "x-correlator" has same value as the request header "x-correlator"
And the response body complies with the OAS schema at "/components/schemas/CheckSimSwapInfo"

# Scenarios testing specific situations

@check_sim_swap_2_valid_sim_swap_no_max_age
Scenario: Check that the response shows that the SIM has been swapped
Given the request body property "$.phoneNumber" is set to a phone number connected to the Operator's network that has been swapped in the last 240 hours
When the request "checkSimSwap" is sent
Then the response status code is 200
And the value of response property "$.swapped" == true

@check_sim_swap_3_valid_sim_swap_max_age
Scenario Outline: Check that the response shows that the SIM has been swapped
Given the request body property "$.phoneNumber" is set to a phone number connected to the Operator's network that has been swapped in the last "<hours>" hours, where "<hours>" is equal or less than provided "maxAge" request body parameter
When the request "checkSimSwap" is sent
Then the response status code is 200
And the value of response property "$.swapped" == true

Examples:
| hours |
| 260 |
| 120 |
| 24 |
| 12 |

@check_sim_swap_4_more_than_240_hours
Scenario: Check that the response shows that the SIM has not been swapped when the last swap was more than 240 hours ago
Given the request body property "$.phoneNumber" is set to a phone number connected to the Operator's network that has been swapped more than 240 hours ago
When the request "checkSimSwap" is sent
Then the response status code is 200
And the value of response property "$.swapped" == false

@check_sim_swap_5_out_of_max_age
Scenario: Check that the response shows that the SIM has not been swapped when the last swap was before the maxAge field
Given the request body property "$.maxAge" is set to the number of hours since the last SIM swap minus 1
And the request body property "$.phoneNumber" is set to a phone number connected to the network whose last SIM swap was more than "$.maxAge" hours ago
When the request "checkSimSwap" is sent
Then the response status code is 200
And the value of response property "$.swapped" == false

@check_sim_swap_6_no_sim_swap
Scenario: Check that the response shows that the SIM has not been swapped
Given the request body property "$.phoneNumber" is set to a phone number with a SIM that has never been swapped
When the request "checkSimSwap" is sent
Then the response status code is 200
And the value of response property "$.swapped" == false

# Specific errors

@check_sim_swap_7_unknown_phone_number
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @fernandopradocabrillo @bigludo7,

Regarding the test cases where there is a mismatch between access token and phoneNumber - are there wider CAMARA guidelines for the expected error responses (e.g. UNKNOWN_PHONE_NUMBER / NOT_FOUND / INVALID_TOKEN_CONTEXT)? Is it based on Commonalities' "Standardized use of CAMARA error responses"?

Some of the operator implementations I've experienced return a 403 in every case where there is a mismatch between access token (they do not implement the concept of a user owning multiple phone numbers at the CAMARA level), so I'm thinking these different mismatch error responses might be a little difficult to adopt across operators. If there is some guidelines I could point operators towards for these cases then that would be helpful.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there is a clear guideline on this I'm afraid, but I'll check with my collegues if they have more info.

I have included our view on the scenarios and this is what we are implementing in TEF. We have a more complex filtering structure in the apigw so we are able to differenciate between these cases.

Maybe we can place a comment and clarify that depending on the auth implementations, the operator can return 404 or 403.

Edit: as stated in the issue #119 we are going to remove the unkown_phon_ number error

Scenario: Error when the phone number does not belong to the Operator
Given the request body property "$.phoneNumber" is set to a phone number that does not belong to the Operator
When the request "checkSimSwap" is sent
Then the response status code is 404
And the response property "$.status" is 404
And the response property "$.code" is "SIM_SWAP.UNKNOWN_PHONE_NUMBER"
And the response property "$.message" contains a user friendly text

@check_sim_swap_8_phone_number_provided_does_not_belong_to_the_user
Scenario: Error when provided phone number does not belong to the user
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To which "the user" the phone number must belong? The client does not provide a user in the input.
Th test case logic is correct: "the header "Authorization" is set to a valid access token emitted for a different phone number" => access token does not matches provided phoenNumber (both can belong to the same person or nor - does not matter).
Suggest: adjust the name and scenario of the test to something like "access token does not match phone number".

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For us this is a step we have prior to any other validation.
To be a valid token, the phoneNumber needs to be related to a subscriber, so we have a validation to check that indeed the phoneNumber belongs to the subscriber.
I can remove this test to not block the release, but maybe we'll like to revisit the possibility to include it in the future.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, let's remove this for now.

to be honest, I do not fully understand how this could work. To check that phoneNumber belongs to the user, you need to get the user (user id) from somewhere. But user ID is not part of the input. Probably, you can find the user by access token, but access token is already for the exact phone number anyway. So, if the access token is generated correctly user id amd phone number "in" it must match...

Given the request body property "$.phoneNumber" is set to a valid testing phoneNumber that does not belong to the user
And the header "Authorization" is set to a valid access token emitted for a different phone number
When the request "checkSimSwap" is sent
Then the response status code is 404
fernandopradocabrillo marked this conversation as resolved.
Show resolved Hide resolved
And the response property "$.status" is 404
And the response property "$.code" is "NOT_FOUND"
And the response property "$.message" contains a user friendly text

@check_sim_swap_9_phone_number_provided_does_not_match_the_token
Scenario: Error when the phone number provided in the request body belongs to the user but does not match the phone number asssociated with the access token
Given the request body property "$.phoneNumber" is set to a valid testing phoneNumber that belongs to the user but does not match the one associated with the token
fernandopradocabrillo marked this conversation as resolved.
Show resolved Hide resolved
And the header "Authorization" is set to a valid access token
When the request "checkSimSwap" is sent
Then the response status code is 403
And the response property "$.status" is 403
And the response property "$.code" is "INVALID_TOKEN_CONTEXT"
And the response property "$.message" contains a user friendly text

@check_sim_swap_10_phone_number_provided_cannot_be_deducted_from_access_token
fernandopradocabrillo marked this conversation as resolved.
Show resolved Hide resolved
Scenario: Error when the phone number is provided in the request body but cannot be deducted from the access token
Given the request body property "$.phoneNumber" is set to a valid testing phoneNumber of the user
And the header "Authorization" is set to a valid access token from which the phone number cannot be deduced
When the request "checkSimSwap" is sent
Then the response status code is 403
And the response property "$.status" is 403
And the response property "$.code" is "INVALID_TOKEN_CONTEXT"
And the response property "$.message" contains a user friendly text

@check_sim_swap_11_phone_number_not_provided_and_cannot_be_deducted_from_access_token
Scenario: Error when phone number can not be deducted from access token and it is not provided in the request body
Given the header "Authorization" is set to a valid access token from which the phone number cannot be deduced
When the request "checkSimSwap" is sent
Then the response status code is 403
And the response property "$.status" is 403
And the response property "$.code" is "INVALID_TOKEN_CONTEXT"
And the response property "$.message" contains a user friendly text

@check_sim_swap_12_phone_number_conflict
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @fernandopradocabrillo,

Apologies, I'm not able to comprehend this test case. Does it mean sending 2 requests using the same access token? If so, how does this result in 409?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @trehman-gsma!

I included this test just to cover all the errors defined in the API but, actually, I don't see any use for it.
I have an open issue to discuss whether we should remove it or not #119

Scenario: Error when another request is created for the same phoneNumber
Given the request body property "$.phoneNumber" is set to a valid testing phoneNumber
And the header "Authorization" is set to a valid access token emitted
And another request is created for the same phoneNumber
When the request "checkSimSwap" is sent
Then the response status code is 409
And the response property "$.status" is 409
And the response property "$.code" is "CONFLICT"
And the response property "$.message" contains a user friendly text

# Generic 401 errors

@check_sim_swap_401.1_no_authorization_header
Scenario: No Authorization header
Given the header "Authorization" is removed
And the request body is set to a valid request body
When the request "checkSimSwap" is sent
Then the response status code is "401"
And the response property "$.status" is 401
And the response property "$.code" is "UNAUTHENTICATED"
And the response property "$.message" contains a user friendly text

@check_sim_swap_401.2_expired_access_token
Scenario: Expired access token
Given the header "Authorization" is set to an expired access token
And the request body is set to a valid request body
When the request "checkSimSwap" is sent
Then the response status code is "401"
And the response property "$.status" is 401
And the response property "$.code" is "UNAUTHENTICATED"
bigludo7 marked this conversation as resolved.
Show resolved Hide resolved
And the response property "$.message" contains a user friendly text

@check_sim_swap_401.3_invalid_access_token
Scenario: Invalid access token
Given the header "Authorization" is set to an invalid access token
And the request body is set to a valid request body
When the request "checkSimSwap" is sent
Then the response status code is "401"
And the response property "$.status" is 401
And the response property "$.code" is "UNAUTHENTICATED"
And the response property "$.message" contains a user friendly text

# Generic 400 errors

@check_sim_swap_400.1_invalid_phone_number
Scenario: Check that the response shows an error when the phone number is invalid
Given the request body property "$.phoneNumber" does not comply with the OAS schema at "/components/schemas/PhoneNumber"
When the request "checkSimSwap" is sent
Then the response status code is 400
And the response property "$.status" is 400
And the response property "$.code" is "INVALID_ARGUMENT"
And the response property "$.message" contains a user friendly text
fernandopradocabrillo marked this conversation as resolved.
Show resolved Hide resolved
163 changes: 163 additions & 0 deletions code/Test_definitions/retrieveSimSwapDate.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
Feature: CAMARA SIM Swap API, 0.5.0 - Operation retrieveSimSwapDate

# Input to be provided by the implementation to the tester
#
# Testing assets:
#
# References to OAS spec schemas refer to schemas specifies in sim_swap.yaml, version 0.5.0

Get timestamp of last MSISDN <-> IMSI pairing change for a mobile user account provided with MSIDN.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks a bit misleading: we do not user user accounts in camara
Suggestion:
Get timestamp of last MSISDN <-> IMSI pairing change for the provided phone number.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I basically used the endpoint description. I don't see any problem in changing it to the one you proposed.
We can update the description in the yaml in the future


Background: Common retrieveSimSwapDate setup
Given the resource "sim-swap/v0/retrieve-date"
And the header "Content-Type" is set to "application/json"
And the header "Authorization" is set to a valid access token
And the header "x-correlator" is set to a UUID value
And the request body is set by default to a request body compliant with the schema

# This first scenario serves as a minimum, not testing any specific verificationResult
@retrieve_sim_swap_date_1_generic_success_scenario
Scenario: Common validations for any sucess scenario
Given the request body property "$.phoneNumber" is set to a valid testing phone number
fernandopradocabrillo marked this conversation as resolved.
Show resolved Hide resolved
When the request "retrieveSimSwapDate" is sent
Then the response status code is 200
And the response header "Content-Type" is "application/json"
And the response header "x-correlator" has same value as the request header "x-correlator"
And the response body complies with the OAS schema at "/components/schemas/SimSwapInfo"


# Scenarios testing specific situations

@retrieve_sim_swap_date_2_valid_sim_swap
Scenario: Check SIM swap date for a valid SIM swap
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we probably use "retrieve" instead of "check" here (and for the following test cases)?

Given the request body property "$.phoneNumber" is set to a phone number connected to the Operator's network
When the request "retrieveSimSwapDate" is sent
Then the response status code is 200
And the response property "$.latestSimChange" contains a valid timestamp

@retrieve_sim_swap_date_3_no_sim_swap_returns_activation_date

This comment was marked as resolved.

Sign in to view

Scenario: Response contains the sim's activation date when it hasn't been swapped
Given the request body property "$.phoneNumber" is set to a phone number connected to the Operator's network whose sim has never been swapped
When the request "retrieveSimSwapDate" is sent
Then the response status code is 200
And the response property "$.latestSimChange" contains the sim's activation timestamp

# This test applies if the operator allows to do the request for a sim that has never been connected to the network
@retrieve_sim_swap_date_4_sim_not_activated
Scenario: Check SIM swap date for a non-activated sim
Given the request body property "$.phoneNumber" is set to a phone number never connected to the Operator's network
When the request "retrieveSimSwapDate" is sent
Then the response status code is 200
And the response property "$.latestSimChange" is null

# This scenario applies when there is a local regulation with a time limitation on the information that can be returned
@retrieve_sim_swap_date_5_no_sim_swap_or_activation_date_due_to_legal_constrain
Scenario: Check SIM swap date for a valid SIM swap
Given the request body property "$.phoneNumber" is set to a valid testing phone number
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the request body property "$.phoneNumber" is set to a valid testing phone number
=>
Given the request body property "$.phoneNumber" is set to a valid testing phone number and SimSwap event happened before the limited history window threshold.

When the request "retrieveSimSwapDate" is sent
Then the response status code is 200
And the response property "$.latestSimChange" is null

# Specific errors

@retrieve_sim_swap_date_6_unknown_phone_number
Scenario: Check that the response shows an error when the phone number does not belong to the Operator
Given the request body property "$.phoneNumber" is set to a phone number that does not belong to the Operator
When the request "retrieveSimSwapDate" is sent
Then the response status code is 404
And the response property "$.status" is 404
And the response property "$.code" is "SIM_SWAP.UNKNOWN_PHONE_NUMBER"
And the response property "$.message" contains a user friendly text

@retrieve_sim_swap_date_7_phone_number_provided_does_not_belong_to_the_user
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same concern as for the corresponding /check tests

Scenario: Error when provided phone number does not belong to the user
Given the request body property "$.phoneNumber" is set to a valid testing phoneNumber that does not belong to the user
And the header "Authorization" is set to a valid access token emitted for a different phone number
When the request "retrieveSimSwapDate" is sent
Then the response status code is 404
And the response property "$.status" is 404
And the response property "$.code" is "NOT_FOUND"
And the response property "$.message" contains a user friendly text

@retrieve_sim_swap_date_8_phone_number_provided_does_not_match_the_token
Scenario: Error when the phone number provided in the request body belongs to the user but does not match the phone number asssociated with the access token
Given the request body property "$.phoneNumber" is set to a valid testing phoneNumber that belongs to the user but does not match the one associated with the token
And the header "Authorization" is set to a valid access token emitted for a different phone number
When the request "retrieveSimSwapDate" is sent
Then the response status code is 403
And the response property "$.status" is 403
And the response property "$.code" is "INVALID_TOKEN_CONTEXT"
And the response property "$.message" contains a user friendly text

@retrieve_sim_swap_date_9_phone_number_provided_cannot_be_deducted_from_access_token
Scenario: Error when the phone number is provided in the request body but cannot be deducted from the access token
Given the request body property "$.phoneNumber" is set to a valid testing phoneNumber of the user
And the header "Authorization" is set to a valid access token from which the phone number cannot be deduced
When the request "retrieveSimSwapDate" is sent
Then the response status code is 403
And the response property "$.status" is 403
And the response property "$.code" is "INVALID_TOKEN_CONTEXT"
And the response property "$.message" contains a user friendly text

@retrieve_sim_swap_date_10_phone_number_not_provided_and_cannot_be_deducted_from_access_token
Scenario: Error when phone number can not be deducted from access token and it is not provided in the request body
Given the header "Authorization" is set to a valid access token from which the phone number cannot be deduced
When the request "retrieveSimSwapDate" is sent
Then the response status code is 403
And the response property "$.status" is 403
And the response property "$.code" is "INVALID_TOKEN_CONTEXT"
And the response property "$.message" contains a user friendly text

@retrieve_sim_swap_date_11_phone_number_conflict
Scenario: Check that the response shows an error when another request is created for the same phoneNumber
Given the request body property "$.phoneNumber" is set to a valid testing phone number
And the header "Authorization" is set to a valid access token emitted
And another request is created for the same phoneNumber
When the request "retrieveSimSwapDate" is sent
Then the response status code is "409"
And the response property "$.status" is 409
And the response property "$.code" is "CONFLICT"
And the response property "$.message" contains a user friendly text

# Generic 401 errors

@retrieve_sim_swap_date_401.1_no_authorization_header
Scenario: No Authorization header
Given the header "Authorization" is removed
And the request body is set to a valid request body
When the request "retrieveSimSwapDate" is sent
Then the response status code is 401
And the response property "$.status" is 401
And the response property "$.code" is "UNAUTHENTICATED"
And the response property "$.message" contains a user friendly text

@retrieve_sim_swap_date_401.2_expired_access_token
Scenario: Expired access token
Given the header "Authorization" is set to an expired access token
And the request body is set to a valid request body
When the request "retrieveSimSwapDate" is sent
Then the response status code is 401
And the response property "$.status" is 401
And the response property "$.code" is "UNAUTHENTICATED"
bigludo7 marked this conversation as resolved.
Show resolved Hide resolved
And the response property "$.message" contains a user friendly text

@retrieve_sim_swap_date_401.3_invalid_access_token
Scenario: Invalid access token
Given the header "Authorization" is set to an invalid access token
And the request body is set to a valid request body
When the request "retrieveSimSwapDate" is sent
Then the response status code is 401
And the response property "$.status" is 401
And the response property "$.code" is "UNAUTHENTICATED"
And the response property "$.message" contains a user friendly text

# Generic 400 errors

@retrieve_sim_swap_date_4_invalid_phone_number
Scenario: Check that the response shows an error when the phone number is invalid
Given the request body property "$.phoneNumber" does not comply with the OAS schema at "/components/schemas/PhoneNumber"
When the request "retrieveSimSwapDate" is sent
Then the response status code is 400
And the response property "$.status" is 400
And the response property "$.code" is "INVALID_ARGUMENT"
And the response property "$.message" contains a user friendly text