From f4c51003d3952a7eba1f3a7a4dbf567fe07686b7 Mon Sep 17 00:00:00 2001 From: Vilem Obratil Date: Mon, 13 Jan 2025 14:05:15 +0100 Subject: [PATCH 1/4] Added test plans for Advisory and Vulnerability explorers. --- tests/features/advisory-explorer.feature | 52 +++++++++++++++++++ tests/features/vulnerability-explorer.feature | 44 ++++++++++++++++ 2 files changed, 96 insertions(+) create mode 100644 tests/features/advisory-explorer.feature create mode 100644 tests/features/vulnerability-explorer.feature diff --git a/tests/features/advisory-explorer.feature b/tests/features/advisory-explorer.feature new file mode 100644 index 0000000..5bc2d18 --- /dev/null +++ b/tests/features/advisory-explorer.feature @@ -0,0 +1,52 @@ +Feature: Advisory Explorer + As a Developer or a Devsecops Engineer + I want to be able to display all available information about a single advisory - metadata and related vulnerabilites + +Background: + Given User is using an instance of the TPA Application + And User has successfully uploaded an SBOM + And User has successfully uploaded a vulnerability dataset + And User has successfully uploaded an advisory dataset + +# Advisory Explorer +Scenario: Display an overview of all advisories + Given User has navigated to the Advisory Explorer page + Then The ID and Title of each advisory is visible on each row + And "Show more" button is visible for advisories with a long title + And Advisory metadata (Severity, Revision, Vulnerabilites) is visible on each row + And A context menu icon is visible on each row + +Scenario: Download an advisory from the Advisory Explorer page + Given User has navigated to the Advisory Explorer page + And User has clicked the Download button in the context menu of an advisory + Then The advisory should download as a JSON file + +# Advisory Overview +Scenario: Display detailed information about a single advisory + Given User has selected an advisory on the Advisory Explorer page + Then The Overview, Publisher, Tracking, References and Product Info panels should display + And The Product Info panel should display a list of impacted products + And The Product Info panel items should be collapsable + +# Advisory Notes +Scenario: Display notes about a single advisory + Given User has selected an advisory on the Advisory Explorer page + And User has navigated to the Notes tab on the Advisory Overview page + Then All notes on an advisory should display + +# Advisory Vulnerabilities +Scenario: Display vulnerabilities tied to a single advisory + Given User has selected an advisory on the Advisory Explorer page + And User has navigated to the Vulnerabilites tab on the Advisory Overview page + Then A list of all vulnerabilites tied to the advisory should display + And The ID, Title, Discovery, Release, Score and CWE information should be visible for each vulnerability + And The vulnerabilities should be sorted by ID by default + And Each vulnerability should be expandable + +Scenario: Display detailed information about a single vulnerability tied to a single advisory + Given User has selected an advisory on the Advisory Explorer page + And User has navigated to the Vulnerabilites tab on the Advisory Overview page + And User has expanded the row of a selected vulnerability + Then A list of all affected products, separated by status, should display + And Each fixed product should display a link to a remediation + And All notes on a vulnerability should be displayed underneath the list of products diff --git a/tests/features/vulnerability-explorer.feature b/tests/features/vulnerability-explorer.feature new file mode 100644 index 0000000..979f569 --- /dev/null +++ b/tests/features/vulnerability-explorer.feature @@ -0,0 +1,44 @@ +Feature: Vulnerability Explorer + As a Developer or a Devsecops Engineer + I want to be able to display all available information about a vulnerability - metadata, related products and advisories + +Background: + Given User is using an instance of the TPA Application + And User has successfully uploaded an SBOM + And User has successfully uploaded a vulnerability dataset + And User has successfully uploaded an advisory dataset + +# Vulnerabilities Explorer +Scenario: Display an overview of all vulnerabilities + Given User has navigated to the Vulnerabilities Explorer page + Then The ID and Title of each vulnerability is visible on each row + And "Show more" button is visible for vulnerabilities with a long title + And Advisory metadata (CVSS, Date published, Related documents) is visible on each row + And A context menu icon is visible on each row + +# This is how it's described in the User Stories document, but no such button actually appears in either the screenshot or the Trustify app +Scenario: Download a vulnerability CVE from the Vulnerability Explorer page + Given User has navigated to the Vulnerabilities Explorer page + And User has clicked the Download button in the context menu of a vulnerability + Then The vulnerability CVE should download as a JSON file + +# Vulnerability-related Products +Scenario: Display a list of products related to a single vulnerability + Given User has navigated to the Vulnerabilities Explorer page + And User has selected a vulnerability + Then The Name, Version, Status, Dependencies, Supplier and Created On data is visible for each product + And The name of each product should be a link to the overview of a corresponding SBOM Overview page + And The dependencies column should be expandable + +Scenario: Display a list of dependencies of a product related to a single vulnerability + Given User has navigated to the Vulnerabilities Explorer page + And User has selected a vulnerability + And User has clicked on the Dependencies column of a product + Then The Type, Namespace, Name, Version, Path, Qualifiers and Dependency tree position data should display for each dependency + +# Vulnerability-related Advisories +Scenario: Display advisories tied to a single vulnerability + Given User has navigated to the Vulnerabilities Explorer page + And User has navigated to the Related Advisories tab + Then The ID, Title, Aggregated severity, Revision and Vulnerabilities data should display for each advisory tied to the vulnerability + And The ID should be a link a corresponding Advisory Overview page \ No newline at end of file From e85ba2f308312db4b8b062baa900a4a1d21a0f98 Mon Sep 17 00:00:00 2001 From: Vilem Obratil Date: Wed, 15 Jan 2025 14:24:32 +0100 Subject: [PATCH 2/4] Updated the test cases to use keywords more in accordance with the documentation. --- tests/features/advisory-explorer.feature | 32 ++++++++++++------- tests/features/vulnerability-explorer.feature | 25 +++++++++------ 2 files changed, 35 insertions(+), 22 deletions(-) diff --git a/tests/features/advisory-explorer.feature b/tests/features/advisory-explorer.feature index 5bc2d18..d7518b2 100644 --- a/tests/features/advisory-explorer.feature +++ b/tests/features/advisory-explorer.feature @@ -10,43 +10,51 @@ Background: # Advisory Explorer Scenario: Display an overview of all advisories - Given User has navigated to the Advisory Explorer page + Given User is on the Home page + When User navigates to the Advisory Explorer page Then The ID and Title of each advisory is visible on each row And "Show more" button is visible for advisories with a long title And Advisory metadata (Severity, Revision, Vulnerabilites) is visible on each row And A context menu icon is visible on each row Scenario: Download an advisory from the Advisory Explorer page - Given User has navigated to the Advisory Explorer page - And User has clicked the Download button in the context menu of an advisory + Given User is on the Home page + When User navigates to the Advisory Explorer page + And User clicks the Download button in the context menu of an advisory Then The advisory should download as a JSON file # Advisory Overview Scenario: Display detailed information about a single advisory - Given User has selected an advisory on the Advisory Explorer page + Given User is on the Home page + When User navigates to the Advisory Explorer page + And User selects an advisory on the Advisory Explorer page Then The Overview, Publisher, Tracking, References and Product Info panels should display And The Product Info panel should display a list of impacted products And The Product Info panel items should be collapsable # Advisory Notes Scenario: Display notes about a single advisory - Given User has selected an advisory on the Advisory Explorer page - And User has navigated to the Notes tab on the Advisory Overview page + Given User is on the Home page + When User navigates to the Advisory Explorer page + And User selects an advisory on the Advisory Explorer page + And User navigates to the Notes tab on the Advisory Overview page Then All notes on an advisory should display # Advisory Vulnerabilities Scenario: Display vulnerabilities tied to a single advisory - Given User has selected an advisory on the Advisory Explorer page - And User has navigated to the Vulnerabilites tab on the Advisory Overview page - Then A list of all vulnerabilites tied to the advisory should display + Given User is on the Home page + When User selects an advisory on the Advisory Explorer page + And User navigates to the Vulnerabilites tab on the Advisory Overview page + Then A list of all active vulnerabilites tied to the advisory should display And The ID, Title, Discovery, Release, Score and CWE information should be visible for each vulnerability And The vulnerabilities should be sorted by ID by default And Each vulnerability should be expandable Scenario: Display detailed information about a single vulnerability tied to a single advisory - Given User has selected an advisory on the Advisory Explorer page - And User has navigated to the Vulnerabilites tab on the Advisory Overview page - And User has expanded the row of a selected vulnerability + Given User is on the Home page + When User selects an advisory on the Advisory Explorer page + And User navigates to the Vulnerabilites tab on the Advisory Overview page + And User expands the row of a selected vulnerability Then A list of all affected products, separated by status, should display And Each fixed product should display a link to a remediation And All notes on a vulnerability should be displayed underneath the list of products diff --git a/tests/features/vulnerability-explorer.feature b/tests/features/vulnerability-explorer.feature index 979f569..9582bf1 100644 --- a/tests/features/vulnerability-explorer.feature +++ b/tests/features/vulnerability-explorer.feature @@ -10,7 +10,8 @@ Background: # Vulnerabilities Explorer Scenario: Display an overview of all vulnerabilities - Given User has navigated to the Vulnerabilities Explorer page + Given User is on the Home page + When User navigates to the Vulnerabilities Explorer page Then The ID and Title of each vulnerability is visible on each row And "Show more" button is visible for vulnerabilities with a long title And Advisory metadata (CVSS, Date published, Related documents) is visible on each row @@ -18,27 +19,31 @@ Scenario: Display an overview of all vulnerabilities # This is how it's described in the User Stories document, but no such button actually appears in either the screenshot or the Trustify app Scenario: Download a vulnerability CVE from the Vulnerability Explorer page - Given User has navigated to the Vulnerabilities Explorer page - And User has clicked the Download button in the context menu of a vulnerability + Given User is on the Home page + When User navigates to the Vulnerabilities Explorer page + And User clicks the Download button in the context menu of a vulnerability Then The vulnerability CVE should download as a JSON file # Vulnerability-related Products Scenario: Display a list of products related to a single vulnerability - Given User has navigated to the Vulnerabilities Explorer page - And User has selected a vulnerability + Given User is on the Home page + When User navigates to the Vulnerabilities Explorer page + And User selects a vulnerability Then The Name, Version, Status, Dependencies, Supplier and Created On data is visible for each product And The name of each product should be a link to the overview of a corresponding SBOM Overview page And The dependencies column should be expandable Scenario: Display a list of dependencies of a product related to a single vulnerability - Given User has navigated to the Vulnerabilities Explorer page - And User has selected a vulnerability - And User has clicked on the Dependencies column of a product + Given User is on the Home page + When User navigates to the Vulnerabilities Explorer page + And User selects a vulnerability + And User clicks on the Dependencies column of a product Then The Type, Namespace, Name, Version, Path, Qualifiers and Dependency tree position data should display for each dependency # Vulnerability-related Advisories Scenario: Display advisories tied to a single vulnerability - Given User has navigated to the Vulnerabilities Explorer page - And User has navigated to the Related Advisories tab + Given User is on the Home page + When User navigates to the Vulnerabilities Explorer page + And User then navigates to the Related Advisories tab Then The ID, Title, Aggregated severity, Revision and Vulnerabilities data should display for each advisory tied to the vulnerability And The ID should be a link a corresponding Advisory Overview page \ No newline at end of file From a7cae26915d617a1c87b62b2d17f225a6805c581 Mon Sep 17 00:00:00 2001 From: Vilem Obratil Date: Thu, 16 Jan 2025 15:07:31 +0100 Subject: [PATCH 3/4] Applied changes suggested in the review and added search scenarios. --- tests/features/advisory-explorer.feature | 44 ++++++++++++++----- tests/features/vulnerability-explorer.feature | 42 +++++++++++++----- 2 files changed, 64 insertions(+), 22 deletions(-) diff --git a/tests/features/advisory-explorer.feature b/tests/features/advisory-explorer.feature index d7518b2..521b1f8 100644 --- a/tests/features/advisory-explorer.feature +++ b/tests/features/advisory-explorer.feature @@ -9,25 +9,46 @@ Background: And User has successfully uploaded an advisory dataset # Advisory Explorer -Scenario: Display an overview of all advisories +Scenario: Navigating to the Advisory Explorer page by selecting it from the list of all advisories Given User is on the Home page - When User navigates to the Advisory Explorer page - Then The ID and Title of each advisory is visible on each row - And "Show more" button is visible for advisories with a long title - And Advisory metadata (Severity, Revision, Vulnerabilites) is visible on each row - And A context menu icon is visible on each row + When User clicks on the Search button in the menu + And User selects the Advisories tab + And User selects an advisory from the list + Then The Advisory Explorer page should display + +Scenario: Navigating to the Advisory Explorer page by searching for it in the search bar + Given User is on the Home page + When User clicks on the Search textbox + And User enters the ID of an advisory + And User searches for the advisory + And User selects the advisory returned by the search + Then The Advisory Explorer page should display + +Scenario: Navigating to the Advisory Explorer page by filtering advisories in the list of all advisories + Given User is on the Home page + When User clicks on the Search button in the menu + And User selects the Advisories tab + And User selects a filter + And User selects an advisory returned by the filter + Then The Advisory Explorer page should display + +Scenario: Display an overview of an advisory + Given User is on the Home page + When User navigates to a Advisory Explorer page + Then The ID and description of the advisory should be visible + And "Show more" button should be visible for advisories with a long title + And Download button should be visible Scenario: Download an advisory from the Advisory Explorer page Given User is on the Home page When User navigates to the Advisory Explorer page - And User clicks the Download button in the context menu of an advisory + And User clicks the Download button Then The advisory should download as a JSON file # Advisory Overview Scenario: Display detailed information about a single advisory Given User is on the Home page When User navigates to the Advisory Explorer page - And User selects an advisory on the Advisory Explorer page Then The Overview, Publisher, Tracking, References and Product Info panels should display And The Product Info panel should display a list of impacted products And The Product Info panel items should be collapsable @@ -36,14 +57,13 @@ Scenario: Display detailed information about a single advisory Scenario: Display notes about a single advisory Given User is on the Home page When User navigates to the Advisory Explorer page - And User selects an advisory on the Advisory Explorer page And User navigates to the Notes tab on the Advisory Overview page Then All notes on an advisory should display # Advisory Vulnerabilities Scenario: Display vulnerabilities tied to a single advisory Given User is on the Home page - When User selects an advisory on the Advisory Explorer page + When User navigates to the Advisory Explorer page And User navigates to the Vulnerabilites tab on the Advisory Overview page Then A list of all active vulnerabilites tied to the advisory should display And The ID, Title, Discovery, Release, Score and CWE information should be visible for each vulnerability @@ -52,9 +72,9 @@ Scenario: Display vulnerabilities tied to a single advisory Scenario: Display detailed information about a single vulnerability tied to a single advisory Given User is on the Home page - When User selects an advisory on the Advisory Explorer page + When User navigates to the Advisory Explorer page And User navigates to the Vulnerabilites tab on the Advisory Overview page And User expands the row of a selected vulnerability Then A list of all affected products, separated by status, should display - And Each fixed product should display a link to a remediation + And Each product should display a link to a remediation And All notes on a vulnerability should be displayed underneath the list of products diff --git a/tests/features/vulnerability-explorer.feature b/tests/features/vulnerability-explorer.feature index 9582bf1..ef0bdd3 100644 --- a/tests/features/vulnerability-explorer.feature +++ b/tests/features/vulnerability-explorer.feature @@ -9,26 +9,48 @@ Background: And User has successfully uploaded an advisory dataset # Vulnerabilities Explorer -Scenario: Display an overview of all vulnerabilities +Scenario: Navigating to the Vulnerability Explorer page by selecting it from the list of all vulnerabilities + Given User is on the Home page + When User clicks on the Search button in the menu + And User selects the Vulnerabilities tab + And User selects an Vulnerability from the list + Then The Vulnerability Explorer page should display + +Scenario: Navigating to the Vulnerability Explorer page by searching for it in the search bar + Given User is on the Home page + When User clicks on the Search textbox + And User enters an ID of a Vulnerability + And User searches for the Vulnerability + And User selects the Vulnerability returned by the search + Then The Vulnerability Explorer page should display + +Scenario: Navigating to the Vulnerability Explorer page by filtering vulnerabilities in the list of all vulnerabilities + Given User is on the Home page + When User clicks on the Search button in the menu + And User selects the Vulnerabilities tab + And User selects a filter + And User selects an Vulnerability returned by the filter + Then The Vulnerability Explorer page should display + +Scenario: Display an overview of a vulnerability Given User is on the Home page When User navigates to the Vulnerabilities Explorer page - Then The ID and Title of each vulnerability is visible on each row - And "Show more" button is visible for vulnerabilities with a long title - And Advisory metadata (CVSS, Date published, Related documents) is visible on each row - And A context menu icon is visible on each row + Then CVE name, CVSS score and CVE description should be visible + And "Show more" button is visible for vulnerabilities with a long descriptions + And Advisory metadata (CVE name, Date published, Related documents) should be visible + And Download button should be visible -# This is how it's described in the User Stories document, but no such button actually appears in either the screenshot or the Trustify app Scenario: Download a vulnerability CVE from the Vulnerability Explorer page Given User is on the Home page When User navigates to the Vulnerabilities Explorer page - And User clicks the Download button in the context menu of a vulnerability + And User clicks the Download button Then The vulnerability CVE should download as a JSON file # Vulnerability-related Products Scenario: Display a list of products related to a single vulnerability Given User is on the Home page When User navigates to the Vulnerabilities Explorer page - And User selects a vulnerability + And User clicks on the Related Products tab Then The Name, Version, Status, Dependencies, Supplier and Created On data is visible for each product And The name of each product should be a link to the overview of a corresponding SBOM Overview page And The dependencies column should be expandable @@ -36,7 +58,7 @@ Scenario: Display a list of products related to a single vulnerability Scenario: Display a list of dependencies of a product related to a single vulnerability Given User is on the Home page When User navigates to the Vulnerabilities Explorer page - And User selects a vulnerability + And User clicks on the Related Products tab And User clicks on the Dependencies column of a product Then The Type, Namespace, Name, Version, Path, Qualifiers and Dependency tree position data should display for each dependency @@ -44,6 +66,6 @@ Scenario: Display a list of dependencies of a product related to a single vulner Scenario: Display advisories tied to a single vulnerability Given User is on the Home page When User navigates to the Vulnerabilities Explorer page - And User then navigates to the Related Advisories tab + And User clicks on the Related Advisories tab Then The ID, Title, Aggregated severity, Revision and Vulnerabilities data should display for each advisory tied to the vulnerability And The ID should be a link a corresponding Advisory Overview page \ No newline at end of file From 4b1b60609ecbef24e66619ce6ef8f8bb9666d98c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Vil=C3=A9m=20Obr=C3=A1til?= <41943007+vobratil@users.noreply.github.com> Date: Fri, 17 Jan 2025 14:32:57 +0100 Subject: [PATCH 4/4] Update tests/features/vulnerability-explorer.feature Co-authored-by: Rajan Ravi <86094767+mrrajan@users.noreply.github.com> --- tests/features/vulnerability-explorer.feature | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/features/vulnerability-explorer.feature b/tests/features/vulnerability-explorer.feature index ef0bdd3..e05d9fe 100644 --- a/tests/features/vulnerability-explorer.feature +++ b/tests/features/vulnerability-explorer.feature @@ -37,7 +37,7 @@ Scenario: Display an overview of a vulnerability When User navigates to the Vulnerabilities Explorer page Then CVE name, CVSS score and CVE description should be visible And "Show more" button is visible for vulnerabilities with a long descriptions - And Advisory metadata (CVE name, Date published, Related documents) should be visible + And Vulnerability metadata Reserved, Last Published Date and Last Modified should be visible And Download button should be visible Scenario: Download a vulnerability CVE from the Vulnerability Explorer page