Epic: Comprehensive Test Plan Implementation

[gracekretschmer-metrostar]

Status

Update each sprint until completed

Date	Status	Launch Date (see above)	Notes
10/28/2024	In-Progress	On-track	To minimize the dependency of relying on the Sitewide team, CMS team is exploring using a testing module that can allow for future creation of test scripts without content type dependencies.

Background

For CMS development, test plans and test scripts have historically been created by developers without QA engineering oversight and right now, flakey tests and testing gaps have created errors during the daily deploy process. @keisterj-oddball did discovery work to determine how to approach a comprehensive test plan and now the comprehensive test plan needs to be implemented.

User Story or Problem Statement

As an editor in Drupal CMS, I need to know that when there are updates to Drupal CMS, it will not unintentionally impact other areas of Drupal CMS that are critical for my day-to-day work.

As a developer in Drupal CMS, I need to know if and how my code changes in Drupal CMS unintentionally impact other areas of Drupal CMS.

As a developer in Drupal CMS, I need to have confidence that the test scripts in Drupal CMS are providing accurate and quick results that I can use to shape my code.

Affected users and stakeholders

• Developers of CMS and CMS products (e.g. CMS team, Sitewide team)

Hypothesis

By implementing a comprehensive test plan for CMS, we will be able to better catch bugs and bad code before it goes into production and impacts editors and veterans.

Desired Outcomes

• Increase the number of test coverage for critical pathways for Prod CMS
• Decrease the test execution time.

OCTO OKR

Objective: Our platforms are the best way to deliver products at VA
Key Results: Our platforms hit the “elite” level (as defined by DORA) on Deployment Frequency, Lead Time for Changes, Change Failure Rate, and Time to Restore Service

Reference Links

• https://vfs.atlassian.net/wiki/spaces/PCMS/pages/3353673764/CMS+Test+Plan

Definition of Done

• 50% of critical pathways within Drupal CMS have automated tests.
• Existing Cypress test suite is separated into separate test runs, to improve the overall performance of tests and it's feedback to developers.

Tasks

Preview Give feedback

1. Discovery: Comprehensive Test Plan #17563
   1 of 1
   CMS Team Epic +2
   
2. Incorporate Most Common Workflows in Comprehensive Test Plan #18275
   CMS Team +1
   
3. Opportunities and Priorities for Comprehensive Test Plan #18808
   CMS Team +1
   
4. Comprehensive Test Plan: Identify and Script E2E Tests For Additional Content Types #19002
   Automated testing CMS Team +2
   
5. Comprehensive Test Plan: Divide Existing Cypress Test Suite into Separate Test Runs #19004
   Automated testing CMS Team Needs refining +3
   
6. Spike: Understand level of effort to seperate test runs #19326
   CMS Team +1
   
7. Spike: Investigate Content Sync Module for Reducing Dependency on Production Content #19496
   CMS - QA Standards CMS Team +2
   
8. Comprehensive Test Plan: Dump the cypress_accessibility_violations.json file into the PR after cypress tests have run in tugboat #19412
   Automated testing CMS Team accessibility +3
   
9. Comprehensive Test Plan: Refactor Cypress Tests to Bypass the Login Page #19641
   Automated testing CMS Team +2
   
10. Comprehensive Test Plan: Investigate Existing Script for Reporting Accessibility Violations #19813
    +0
    
11. Comprehensive Test Plan: Import Dedicated Content for Testing with Content-Sync #19746
    Automated testing CMS Team +2
    
12. Comprehensive Test Plan: Create static test users for cypress testing #19753
    Automated testing CMS Team +2
    
13. Comprehensive Test Plan: Build out an automated regression test suite covering critical path scenarios within the CMS #12611
    Automated testing CMS Team Quality Assurance +3
14. Discovery: Persist Automated Testing History #19006
    Automated testing CMS Team +2
    
15. Comprehensive Test Plan: Improve Accessibility Violation Logging #20214
    +0