Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix flaky E2E test #90860

Merged
merged 2 commits into from
May 20, 2024
Merged

Fix flaky E2E test #90860

merged 2 commits into from
May 20, 2024

Conversation

DustyReagan
Copy link
Member

@DustyReagan DustyReagan commented May 17, 2024 •
edited
Loading

Related to failed test.

Slack thread: p1715984606123039-slack-C7HH3V5AS

Proposed Changes

  • The flaky e2e test is failing due to the Beginner | Starter name change experiment pbxNRc-3E6-p2. Sometimes the e2e test user is placed in the treatment group, and sometimes it is not. This PR checks for either the Beginner or Starter plan names so the test will pass if either is present.

Why are these changes being made?

  • Fix failing Calypso e2e test.

Testing Instructions

  • Follow instructions here to setup e2e tests locally.
  • Run: yarn workspace wp-e2e-tests test test/e2e/specs/onboarding/lifecycle__signup-onboarding-launch-cancel.ts

Pre-merge Checklist

  • Has the general commit checklist been followed? (PCYsg-hS-p2)
  • Have you written new tests for your changes?
  • Have you tested the feature in Simple (P9HQHe-k8-p2), Atomic (P9HQHe-jW-p2), and self-hosted Jetpack sites (PCYsg-g6b-p2)?
  • Have you checked for TypeScript, React or other console errors?
  • Have you used memoizing on expensive computations? More info in Memoizing with create-selector and Using memoizing selectors and Our Approach to Data
  • Have we added the "[Status] String Freeze" label as soon as any new strings were ready for translation (p4TIVU-5Jq-p2)?
  • For changes affecting Jetpack: Have we added the "[Status] Needs Privacy Updates" label if this pull request changes what data or activity we track or use (p4TIVU-aUh-p2)?

@github-actions GitHub Actions
Copy link

github-actions bot commented May 17, 2024 •
edited by matticbot
Loading

Calypso Live (direct link)
https://calypso.live?image=registry.a8c.com/calypso/app:build-108154
Jetpack Cloud live (direct link)
https://calypso.live?image=registry.a8c.com/calypso/app:build-108154&env=jetpack
Automattic for Agencies live (direct link)
https://calypso.live?image=registry.a8c.com/calypso/app:build-108154&env=a8c-for-agencies

@DustyReagan DustyReagan marked this pull request as ready for review May 17, 2024 22:11
@DustyReagan DustyReagan changed the title Try fix flakey test Try fix flaky test May 17, 2024
@DustyReagan DustyReagan changed the title Try fix flaky test Try fix flaky E2E test May 17, 2024
@DustyReagan DustyReagan self-assigned this May 17, 2024
@matticbot
Copy link
Contributor

This PR does not affect the size of JS and CSS bundles shipped to the user's browser.

Generated by performance advisor bot at iscalypsofastyet.com.

@DustyReagan DustyReagan requested a review from tyxla May 17, 2024 22:23
@matticbot matticbot added the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label May 17, 2024
@DustyReagan DustyReagan requested review from niranjan-uma-shankar and a team May 17, 2024 22:23
@DustyReagan
Copy link
Member Author

I won't YOLO-merge this at 5:40pm on Friday, but if anyone wants to modify this PR or ship it before I'm back on Monday morning, please go ahead. ☺️

roo2
roo2 approved these changes May 20, 2024
View reviewed changes
Copy link
Contributor

@roo2 roo2 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice fix for now! I'll merge it, worst case, the test will remain broken :)

@roo2 roo2 merged commit e7151f4 into trunk May 20, 2024
10 checks passed
@roo2 roo2 deleted the try/fix-flakey-test branch May 20, 2024 04:32
@github-actions github-actions bot removed the [Status] Needs Review The PR is ready for review. This also triggers e2e canary tests and wp-desktop tests automatically. label May 20, 2024
@roo2
Copy link
Contributor

roo2 commented May 20, 2024

nice, it looks like it handles it well

@tyxla
Copy link
Member

tyxla commented May 20, 2024

Thanks for taking care of it 👍

niranjan-uma-shankar added a commit that referenced this pull request May 31, 2024
@niranjan-uma-shankar
Revert "Try fix flaky E2E test (#90860)"
51e9d77 
This reverts commit e7151f4.
@niranjan-uma-shankar niranjan-uma-shankar mentioned this pull request May 31, 2024
Merged
niranjan-uma-shankar added a commit that referenced this pull request May 31, 2024
@niranjan-uma-shankar
Revert "Try fix flaky E2E test (#90860)" (#91336)
e7fa3c1 
This reverts commit e7151f4.
@DustyReagan DustyReagan changed the title Try fix flaky E2E test Fix flaky E2E test Sep 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@roo2 roo2 roo2 approved these changes

@tyxla tyxla Awaiting requested review from tyxla

@niranjan-uma-shankar niranjan-uma-shankar Awaiting requested review from niranjan-uma-shankar

Assignees

@DustyReagan DustyReagan

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@DustyReagan @matticbot @roo2 @tyxla