Stepper: Seah's onboarding flow test feedback

[skim1220]

Hi Team,

I’d like to share my findings from comparing the Stepper and Classic onboarding flows. @travisw tested the es locale in this GitHub issue, and I wanted to test another non-EN locale. However, I wasn’t able to load it, so I tested the EN version instead. I’ll get help from Travis to conduct another audit for non-EN locales.

Test Scenarios:

• Stepper flow test 1: https://wordpress.com/setup/onboarding/ (selected custom domain, premium plan)
• Stepper flow test 2: https://wordpress.com/setup/onboarding/?ref=test (selected custom domain, business plan)
• Classic flow test: https://wordpress.com/start/onboarding/?ref=test (selected custom domain, premium plan)

Issue Color Codes:

• 🟢 Works as expected
• 🔴 Major issue – needs to be resolved before setting up the experiment
• 🟡 Minor issue – can be added to the backlog

@escapemanuele, @renancarvalho (cc: @chriskmnds), Could we prioritize resolving the 🔴 red-coded issues below asap? The most significant problem is that I was sent to https://wordpress.com/sites instead of the checkout page, which Travis also experienced. I retried the same test scenario, but still couldn’t reach the checkout page.

---

Signup Start

🟢 calypso_signup_start fires with the correct attributes (flow:onboarding; ref:test)
🔴 However, as Travis mentioned in his report, we need to address calypso_signup_step_start firing before calypso_signup_start.

Account Creation Step

🟢 calypso_signup_step_start fires with the correct attributes (flow:onboarding; step:user)
🟡 In the Classic flow, calypso_page_view event at the Account Creation step logs flow:onboarding, but the Stepper flow doesn’t. It would be helpful if the Stepper flow also logged this property.

Account Created (Registration Complete)

🟢 calypso_user_registration_complete fires with the correct attributes (flow:onboarding; type:passwordless)
🟡 calypso_signup_actions_submit_step fires with the correct attributes (flow:onboarding; step:user), but the step value differs from the Classic flow (cf. step:user-social). I agree with Travis that the Stepper flow makes more sense here. While we don’t use this event much for Registration Complete in reporting and analysis, aligning them would be good.
🟡 calypso_signup_actions_complete_step (flow:onboarding; step:user-social) fires only in the Classic flow. We don’t currently rely on this event to check a user’s registration completion, but firing it in the Stepper flow would make the use of this event more consistent.

Domain Selection Step

🟢 calypso_signup_step_start fires with the correct attributes (flow:onboarding; step:domains)
🟡 calypso_page_view event at Domain Selection step logs flow:onboarding in the Classic flow, but not in the Stepper flow. It would be helpful to log it in the Stepper flow as well.

Domain Submitted

🔴 In the Stepper flow, calypso_signup_actions_submit_step (flow:onboarding; step:domains) doesn’t log the domain details submitted. It should log the details of the selected domain(s). In the Classic flow, the following properties are logged:

• product_slug:dotblog_domain,meta:septembertest.blog,is_domain_registration:true
• signup_domain_origin: custom

While signup_domain_origin isn’t required for this event, it is required for calyspo_signup_complete event later in the flow. I assume the Classic flow logs signup_domain_origin here and passes it to the calypso_signup_complete event?

Nonetheless, product_slug (i.e., the domains submitted) should be recorded in the Stepper flow.

🟡 calypso_signup_actions_submit_step event firing upon domain submission in the Stepper flow includes both step and step_name properties, which seem redundant. Cleaning this up when possible would be ideal.

Plan Selection Step

🟢 calypso_signup_step_start fires with the correct attributes (flow:onboarding; step:plans)
🔴 Plan Selection step differs between the Classic and Stepper flows. The Stepper flow prominently displays the Free Plan, and the plan description text also differs.

(Plan Selection Step in the Stepper flow: click to see the image bigger)

(Plan Selection Step in the Classic flow: click to see the image bigger)

Signup Complete & Checkout

🔴 I was unable to reach the checkout page and was instead redirected to https://wordpress.com/sites. The following errors were recorded in Vigilante:

error: {"beacons":["pinghub.conn_open_dc.dfw.chrome:124|ms"]}
error: https://public-api.wordpress.com/pinghub
error: 0.9

error: {"beacons":["pinghub.restproxy.cookie.chrome:1|c"]}
error: https://public-api.wordpress.com/pinghub
error: 0.9

error: {"beacons":["pinghub.restproxy.ws_supported.chrome:1|c"]}
error: https://public-api.wordpress.com/pinghub
error: 0.9

[escapemanuele]

Hey @skim1220, could you record a short video about "Signup Complete & Checkout"?

[skim1220]

Here it is, @escapemanuele

stepper_flow_test.mp4

[escapemanuele]

Opened an issue to investigate the checkout problem: #94432

The plan not showing the free plan as in /start will be fixed by #94319 ✌️

[renancarvalho]

@escapemanuele and @skim1220 , maybe we can close this?

[renancarvalho]

Closing since it was addressed in many other smaller issues 👍