Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KUTTL: don't bail out immediately, save the test reports #2628

Merged
merged 1 commit into from
Jan 9, 2025
Merged

KUTTL: don't bail out immediately, save the test reports #2628

merged 1 commit into from
Jan 9, 2025

Conversation

tosky
Copy link
Contributor

@tosky tosky commented Jan 2, 2025

Execute all KUTTL tests without bailing out at the first error. Instead make sure that the tests are executed and their JUnitXML reports are collected if available.
Move the accept/fail condition at the end, following the same pattern used elsewhere when running tests.

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 2, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@tosky tosky mentioned this pull request Jan 2, 2025
Merged
@softwarefactory-project-zuul softwarefactory-project-zuul
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/1c37781eb08c4c1f9dbcd217a71fd163

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 31m 23s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 18m 32s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 17m 25s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 28s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 40s
❌ cifmw-multinode-kuttl FAILURE in 1h 56m 28s
✔️ build-push-container-cifmw-client SUCCESS in 20m 45s

@tosky tosky force-pushed the kuttl-always-save-reports branch from 599dd07 to 3c78811 Compare January 2, 2025 15:10
@softwarefactory-project-zuul softwarefactory-project-zuul
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/78b4cef704b0424fa71916c5c0abf649

❌ openstack-k8s-operators-content-provider FAILURE in 4m 40s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 10s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 31s
❌ cifmw-multinode-kuttl FAILURE in 16m 23s
✔️ build-push-container-cifmw-client SUCCESS in 38m 04s

@tosky
Copy link
Contributor Author

tosky commented Jan 2, 2025

recheck

@softwarefactory-project-zuul softwarefactory-project-zuul
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/cb77788148644f8189f336b051fdd334

❌ openstack-k8s-operators-content-provider FAILURE in 4m 45s
⚠️ podified-multinode-edpm-deployment-crc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
⚠️ cifmw-crc-podified-edpm-baremetal SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 59s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 05s
❌ cifmw-multinode-kuttl FAILURE in 15m 43s
✔️ build-push-container-cifmw-client SUCCESS in 21m 03s

@tosky
Copy link
Contributor Author

tosky commented Jan 3, 2025

recheck

@softwarefactory-project-zuul softwarefactory-project-zuul
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/12fbd9c7a8aa437b8a1e9f96434ca8be

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 48m 47s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 21m 17s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 36m 40s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 10s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 20s
❌ cifmw-multinode-kuttl FAILURE in 31m 26s
✔️ build-push-container-cifmw-client SUCCESS in 20m 49s

@tosky tosky force-pushed the kuttl-always-save-reports branch 2 times, most recently from 9e77be1 to c8be930 Compare January 3, 2025 11:52
@softwarefactory-project-zuul softwarefactory-project-zuul
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/af7a164b0683496d8b05b53fa2587cd7

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 41m 53s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 16m 25s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 28m 30s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 7m 57s
✔️ cifmw-pod-pre-commit SUCCESS in 7m 12s
❌ cifmw-multinode-kuttl FAILURE in 1h 55m 15s
✔️ build-push-container-cifmw-client SUCCESS in 20m 43s

@tosky tosky force-pushed the kuttl-always-save-reports branch 3 times, most recently from 8e498a8 to fd9cefc Compare January 7, 2025 18:53
@tosky
KUTTL: don't bail out immediately, save the test reports
b58a7c1 
Execute all KUTTL tests without bailing out at the first error.
Instead make sure that the tests are executed and their JUnitXML
reports are collected if available.
Move the accept/fail condition at the end, following the same
pattern used elsewhere when running tests.
@tosky tosky force-pushed the kuttl-always-save-reports branch from fd9cefc to b58a7c1 Compare January 7, 2025 23:40
@tosky tosky marked this pull request as ready for review January 7, 2025 23:40
@tosky tosky requested a review from fmount January 7, 2025 23:41
@tosky
Copy link
Contributor Author

tosky commented Jan 7, 2025

As the kuttl tests executed here don't generate JUnitXML reports yet, I've tested the change here: openstack-k8s-operators/glance-operator#678

This was referenced Jan 7, 2025
Open
Open
@pablintino
Copy link
Collaborator

/approve

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Jan 9, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pablintino

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved label Jan 9, 2025
@frenzyfriday
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm label Jan 9, 2025
@openshift-merge-bot openshift-merge-bot bot merged commit 52af24e into openstack-k8s-operators:main Jan 9, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pablintino pablintino Awaiting requested review from pablintino pablintino is a code owner

@lewisdenny lewisdenny Awaiting requested review from lewisdenny lewisdenny is a code owner

@fmount fmount Awaiting requested review from fmount

Assignees

@frenzyfriday frenzyfriday

Labels
approved lgtm Ready For Review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tosky @pablintino @frenzyfriday