Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Add mutation testing #648

Draft
wants to merge 38 commits into
base: main
Choose a base branch
from
Draft

test: Add mutation testing #648

wants to merge 38 commits into from

Conversation

WillGibson
Copy link
Contributor

@WillGibson WillGibson commented Nov 20, 2024

An experiment @WillGibson is running, mostly when he wakes up too early.

Please do not delete.


Checklist:

Title:

  • Scope included as per conventional commits
  • Ticket reference included (unless it's a quick out of ticket thing)

Description:

  • Link to ticket included (unless it's a quick out of ticket thing)
  • Includes tests (or an explanation for why it doesn't)
  • If the work includes user interface changes, before and after screenshots included in description
  • Includes any applicable changes to the documentation in this code base
  • Includes link(s) to any applicable changes to the documentation in the DBT Platform Documentation (can be to a pull request)

Tasks:

  • Run the end to end tests for this branch and confirm that they are passing

@WillGibson WillGibson changed the title Add mutation testing test: Add mutation testing Nov 20, 2024
Comment on lines +70 to +75
We use [Cosmic Ray](https://cosmic-ray.readthedocs.io/en/latest/index.html) to run [Python mutation tests in GitHub Actions](https://github.com/uktrade/platform-tools/actions/workflows/python-mutation-tests.yml) on a schedule to help identify areas of our code which are not covered by our tests.

You can run the mutation tests locally, but:

- they take a very long time to run as they run the whole unit test suite for each mutation.
- they actually your source files while they run, so you cannot work on the code until it's finished.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be confirmed...

@codecov-commenter
Copy link

codecov-commenter commented Nov 21, 2024

❌ 12 Tests Failed:

Tests completed Failed Passed Skipped
613 12 601 0
View the top 3 failed tests by shortest run time
tests/platform_helper/test_command_conduit.py::::test_conduit_command_flags[write]
Stack Traces | 0.014s run time
tests/platform_helper/test_command_conduit.py:290: in test_conduit_command_flags
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
tests/platform_helper/test_command_conduit.py:299: in if_not_mutmut_run_assert_validate_version_called_once
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
tests/platform_helper/test_command_conduit.py:299: in if_not_mutmut_run_assert_validate_version_called_once
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
E   RecursionError: maximum recursion depth exceeded
!!! Recursion detected (same locals & position)
tests/platform_helper/test_command_conduit.py::::test_conduit_command_when_client_task_fails_to_start
Stack Traces | 0.014s run time
tests/platform_helper/test_command_conduit.py:155: in test_conduit_command_when_client_task_fails_to_start
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
tests/platform_helper/test_command_conduit.py:299: in if_not_mutmut_run_assert_validate_version_called_once
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
tests/platform_helper/test_command_conduit.py:299: in if_not_mutmut_run_assert_validate_version_called_once
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
E   RecursionError: maximum recursion depth exceeded
!!! Recursion detected (same locals & position)
tests/platform_helper/test_command_conduit.py::::test_conduit_command_flags[read]
Stack Traces | 0.015s run time
tests/platform_helper/test_command_conduit.py:290: in test_conduit_command_flags
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
tests/platform_helper/test_command_conduit.py:299: in if_not_mutmut_run_assert_validate_version_called_once
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
tests/platform_helper/test_command_conduit.py:299: in if_not_mutmut_run_assert_validate_version_called_once
    if_not_mutmut_run_assert_validate_version_called_once(validate_version)
E   RecursionError: maximum recursion depth exceeded
!!! Recursion detected (same locals & position)

To view more test analytics, go to the Test Analytics Dashboard
Got feedback? Let us know on Github

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants