Is it possible to attach a screenshot of failed tests?

[doston12]

Hi, I have two questions related to attaching logs & screenshots to test execution.

1. Is it possible to attach a screenshot of the failed tests on x-ray test execution?
2. Right now, I see logs of the failed tests under the "comment" section and the logs are not complete - only a small portion of the log messages are attached. Is it possible to send a complete test execution log?

[fundakol]

Comment contains stack trace from pytest, it should be the same what you see in a terminal code. Do you have something different reported in Xray?

It is not possible to attache a screenshot or any other test evidence. There is no such functionality implemented in the plugin.

[doston12]

Thanks for your response!
In the comments section it just shows the only stacktrace. It does not show the logs from previous steps

[fundakol]

What do you mean by steps? Do you use gherkin, or just logs from a python logger?

(I can try to add possibility to include screenshots as a test evidence)

[doston12]

I mean logs from python logger.
If you see the screenshot I attached there for the failed test in the comments section the logs showing only why the verify failed. However, I have in my test several steps before this verify statement. For each test step there is a log from python logger and they are not transferred/displayed in the comments section.

That would be really great to be able to attach a screenshot as test evidence, thanks!

[fundakol]

I created a branch attach-test-evidences with implementation for attaching test evidence. It could be jpeg or text as for now.
Can you test it? I have no access to Xray right now.

Here is an example how to use it:

# FILE: conftest.py
import pytest
from pytest_xray import evidence

@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
    outcome = yield
    report = outcome.get_result()
    evidences = getattr(report, "evidences", [])
    if report.when == "call":
        xfail = hasattr(report, "wasxfail")
        if (report.skipped and xfail) or (report.failed and not xfail):
            data = open('screenshot.jpg', 'rb').read()
            evidences.append(
                evidence.jpeg(data=data, filename="image21.jpg",)
            )
        report.evidences = evidences

[doston12]

Sorry, maybe I am not catching up well. But, suppose the screenshot taken once the test failed is "xyz.png" where I should refer to it:
in this line:
data = open('xyz.png', 'rb').read()
or, here:

evidences.append(
                evidence.jpeg(data=data, filename="xyz.png",)
)

[fundakol]

data = open('xyz.png', 'rb').read() -> zyz.png is our screenshot's filename
evidence.jpeg(data=data, filename="xyz.png",) -> xyz.png is a filename which you would see in XRAY

XRAY API required evidence as dict:

"evidences" : [
          {
              "data": "(... base 64 encoded ...)",
              "filename": "screenshot1.jpg",
              "contentType": "image/jpeg"
          }
      ]

BTW, you can attach logs from test in the same way:

logs = report.caplog
evidences.append(evidence.text(data=logs, filename="test.log"))

[doston12]

Hi, for the failed tests no screenshot is attached.
I had to change slightly your code to adjust to our test framework:

`@pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
outcome = yield
report = outcome.get_result()
evidences = getattr(report, "evidences", [])
if report.when == "call":
xfail = hasattr(report, "wasxfail")
if (report.skipped and xfail) or (report.failed and not xfail):
screenshot_file = f"{TEST_METHOD_NAME}" + ".jpg"
time.sleep(1)
DRIVER.save_screenshot(screenshot_file)
logger.info("Screenshot is taken with name: %s", screenshot_file)
logger.info("Screenshot absolute path: %s", os.path.abspath(screenshot_file))

        with open(screenshot_file, mode="rb") as the_file:
            data = the_file.read()
            evidences.append(
                jpeg(data=data, filename=screenshot_file,)
            )

    report.evidences = evidences`

[doston12]

We launch tests on Jenkins using docker containers. I think the problem is that there is something going wrong while reading the file from the saved location.
I see the logs that say the screenshot is saved in a particular path. But, when I try to read file from that path FileNotFound error was being returned.

[fundakol]

Did you try to use full path to a screenshot file, e.g.:

from pathlib import Path
screenshot_path = Path.cwd() /  f"{TEST_METHOD_NAME}" + ".jpg"

Also if DRIVER is Selenium webdriver than you should use PNG format

[doston12]

Thanks. I will try that.
I use Selenium Remote webdriver.

Changing screenshot format from "jpg" to "png" also requires changes these parts of the code as well, right?
`
IMAGE_JPEG: str = "image/jpeg"

def jpeg(data: AnyStr, filename: str) -> Dict[str, str]:
return evidence(data, filename, IMAGE_JPEG)
`

[fundakol]

Yes, I commited that change to the branch.