Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: fix occasional LVM2-thinp test failures #66

Merged
merged 1 commit into from
Jan 8, 2025

Conversation

coiby
Copy link
Member

@coiby coiby commented Jan 7, 2025

Somehow LVM2-thinp test fails occasionally. It turns out the test fails
because /etc/lvm/devices/system.devices somehow becomes empty file after
a kernel panic,

[root@client ~]# journalctl -b0 |grep vdb
Aug 02 08:10:52 localhost kernel: virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)
Aug 02 08:10:54 client lvm[712]: /dev/vdb excluded by devices file (checking PVID).

[root@client ~]# cat /etc/lvm/devices/system.devices
# empty result

[root@client ~]# pvscan -vvv
  /dev/vdb: Skipping (deviceid)

"sync -f /etc/lvm/devices/system.devices" doesn't work for this case. So
simply delete it to prevent an empty /etc/lvm/devices/system.devices. An
empty system.devices will filter out the LVM device (dev/vdb) [1].

Note the failure of this flaky test can happen 1/15 times,

log_dir=logs_dir
mkdir -p $log_dir
for i in {01..15}; do
  if tmt run  --environment KDUMP_UTILS_RPM=kdump-utils plan --name /plans/lvm2_thinp &> $log_dir/$i.log; then
    echo $i success
  else
    echo $i failure
    break
  fi
done

[1] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/configuring_and_managing_logical_volumes/limiting-lvm-device-visibility-and-usage_configuring-and-managing-logical-volumes?extIdCarryOver=true&sc_cid=701f2000001Css5AAC#disabling-the-system-devices-file_the-lvm-devices-file

@coiby coiby requested a review from liutgnu January 7, 2025 08:20
@coiby
Copy link
Member Author

coiby commented Jan 7, 2025

Hi @liutgnu, do you know why /etc/lvm/devices/system.devices will become empty?

@liutgnu
Copy link
Collaborator

liutgnu commented Jan 7, 2025

Hi @liutgnu, do you know why /etc/lvm/devices/system.devices will become empty?

Hi coiby, Thanks for fixing the issue. I'm not sure, because I never encountered it. And I think delete the file is a good solution because according to 1, when the file is removed, it will use the device filter in lvm.conf instead, which is none in this case. So Ack for your patch.

Somehow LVM2-thinp test fails occasionally. It turns out the test fails
because /etc/lvm/devices/system.devices somehow becomes empty file after
a kernel panic,

    [root@client ~]# journalctl -b0 |grep vdb
    Aug 02 08:10:52 localhost kernel: virtio_blk virtio2: [vdb] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)
    Aug 02 08:10:54 client lvm[712]: /dev/vdb excluded by devices file (checking PVID).

    [root@client ~]# cat /etc/lvm/devices/system.devices
    # empty result

    [root@client ~]# pvscan -vvv
      /dev/vdb: Skipping (deviceid)

/etc/lvm/devices/system.devices still has valid content before
triggering a kernel panic and it only becomes empty after a kernel
panic. "sync -f /etc/lvm/devices/system.devices" unfortunately doesn't
work for this case. So simply delete it to prevent an empty
/etc/lvm/devices/system.devices. An empty system.devices will filter out
the LVM device (dev/vdb) [1].

Note the failure of this flaky test can happen 1/15 times,

    log_dir=logs_dir
    mkdir -p $log_dir
    for i in {01..15}; do
      if tmt run  --environment KDUMP_UTILS_RPM=kdump-utils plan --name /plans/lvm2_thinp &> $log_dir/$i.log; then
        echo $i success
      else
        echo $i failure
        break
      fi
    done

[1] https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/configuring_and_managing_logical_volumes/limiting-lvm-device-visibility-and-usage_configuring-and-managing-logical-volumes?extIdCarryOver=true&sc_cid=701f2000001Css5AAC#disabling-the-system-devices-file_the-lvm-devices-file

Signed-off-by: Coiby Xu <coxu@redhat.com>
@coiby
Copy link
Member Author

coiby commented Jan 8, 2025

@liutgnu Thanks for ack'ing the patch! Can you explicit approve this PR which is required by the workflow? Btw, I should have noted that /etc/lvm/devices/system.devices still has valid content before triggering the kernel panic. It's a kind of mysterious that sync -f /etc/lvm/devices/system.devices doesn't work.

@coiby coiby merged commit 3c7b8f7 into rhkdump:main Jan 8, 2025
6 of 9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants