You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Trying to run the downloadable deepdanbooru-v3/v4 always ends up in the same error
Loading deepdanbooru-v3-20211112-sgd-e28 from <DirEntry 'deepdanbooru-v3-20211112-sgd-e28'> 2024-03-30 17:12:30.152946: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.174784: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.178338: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.182152: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.185918: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.189107: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.192417: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.195551: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.199125: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.202313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1929] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 6744 MB memory: -> device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:10:00.0, compute capability: 8.6 Loaded deepdanbooru-v3-20211112-sgd-e28 model from <DirEntry 'deepdanbooru-v3-20211112-sgd-e28'> 2024-03-30 17:12:38.330697: W external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:504] Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice. This may result in compilation or runtime failures, if the program we try to run uses routines from libdevice. Searched for CUDA in the following directories: ./cuda_sdk_lib /usr/local/cuda-12.2 /usr/local/cuda /mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorflow/python/platform/../../../nvidia/cuda_nvcc /mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorflow/python/platform/../../../../nvidia/cuda_nvcc You can choose the search directory by setting xla_gpu_cuda_data_dir in HloModule's DebugOptions. For most apps, setting the environment variable XLA_FLAGS=--xla_gpu_cuda_data_dir=/path/to/cuda will work. 2024-03-30 17:12:38.516224: W external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:542] libdevice is required by this HLO module but was not found at ./libdevice.10.bc error: libdevice not found at ./libdevice.10.bc 2024-03-30 17:12:38.516496: E tensorflow/compiler/mlir/tools/kernel_gen/tf_framework_c_interface.cc:207] INTERNAL: Generating device code failed. 2024-03-30 17:12:38.516839: W tensorflow/core/framework/op_kernel.cc:1827] UNKNOWN: JIT compilation failed. *** Error completing request *** Arguments: (<PIL.Image.Image image mode=RGB size=850x1363 at 0x7B5A310A8BE0>, 'deepdanbooru-v3-20211112-sgd-e28', 'ba', '', '', '', '', '') {} Traceback (most recent call last): File "/mnt/ts512/stable-diffusion-webui-forge/modules/call_queue.py", line 57, in f res = list(func(*args, **kwargs)) File "/mnt/ts512/stable-diffusion-webui-forge/modules/call_queue.py", line 36, in f res = func(*args, **kwargs) File "/mnt/ts512/stable-diffusion-webui-forge/extensions/stable-diffusion-webui-wd14-tagger/tagger/ui.py", line 113, in on_interrogate_image_submit interrogator.interrogate_image(image) File "/mnt/ts512/stable-diffusion-webui-forge/extensions/stable-diffusion-webui-wd14-tagger/tagger/interrogator.py", line 150, in interrogate_image data = ('', '', fi_key) + self.interrogate(image) File "/mnt/ts512/stable-diffusion-webui-forge/extensions/stable-diffusion-webui-wd14-tagger/tagger/interrogator.py", line 309, in interrogate image = ddd.load_image_for_evaluate( File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/deepdanbooru/data/__init__.py", line 26, in load_image_for_evaluate image = tf.image.resize( File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorflow/python/framework/ops.py", line 5883, in raise_from_not_ok_status raise core._status_to_exception(e) from None # pylint: disable=protected-access tensorflow.python.framework.errors_impl.UnknownError: {{function_node __wrapped__Round_device_/job:localhost/replica:0/task:0/device:GPU:0}} JIT compilation failed. [Op:Round] name:
Traceback (most recent call last): File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict output = await app.get_blocks().process_api( File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1434, in process_api data = self.postprocess_data(fn_index, result["prediction"], state) File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1297, in postprocess_data self.validate_outputs(fn_index, predictions) # type: ignore File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1272, in validate_outputs raise ValueError( ValueError: An event handler (on_interrogate_image_submit) didn't receive enough output values (needed: 7, received: 3). Wanted outputs: [state, html, html, label, label, label, html] Received outputs: [None, "", "<div class='error'>UnknownError: {{function_node __wrapped__Round_device_/job:localhost/replica:0/task:0/device:GPU:0}} JIT compilation failed. [Op:Round] name: </div><div class='performance'><p class='time'>Time taken: <wbr><span class='measurement'>8.4 sec.</span></p><p class='vram'><abbr title='Active: peak amount of video memory used during generation (excluding cached data)'>A</abbr>: <span class='measurement'>1.73 GB</span>, <wbr><abbr title='Reserved: total amout of video memory allocated by the Torch library '>R</abbr>: <span class='measurement'>1.75 GB</span>, <wbr><abbr title='System: peak amout of video memory allocated by all running programs, out of total capacity'>Sys</abbr>: <span class='measurement'>5.5/11.5391 GB</span> (47.9%)</p></div>"]
The text was updated successfully, but these errors were encountered:
Trying to run the downloadable
deepdanbooru-v3/v4
always ends up in the same errorLoading deepdanbooru-v3-20211112-sgd-e28 from <DirEntry 'deepdanbooru-v3-20211112-sgd-e28'> 2024-03-30 17:12:30.152946: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.174784: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.178338: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.182152: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.185918: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.189107: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.192417: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.195551: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.199125: I external/local_xla/xla/stream_executor/cuda/cuda_executor.cc:901] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 2024-03-30 17:12:30.202313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1929] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 6744 MB memory: -> device: 0, name: NVIDIA GeForce RTX 3060, pci bus id: 0000:10:00.0, compute capability: 8.6 Loaded deepdanbooru-v3-20211112-sgd-e28 model from <DirEntry 'deepdanbooru-v3-20211112-sgd-e28'> 2024-03-30 17:12:38.330697: W external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:504] Can't find libdevice directory ${CUDA_DIR}/nvvm/libdevice. This may result in compilation or runtime failures, if the program we try to run uses routines from libdevice. Searched for CUDA in the following directories: ./cuda_sdk_lib /usr/local/cuda-12.2 /usr/local/cuda /mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorflow/python/platform/../../../nvidia/cuda_nvcc /mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorflow/python/platform/../../../../nvidia/cuda_nvcc You can choose the search directory by setting xla_gpu_cuda_data_dir in HloModule's DebugOptions. For most apps, setting the environment variable XLA_FLAGS=--xla_gpu_cuda_data_dir=/path/to/cuda will work. 2024-03-30 17:12:38.516224: W external/local_xla/xla/service/gpu/llvm_gpu_backend/gpu_backend_lib.cc:542] libdevice is required by this HLO module but was not found at ./libdevice.10.bc error: libdevice not found at ./libdevice.10.bc 2024-03-30 17:12:38.516496: E tensorflow/compiler/mlir/tools/kernel_gen/tf_framework_c_interface.cc:207] INTERNAL: Generating device code failed. 2024-03-30 17:12:38.516839: W tensorflow/core/framework/op_kernel.cc:1827] UNKNOWN: JIT compilation failed. *** Error completing request *** Arguments: (<PIL.Image.Image image mode=RGB size=850x1363 at 0x7B5A310A8BE0>, 'deepdanbooru-v3-20211112-sgd-e28', 'ba', '', '', '', '', '') {} Traceback (most recent call last): File "/mnt/ts512/stable-diffusion-webui-forge/modules/call_queue.py", line 57, in f res = list(func(*args, **kwargs)) File "/mnt/ts512/stable-diffusion-webui-forge/modules/call_queue.py", line 36, in f res = func(*args, **kwargs) File "/mnt/ts512/stable-diffusion-webui-forge/extensions/stable-diffusion-webui-wd14-tagger/tagger/ui.py", line 113, in on_interrogate_image_submit interrogator.interrogate_image(image) File "/mnt/ts512/stable-diffusion-webui-forge/extensions/stable-diffusion-webui-wd14-tagger/tagger/interrogator.py", line 150, in interrogate_image data = ('', '', fi_key) + self.interrogate(image) File "/mnt/ts512/stable-diffusion-webui-forge/extensions/stable-diffusion-webui-wd14-tagger/tagger/interrogator.py", line 309, in interrogate image = ddd.load_image_for_evaluate( File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/deepdanbooru/data/__init__.py", line 26, in load_image_for_evaluate image = tf.image.resize( File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler raise e.with_traceback(filtered_tb) from None File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/tensorflow/python/framework/ops.py", line 5883, in raise_from_not_ok_status raise core._status_to_exception(e) from None # pylint: disable=protected-access tensorflow.python.framework.errors_impl.UnknownError: {{function_node __wrapped__Round_device_/job:localhost/replica:0/task:0/device:GPU:0}} JIT compilation failed. [Op:Round] name:
Traceback (most recent call last): File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/gradio/routes.py", line 488, in run_predict output = await app.get_blocks().process_api( File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1434, in process_api data = self.postprocess_data(fn_index, result["prediction"], state) File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1297, in postprocess_data self.validate_outputs(fn_index, predictions) # type: ignore File "/mnt/ts512/stable-diffusion-webui-forge/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1272, in validate_outputs raise ValueError( ValueError: An event handler (on_interrogate_image_submit) didn't receive enough output values (needed: 7, received: 3). Wanted outputs: [state, html, html, label, label, label, html] Received outputs: [None, "", "<div class='error'>UnknownError: {{function_node __wrapped__Round_device_/job:localhost/replica:0/task:0/device:GPU:0}} JIT compilation failed. [Op:Round] name: </div><div class='performance'><p class='time'>Time taken: <wbr><span class='measurement'>8.4 sec.</span></p><p class='vram'><abbr title='Active: peak amount of video memory used during generation (excluding cached data)'>A</abbr>: <span class='measurement'>1.73 GB</span>, <wbr><abbr title='Reserved: total amout of video memory allocated by the Torch library '>R</abbr>: <span class='measurement'>1.75 GB</span>, <wbr><abbr title='System: peak amout of video memory allocated by all running programs, out of total capacity'>Sys</abbr>: <span class='measurement'>5.5/11.5391 GB</span> (47.9%)</p></div>"]
The text was updated successfully, but these errors were encountered: