You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When i use DS_BUILD_AIO=1 CFLAGS="-I$CONDA_PREFIX/include/ -I/usr/include/" LDFLAGS="-L$CONDA_PREFIX/lib/ -L/usr/lib/x86_64-linux-gnu/" pip install -e . to install async_io op, i get fake successful msg.
it indeed displays Successfully installed deepspeed , but i use ds_report and only get .
And i use print stderr msg and i find that
To figure out how to result in this case's coming. I read the source code such as "setup.py"...
and i find problem in "setup.py line 182" for op_name, builder in ALL_OPS.items(): op_compatible = builder.is_compatible()
When op_name is "async_io", builder.is_compatible() returns false. And i open the "DeepSpeed/deepspeed/ops/op_builder/async_io.py" and find "line 93" def is_compatible(self, verbose=False) . It's result depends on "line 99" aio_compatible = self.has_function('io_submit', ('aio', )) .
Go on to find def has_function() in "DeepSpeed/deepspeed/ops/op_builder/builder.py line308" , and i confirm it raise linkerror in line362 compiler.link_executable(objs, os.path.join(tempdir, 'a.out'), extra_preargs=self.strip_empty_entries(ldflags), libraries=libraries, library_dirs=library_dirs) by "distutils.unixccompiler.UnixCCompiler"
I don't know why it happened and to address this issue i had to change the "class AsyncIOBuilder"("DeepSpeed/deepspeed/ops/op_builder/async_io.py") like the following picture .
And i install it again and get the correct result.
I hope u can figure out why it caused link error. And i don't know my change whether to cause aio disabled when i use offload.
The text was updated successfully, but these errors were encountered:
LZhengguo
changed the title
{{ env.GITHUB_WORKFLOW }} Cannot install async_io op even if it's compatible flag is displaying OK by ds_report cmd!
Cannot install async_io op even if it's compatible flag is displaying OK by ds_report cmd!
Dec 31, 2024
When i use
DS_BUILD_AIO=1 CFLAGS="-I$CONDA_PREFIX/include/ -I/usr/include/" LDFLAGS="-L$CONDA_PREFIX/lib/ -L/usr/lib/x86_64-linux-gnu/" pip install -e .
to install async_io op, i get fake successful msg.it indeed displays
Successfully installed deepspeed
, but i useds_report
and only get .And i use print stderr msg and i find that
To figure out how to result in this case's coming. I read the source code such as "setup.py"...
and i find problem in "setup.py line 182"
for op_name, builder in ALL_OPS.items(): op_compatible = builder.is_compatible()
When op_name is "async_io", builder.is_compatible() returns false. And i open the "DeepSpeed/deepspeed/ops/op_builder/async_io.py" and find "line 93"
def is_compatible(self, verbose=False)
. It's result depends on "line 99"aio_compatible = self.has_function('io_submit', ('aio', ))
.Go on to find
def has_function()
in "DeepSpeed/deepspeed/ops/op_builder/builder.py line308" , and i confirm it raise linkerror in line362compiler.link_executable(objs, os.path.join(tempdir, 'a.out'), extra_preargs=self.strip_empty_entries(ldflags), libraries=libraries, library_dirs=library_dirs)
by "distutils.unixccompiler.UnixCCompiler"I don't know why it happened and to address this issue i had to change the "class AsyncIOBuilder"("DeepSpeed/deepspeed/ops/op_builder/async_io.py") like the following picture .
And i install it again and get the correct result.
I hope u can figure out why it caused link error. And i don't know my change whether to cause aio disabled when i use offload.
The text was updated successfully, but these errors were encountered: