Fixed package naming for variants, fixed html install documentation

Signed-off-by: Bharath Ramaswamy <quic_bharathr@quicinc.com>
quic · Jun 4, 2024 · d74961d · d74961d
1 parent d942f8e
commit d74961d
Show file tree

Hide file tree

Showing 5 changed files with 115 additions and 36 deletions.
diff --git a/Docs/install/index.rst b/Docs/install/index.rst
@@ -1,7 +1,7 @@
 .. # =============================================================================
    #  @@-COPYRIGHT-START-@@
    #
-   #  Copyright (c) 2022-2023, Qualcomm Innovation Center, Inc. All rights reserved.
+   #  Copyright (c) 2022-2024, Qualcomm Innovation Center, Inc. All rights reserved.
    #
    #  Redistribution and use in source and binary forms, with or without
    #  modification, are permitted provided that the following conditions are met:
@@ -40,34 +40,74 @@
 AIMET Installation
 ###################
 
-Release packages
-~~~~~~~~~~~~~~~~
+Quick Install
+~~~~~~~~~~~~~
+
+The AIMET PyTorch GPU PyPI packages are available for environments that meet the following requirements:
 
-AIMET release packages are hosted at https://github.com/quic/aimet/releases. Each release includes multiple python packages of the following format:
+* 64-bit Intel x86-compatible processor
+* Linux Ubuntu 22.04 LTS [Python 3.10] or Linux Ubuntu 20.04 LTS [Python 3.8]
+* Torch 1.13+cu117
+
+**Pip install:**
 
 .. code-block::
 
-    <PACKAGE_PREFIX>-<VARIANT>_<VERSION>-cp310-cp310-linux_x86_64.whl
+    apt-get install liblapacke
+    python3 -m pip install aimet-torch
+
+
+Release Packages
+~~~~~~~~~~~~~~~~
+
+For other aimet variants, install the latest version from the .whl files hosted at https://github.com/quic/aimet/releases
+
+**PyTorch**
+
+.. parsed-literal::
+
+    # Pytorch 1.13 with CUDA 11.x
+    python3 -m pip install |download_url|\ |version|/aimet_torch_gpu\_\ |version|\ |whl_suffix| -f https://download.pytorch.org/whl/torch_stable.html
+    # Pytorch 1.13 CPU only
+    python3 -m pip install |download_url|\ |version|/aimet_torch_cpu\_\ |version|\ |whl_suffix|
+
+    # Pytorch 2.1 with CUDA 11.x
+    python3 -m pip install |download_url|\ |version|/aimet_torch_gpu_pt21\_\ |version|\ |whl_suffix| -f https://download.pytorch.org/whl/torch_stable.html
+    # Pytorch 2.1 CPU only
+    python3 -m pip install |download_url|\ |version|/aimet_torch_cpu_pt21\_\ |version|\ |whl_suffix|
+
+
+**TensorFlow**
+
+.. parsed-literal::
+
+    # Tensorflow 2.10 GPU with CUDA 11.x
+    python3 -m pip install |download_url|\ |version|/aimet_tensorflow_gpu\_\ |version|\ |whl_suffix|
+    # Tensorflow 2.10 CPU only
+    python3 -m pip install |download_url|\ |version|/aimet_tensorflow_cpu\_\ |version|\ |whl_suffix|
+
 
-Please find more information below about each *VARIANT*.
+**Onnx**
 
-PyTorch
+.. parsed-literal::
 
-#. **torch-gpu** for PyTorch 1.13 GPU package with Python 3.10 and CUDA 11.x - *Recommended for use* with PyTorch models
-#. **torch-cpu** for PyTorch 1.13 CPU package with Python 3.10 - If installing on a machine without CUDA
-#. **torch-gpu-pt21** for PyTorch 2.1 GPU package with Python 3.10 and CUDA 11.x
-#. **torch-cpu-pt21** for PyTorch 2.1 CPU package with Python 3.10 - If installing on a machine without CUDA
+    # ONNX 1.14 GPU
+    python3 -m pip install |download_url|\ |version|/aimet_onnx_gpu\_\ |version|\ |whl_suffix|
+    # ONNX 1.14 CPU
+    python3 -m pip install |download_url|\ |version|/aimet_onnx_cpu\_\ |version|\ |whl_suffix|
 
+For previous AIMET releases, browse packages at https://github.com/quic/aimet/releases. Each release includes multiple python packages of the following format:
 
-TensorFlow
+.. parsed-literal::
 
-#. **tf-gpu** for TensorFlow 2.10 GPU package with Python 3.10 - *Recommended for use* with TensorFlow models
-#. **tf-cpu** for TensorFlow 2.10 CPU package with Python 3.10 - If installing on a machine without CUDA
+    # VARIANT in {torch_gpu, torch_cpu, tf_gpu, tf_cpu, onnx_gpu, onnx_cpu}
+    # PACKAGE_PREFIX in {aimet_torch, aimet_tensorflow, aimet_onnx}
+    <PACKAGE_PREFIX>-<VARIANT>_<VERSION>\ |whl_suffix|
 
-ONNX
 
-#. **onnx-gpu** for ONNX 1.14 GPU package with Python 3.10 - *Recommended for use* with ONNX models
-#. **onnx-cpu** for ONNX 1.14 CPU package with Python 3.10 - If installing on a machine without CUDA
+.. |version| replace:: 1.31.0
+.. |whl_suffix| replace:: -cp38-cp38-manylinux_2_34_x86_64.whl
+.. |download_url| replace:: \https://github.com/quic/aimet/releases/download/
 
 System Requirements
 ~~~~~~~~~~~~~~~~~~~
@@ -93,7 +133,7 @@ Recommended host system hardware requirements:
 
 While these are not minimum requirements, they are recommended for good performance when training large networks.
 
-Installation Instructions
+Advanced Installation Instructions
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 There are two ways to setup and install AIMET:

diff --git a/Docs/install/install_docker.rst b/Docs/install/install_docker.rst
@@ -1,7 +1,7 @@
 .. # =============================================================================
    #  @@-COPYRIGHT-START-@@
    #
-   #  Copyright (c) 2022-2023, Qualcomm Innovation Center, Inc. All rights reserved.
+   #  Copyright (c) 2022-2024, Qualcomm Innovation Center, Inc. All rights reserved.
    #
    #  Redistribution and use in source and binary forms, with or without
    #  modification, are permitted provided that the following conditions are met:
@@ -144,8 +144,8 @@ Set the <variant_string> to ONE of the following depending on your desired varia
 
 #. For the PyTorch 1.13 GPU variant, use "torch_gpu"
 #. For the PyTorch 1.13 CPU variant, use "torch_cpu"
-#. For the PyTorch 2.1 GPU variant, use "torch_gpu-pt21"
-#. For the PyTorch 2.1 CPU variant, use "torch_cpu-pt21"
+#. For the PyTorch 2.1 GPU variant, use "torch_gpu_pt21"
+#. For the PyTorch 2.1 CPU variant, use "torch_cpu_pt21"
 #. For the TensorFlow GPU variant, use "tf_gpu"
 #. For the TensorFlow CPU variant, use "tf_cpu"
 #. For the ONNX GPU variant, use "onnx_gpu"
@@ -171,7 +171,7 @@ Set the common suffix for the package files as follows:
 
 .. code-block:: bash
 
-    export wheel_file_suffix="cp310-cp310-linux_x86_64.whl"
+    export wheel_file_suffix="cp310-cp310-manylinux_2_34_x86_64.whl"
 
 Install the AIMET packages in the order specified below:
 
@@ -183,11 +183,11 @@ Install the AIMET packages in the order specified below:
 .. code-block:: bash
 
     # Install ONE of the following depending on the variant
-    python3 -m pip install ${download_url}/aimet_torch-${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix} -f https://download.pytorch.org/whl/torch_stable.html
+    python3 -m pip install ${download_url}/aimet_${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix} -f https://download.pytorch.org/whl/torch_stable.html
     # OR
-    python3 -m pip install ${download_url}/aimet_tensorflow-${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix}
+    python3 -m pip install ${download_url}/aimet_${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix}
     # OR
-    python3 -m pip install ${download_url}/aimet_onnx-${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix}
+    python3 -m pip install ${download_url}/aimet_${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix}
 
 Environment setup
 ~~~~~~~~~~~~~~~~~

diff --git a/Docs/install/install_host.rst b/Docs/install/install_host.rst
@@ -1,7 +1,7 @@
 .. # =============================================================================
    #  @@-COPYRIGHT-START-@@
    #
-   #  Copyright (c) 2022-2023, Qualcomm Innovation Center, Inc. All rights reserved.
+   #  Copyright (c) 2022-2024, Qualcomm Innovation Center, Inc. All rights reserved.
    #
    #  Redistribution and use in source and binary forms, with or without
    #  modification, are permitted provided that the following conditions are met:
@@ -122,6 +122,28 @@ Install GPU packages for PyTorch 2.1 or TensorFlow
 Install AIMET packages
 ~~~~~~~~~~~~~~~~~~~~~~~
 
+From PyPI
+=========
+
+Aimet Torch GPU can install from pypi through the following method:
+
+Go to https://pypi.org/project/aimet-torch to identify a version you wish to install
+
+    - For PyTorch 1.13 GPU you should use aimet-torch==1.31.1
+    - For Pytorch 2.1.2 GPU you should use aimet-torch >= 1.32.0
+
+.. code-block:: bash
+
+    sudo apt-get install liblapacke -y
+    pip install aimet-torch
+
+
+From Release Package
+====================
+
+Alternatively, we host .whl packages for each release at https://github.com/quic/aimet/releases. Identify the release tag
+of the package you wish to install, then follow the instructions below to install AIMET from the .whl file.
+
 Go to https://github.com/quic/aimet/releases and identify the release tag of the package you want to install.
 
 Set the <variant_string> to ONE of the following depending on your desired variant
@@ -166,11 +188,11 @@ Install the AIMET packages in the order specified below:
 .. code-block:: bash
 
     # Install ONE of the following depending on the variant
-    python3 -m pip install ${download_url}/aimet_torch-${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix} -f https://download.pytorch.org/whl/torch_stable.html
+    python3 -m pip install ${download_url}/aimet_${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix} -f https://download.pytorch.org/whl/torch_stable.html
     # OR
-    python3 -m pip install ${download_url}/aimet_tensorflow-${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix}
+    python3 -m pip install ${download_url}/aimet_${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix}
     # OR
-    python3 -m pip install ${download_url}/aimet_onnx-${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix}
+    python3 -m pip install ${download_url}/aimet_${AIMET_VARIANT}_${release_tag}-${wheel_file_suffix}
 
 
 Install common debian packages

diff --git a/packaging/package_aimet.cmake b/packaging/package_aimet.cmake
@@ -47,6 +47,7 @@ file(REMOVE_RECURSE ${build_packaging_dir})
 # set variant name
 if(DEFINED ENV{AIMET_VARIANT})
   set(variant_name $ENV{AIMET_VARIANT})
+  message(WARNING "*** variant_name was set to AIMET_VARIANT ${variant_name} ***")
 else()
   if(ENABLE_TORCH AND ENABLE_TENSORFLOW)
     set(variant_name "tf-torch")
@@ -65,6 +66,8 @@ else()
   else()
     set(variant_name ${variant_name}-cpu)
   endif()
+
+  message(WARNING "*** AIMET_VARIANT not defined. Variant_name was set to ${variant_name} ***")
 endif()
 
 # Common dependencies
@@ -143,7 +146,8 @@ endif()
 # Pull just the version from the string (ex. 1.13+c116 --> 1.13)
 execute_process(COMMAND bash -c "echo ${FMWORK_VERSION} | tr -s ' ' | cut -d\"+\" -f1"
                 OUTPUT_VARIABLE fmwork_ver_bare
-                OUTPUT_STRIP_TRAILING_WHITESPACE)
+                OUTPUT_STRIP_TRAILING_WHITESPACE
+                COMMAND_ERROR_IS_FATAL ANY)
 
 
 # Loop over the package array list to generate wheel files
@@ -161,6 +165,7 @@ foreach(package ${package_name_list})
     COMMAND ${CMAKE_COMMAND} -E copy_directory "${AIMET_PACKAGE_PATH}/lib/x86_64-linux-gnu" "${pkg_common_staging_path}/x86_64-linux-gnu"
     COMMAND ${CMAKE_COMMAND} -E copy "${src_packaging_dir}/NOTICE.txt" "${pkg_staging_path}/"
     COMMAND ${CMAKE_COMMAND} -E copy "${src_packaging_dir}/README.txt" "${pkg_staging_path}/"
+    COMMAND_ERROR_IS_FATAL ANY
     )
   execute_process(
     # Delete binaries from aimet_common which should not be part of the package
@@ -170,13 +175,15 @@ foreach(package ${package_name_list})
   list(TRANSFORM deps_name_list_${package} PREPEND "${src_deps_dir}/${variant_name}/")
 
   # Copy the dependency and other files into the staging subfolder
-  execute_process(COMMAND ${CMAKE_COMMAND} -E make_directory "${pkg_deps_staging_path}")
+  execute_process(COMMAND ${CMAKE_COMMAND} -E make_directory "${pkg_deps_staging_path}"
+                  COMMAND_ERROR_IS_FATAL ANY)
   execute_process(
     COMMAND ${CMAKE_COMMAND} -E copy 
       ${deps_name_list_${package}}
       "${src_packaging_dir}/INSTALL.txt"
       "${src_packaging_dir}/envsetup.sh"
       "${pkg_deps_staging_path}/"
+      COMMAND_ERROR_IS_FATAL ANY
   )
 
   if(EXISTS "${src_packaging_dir}/LICENSE.pdf")
@@ -205,6 +212,7 @@ foreach(package ${package_name_list})
     WORKING_DIRECTORY ${pkg_staging_path}
     OUTPUT_VARIABLE output_var
     RESULT_VARIABLE setup_return_string
+    COMMAND_ERROR_IS_FATAL ANY
   )
 endforeach()
 
@@ -213,7 +221,5 @@ message(STATUS "Package rename command: ${rename_package_cmd}")
 execute_process(COMMAND bash "-c" "${src_packaging_dir}/convert_wheel_format.sh ${CMAKE_BINARY_DIR}"
                 OUTPUT_VARIABLE convert_wheel_output
                 RESULT_VARIABLE convert_wheel_result
-                OUTPUT_STRIP_TRAILING_WHITESPACE)
-if(convert_wheel_result EQUAL "1")
-  message( FATAL_ERROR "Wheel package conversion failed")
-endif()
+                OUTPUT_STRIP_TRAILING_WHITESPACE
+                COMMAND_ERROR_IS_FATAL ANY)
diff --git a/packaging/setup.py.in b/packaging/setup.py.in
@@ -266,6 +266,15 @@ def get_package_version():
     return package_version_with_variant
 
 
+def get_wheel_package_name():
+    whl_package_name = "aimet_@package@"
+    if "pypi" not in "@PIP_INDEX@":
+        aimet_variant = os.environ.get("AIMET_VARIANT", None)
+        if aimet_variant is not None:
+            whl_package_name = "aimet_" + aimet_variant
+    return whl_package_name
+
+
 # Obtain list of package and sub-packages (including common dependency ones)
 packages_found = get_package_list(package_name)
 
@@ -284,14 +293,16 @@ dependency_url_list.extend(get_all_dependency_urls(common_package_name))
 # Obtain the package home page URL
 package_url_base = get_package_url()
 
+# Obtain the name of the wheel package
+wheel_package_name = get_wheel_package_name()
 
 if "--gpu" in sys.argv:
     sys.argv.remove("--gpu")
 package_version = get_package_version()
 
 
 setup(
-    name='aimet-@package@',
+    name=wheel_package_name,
     version=package_version,
     author='Qualcomm Innovation Center, Inc.',
     author_email='aimet.os@quicinc.com',