install_plugin fails due to externally-managed-environment

[Thinkpiet]

Hi,

when I execute fabsim localhost install_plugin:FabMaMiCo, I get this:

[Executing task] install_plugin
[local] mkdir -p /home/piet/repos/FabSim3/plugins
[local] rm -rf /home/piet/repos/FabSim3/plugins/FabMaMiCo
[local] git clone https://github.com/HSU-HPC/FabMaMiCo.git '/home/piet/repos/FabSim3/plugins/FabMaMiCo'
Cloning into '/home/piet/repos/FabSim3/plugins/FabMaMiCo'...
remote: Enumerating objects: 931, done.
remote: Counting objects: 100% (94/94), done.
remote: Compressing objects: 100% (62/62), done.
remote: Total 931 (delta 35), reused 44 (delta 23), pack-reused 837 (from 1)
Receiving objects: 100% (931/931), 36.38 MiB | 21.31 MiB/s, done.
Resolving deltas: 100% (501/501), done.
FabMaMiCo plugin installed...
Installing plugin requirements...
[local] pip install -r /home/piet/repos/FabSim3/plugins/FabMaMiCo/requirements.txt
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try 'pacman -S
    python-xyz', where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Arch-packaged Python package,
    create a virtual environment using 'python -m venv path/to/venv'.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip.
    
    If you wish to install a non-Arch packaged Python application,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. Make sure you have python-pipx
    installed via pacman.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
Traceback (most recent call last):
  File "/home/piet/repos/FabSim3/fabsim/bin/fabsim", line 46, in <module>
    sys.exit(fabsim_main.main())
             ^^^^^^^^^^^^^^^^^^
  File "/home/piet/repos/FabSim3/fabsim/base/fabsim_main.py", line 162, in main
    env.exec_func(*env.task_args, **env.task_kwargs)
  File "/home/piet/repos/FabSim3/fabsim/base/decorators.py", line 75, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/piet/repos/FabSim3/fabsim/base/setup_fabsim.py", line 86, in install_plugin
    local(f"pip install -r {plugin_dir}/{plugin_name}/requirements.txt")
  File "<@beartype(fabsim.base.networks.local) at 0x785a9afab880>", line 75, in local
  File "/home/piet/repos/FabSim3/fabsim/base/networks.py", line 62, in local
    raise RuntimeError(
RuntimeError: 
local() encountered an error (return code 1)while executing 'pip install -r /home/piet/repos/FabSim3/plugins/FabMaMiCo/requirements.txt'

This is surprising for me because the FabSim3 installation instructions do not say that I am supposed to run FabSim3 in a virtual environment. Thus, I was expecting that FabSim3 would manage a virtual environment, if necessary.

I could install the requirements manually with my system package manager (or with pip --break-system-packages), but I assume that then this FabSim command would still fail, since it still calls pip install in the externally-managed-environment. How am I supposed to handle this? Should I always launch FabSim in a Python virtual environment?

Platform details (optional)

• OS: Arch Linux
• FabSim3 commit 20c62f5
• Python version 3.12.6
• Pip version pip 24.2

[Thinkpiet]

Hi,

I want to suggest a solution for this:

1. Make sure that $FABSIM3_HOME/fabsim/bin is NOT in $PATH
2. Create a file ~/.local/bin/fabsim with this content:

   #!/bin/bash
   
   if [[ -z "$FABSIM3_HOME" ]]; then
       echo "ERROR environment variable FABSIM3_HOME is not set" 1>&2
       exit 1
   fi
   
   # check if virtual environment exists, otherwise create one 
   if [[ ! -e "$FABSIM3_HOME/venv" ]]; then
       python -m venv --system-site-packages $FABSIM3_HOME/venv
   fi
   
   # execute fabsim, wrapped in activated venv
   source $FABSIM3_HOME/venv/bin/activate
   $FABSIM3_HOME/fabsim/bin/fabsim "$@"
   deactivate
   

3. Make it executable, chmod +x ~/.local/bin/fabsim

That works for me :-) I am hoping it should work for anyone 👍

Best, Piet