Skip to content

Commit

Permalink
Add config example entry for Open Science Pool
Browse files Browse the repository at this point in the history
Open Science Pool is an open computing pool maintained by the OSG-Consortium.
This configuration uses CondorProvider with apptainer.
  • Loading branch information
yadudoc authored and khk-globus committed Jan 10, 2025
1 parent 4fd129d commit a0482d2
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 0 deletions.
39 changes: 39 additions & 0 deletions docs/endpoints/configs/ospool.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
display_name: OSPool
engine:
type: GlobusComputeEngine
max_workers_per_node: 1

# This config uses apptainer containerization to ensure a consistent
# python environment on the worker side. Since apptainer limits writable
# directory paths, set working directory paths paths used by the worker to /tmp
# P.S: These filepaths remain private to the container and will not be
# accessible on the host system
worker_logdir_root: /tmp/logs
working_dir: /tmp/tasks_dir

# GlobusComputeEngine relies on a shared-filesystem to distribute keys used
# for encrypting communication between the endpoint and workers.
# Since OSPool does not support a writable shared-filesystem,
# **encryption** is disabled in the configuration below.
encrypted: False

provider:

type: CondorProvider
init_blocks: 1
max_blocks: 1
min_blocks: 0

# Specify ProjectName and Apptainer image
scheduler_options: >
+ProjectName = {{ PROJECT_NAME }}
# To use apptainer on OSPool, build an apptainer image and copy it to
# OSDF and specify the full Specify the apptainer image path for eg.:
# "osdf:///ospool/ap20/data/USERNAME/globus_compute_py3.11.v1.sif"
+SingularityImage = {{ APPTAINER_IMAGE_PATH }}
# Add a condor requirement to guarantee that worker nodes support apptainer
Requirements = HAS_SINGULARITY == True && OSG_HOST_KERNEL_VERSION >= 31000
21 changes: 21 additions & 0 deletions docs/endpoints/endpoint_examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,27 @@ scheduler, and uses the ``SrunLauncher`` to launch workers.
.. literalinclude:: configs/faster.yaml
:language: yaml


Open Science Pool
^^^^^^^^^^^^^^^^^

The `Open Science Pool <https://osg-htc.org/services/open_science_pool.html>`_ is a pool of opportunistic
computing resources operated for all US-associated open science by the
`OSG consortium <https://osg-htc.org/about/introduction/>`_.
Unlike traditional HPC clusters, these computational resources are offered from campus and research cluster
resources that are loosely connected. The configuration below uses the ``CondorProvider``
to interface with the scheduler, and uses ``apptainer`` to distribute the computational environment
to the workers.

.. warning::
``GlobusComputeEngine`` relies on a shared-filesystem to distribute keys used for encrypting
communication between the endpoint and workers. Since OSPool does not support a writable
shared-filesystem, **encryption** is disabled in the configuration below.

.. literalinclude:: configs/ospool.yaml
:language: yaml


Pinning Workers to devices
^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down

0 comments on commit a0482d2

Please sign in to comment.