Performance Tests

Architecture and implementation needs to be performance test driven so that the final product satisfies performance requirements. Performance is measured in terms of scalability along two axes: number of tasks and number of agents.

We define three test categories:

• Large number of cores owned by one agent, represented through one task queue (single HPC scenarios)
• Large number of cores owned by small number of agents (multiple HPC scenarios)
  • through one aggregate task queue (saga-pilot only scenario)
  • through one task queue per agent (saga-pilot + TROY)
• Small number of cores owned by large number of agents (OSG / Cloud scenarios)
  • through one aggregate task queue (saga-pilot only scenario)
  • through one task queue per agent (saga-pilot + TROY)

Tests shall be defined as soon as the user-facing REST API has been defined and periodically run during all stages of the implementation period to ensure performance QoS and get an early handle on overall scalability and performance numbers and limits.