Data Transfer Test Campaigns

CampaignFocusStart DateEnd Date/FrequencySoftware/Data ResourcesInfrastructure ResourcesActivities
Network and Storage Performance Dress RehearsalNetwork and storage performanceApril 1April 15Rucio task manager, various sized data sets, monitoring dashboards
ResourcePercentage
Storage50%
Network50%
Computing10%
  • Conduct full network mesh tests with data of various sizes using Rucio task manager.
  • Monitor data transfer and storage performance through dashboards.
Network and Storage PerformanceNetwork and storage performanceMay 1May 7/first week of every monthRucio task manager, various sized data sets, monitoring dashboards
ResourcePercentage
Storage100%
Network100%
Computing10%
  • Conduct full network mesh tests with data of various sizes using Rucio task manager.
  • Monitor data transfer and storage performance through dashboards.
Scalability and Stress TestsSystem scalability and stress toleranceJuly 15July 31Predefined workload profiles with increasing data and user demands, monitoring tools
ResourcePercentage
Storage100%
Network100%
Computing100%
  • Gradually increase data volumes and user concurrency to test system capacity.
  • Monitor performance metrics for resource usage, data transfer, and response times. Identify potential bottlenecks and areas for improvement.
Wrap-up and ReportingFinalize test results and prepare reportsSeptember 1September 30Consolidated test data, reporting toolsN/A
  • Consolidate results from all previous campaigns. Identify overall system performance and potential areas for improvement.
  • Generate reports summarizing test results, findings, and recommendations.

Data Lifecycle Test campaigns

CampaignFocusStart DateEnd DateSoftware/Data ResourcesInfrastructure ResourcesActivities
Ingestion and DisseminationData ingestion, replication, and transferApril 1April 30Precursor/synthetic data, ingestion service, data transfer tools, monitoring dashboards
ResourcePercentage
Storage50%
Network50%
Computing10%
  • Ingest precursor and synthetic test data (Karabo) via the ingestion service.
  • Replicate data to a designated set of sites.
  • Monitor data transfer performance through dashboards.
Integration, User Workflows, and Data LifecycleIntegration, workflows, data lifecycleJune 15June 30Workloads repository, data for lifecycle testing, Jupyter Notebooks, visualization tools
ResourcePercentage
Storage50%
Network50%
Computing75%
  • Run pre-defined workflows from a common repository using public precursor or synthetic data.
  • Document workflow execution details (local environments, resources, sites involved).
  • Test interactive workflows involving visualization tools like VisIVO or CARTA.
  • Conduct end-to-end data lifecycle tests with all v0.1 sites participating.
  • Process data using Jupyter Notebooks, and visualise results.
Wrap-up and ReportingFinalize test results and prepare reportsSeptember 1September 30Consolidated test data, reporting toolsN/A
  • Consolidate results from all previous campaigns. Identify overall system performance and potential areas for improvement.
  • Generate reports summarizing test results, findings, and recommendations.





  • No labels