Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lift methylation meta job to WFMS level #62

Open
vinjana opened this issue Jun 25, 2021 · 0 comments
Open

Lift methylation meta job to WFMS level #62

vinjana opened this issue Jun 25, 2021 · 0 comments

Comments

@vinjana
Copy link
Member

vinjana commented Jun 25, 2021

The methylationcalling meta job starts bisulfite calling processes with bash -- always in pairs of largest (front) and smallest (back) chromosome. This has a number of disadvantages

  • Chromosomes are processed sequentially (in pairs), which increases the throughput of the workflow.
  • The batch processing system's scheduling is not exploited. Instead manual scheduling is used!
  • There is job-management code in Bash (argh)
  • Checkpointing does not work correctly (Why restart both chromosomes, if only one chromosome should be restarted?)

This job should be done by the workflow manager! Move the logic from the meta job into the workflow manager!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant