Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add performances comparisons with previous packages #500

Closed
3 tasks
gcroci2 opened this issue Sep 21, 2023 · 4 comments
Closed
3 tasks

Add performances comparisons with previous packages #500

gcroci2 opened this issue Sep 21, 2023 · 4 comments
Labels
disc Discussion needed, to better define the task/s docs Improvements or additions to documentation framework paper stale issue not touched from too much time

Comments

@gcroci2
Copy link
Collaborator

gcroci2 commented Sep 21, 2023

Table implemented in PR #493 shows the timings obtained for generating graphs/graphs+grids, atomic resolution, with all features except for the ones in the conservation module, because we don't have the pssm files for the data in the tutorials (for computing the performances of deeprank2, I used the raw data available at this address).

We need some discussion here:

  1. Is this a satisfying way of showing performances? Do we need to generate all the features possible (by adding conservation module features), and to add performances for residue resolution as well?
  2. How do we do a fair comparison with the previously developed packages? Features are different in number and in how they are calculated, so if we use all features in all packages we can't know if the comparison is fair. Maybe we could just pick a couple of them which are the same in all packages (e.g., distance, residue type)?

When we'll have clearer ideas/plans about 1. and 2., compare deeprank2 with:

  • deeprank
    • PPIs, grid
  • deeprank-gnn
    • PPIs, graph
  • deeprank-mut
    • variants, grid
@gcroci2
Copy link
Collaborator Author

gcroci2 commented Sep 21, 2023

How would you advice to proceed here, especially for question 2.? @sonjageorgievska, @DaniBodor

@gcroci2 gcroci2 added docs Improvements or additions to documentation disc Discussion needed, to better define the task/s framework paper labels Sep 21, 2023
@DaniBodor
Copy link
Collaborator

I don't have a great idea about this, apart from just doing a "not fair" comparison and being open about it and explaining the differences.

@gcroci2
Copy link
Collaborator Author

gcroci2 commented Sep 27, 2023

I don't have a great idea about this, apart from just doing a "not fair" comparison and being open about it and explaining the differences.

Actually I agree, I also think this is the only realistic option we have.

@github-actions
Copy link

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale issue not touched from too much time label Oct 30, 2023
@gcroci2 gcroci2 added SS and removed SS labels Jan 10, 2024
@gcroci2 gcroci2 closed this as not planned Won't fix, can't repro, duplicate, stale Jun 27, 2024
@gcroci2 gcroci2 moved this to Done in Development Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
disc Discussion needed, to better define the task/s docs Improvements or additions to documentation framework paper stale issue not touched from too much time
Projects
Status: Done
Development

No branches or pull requests

2 participants