Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Farmer plot cache #2550

Merged
merged 3 commits into from
Feb 26, 2024
Merged

Farmer plot cache #2550

merged 3 commits into from
Feb 26, 2024

Conversation

nazar-pc
Copy link
Member

@nazar-pc nazar-pc commented Feb 23, 2024

Previously farmer only had piece cache as it supposed to for DSN, however, as described in #2399 we have a temporary additional space that we can exploit to cache even more pieces, helping both with plotting performance and with retrieval from DSN.

First two commits here are doing some cleanup/refactoring, the last commit implements plot cache. Since logic of plot cache contained in FarmerCache is quite trivial, no new FarmerCache-specific tests were added at this time, but DiskPlotCache itself has corresponding tests added to make sure lower-level component behaves as expected.

I also tested manually locally with dev chain and will do extra tests with Gemini 3h farmer client before merging for extra confidence.
UPD: Tested, seems to work fine on 3h as well.

Resolves #2399

Code contributor checklist:

@nazar-pc
Copy link
Member Author

nazar-pc commented Feb 26, 2024

This causes problems on Windows, see al8n/fs4-rs#10

If I can't resolve that, I'll update PR to not cache pieces into plot file on Windows temporarily.

UPD: Seems to be related to VM environment (likely virtio).

@nazar-pc nazar-pc enabled auto-merge February 26, 2024 02:39
}

#[derive(Debug)]
pub(crate) enum MaybePieceStoredResult {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ProbabilisticStoredPieceResult ??

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree the name is a bit awkward. It is not necessarily probabilistic at the moment it is returned, but since there is no lock held throughout, things might change by the time this information is acted upon. Not sure ProbabilisticStoredPieceResult is much better option, but definitely looking for better alternatives.

@nazar-pc nazar-pc added this pull request to the merge queue Feb 26, 2024
Merged via the queue into main with commit 4c8b66d Feb 26, 2024
24 checks passed
@nazar-pc nazar-pc deleted the farmer-plot-cache branch February 26, 2024 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exploit unplotted plot space as extra caching space
2 participants