Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use mimalloc allocator on all platforms for node and farmer #2105

Merged
merged 1 commit into from
Oct 13, 2023
Merged

Conversation

nazar-pc
Copy link
Member

I noticed that without jemalloc farmer is using ridiculous amounts of memory on Linux. It looks like it leaks memory, but this is just system allocator behaving badly, when application stops it goes release the expected memory just fine.

I thought about switching to jemalloc again, but the crate is not working on Windows and https://github.com/microsoft/mimalloc while working there should also be one of the highest performing allocators out there. Let's use it on both node and farmer.

I was running farmer over night with milalloc and it plotted ~100 sectors with current memory usage of ~2G (goes up and down depending on what app is doing), with system allocator farmer was using over 10G of RAM after plotting just a few sectors.

Code contributor checklist:

Copy link
Contributor

@shamil-gadelshin shamil-gadelshin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you consider using jemalloc on linux and mimalloc on windows?

@nazar-pc
Copy link
Member Author

I did, but I'd expect mimalloc to have superior performance on all platforms, so decided to go with one for all platforms instead.

@shamil-gadelshin
Copy link
Contributor

I did, but I'd expect mimalloc to have superior performance on all platforms, so decided to go with one for all platforms instead.

Indeed: https://github.com/microsoft/mimalloc#Performance

@nazar-pc nazar-pc added this pull request to the merge queue Oct 13, 2023
Merged via the queue into main with commit 014ee55 Oct 13, 2023
10 checks passed
@nazar-pc nazar-pc deleted the mimalloc branch October 13, 2023 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants