Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create KB article (Best Practices for Optimizing Longhorn Disk Performance) #51

Merged

Conversation

jillian-maroket
Copy link
Contributor

This KB article addresses a customer request for best practice recommendations regarding Harvester configuration with disk performance in mind.

Jira: SURE-6729
GitHub issue: harvester/harvester#3356

David Ko and the Longhorn engineering team provided most of the content. I added supporting details from the Longhorn documentation.

Copy link

netlify bot commented Dec 27, 2023

Deploy Preview for harvester-home-preview ready!

Name Link
🔨 Latest commit 1f04d45
🔍 Latest deploy log https://app.netlify.com/sites/harvester-home-preview/deploys/659e4121c10fa5000825bfeb
😎 Deploy Preview https://deploy-preview-51--harvester-home-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link
Member

@bk201 bk201 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's in Harvester KB, I suggest adding Harvester doc links in the KB if possible (we can always add Longhorn doc as a reference too).

kb/2023-12-27/best_practices_disk_performance.md Outdated Show resolved Hide resolved
kb/2023-12-27/best_practices_disk_performance.md Outdated Show resolved Hide resolved
kb/2023-12-27/best_practices_disk_performance.md Outdated Show resolved Hide resolved

For data-intensive applications, you can use pod scheduling functions such as node selector or taint toleration. These functions allow you to schedule the workload to a specific storage-tagged node together with one replica.

- **Revision counter**: You can disable the [revision counter](https://longhorn.io/docs/1.5.3/advanced-resources/deploy/revision_counter/) to improve IO performance, especially for write-intensive applications. When the revision counter is disabled, Longhorn does not track write operations for replicas and the Longhorn Engine does not check the `revision.counter` file after restarting.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@innobead Do we want to recommend this for production environments?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@innobead Can you comment on this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this from this practice, as the focus here should be on actions that are 'recommended' be taken.

However, this depends on a very strict requirement already mentioned in the 'important information' below, and it could be challenging to achieve in a prod setup. Additionally, it's not a recommendation but rather an optional method to enhance performance.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed this item.

kb/2023-12-27/best_practices_disk_performance.md Outdated Show resolved Hide resolved

- **Recurring snapshots**: Periodically clean up system-generated snapshots and retain only the number of snapshots that makes sense for your implementation. For applications with replication capability, periodically [delete all types of snapshots](https://longhorn.io/docs/1.5.3/concepts/#243-deleting-snapshots).

- **Recurring filesystem trim**: Periodically [trim the filesystem](https://longhorn.io/docs/1.5.3/volumes-and-nodes/trim-filesystem/) inside volumes to reclaim disk space.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@innobead I suggest we remove this. Most harvester VMs use block mode volume, I'm not sure if the trim operation works here or not. (Probably doing a trim inside VM?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@innobead Can you comment on this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, let's remove this as it only works for the fs mount point (volume mode: fs).

Longhorn volume supports unmap, which should continue functioning if users perform trim on the VM fs. However, this should not be recommended, as there is a known issue at harvester/harvester#4739. Additionally, the user experience is not good.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have removed this item.

Copy link
Collaborator

@LucasSaintarbor LucasSaintarbor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Copy link
Contributor

@innobead innobead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, LGTM. After resolve the comments, we can merge this.

However, we continue improving this doc over time, since there will be some new practices to be introduced later. For example, snapshot space management, default priority class, etc.

@jillian-maroket We should have one in Longhorn doc as well, but it should be in a dedicated in https://longhorn.io/docs/1.6.0/best-practices/. Please help create a ticket for that and plan it.

@jillian-maroket jillian-maroket merged commit cb6902f into harvester:main Jan 10, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants