Added Aurora DeepSpeed content #581
Open
+154
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
felker
requested changes
Jan 6, 2025
|
|
||
| The base `frameworks` environment on Aurora does not come with Microsoft's | ||
| [DeepSpeed](https://github.com/microsoft/DeepSpeed) pre-installed and it needs to be installed by the user. Instructions | ||
| for using / cloning the base environment can be found [here](../python.md). |
There was a problem hiding this comment.
Suggested change
| for using / cloning the base environment can be found [here](../python.md). | |
| for using and cloning the base environment can be found [here](../python.md). |
|
|
||
| !!! example "Launching DeepSpeed" | ||
| In both examples the 'train_batch_size' variable needs to be modified from 16 to 12 in the deepspeed | ||
| config embedded in the python file cifar10_deepspeed.py. This is because the default of 16 is not |
There was a problem hiding this comment.
Suggested change
| config embedded in the python file cifar10_deepspeed.py. This is because the default of 16 is not | |
| config embedded in the Python file `cifar10_deepspeed.py`. This is because the default of 16 is not |
| ``` | ||
|
|
||
| !!! example "Launching DeepSpeed" | ||
| In both examples the 'train_batch_size' variable needs to be modified from 16 to 12 in the deepspeed |
There was a problem hiding this comment.
Suggested change
| In both examples the 'train_batch_size' variable needs to be modified from 16 to 12 in the deepspeed | |
| In both examples the 'train_batch_size' variable needs to be modified from 16 to 12 in the DeepSpeed |
| sed -e 's/$/ slots=12/' -i hostfile | ||
| ``` | ||
|
|
||
| 2. Create a `#!bash .deepspeed_env` containing the environment |
There was a problem hiding this comment.
I had a conversation with @saforem2 about the use of syntax highlighting for inline code markup like you use here with #!bash ...
We concluded that it isnt really worth the marginal gains for such short pieces of text, since it doesnt render in GitHub Flavored Markdown and has different escaping conventions.
Kinda degrades the readability when you are editing someone else's source text. Have you found it to be helpful?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
First draft for DeepSpeed on Aurora. Modified from Polaris instructions and tested to work.