Skip to content

Commit

Permalink
Update README.md to be more detailed
Browse files Browse the repository at this point in the history
  • Loading branch information
@jshuadvd
jshuadvd committed Jul 20, 2024
1 parent 54d8b56 commit 9a13452
Showing 1 changed file with 0 additions and 5 deletions.
5 changes: 0 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -208,11 +208,6 @@ LongRoPE's performance can be sensitive to hyperparameters. Key parameters to tu
Learning rate and scheduler parameters for fine-tuning
gradient_accumulation_steps for training stability

- **Base Length**: The initial context length used for fine-tuning.
- **Target Length**: The final context length to extend to.
- **Population Size**: The number of candidate solutions to search for during the search algorithm.
- **Number of Mutations**: The number of mutations to perform during the search algorithm.

## Results

My implementation of LongRoPE achieves the following results:
Expand Down

0 comments on commit 9a13452

Please sign in to comment.