Skip to content

Commit

Permalink
Update README.md to be more detailed
Browse files Browse the repository at this point in the history
  • Loading branch information
jshuadvd committed Jul 18, 2024
1 parent 5353731 commit 7ab4edd
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ Key components include:
```

3. Progressive Extension Strategy:

```python
def progressive_extension(model, data, base_length, target_length, population_size, num_mutations, num_crossovers, max_iterations):
# Extend to 128k
Expand Down Expand Up @@ -155,7 +156,6 @@ The architecture incorporates several structural modifications to handle the inc

### Performance and Applications


Experiments demonstrate that LongRoPE maintains low perplexity across evaluation lengths from 4k to 2048k tokens and achieves high accuracy in tasks requiring long contexts. This makes it suitable for various applications, including in-context learning, long document summarization, and few-shot learning.


Expand Down

0 comments on commit 7ab4edd

Please sign in to comment.