From 7ab4edd59257107bc3b53bc1c9fa424edbabec8a Mon Sep 17 00:00:00 2001 From: Joshua David Date: Wed, 17 Jul 2024 22:41:11 -0700 Subject: [PATCH] Update README.md to be more detailed --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d67cdea..add9fd4 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,7 @@ Key components include: ``` 3. Progressive Extension Strategy: + ```python def progressive_extension(model, data, base_length, target_length, population_size, num_mutations, num_crossovers, max_iterations): # Extend to 128k @@ -155,7 +156,6 @@ The architecture incorporates several structural modifications to handle the inc ### Performance and Applications - Experiments demonstrate that LongRoPE maintains low perplexity across evaluation lengths from 4k to 2048k tokens and achieves high accuracy in tasks requiring long contexts. This makes it suitable for various applications, including in-context learning, long document summarization, and few-shot learning.