Skip to content

Commit

Permalink
Update awq.md
Browse files Browse the repository at this point in the history
  • Loading branch information
jklj077 authored Dec 4, 2024
1 parent 950bb45 commit a7b5155
Showing 1 changed file with 1 addition and 4 deletions.
5 changes: 1 addition & 4 deletions docs/source/quantization/awq.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,9 @@ print("Chat response:", chat_response)
## Quantize Your Own Model with AutoAWQ

If you want to quantize your own model to AWQ quantized models, we advise you to use AutoAWQ.
It is suggested installing the latest version of the package by installing from source code:

```bash
git clone https://github.com/casper-hansen/AutoAWQ.git
cd AutoAWQ
pip install -e .
pip install "autoawq<0.2.7"
```

Suppose you have finetuned a model based on `Qwen2.5-7B`, which is named `Qwen2.5-7B-finetuned`, with your own dataset, e.g., Alpaca.
Expand Down

0 comments on commit a7b5155

Please sign in to comment.