Which model loader to choose to upload several articles ?

[lofti198]

Just installed Oobabooga. Searching for tutorials for beginners. My task is to upload big bunch of text (several big articles) to ask against their content. Which Model loader to choose for those purpose? Also need to consider, that I use CPU.

[SlowLogicBoy]

When I use CPU, I use 7B model in GGUF format Q4.K_M quant and llama.cpp loader (I think this is the only good option for CPU?).
Current 7B models I use interchangeably:

• OpenHermes 2.5 mistral
• NeuralHermes 2.5 mistral (I don't see much difference between this and OpenHermes)
• OpenChat 3.5 0106

[Alkohole]

All models are downloaded from huggingface (first link of your search), you can either download the model yourself from the huggingface or in webui under the "Model" tab in the download section. If you're looking for documentation, you can go to the wiki tab in this github repository.

[lofti198]

Stuck on the next step with the wiki.

1. Downloaded model by link https://huggingface.co/TheBloke/Mistral-7B-v0.1-GGUF
2. Clicked get file list , copied one of files ( mistral-7b-v0.1.Q2_K.gguf ) to the corresponding field
   
3. What is the next step? Clicking again "Download" brings nothing new
   (Trying to follow this instruction https://github.com/oobabooga/text-generation-webui/wiki/04-%E2%80%90-Model-Tab#model-dropdown )

Thank you in advance!

[Alkohole]

Use a shortened link like TheBloke/Mistral-7B-v0.1-GGUF, after selecting the model file, click Download (all subsequent downloads will take place in the terminal).

[lofti198]

emperically I found that I should had to relaunch the webui to see the downloaded model in the dropbox.

[Alkohole]

just use the update button "🔄", it's opposite the model selection.

[lofti198]

All models are downloaded from huggingface (first link of your search), you can either download the model yourself from the huggingface or in webui under the "Model" tab in the download section. If you're looking for documentation, you can go to the wiki tab in this github repository.

Thanks a lot for clear explanation!!!