llama.cpp error: 'llama_model_loader: failed to load model' on Snapdragon X Elite with Q4_0_4_8 models

[JakoDel]

this happens with everything set to default. ctx 4096, 0 layers offloaded to GPU, 9 threads, FA disabled, etc.

LM Studio 0.3.6

🥲 Failed to load the model

Failed to load model

llama.cpp error: 'llama_model_loader: failed to load model from C:\Users\user\.lmstudio\models\bartowski\QwQ-32B-Preview-GGUF\QwQ-32B-Preview-Q4_0_4_8.gguf
'

edit: this seems to be an issue with all q4_0_4_8 models as this is happening with llama 3.2 3b too. 3b q8 runs painfully slow but fine.

[MovGP0]

I have a Snapdragon X Elite myself running llama.cpp v1.8.0.

Q4_0 and Q4_K_M quantization seems to works fine, while q4_0_4_8 is not working.

Note

Most models from HuggingFace are not working.
Models under staff picks , which are not marked as Likely too large for this machine, seem to work fine.