(Exit code 133) Error when loading large LLM models

[yushijinhun]

When loading large LLMs (for example, Meta-Llama-3.1-70B-Instruct-IQ2_S with context window 32768), I would encounter the error (Exit code: 133). Please check settings and try loading the model again.. My machine has 64G RAM and 16G vRAM, and I can load the model with same configuration with llama.cpp. Therefore, the problem should not be caused by insufficient RAM or vRAM.

Further investigation into the coredump shows that, the application crashes in function _ZN15partition_alloc8internal32PartitionExcessiveAllocationSizeEm, which means the application is trying to allocate an excessive amount of memory using PartitionAlloc in a call to posix_memalign. This explains why the problem occurs only on LM Studio but not on llama.cpp, as Electron uses PartitionAlloc by default. This is tracked in issue electron/electron#44291.

Apart from waiting for upstream to fix this issue, a potential workaround is to use a customized Electron build to disable PartitionAlloc.

[yagil]

@yushijinhun which operating system are you on? Assuming Linux?

[yushijinhun]

@yushijinhun which operating system are you on? Assuming Linux?

Yes. I'm on Linux.

[yagil]

@yushijinhun which operating system are you on? Assuming Linux?

Yes. I'm on Linux.

Thanks. @neilmehta24 from our team is investigating this with priority as we speak.

[yushijinhun]

I built Electron v33.0.2 with use_allocator_shim = false and enable_backup_ref_ptr_support = false. After replacing LM Studio's electron with the one I built, the problem is solved.

[krones9000]

Could you give a little more specific detail about exactly how you resolved this? I'm in a similar situation, on linux, 16gbVRAM 128GB system RAM, could load models in Oobabooga but getting the 133 error in LM studio. Not sure where to start or how I would "build" Electron or what Electron is and how it relates to the Linux app image of LM Studio that I load.

[laushunyu]

I run LM Studio v0.3.6 on Ubuntu 24.04, with 7840HS CPU, 96GB RAM and 16GB GPU VRAM, encountered the same issue: small models can be loaded normally but large models cannot be loaded.

I would appreciate it if @yushijinhun share your compiled Electron version.

[yushijinhun]

Not sure where to start or how I would "build" Electron or what Electron is and how it relates to the Linux app image of LM Studio that I load.

Electron is a UI framework built on Chromium used by LM Studio.

I would appreciate it if @yushijinhun share your compiled Electron version.

Here is my Electron v33.0.2 Linux x86_64 build (without allocator shim): Google Drive

You can also follow Electron's Build Instructions to build it yourself.

To replace LM Studio' stock Electron, first unpack the AppImage:

/path/to/LM-Studio-0.3.6-8-x64.AppImage --appimage-extract

Then you would see a squashfs-root directory. Go into it, extract the Electron distribution zip, and overwrite all existent files. Run the electron executable (you may need to add --no-sandbox flag), and you will see LM Studio starting.

[daxime]

@yushijinhun Thank you for that :)

[CHesketh76]

Why is this not set to default? LMstudio exclusively uses GGUF files which implies most users are limited on Vram.

[yagil]

This is a new bug in 0.3.6 because we updated our Electron version.
Our recommendation: consider staying on 0.3.5 until we fix it properly in a new release (it's in the works)

[CHesketh76]

Same issue when downgrading.

[aamir-gmail]

I am using linux version of LM studio 0 3.6 , with Ubuntu 22.04 LTS , I have two 3090 card and 256 GB RAM and AMD 24 Core CPU , for some reason the Qwen2 VL model not load 7B or 70B I get the following message (Exit code: 133). Please check settings and try loading the model again. I was able to load 7B version of the same model on the windows machine (0.3.6) with 64 GB RAM and 12 GB 4070 card , here neither 72B or 4B does not load my NVidia driver is 550, I thought it was a memory issues , them I tried LLAMA 70B , which loaded just fine , the GPU utilisation was around 15 GB each . Are there any logs collected by LM studion which I can share to help with this problem.

[yagil]

Thanks @aamir-gmail! We are aware of this issue and we are working on a fix. The recommendation is to stay on 0.3.5 until it's out. Get 0.3.5 from https://lmstudio.ai/download#beta-and-experimental-releases

[aamir-gmail]

FYI, I was able to load LLAMA 70B with 0.3.6 without a problem. I will keep you posted on how go with 0.3.5

…

On Mon, Jan 13, 2025 at 12:37 PM Yagil Burowski ***@***.***> wrote: Thanks @aamir-gmail <https://github.com/aamir-gmail>! We are aware of this issue and we are working on a fix. The recommendation is to stay on 0.3.5 until it's out. Get 0.3.5 from https://lmstudio.ai/download#beta-and-experimental-releases — Reply to this email directly, view it on GitHub <#285 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJA2ECIRZUWL2M6CV27FQXT2KMKG7AVCNFSM6AAAAABUWBROFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBWGAZTAMZSG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Kind Regards Aamir Mirza

[aamir-gmail]

Tried your link for Qwen2 VL 7B and 70B , on the download link you provided still got the same error message. Are there any logs I can send you. Let me know where to find them if you require them. (Exit code: 133). Please check the settings and try loading the model again.

…

On Mon, Jan 13, 2025 at 4:24 PM Aamir mirza ***@***.***> wrote: FYI, I was able to load LLAMA 70B with 0.3.6 without a problem. I will keep you posted on how go with 0.3.5 On Mon, Jan 13, 2025 at 12:37 PM Yagil Burowski ***@***.***> wrote: > Thanks @aamir-gmail <https://github.com/aamir-gmail>! We are aware of > this issue and we are working on a fix. The recommendation is to stay on > 0.3.5 until it's out. Get 0.3.5 from > https://lmstudio.ai/download#beta-and-experimental-releases > > — > Reply to this email directly, view it on GitHub > <#285 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AJA2ECIRZUWL2M6CV27FQXT2KMKG7AVCNFSM6AAAAABUWBROFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKOBWGAZTAMZSG4> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > -- Kind Regards Aamir Mirza

-- Kind Regards Aamir Mirza