How to load the model?

pinned

by Blithe8002 - opened Sep 11

Sep 11

•

I want to use the 4bit quantized model to run an OCR related task on an image. Can you please tell me how to do that? I tried llama.cpp but I was not able to make it work.

csabakecskemeti pinned discussion Sep 15

csabakecskemeti

DevQuasar org Sep 17

•

edited Sep 17

Thanks for raising the issue
The vision adapter has missed the 'mmproj' prefix so that could cause issue, that like the llama.cpp server were not automatically recognizing the vision adapter as they rely on filename patterns.
Filname fixed
Easy way to use is with LMStudio, but it should work similarly with llama.cpp

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment