RedHatAI
/

Mistral-Small-24B-Instruct-2501-quantized.w8a8

Text Generation

text-generation-inference

compressed-tensors

8-bit precision

Model card Files Files and versions

alexmarques commited on Oct 29, 2025

Commit

3ef5abe

·

verified ·

1 Parent(s): 800fcca

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -100,7 +100,7 @@ client = OpenAI(
     base_url=openai_api_base,
 )
-model = "RedHatAI/Mistral-Small-24B-Instruct-2501-quantized.w4a16"
 messages = [

     base_url=openai_api_base,
 )
+model = "RedHatAI/Mistral-Small-24B-Instruct-2501-quantized.w8a8"
 messages = [