Spaces:

rphrp1985
/

zerogpu

Running on Zero

rphrp1985 commited on Jan 30

Commit

1b0ef2e

verified ·

1 Parent(s): 08206a7

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -135,7 +135,8 @@ model = AutoModelForCausalLM.from_pretrained(
     MODEL_ID,
     device_map="auto",
      low_cpu_mem_usage=True,
-    quantization_config=quantization_config,
     attn_implementation="flash_attention_2",
 )

     MODEL_ID,
     device_map="auto",
      low_cpu_mem_usage=True,
+    torch_dtype=torch.bfloat16,
+    # quantization_config=quantization_config,
     attn_implementation="flash_attention_2",
 )