Spaces:

rphrp1985
/

zerogpu

Running on Zero

rphrp1985 commited on Jan 29

Commit

bc7e35a

verified ·

1 Parent(s): 59f7ef5

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -134,6 +134,7 @@ tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
 model = AutoModelForCausalLM.from_pretrained(
     MODEL_ID,
     device_map="auto",
     quantization_config=quantization_config,
     attn_implementation="flash_attention_2",
 )

 model = AutoModelForCausalLM.from_pretrained(
     MODEL_ID,
     device_map="auto",
+     low_cpu_mem_usage=True,
     quantization_config=quantization_config,
     attn_implementation="flash_attention_2",
 )