Hugging Face's logo

giladgd
/
Apertus-70B-Instruct-2509-GGUF

Text Generation
GGUF
node-llama-cpp
llama.cpp
apertus
multilingual
swiss-ai
compliant
conversational
Model card Files
xet
Community
giladgd commited on
Commit
43a53c7
·
verified ·
1 Parent(s): b0fec72

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -5
README.md CHANGED
@@ -54,7 +54,6 @@ Static quants of [`swiss-ai/Apertus-70B-Instruct-2509`](https://huggingface.co/s
54
  | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q5_K_M.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q5_K_M` | Q5_K_M | 50.6GB |
55
  | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q6_K.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q6_K` | Q6_K | 57.9GB |
56
  | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q8_0.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q8_0` | Q8_0 | 75.0GB |
57
- | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.MXFP4.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:MXFP4` | MXFP4 | 75.0GB |
58
  | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.F16.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:F16` | F16 | 141.2GB |
59
 
60
  > [!TIP]
@@ -74,7 +73,7 @@ brew install nodejs
74
  ### CLI
75
  Chat with the model:
76
  ```bash
77
- npx -y node-llama-cpp chat hf:giladgd/Apertus-70B-Instruct-2509-GGUF:MXFP4
78
  ```
79
 
80
  ### Code
@@ -86,7 +85,7 @@ npm install node-llama-cpp
86
  ```typescript
87
  import {getLlama, resolveModelFile, LlamaChatSession} from "node-llama-cpp";
88
 
89
- const modelUri = "hf:giladgd/Apertus-70B-Instruct-2509-GGUF:MXFP4";
90
 
91
 
92
  const llama = await getLlama();
@@ -119,10 +118,10 @@ brew install llama.cpp
119
 
120
  ### CLI
121
  ```bash
122
- llama-cli -hf giladgd/Apertus-70B-Instruct-2509-GGUF:MXFP4 -p "The meaning to life and the universe is"
123
  ```
124
 
125
  ### Server
126
  ```bash
127
- llama-server -hf giladgd/Apertus-70B-Instruct-2509-GGUF:MXFP4 -c 2048
128
  ```
 
54
  | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q5_K_M.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q5_K_M` | Q5_K_M | 50.6GB |
55
  | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q6_K.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q6_K` | Q6_K | 57.9GB |
56
  | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.Q8_0.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q8_0` | Q8_0 | 75.0GB |
 
57
  | [GGUF](https://huggingface.co/giladgd/Apertus-70B-Instruct-2509-GGUF/resolve/main/Apertus-70B-Instruct-2509.F16.gguf) | `hf:giladgd/Apertus-70B-Instruct-2509-GGUF:F16` | F16 | 141.2GB |
58
 
59
  > [!TIP]
 
73
  ### CLI
74
  Chat with the model:
75
  ```bash
76
+ npx -y node-llama-cpp chat hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M
77
  ```
78
 
79
  ### Code
 
85
  ```typescript
86
  import {getLlama, resolveModelFile, LlamaChatSession} from "node-llama-cpp";
87
 
88
+ const modelUri = "hf:giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M";
89
 
90
 
91
  const llama = await getLlama();
 
118
 
119
  ### CLI
120
  ```bash
121
+ llama-cli -hf giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M -p "The meaning to life and the universe is"
122
  ```
123
 
124
  ### Server
125
  ```bash
126
+ llama-server -hf giladgd/Apertus-70B-Instruct-2509-GGUF:Q4_K_M -c 2048
127
  ```