Upload folder using huggingface_hub

README.md

@@ -0,0 +1,30 @@
+# DeepSeek-3B-MoE-Decoder
+
+This is the decoder component of DeepSeek-OCR, a 3B parameter Mixture-of-Experts (MoE) language model.
+
+## Architecture
+
+- **Model**: DeepSeek 3B MoE
+- **Active Parameters**: ~570M per token
+- **Total Parameters**: ~3B
+- **Architecture**: Mixture-of-Experts with routing
+
+## Usage
+
+This decoder should be used with vision embeddings from the encoder component.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Load decoder
+model = AutoModelForCausalLM.from_pretrained("junkim100/DeepSeek-3B-MoE-decoder")
+tokenizer = AutoTokenizer.from_pretrained("junkim100/DeepSeek-3B-MoE-decoder")
+
+# Use with vision embeddings from encoder
+# vision_embeddings = ... (from DeepEncoder)
+# outputs = model(inputs_embeds=vision_embeddings, ...)
+```
+
+## Source
+
+Extracted from [deepseek-ai/DeepSeek-OCR](https://huggingface.co/deepseek-ai/DeepSeek-OCR)

config.json

@@ -0,0 +1,5 @@
+{
+  "architectures": [
+    "DeepseekOCRForCausalLM"
+  ]
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f85521f2e6c344c36ffa997e4d55b889dcb59284a22ecf0748cc5e32ac283e8e
+size 5869729208