modelcollapse/gemma-3-270m-it-cpt-annealing-sft-251126

🧪 Model Behavior Incident Log

Collapse Symptoms After Annealing + SFT (Gemma-3-270M-TW)

This incident report documents multiple unexpected behaviors observed during the following pipeline:

• Gemma-3-270M (base) → CPT → Dialogue-Annealing → SFT
• Training performed with LLaMA-Factory on multiple Traditional Chinese datasets.

The purpose of this log is to document:

• what happened
• why the model collapsed
• representative failure outputs
• suspected root causes
• mitigation steps and future plans

---

📉 Summary of the Incident

After the stage CPT → Phase-3 Dialogue Annealing → SFT, the model displayed severe conversational degradation, including:

• repetitive loops
• meaningless vocalization (e.g., hihihihi...)
• empty or nearly empty responses
• echoing the user query
• hallucinated list expansions
• loss of chat-template consistency
• collapse-like distribution drift

These issues did not appear in earlier checkpoints (e.g., CPT → SFT without annealing).

---

🔍 Representative Failure Outputs

1. Character-level repetition

Prompt:

hi

Output:

hihihihihihihihihihihihihihihi...

---

2. Garbled symbols / noise

Prompt:

你好

Output:

總！！」！」！」！」!”！」！”！”！」!”」!”

---

3. Semantic drift + self-looping

Prompt:

圓的積分公式

Output:

是什麼？
有，他們都是那個。
你們都 是那個。
我 還是在那個裡頭，
所以，沒了，沒有，沒了。。
我也不知道。
我知道。
知道。
知道了。

---

4. Query echoing + hallucinated QA scaffolding

Prompt:

請用一句話解釋地球為什麼會自轉。

Output:

請用一句話解釋地球為什麼會自轉。
- 根據物理學，什麼是「氣體」？
- 什麼是「磁場」？

---

5. Baseline sanity check (pre-annealing) behaves normally

Prompt:

你好阿

Earlier output:

你好！很高興能和你一起交流，我是台灣大語言模型 Formosa-1...

---

🧭 Suspected Root Causes

1. Annealing Dataset Format Drift

Use of raw:

user: ...
assistant: ...

instead of structured JSON messages.

2. Overwriting Instruction Alignment

Annealing is fragile for 270M models → possible distribution collapse.

3. SFT Unable to Recover

Loss decreased, but entropy collapse persisted.

---

🛠 Actions Taken

• Rebuilt CPT dataset (2.4M samples, 70/30 mix)
• Adopted stable LR tail (cosine_with_min_lr)
• Will rebuild annealing data with strict chat format
• Full SFT pipeline will be repeated after clean CPT

---

🔄 Next Steps

1. Complete clean CPT
2. Rebuild dialogue annealing
3. Annealing from min_lr → 0
4. Re-run SFT
5. Compare ablation paths
6. Add regression tests

---

🧭 Closing Note

Small multilingual models are extremely sensitive to formatting + LR scheduling.
This report documents the collapse event for transparency and future reproducibility.