Upload README.md with huggingface_hub

README.md

@@ -1,42 +1,73 @@
 ---
-base_model: []
-library_name: transformers
+language:
+- en
+license: apache-2.0
 tags:
-- mergekit
 - merge
-
+- mergekit
+- slerp
+- qwen2.5
+- deepseek-r1
+- reasoning
+base_model:
+- Qwen/Qwen2.5-1.5B-Instruct
+- deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
 ---
+
 # Qwen2.5-1.5B-R1-SLERP
 
-This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
+A SLERP merge (t=0.5) of:
+- [`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) — strong general instruction following
+- [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) — RL-distilled chain-of-thought reasoning
 
-## Merge Details
-### Merge Method
+Part of a systematic merge study on the Qwen2.5-1.5B family. See also:
+- [`Mohaaxa/Qwen2.5-1.5B-R1-SLERP-AWQ`](https://huggingface.co/Mohaaxa/Qwen2.5-1.5B-R1-SLERP-AWQ) — AWQ 4-bit quantized version
 
-This model was merged using the [SLERP](https://en.wikipedia.org/wiki/Slerp) merge method.
+## Benchmarks
 
-### Models Merged
+Evaluated against both parent models on PPL (Wikitext-2) and GSM8K (100 samples):
 
-The following models were included in the merge:
-* /models/qwen2.5-1.5b-instruct
-* /models/deepseek-r1-distill-qwen-1.5b
+| Model | PPL | GSM8K |
+|-------|-----|-------|
+| Qwen2.5-1.5B-Instruct (parent) | 16.141 | 38.0% |
+| DeepSeek-R1-Distill-Qwen-1.5B (parent) | 107.467 | 3.0% |
+| **Qwen2.5-1.5B-R1-SLERP (this model)** | 1205.427 | 2.0% |
 
-### Configuration
+PPL delta vs Instruct parent: +1189.286
+GSM8K delta vs Instruct parent: -36.0%
 
-The following YAML configuration was used to produce this model:
+## Merge Config
 
 ```yaml
-
 merge_method: slerp
 base_model:
-  model: /models/qwen2.5-1.5b-instruct
+  model: Qwen/Qwen2.5-1.5B-Instruct
 slices:
   - sources:
-      - model: /models/qwen2.5-1.5b-instruct
+      - model: Qwen/Qwen2.5-1.5B-Instruct
         layer_range: [0, 28]
-      - model: /models/deepseek-r1-distill-qwen-1.5b
+      - model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
         layer_range: [0, 28]
     parameters:
-      t: 0.3
+      t: 0.5
+```
+
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
 
+model = AutoModelForCausalLM.from_pretrained(
+    "Mohaaxa/Qwen2.5-1.5B-R1-SLERP",
+    torch_dtype="auto",
+    device_map="auto",
+)
+tokenizer = AutoTokenizer.from_pretrained("Mohaaxa/Qwen2.5-1.5B-R1-SLERP")
 ```
+
+## Notes
+
+- t=0.5 gives equal weight to both parents
+- SLERP preserves weight magnitude better than linear interpolation
+- Both parents share identical Qwen2.5 architecture (28 layers, hidden_dim=1536)
+- For a quantized version with ~67% VRAM reduction, use the AWQ variant