🧑‍💻 Qwen2.5-Coder-1.5B — Fine-tuned for General Code Generation

A LoRA fine-tune of Qwen2.5-Coder-1.5B-Instruct, trained on a free Colab T4 GPU using Unsloth for fast, low-memory training.

Model on Hugging Face: https://huggingface.co/div1010/qwen2.5-coder-1.5b-finetuned

🧰 Stack

Layer	Tool	Role
🧩 Base model	Qwen2.5-Coder-1.5B-Instruct	Starting checkpoint
⚡ Training	Unsloth	Fast, low-memory LoRA/QLoRA training
🎛️ Fine-tuning method	LoRA (QLoRA, 4-bit)	Rank 16, targeting attention + MLP projections
🧠 Framework	PEFT	LoRA adapter management
🖥️ Hardware	1x Colab T4 (16GB)	Free-tier GPU
📚 Dataset	ise-uiuc/Magicoder-Evol-Instruct-110K	25K-example subset, Apache 2.0
🤗 Hosting	Hugging Face Hub	Model card + weights

🎯 Goal

Improve general-purpose, multi-language code generation on top of the base Qwen2.5-Coder-1.5B-Instruct model.

⚙️ Setup

Base model	Qwen2.5-Coder-1.5B-Instruct
Method	LoRA (QLoRA, 4-bit) via Unsloth
LoRA rank	16
Target modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Hardware	1x Colab T4 (16GB)
Dataset	ise-uiuc/Magicoder-Evol-Instruct-110K (25K-example subset)
Epochs	1
Batch size	2 (effective 8 with gradient accumulation)
Learning rate	2e-4

📚 Why this dataset

Magicoder-Evol-Instruct-110K is a general, multi-language code-instruction dataset built from "evolved" (progressively more complex) instructions, rather than flat single-difficulty examples. It's Apache 2.0 licensed and widely used for training code models.

📊 Results

[Fill in: before/after example outputs, or a benchmark comparison such as HumanEval pass@1 for base vs. fine-tuned model. Even 2-3 side-by-side prompt comparisons make this section useful.]

Example:

Prompt	Base model	Fine-tuned model
"Write a function that..."	...	...

⚠️ Limitations

• Trained on a subset of the dataset for speed; results may improve with the full 110K examples and more epochs
• Not evaluated on a standardized benchmark yet (see Results section above)
• LoRA adapter requires the base model to run unless merged

📄 License

Base model and dataset are both Apache 2.0 — check their respective licenses before commercial use.