WrestlingMamba
First successful integration of LoRA fine-tuning with the Mamba architecture.
Python PyTorch HuggingFace Transformers
Pioneered (to knowledge) the first successful integration of LoRA fine-tuning with the Mamba architecture, establishing initial benchmarks for fine-tuning stability against leading LLMs (Llama-2, Gemma, Pythia, and Qwen) as part of a 3-person team.
Enhanced open-source infrastructure by implementing quantization support, enabling reproducible fine-tuning of state-space sequence models.