VV
Public post
Ved Vekhande
Sep 2026 • linkedin
Post excerpt
This Github repo teaches you to train a 64M parameter LLM from scratch. Pure PyTorch. Every line of code explained. What the training pipeline covers: Tokenizer → Custom 6,400 token vocabulary → Built entirely from scratch Pretraining → Next-token prediction → Decoder-only Transformer → RMSNorm, SwiGLU, RoPE built in Supervised Fine-Tuning (SFT) → Instruction tuning → Multi-turn dialogue training → Tool-use data included Alignment and RL → LoRA : parameter-efficient fine-tuning → DPO : preference optimization → PP…
Find the people and context behind this post
Sign up for Super Carl to explore relationship context, warm paths, and relevant opportunities.