Thank you for attending Back to Basics: Build Your Own LLM from Scratch

I sent this email to all workshop attendees. It made sense to publish it as a post as well.

Thank you for attending the “Back to Basics: Build Your Own LLM from Scratch” session. It was great to see so much curiosity and many questions in the room, the feedback many of you shared echoed this sentiment. I’m really glad you found it useful. 

As we discussed during the session, I have put together a detailed blog post that walks through everything we covered, including the slides. You can also find the code from the sessions on GitHub and Codeberg.

1. Blog with slides and code: https://thejeshgn.com/2026/06/14/back-to-basics-build-your-own-llm-from-scratch/
2. GitHub: https://github.com/thejeshgn/workshop-back-basics-llm-build-scratch
3. Codeberg: https://codeberg.org/thejeshgn/workshop-back-basics-llm-build-scratch

I would also like all of you to continue building on what you learned. You could do that by just forking the code above and adding more features. Some ideas to improve are

1. Swap the simple character tokenizer we used for a word-level tokenizer, or go further and implement a BERT-style WordPiece/subword tokenizer, then compare vocabulary size and how each handles unseen words. Also, what effect does it have on the model?
2. Train on a relatively larger corpus rather than the very small sample we used in the workshop. Project Gutenberg’s Top 100 is a great source of popular, public-domain books that are not very large.
3. You can also use thematic corpora, such as public-domain poetry collections, recipe collections, or the collected works of Gandhi or Ambedkar, and see how the model’s generated text picks up the style and vocabulary of that specific body of work. For some of it, you will also have to write data cleaners. Also, be mindful when downloading external content; make sure you don’t overload their servers and use public domain content. Check whether they already provide downloads in BitTorrent or in other formats, instead of scraping first.
4. Experiment with model parameters (in GPTConfig) such as context length, heads, and layers once your pipeline works end to end. See how changes in parameters change the model and its output.

If you build something interesting, feel free to reach out. I’d love to hear what you create. Thanks again for being part of the session.

---