Create a llm from the ground up

[nuotianqing]

Select Topic Area

Question

Body

Where should I start if I want to build a large language model from the bottom up?

[yamilobt8]

To build a large language model, start by understanding NLP fundamentals, followed by machine learning and deep learning concepts using frameworks like TensorFlow or PyTorch. Learn about transformer architectures (e.g., GPT), and practice coding simple models before diving into datasets, preprocessing, and large-scale model training.
Resources:

Building LLMs from the Ground Up: A 3-hour Coding Workshop
https://www.youtube.com/watch?v=quh7z1q7-uc

Large Language Models from Scratch
https://www.youtube.com/watch?v=lnA9DMvHtfI

Create a Large Language Model from Scratch with Python – Tutorial
https://www.youtube.com/watch?v=UU1WVnMk4E8

Introduction to Large Language Models
https://www.youtube.com/watch?v=RBzXsQHjptQ

Introduction to Large Language Models (LLMs) - Playlist
https://www.youtube.com/playlist?list=PLDqi6CuDzubwpag1cdbcvaycJneOf96Lg

[mecodeatlas]

Thanks for posting in the GitHub Community, @nuotianqing!

We're happy you're here. You are more likely to get a useful response if you are posting your question in the applicable category, the Discussions category is solely related to conversations around the GitHub product Discussions. This question should be in the Programming Help category. I've gone ahead and moved it for you. Good luck!