Price: $59.99 - $55.92
(as of Nov 06, 2024 13:46:35 UTC – Details)
Learn how to create, train, and tweak large language models (LLMs) by building one from the ground up!
In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You’ll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
• Plan and code all the parts of an LLM
• Prepare a dataset suitable for LLM training
• Fine-tune LLMs for text classification and with your own data
• Use human feedback to ensure your LLM follows instructions
• Load pretrained weights into an LLM
Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you’ll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant.
Purchase of the print book includes a free eBook in PDF and ePub formats from Manning Publications.
About the technology
Physicist Richard P. Feynman reportedly said, “I don’t understand anything I can’t build.” Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning.
About the book
Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself!
What’s inside
• Plan and code an LLM comparable to GPT-2
• Load pretrained weights
• Construct a complete training pipeline
• Fine-tune your LLM for text classification
• Develop LLMs that follow human instructions
About the reader
Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs.
About the author
Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software.
The technical editor on this book was David Caswell.
Table of Contents
1 Understanding large language models
2 Working with text data
3 Coding attention mechanisms
4 Implementing a GPT model from scratch to generate text
5 Pretraining on unlabeled data
6 Fine-tuning for classification
7 Fine-tuning to follow instructions
A Introduction to PyTorch
B References and further reading
C Exercise solutions
D Adding bells and whistles to the training loop
E Parameter-efficient fine-tuning with LoRA
From the Publisher
<img alt="Build a Large Language Model (From Scratch) banner A+" src="https://images-na.ssl-images-amazon.com/images/G/01/x-locale/common/grey-pixel.gif" class="a-spacing-base a-lazy-loaded" data-src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/96265afe-fd61-4a12-9f79-36ffc0807f92.__CR0,0,970,600_PT0_SX970_V1___.jpeg"><img alt="Build a Large Language Model (From Scratch) banner A+" src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/96265afe-fd61-4a12-9f79-36ffc0807f92.__CR0,0,970,600_PT0_SX970_V1___.jpeg">
<img alt="Build a Large Language Model (From Scratch) quote 1" src="https://images-na.ssl-images-amazon.com/images/G/01/x-locale/common/grey-pixel.gif" class="a-spacing-mini a-lazy-loaded" data-src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/716f0f2d-d7da-453d-a8b1-5b1059bea02f.__CR0,0,300,300_PT0_SX300_V1___.jpeg"><img alt="Build a Large Language Model (From Scratch) quote 1" src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/716f0f2d-d7da-453d-a8b1-5b1059bea02f.__CR0,0,300,300_PT0_SX300_V1___.jpeg">
<img alt="Build a Large Language Model (From Scratch) quote 2" src="https://images-na.ssl-images-amazon.com/images/G/01/x-locale/common/grey-pixel.gif" class="a-spacing-mini a-lazy-loaded" data-src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/f849a131-0f10-4fa4-b815-96f546425dba.__CR0,0,300,300_PT0_SX300_V1___.jpeg"><img alt="Build a Large Language Model (From Scratch) quote 2" src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/f849a131-0f10-4fa4-b815-96f546425dba.__CR0,0,300,300_PT0_SX300_V1___.jpeg">
<img alt="Build a Large Language Model (From Scratch) quote 3" src="https://images-na.ssl-images-amazon.com/images/G/01/x-locale/common/grey-pixel.gif" class="a-spacing-mini a-lazy-loaded" data-src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/b411c71a-d708-4d00-b577-eecf03677b00.__CR0,0,300,300_PT0_SX300_V1___.jpeg"><img alt="Build a Large Language Model (From Scratch) quote 3" src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/b411c71a-d708-4d00-b577-eecf03677b00.__CR0,0,300,300_PT0_SX300_V1___.jpeg">
<img alt="Build a Large Language Model (From Scratch) includes" src="https://images-na.ssl-images-amazon.com/images/G/01/x-locale/common/grey-pixel.gif" class="a-spacing-base a-lazy-loaded" data-src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/a9d4c1bf-9377-441c-b3c4-6d4c60d0d89f.__CR0,0,970,600_PT0_SX970_V1___.jpeg"><img alt="Build a Large Language Model (From Scratch) includes" src="https://m.media-amazon.com/images/S/aplus-media-library-service-media/a9d4c1bf-9377-441c-b3c4-6d4c60d0d89f.__CR0,0,970,600_PT0_SX970_V1___.jpeg">
<br> Publisher : Manning (October 29, 2024) <br> Language : English <br> Paperback : 368 pages <br> ISBN-10 : 1633437167 <br> ISBN-13 : 978-1633437166 <br> Item Weight : 1.35 pounds <br> Dimensions : 7.38 x 0.7 x 9.25 inches <br>
Discover more from Thoughts & Reality
Subscribe to get the latest posts sent to your email.
eb_canada –
The AI landscape is evolving quickly, and the opportunity to build and deeply understand a language model from the ground up is exciting.I really like the approach of this book (I have read some parts online and some of the code) because these days when big claims and hype are close to LLMs, exploring architectures, training nuances and getting a hands-on deep understanding is very valuable.I highly recommend it!
Jonathan Reeves –
Builds a great foundation
This book is a great resource for anyone that wants an inside glimpse of the black box that is an LLM. Everyone that has ever used something ChatGPT or Google’s Gemini that is curious about how these tools work internally need to get this book. It will teach you about the inner workings of what makes a great LLM and how you can learn to make your own for all use cases.
Malc –
Learn from Scratch
I really enjoyed this book and the idea of from scratch makes a lot of sense. I found a Discord study group and we parsed out weekly segments. Great ot have these discussions. I would say the only caution I have is information may go out of date as this field moves so fast. But it doesn’t really mater since the idea is to get the basic concepts and deep understanding of th technology. I don’t think yu would every build a genAI from scrathc but knowing how the engine works is really a great mental model to have. Especially when explain it to others within the organization.
Samir Bajaj –
Very good, practical book
I subscribe to the author’s newsletter, and I had been waiting for this book to be published since the day it was announced on the mailing list. And it did not disappoint! I prefer to learn by looking inside a system (as opposed to [only] reading about high-level concepts) — so this book and the accompanying code was exactly what I needed.My only suggestion to the author: please include material related to DPO/RLHF in the second edition.
Aymiee Lee –
Your journey starts with one step…
I couldn’t wait for this book to be released on Amazon, so I purchased it directly from Manningâand it did not disappoint! As a programmer developing RAG and copilot agents, I’ve always wanted to dive deeper into how LLMs are built, particularly attention mechanisms. Sebastian Raschka’s Build a Large Language Model (From Scratch) is a comprehensive and well-structured guide that made complex topics understandable. I coded along with each chapter, which helped visualize the concepts and made the learning experience more clear. This book is a must-read for anyone eager to bridge the gap between using LLMs and understanding their architecture. Highly recommended!
Suman Debnath –
Exceptional book absolutely exception.
An exceptional resource for diving into the world of Large Language Models (LLMs)! I picked up this book to deepen my understanding of embeddings, and the material has been invaluable for my presentations at various conferences. Every time I share these concepts, there’s a genuine “Wow” reaction from the audience. The way Sebi distills complex ideas without overwhelming the reader is truly remarkable. His insight into how students learn and think makes this book a standout.If youâre interested in getting started with LLMs and want to peek inside the “black box” of how these models work, I highly recommend reading this book cover to coverâand coding along! Though I havenât yet completed it, each chapter has already enriched my knowledge tremendously.For anyone serious about understanding LLMs, pair this book with Simon’s *Understanding Deep Learning.* Completing these two with the hands-on exercises will give you a solid foundation to consider yourself a knowledgeable, well-rounded ML engineer.
Roberto Pardo –
Manning publisher is cheap
Although the content is very good, the book is in black and white, soft cover, and cost me more than 50 dollars. Springer’s books are good, hard cover. In color, better quality…. Manning is a low-quality printing company.
Bahman –
A Must-Have Guide to Building Large Language Models from Scratch
Build Large Language Models from Scratch by Sebastian Raschka is an exceptional resource for anyone diving into the world of transformer-based LLMs. This book is packed with practical, hands-on guidance using PyTorch and covers everything you need to know, from foundational concepts to building and fine-tuning models.Each chapter unpacks complex topics with clarity. Starting with an introduction to how LLMs work (Chapter 1) and moving through essential steps like text tokenization (Chapter 2) and the attention mechanism (Chapter 3), Sebastian guides readers toward building their own GPT model to generate text in Chapter 4. The pre-training and fine-tuning chapters (5 through 7) offer real-world insights, illustrating how to optimize models for specific tasks like classification and instruction-following.The book also includes excellent diagrams that make each concept easier to grasp, guiding readers step-by-step through intricate processes. This visual aid is invaluable, adding depth to the already well-explained content and making it accessible even for readers who are newer to deep learning.For anyone serious about understanding and building LLMs, this book is an essential resource. By the time you finish it, youâll have the practical skills and theoretical knowledge to confidently navigate the world of large language models. Highly recommended!
apincsak –
This book makes it all click
This book takes a really fresh approach by walking you through building an LLM from scratch, which I havenât seen anywhere else. Itâs a fun and hands-on way to finally get a solid understanding of how everything works under the hood. It’s been a good time investment!