Distillation (knowledge distillation)
Japanese 蒸留（大きなモデルの力を小さなモデルに移す方法）
Distillation is a way of training a small AI model on the answers of a large one. It is used to make lighter models that can run on a laptop instead of an expensive server.
Distillation treats a large, capable model as the teacher and a small, light model as the student, and trains the student again using the teacher's answers as its textbook. The student does not know as much as the teacher, but it picks up much of the way the teacher answers. It is a bit like turning a thick textbook into a set of summary notes. A large model needs an expensive server, while a distilled small model can run on a personal computer.
Watch out A distilled model is not the same as the original. DeepSeek released small models made by retraining Qwen and Llama models on R1's answers; even though their names include R1, they are built on other models and perform differently. Licenses also differ from model to model: starting with Llama 3.1, Meta changed its license so that developers may use the model's outputs to improve other models. Check the original model's license before you use one.
Source: Distilling the Knowledge in a Neural Network · deepseek-ai/DeepSeek-R1 · Introducing Llama 3.1: Our most capable models to date · deepseek-r1 (checked 2026-10-26)