Instruction tuning

Instruction tuning fine-tunes a pre-trained language model on a dataset of instruction-response pairs, teaching it to follow diverse natural language directives. Models like InstructGPT, Llama-3-Instruct, and Mistral-Instruct are produced through instruction tuning.

This technique dramatically improves zero-shot task performance and the model's ability to act as a helpful assistant.