Foundation Model
A foundation model is a large AI model pretrained on massive amounts of data. It gains broad knowledge and general capabilities that can serve as the basis for many specific tasks.
How it works
It is typically built on the Transformer architecture, learning general patterns and representations by analyzing internet-scale data such as text, images, or code. This pretraining gives it strong generalization ability, and it can later be adapted to a wide range of downstream tasks through methods like fine-tuning — avoiding the enormous cost of training a model from scratch for every new task.
Examples
- 1.Chatbots like GPT-4 or Claude are powered at their core by a powerful text foundation model.
- 2.Image generation tools like DALL-E or Stable Diffusion are built around image foundation models based on diffusion models.
- 3.The code assistant GitHub Copilot is powered at its core by a foundation model trained on code data.