Model Size
The number of parameters in a model, typically expressed in millions (M) or billions (B). Model size correlates loosely with capability but also determines compute and memory requirements.
Why It Matters
Model size is one of the first things practitioners evaluate when choosing a model. It determines hardware requirements, inference cost, and rough capability expectations.
Example
Llama 3 comes in 8B, 70B, and 405B parameter versions — each step up increases capability but also quadruples memory requirements and inference cost.
Think of it like...
Like engine displacement in cars — bigger engines generally mean more power, but also more fuel consumption, weight, and cost.
Related Terms
Parameter
Any learnable value in a machine learning model that is adjusted during training. Parameters include weights and biases in neural networks. Model size is often described by parameter count.
Scaling Laws
Empirical findings showing predictable relationships between model performance and factors like model size (parameters), dataset size, and compute budget. Performance improves as a power law with these factors.
Compute
The computational resources (processing power, memory, time) required to train or run AI models. Compute is measured in FLOPs (floating-point operations) and is a primary constraint and cost in AI development.
Inference
The process of using a trained model to make predictions on new, previously unseen data. Inference is what happens when an AI model is deployed and actively serving results to users.