Sigmoid
An activation function that squashes input values into a range between 0 and 1, creating an S-shaped curve. It is commonly used for binary classification outputs and in certain neural network architectures.
Why It Matters
Sigmoid converts raw model outputs into probabilities, making them interpretable. It is essential for any binary decision-making in ML.
Example
A sigmoid function converting a model's raw score of 2.5 into a probability of 0.92, which could be interpreted as 92% chance of the email being spam.
Think of it like...
Like a volume knob that smoothly maps any input level to a range between mute and maximum — extreme inputs still stay within bounds.
Related Terms
Activation Function
A mathematical function applied to the output of each neuron in a neural network that introduces non-linearity. Without activation functions, a neural network would just be a series of linear transformations.
Logistic Regression
A classification algorithm that uses the sigmoid function to predict the probability of a binary outcome. Despite its name containing 'regression,' it is used for classification tasks.
ReLU
Rectified Linear Unit — the most commonly used activation function in deep learning. It outputs the input directly if positive, and zero otherwise: f(x) = max(0, x).
Softmax
A function that converts a vector of numbers into a probability distribution, where each value is between 0 and 1 and all values sum to 1. It is typically used as the final layer in classification models.