epochs

Learn · Module 03 · Neural networks · Lesson 1 of 7

One neuron

A neural network is built from one small piece repeated millions of times. Here is that piece, by hand.

3 minute read

In short
  • A neuron computes a weighted sum of its inputs, adds a bias and applies an activation.
  • ReLU keeps positive values and zeroes negatives; sigmoid squashes to between 0 and 1.
  • A whole layer of neurons is a matrix multiplication.

In plain words

A neuron takes some numbers in, multiplies each by a weight, adds them up, adds a bias, then passes the total through an activation function. The weights say how much each input matters; the bias shifts the result.

The activation decides what comes out. The most common today is ReLU: keep the number if it's positive, otherwise output 0. Another is sigmoid, which squashes any number into the range 0 to 1.

Worked example: one neuron, three inputs

  1. 1
    Inputs, weights and bias
    inputs  x = [0.5, 1.0, 0.2]
    weights w = [0.4, 0.6, 1.5]
    bias    b = 0.1
  2. 2
    Weighted sum plus bias

    This is a dot product, from the toolkit module.

    z = 0.5×0.4 + 1.0×0.6 + 0.2×1.5 + 0.1
      = 0.2 + 0.6 + 0.3 + 0.1 = 1.2
  3. 3
    Activation

    With ReLU the output is 1.2. With sigmoid it would be 0.769.

    ReLU(1.2) = 1.2
    sigmoid(1.2) = 0.769

Learning means adjusting w and b so the outputs come out right. A single neuron can only draw a straight line between yes and no. That's enough for AND or OR, but not for patterns like 'one or the other, but not both'. That needs layers, the next lesson.

Your turn

A neuron's weighted sum plus bias comes to −0.7. What does ReLU output?

Where you'll see this

A layer is many neurons reading the same inputs, each with its own weights. Stack their weights as rows and the whole layer is one matrix multiplication plus a bias, then an activation. That is what fills a large model's billions of parameters.

Common mistakes

  • Forgetting the bias term, which lets the neuron shift its decision boundary away from the origin.
  • Thinking the activation function is optional. Without it, stacked neurons collapse into one linear model.
  • Saying a neuron 'understands' a feature. It only computes a weighted sum and a simple function of it.
Check yourself

Answer in your own words, out loud or on paper, before you open the model answer.

What does a single neuron compute?

A single neuron takes its inputs, multiplies each by a weight, adds them up with a bias, and passes the result through an activation function. So it's a dot product plus a bias, then a non-linearity like ReLU or sigmoid. The weights decide how much each input matters and in which direction, and the bias shifts the threshold. With a sigmoid activation, one neuron is exactly logistic regression, so it can draw one straight decision boundary. Its power comes from combining many of them in layers, where each neuron's simple function becomes a building block for much richer patterns, and training adjusts all the weights together with backpropagation.

More to think about
  • Why do we need activation functions?
  • Why is ReLU so widely used?

Go further

Was this lesson clear?