CS546 Learning Notes: From Softmax to MLP, Finally Understood Backpropagation

1. Why Output Three Numbers for Three-Class Classification?

Previously we predicted a single number using $\hat y=\mathbf w^\top\mathbf x+b$ . Now, to classify comments as positive, negative, or neutral, we give each category its own set of weights and bias, computing one score for each:

z_i=\mathbf w_i^\top\mathbf x+b_i

Writing them together:

\mathbf z=W\mathbf x+\mathbf b

$\mathbf z$ is the logits, meaning the raw scores before conversion to probabilities.

Is z equal to W * x + b?

Yes. Here $z$ is the score, and $p$ later is the probability. We need to keep them separate.

If a sample has $d$ features and the task has $C$ classes, we write single-sample input uniformly as a column vector:

\underbrace{\mathbf z}_{C\times1} =\underbrace{W}_{C\times d} \underbrace{\mathbf x}_{d\times1} +\underbrace{\mathbf b}_{C\times1}

In this notation, each row of $W$ is responsible for one class, and each column corresponds to one input feature. The number of rows in $W$ equals the number of classes.

Suppose there are three classes (positive, negative, neutral), then $W$ has three rows.

So how is the number of input features determined?

It's determined by how we represent samples numerically. For example, if we represent comments using "number of positive words, number of negative words," the input is two numbers; if we add "number of exclamation marks," the input becomes three numbers. These are handcrafted features just for convenience. Later we'll learn word embeddings.

With three classes and three input features, $W$ is $3\times3$ . The two 3's happen to be the same, but their meanings differ—one is the number of rows, the other the number of columns.

2. Why Can't Raw Scores Be Used Directly as Probabilities?

Suppose the three logits are:

\mathbf z=(2,-1,0)^\top

I think they can't be used as probabilities because they don't sum to one hundred percent.

Let me check the arithmetic: $2-1+0$ happens to equal $1$ . The real problem is that it contains negative numbers and numbers greater than $1$ .

Class probabilities must satisfy two requirements: each value is in $[0,1]$ , and the sum equals $1$ . Softmax performs exactly this conversion:

p_i=\frac{e^{z_i}}{\sum_{j=1}^{C}e^{z_j}}

First we take the exponential to make every term positive; then divide by the sum to make the final sum equal $1$ :

\sum_i p_i=\frac{\sum_i e^{z_i}}{\sum_j e^{z_j}}=1

Substituting the scores from earlier:

Class	Logit	Exponential (approx.)	Probability (approx.)
Positive	2	7.389	0.8438
Negative	−1	0.3679	0.0420
Neutral	0	1	0.1142

The class with the highest score still has the highest probability. However, softmax reflects the relative relationship between scores. If all logits increase by the same constant $c$ :

\frac{e^{z_i+c}}{\sum_j e^{z_j+c}} =\frac{e^c e^{z_i}}{e^c\sum_j e^{z_j}}=p_i

The probabilities don't change at all. So we cannot look at a single logit in isolation; we must consider how it changes relative to other classes.

3. How Do We Calculate Loss for Classification?

For cases where each sample has exactly one correct class, cross-entropy can be written as:

L=-\ln p_{\text{correct class}}

The higher the probability of the correct class, the lower the loss:

Probability of Correct Class	Loss (approx.)
0.9	0.105
0.5	0.693
0.1	2.303

When the probability is $1$ , $-\ln1=0$ ; this is the ideal boundary value. With finite real-valued logits, ordinary softmax probabilities are strictly less than $1$ and can only approach it.

Suppose the correct label is "negative," but the model assigns negative a probability of about $0.042$ :

L=-\ln(0.042)\approx3.17

We take the probability corresponding to the correct label, not the model's highest predicted probability. Even if the model chooses the wrong class, we can still compute the loss.

4. So with Three Classes, Do We Calculate Loss Three Times?

When I saw the complete formula, I got a bit confused:

L=-\sum_{i=1}^{C}y_i\ln p_i

Following the order positive, negative, neutral, if the correct answer is negative, the one-hot label is:

\mathbf y=(0,1,0)^\top

Expanding:

L=-[0\ln p_1+1\ln p_2+0\ln p_3]=-\ln p_2

So it's the sum of three cross-entropies.

Let me clarify: it's three class terms added within one cross-entropy, not three independent cross-entropy losses. The one-hot label zeros out the terms for incorrect classes, keeping only the term for the correct class.

One sample, three classes: output three probabilities, get one loss. Two samples, each with three classes: get one loss for each, then average over the batch.

Using different subscripts to distinguish classes and samples:

\bar L=-\frac1B\sum_{n=1}^{B}\sum_{i=1}^{C}y_i^{(n)}\ln p_i^{(n)}

$B$ is the batch size, $C$ is the number of classes. The rest of this article derives for single samples first, with labels as a fixed one-hot vector, and without regularization terms.

5. Why Is the Gradient of Softmax + Cross-entropy Simply p − y?

This part deserves careful explanation. Simply memorizing the result is easy to forget; deriving it by substitution is actually straightforward.

Still assume the second class is the correct answer:

p_2=\frac{e^{z_2}}{e^{z_1}+e^{z_2}+e^{z_3}},\qquad L=-\ln p_2

Substituting and using $\ln(a/b)=\ln a-\ln b$ :

L=-\ln\frac{e^{z_2}}{e^{z_1}+e^{z_2}+e^{z_3}} =-z_2+\ln(e^{z_1}+e^{z_2}+e^{z_3})

Taking the partial derivative with respect to $z_2$ (the score of the correct class):

\frac{\partial L}{\partial z_2} =-1+\frac{1}{e^{z_1}+e^{z_2}+e^{z_3}}\cdot e^{z_2} =p_2-1

The second part uses the chain rule: the derivative of $\ln u$ is $1/u$ , multiplied by the derivative of the inside with respect to $z_2$ , which is $e^{z_2}$ .

For the incorrect class $z_1$ , the derivative of $-z_2$ is zero:

\frac{\partial L}{\partial z_1} =0+\frac{e^{z_1}}{e^{z_1}+e^{z_2}+e^{z_3}}=p_1

Similarly for $z_3$ , we get $p_3$ . Combining all three results:

\frac{\partial L}{\partial\mathbf z} =\begin{bmatrix}p_1\\p_2-1\\p_3\end{bmatrix} =\mathbf p-\mathbf y

Therefore:

\boxed{\frac{\partial L}{\partial z_i}=p_i-y_i}

Although the loss ultimately reduces to $-\ln p_2$ , $p_2$ 's denominator contains all logits, so other classes also have gradients.

6. From Gradient of Logits to Gradient of Parameters

This distinction is important: $p_i-y_i$ is the gradient with respect to $z_i$ , not yet the gradient with respect to weights.

Because:

z_i=\sum_j W_{ij}x_j+b_i

Applying the chain rule:

\frac{\partial L}{\partial W_{ij}}=(p_i-y_i)x_j, \qquad \frac{\partial L}{\partial b_i}=p_i-y_i

Working through today's exercise: with only one input feature $x=2$ , the correct class is negative, and the negative probability is about $0.042$ :

\frac{\partial L}{\partial z_{\text{negative}}}=0.042-1=-0.958

\frac{\partial L}{\partial w_{\text{negative}}}=-0.958\times2=-1.916, \qquad \frac{\partial L}{\partial b_{\text{negative}}}=-0.958

With learning rate $0.1$ :

w_{\text{new}}=w_{\text{old}}+0.1916, \qquad b_{\text{new}}=b_{\text{old}}+0.0958

So both w and b increase, the negative logit increases, and then the softmax probability increases?

In this single-sample, single-layer classifier update, yes. Here $x>0$ , so increasing both parameters raises the negative logit. The gradients for the other two classes are positive, so their logits decrease, raising the probability of the correct class.

Supplement: After Updating Parameters, How Much Does the Logit Actually Change?

What does this formula mean?

First define the change: $\Delta z_i=z_i'-z_i$ , where primes indicate values after the update. This isn't a new update rule, but rather substituting the parameter update back into the logit formula to compute the score change.

With just one input feature, the original score is:

z_i=w_ix+b_i

We already know the single-sample gradient descent update:

w_i'=w_i-\eta(p_i-y_i)x, \qquad b_i'=b_i-\eta(p_i-y_i)

The probabilities and gradients are all computed before the update. For the same input $x$ , substituting into the updated score:

\begin{aligned} z_i'&=w_i'x+b_i'\\ &=\big[w_i-\eta(p_i-y_i)x\big]x +\big[b_i-\eta(p_i-y_i)\big]\\ &=\underbrace{w_ix+b_i}_{z_i} -\eta(p_i-y_i)x^2-\eta(p_i-y_i) \end{aligned}

Subtracting the old score and combining the two terms:

\boxed{\Delta z_i=-\eta(p_i-y_i)(x^2+1)}

The $x^2$ comes from the contribution of weight change, and the $1$ comes from the contribution of bias change. The weight update is multiplied by the input once; when substituted into $w_ix$ , it's multiplied again, producing the square.

Verifying with the negative class from before: $x=2$ , $\eta=0.1$ , $p_i-y_i=-0.958$ , thus:

\Delta z_{\text{negative}} =-0.1\times(-0.958)\times(2^2+1)=0.479

That is, the negative logit increases by $0.479$ . Using the parameter changes calculated earlier directly also gives the same result:

\Delta z_{\text{negative}} =\underbrace{0.1916}_{\Delta w}\times2 +\underbrace{0.0958}_{\Delta b}=0.479

When there are multiple input features, the contribution of each weight change is summed:

\Delta z_i=\sum_{j=1}^{d}\Delta W_{ij}x_j+\Delta b_i =-\eta(p_i-y_i)\left(\sum_{j=1}^{d}x_j^2+1\right)

And $\|\mathbf x\|^2=\sum_{j=1}^{d}x_j^2$ is the sum of squares of all vector components, so it can be abbreviated as:

\Delta z_i=-\eta(p_i-y_i)(\|\mathbf x\|^2+1)

Since $\|\mathbf x\|^2+1$ is always positive and the learning rate is also positive, the direction of change is determined by $-(p_i-y_i)$ : the logit of the correct class increases, and the logit of incorrect classes decreases.

The prerequisite here is: the input vector remains unchanged, using only the gradient from this single sample, updating the weights and biases of this layer with the same learning rate, and using ordinary gradient descent without momentum or regularization. Batch updates mix in gradients from other samples; when updating the entire MLP simultaneously, the hidden representation received by this layer may also change. Therefore, we cannot directly generalize to "each step of any batch or entire MLP is guaranteed to reduce loss for every sample."

7. Why discuss ReLU before adding hidden layers?

Suppose we only add another linear computation:

h=2x+1,\qquad z=3h+4

Combining them:

z=3(2x+1)+4=6x+7

We computed two layers, but it's still the same form of weighted sum plus bias. Strictly speaking, with bias it's called an affine transformation; the course often calls such layers linear layers.

In matrix form we can also combine them:

W_2(W_1\mathbf x+\mathbf b_1)+\mathbf b_2 =(W_2W_1)\mathbf x+(W_2\mathbf b_1+\mathbf b_2)

Therefore we need nonlinearity in between. ReLU is:

\operatorname{ReLU}(a)=\max(0,a)

Negative numbers become zero, positive numbers stay the same. After adding it:

h=\operatorname{ReLU}(2x+1),\qquad z=3h+4

When $x=-1$ I get $h=0$ , when $x=1$ I get $h=3$ . The entire function becomes:

z=\begin{cases}4,&x\le-0.5\\6x+7,&x>-0.5\end{cases}

A kink point appears—it's no longer a single global straight line. This is the change brought by nonlinearity in this example.

8. Extending from one hidden unit to four

The complete architecture used today is: three input features, four ReLU hidden units, three classes.

\mathbf a=W_1\mathbf x+\mathbf b_1

\mathbf h=\operatorname{ReLU}(\mathbf a)

\mathbf z=W_2\mathbf h+\mathbf b_2, \qquad \mathbf p=\operatorname{softmax}(\mathbf z)

Quantity	Shape	Meaning
$\mathbf x$	$3\times1$	Input features
$W_1$	$4\times3$	Each row computes weighted sum for one hidden unit
$\mathbf b_1,\mathbf a,\mathbf h$	$4\times1$	First layer bias, pre-activation, post-activation
$W_2$	$3\times4$	Each row computes score for one class
$\mathbf b_2,\mathbf z,\mathbf p,\mathbf y$	$3\times1$	Output bias, scores, probabilities, labels

If we transpose h, wouldn't W2 be 3×4?

The shape of $W_2$ is correct, but we don't need to transpose $\mathbf h$ here:

(3\times4)(4\times1)\longrightarrow(3\times1)

If we transpose $\mathbf h$ , we couldn't multiply at all. First stick with the convention of using column vectors, then check that the inner dimensions match.

Without a hidden layer, classifying directly with softmax is usually called Softmax Regression; now with a ReLU hidden layer added, it's an MLP for multiclass classification. The number of hidden units is a structural choice we make, not equal to the number of classes.

9. Backpropagation: multiply along one path, add multiple paths

First, consider a single hidden unit:

a=w_1x+b_1,\qquad h=\operatorname{ReLU}(a),\qquad z=w_2h+b_2

If we already know $\partial L/\partial z$ coming back from later, then:

\frac{\partial L}{\partial w_1} =\frac{\partial L}{\partial z} \frac{\partial z}{\partial h} \frac{\partial h}{\partial a} \frac{\partial a}{\partial w_1}

That is:

\frac{\partial L}{\partial w_1} =\frac{\partial L}{\partial z}\,w_2\,\operatorname{ReLU}'(a)\,x

When $a>0$ , the ReLU derivative is $1$ ; when $a<0$ it is $0$ . At $a=0$ it is not differentiable; implementations typically choose $0$ .

In today's local exercise, given $\partial L/\partial z=2,w_2=3,x=1,a=3$ , the result is $6$ . This is a local practice example of the chain rule with upstream gradient, not the softmax gradient from the three-class sample earlier.

If the same $h$ affects two outputs:

z_1=v_1h+b_1,\qquad z_2=v_2h+b_2

Then:

\frac{\partial L}{\partial h} =\frac{\partial L}{\partial z_1}v_1 +\frac{\partial L}{\partial z_2}v_2

In the exercise, the two paths contribute $2\times3=6$ and $(-1)\times4=-4$ respectively, for a total gradient of $2$ . Different paths can partially cancel each other out.

Therefore, the two rules of backpropagation are: multiply along a path, add when multiple paths converge.

10. Today's most valuable insight: why use the transpose in backpropagation?

The forward pass is:

\mathbf z=W_2\mathbf h+\mathbf b_2

Let $\mathbf g_z=\partial L/\partial\mathbf z=\mathbf p-\mathbf y$ . The gradient passed back to the hidden layer is:

\boxed{\mathbf g_h=W_2^\top\mathbf g_z}

Shape check:

\underbrace{W_2^\top}_{4\times3} \underbrace{\mathbf g_z}_{3\times1} =\underbrace{\mathbf g_h}_{4\times1}

But just saying "so the dimensions multiply" is not enough. The real reason comes from looking at a single hidden unit $h_j$ :

\frac{\partial L}{\partial h_j} =\sum_{i=1}^{3}W_{2,ij}\frac{\partial L}{\partial z_i}

$h_j$ connects to three outputs; the weights of these three connections are in column $j$ of $W_2$ . We need to multiply this column element-wise with the output gradients and sum them.

Taking the transpose converts this column into a row, so we can use matrix multiplication to simultaneously compute the aggregation for all hidden units at once.

In the forward pass, each row of weights collects inputs to compute one output; in the backward pass, each column of weights collects the gradient contributions from all outputs to the same input.

This is not inversion, nor is it working backward through the forward computation to recover the original input. The matrix is not even necessarily square. We are computing how sensitive the loss is to intermediate quantities.

11. Gradient through ReLU: element-wise multiplication

We also need to pass from $\mathbf h$ to the pre-activation $\mathbf a$ :

\boxed{\mathbf g_a=\mathbf g_h\odot\operatorname{ReLU}'(\mathbf a)}

$\odot$ denotes element-wise multiplication, not matrix multiplication.

Another local exercise today gives:

\mathbf a=\begin{bmatrix}2\\-1\\3\\-2\end{bmatrix}, \qquad \mathbf g_h=\begin{bmatrix}0.5\\-0.2\\-0.8\\0.4\end{bmatrix}

Thus:

\mathbf g_a= \begin{bmatrix}0.5\\-0.2\\-0.8\\0.4\end{bmatrix} \odot\begin{bmatrix}1\\0\\1\\0\end{bmatrix} =\begin{bmatrix}0.5\\0\\-0.8\\0\end{bmatrix}

The second and fourth units have negative pre-activation values, so the gradient for this sample is zeroed out at these locations. This doesn't mean these parameters never change: other samples might activate them; updates with momentum, regularization, etc. cannot be judged by this term alone.

12. Finally, how do we get the weight gradients for the first layer?

Because:

a_i=\sum_j W_{1,ij}x_j+b_{1,i}

We have:

\frac{\partial L}{\partial W_{1,ij}}=g_{a,i}x_j, \qquad \frac{\partial L}{\partial b_{1,i}}=g_{a,i}

Given the input:

\mathbf x=\begin{bmatrix}2\\1\\-1\end{bmatrix}

The weight gradient for the first hidden unit is $0.5[2,1,-1]=[1,0.5,-0.5]$ .

Is the third one −1.6, −0.8, 0.8?

Yes, $-0.8[2,1,-1]=[-1.6,-0.8,0.8]$ . Putting all four rows together:

\boxed{\frac{\partial L}{\partial W_1}=\mathbf g_a\mathbf x^\top} =\begin{bmatrix} 1&0.5&-0.5\\ 0&0&0\\ -1.6&-0.8&0.8\\ 0&0&0 \end{bmatrix}

The shape is $(4\times1)(1\times3)=4\times3$ , same as $W_1$ . This operation of multiplying a column vector by a row vector is called an outer product.

The transpose here serves a different purpose than before:

$W_2^\top\mathbf g_z$ : Aggregates gradients across different output paths to obtain the gradient for the hidden output.
$\mathbf g_a\mathbf x^\top$ : Multiplies the gradient of each hidden unit by each input respectively to get the weight gradient for each connection.

13. Computing gradients is not the same as updating parameters

You also need to update parameters using the previous formula: 1 − learning rate × gradient.

The operation is understood correctly, but $1$ is just the initial weight in the previous example. The general formula is:

\theta_{\text{new}}=\theta_{\text{old}}-\eta\frac{\partial L}{\partial\theta}

There is also a key distinction: $\mathbf h$ and $\mathbf a$ are computed intermediate results, not directly learned parameters. After computing $\mathbf g_h$ , you must continue to compute the gradients for $W_1,\mathbf b_1$ , and then update the parameters. The next forward pass will recompute the hidden layer outputs.

For the network here, what is learned is $W_1,\mathbf b_1,W_2,\mathbf b_2$ .

Backpropagation is responsible for computing gradients; gradient descent is responsible for using gradients to update parameters. A typical training step should first compute all gradients using the same set of forward parameters, then update them uniformly. Do not modify $W_2$ first and then use the new $W_2$ to compute $\mathbf g_h$ for this step.

14. Complete formulas together; during review you can explain line by line

Here we complete the output layer parameter gradients. It follows the same "gradient times input" rule as the first layer; the input to the output layer is $\mathbf h$ .

Forward: compute predictions and loss

\begin{aligned} \mathbf a&=W_1\mathbf x+\mathbf b_1\\ \mathbf h&=\operatorname{ReLU}(\mathbf a)\\ \mathbf z&=W_2\mathbf h+\mathbf b_2\\ \mathbf p&=\operatorname{softmax}(\mathbf z)\\ L&=-\sum_i y_i\ln p_i \end{aligned}

Backward: compute gradients of intermediate quantities and parameters

\begin{aligned} \mathbf g_z&=\mathbf p-\mathbf y\\ \nabla_{W_2}L&=\mathbf g_z\mathbf h^\top\\ \nabla_{\mathbf b_2}L&=\mathbf g_z\\ \mathbf g_h&=W_2^\top\mathbf g_z\\ \mathbf g_a&=\mathbf g_h\odot\operatorname{ReLU}'(\mathbf a)\\ \nabla_{W_1}L&=\mathbf g_a\mathbf x^\top\\ \nabla_{\mathbf b_1}L&=\mathbf g_a \end{aligned}

Update: each parameter minus learning rate times its own gradient

W_\ell\leftarrow W_\ell-\eta\nabla_{W_\ell}L, \qquad \mathbf b_\ell\leftarrow\mathbf b_\ell-\eta\nabla_{\mathbf b_\ell}L, \quad\ell\in\{1,2\}

For batch-averaged loss, average the gradients of each sample with respect to the same parameter, then update. All samples must be computed based on the parameters before this update.

15. Where I am in learning now, and how to review

What was initially worked through today is an "MLP with one ReLU hidden layer plus softmax multiclass output." I can manually compute local examples and understand gradient paths and shapes, but have not yet independently completed a full network numerical computation or code training.

During review, you can first cover the answers and ask yourself:

Why is a single sample loss for three classes one loss rather than three separate losses?
Can you derive $\mathbf p-\mathbf y$ from $-\ln p_{\text{correct class}}$ ?
What does one column of $W_2$ represent, and why do we use it to aggregate in the backward pass?
Which intermediate quantity's gradient does ReLU zero out?
Why is the weight gradient "the incoming gradient times this layer's input"?
After computing $\mathbf g_h$ , why can't we directly say the parameters are already updated?

Explaining these clearly and then computing a complete example yourself will validate your understanding better than just knowing the formulas. Later when connecting word embeddings, you can continue asking: where do input vectors come from, and can they also be learned through gradients?

Corresponding slides

Source from course Lecture3_DeepLearning_Background.pdf, page numbers according to PDF reader counting, with cover page as page 1.

Pages	Corresponding content
17–20	Multiclass, one-hot, class scores, and softmax
22–24	Nonlinearity and why activation functions are needed
27–38	Multilayer networks, layer notation, and forward passing
39–42	Loss, forward and backpropagation, chain rule
44–60	Layered computation and gradient propagation

The ReLU numerical examples, softmax cross-entropy derivation, and $W_1,W_2$ notation in this document come from today's supplementary explanations and are not verbatim transcriptions from the slides. The Lecture 5 uploaded today covers Sequence Modeling; we first consolidate the MLP fundamentals needed to understand it, and have not yet begun CNN, RNN, LSTM, and GRU.