What Is The Derivative W.R.T Matrix Calculus You Need For Deep Learning

Where $mathbf{W} in mathcal{R}^{d imes D}$ and $mathbf(x)in mathcal{R}^{d imes 1}$

How to calculate $partial mathbf{Y}/partial mathbf{W}$ ?



Matrix calculus is used in such cases. Your equation looks like it”s from OLS (least squares) theory. In those you differentiate by vector $x$ some quadratic forms like $frac{partial (x”A”Ax)}{partial x}$. Look up relevant formulae in my link above.

Đang xem: Derivative w.r.t matrix

If you really are up to differentiating by matrices not vectors, you”ll end up with tensors. Tensors are fun, but so far I haven”t seem them used a lot in statistics. They”re ubiquitous in physics, btw. Again, follow the link I gave.


Thanks for contributing an answer to Cross Validated!

Please be sure to answer the question. Provide details and share your research!

But avoid

Asking for help, clarification, or responding to other answers.Making statements based on opinion; back them up with references or personal experience.

Xem thêm: 0 Student Loan Interest Rate, Student Loan Interest Rates Increase On July 1

Use MathJax to format equations. MathJax reference.

Xem thêm: Student Beans (@Studentbeans) · Twitter, Student Beans

To learn more, see our tips on writing great answers.

Post Your Answer Discard

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

See also  Maximum Fi R Data Frame Derivative S Of Fitted Gam Functions — Fderiv

Not the answer you're looking for? Browse other questions tagged machine-learning optimization derivative matrix-calculus or ask your own question.

Is my step by step derivation of quadratic cost function's (Neural Networks) partial derivative with respect to some weights matrix correct?


site design / logo © 2021 Stack Exchange Inc; user contributions licensed under cc by-sa. rev2021.7.23.39827

Cross Validated works best with JavaScript enabled


Your privacy

By clicking “Accept all cookies”, you agree Stack Exchange can store cookies on your device and disclose information in accordance with our Cookie Policy.

See more articles in category: Derivative

Leave a Reply

Back to top button