I want to know how $$\frac{\delta(x^TAx)}{\delta(x)}=2Ax$$  I think here's what happens (please correct me where wrong):  (by: the rule for matrix derivative) $$ \frac{\delta(x^TAx)}{\delta(w)}=x^T(A^T+A) $$  (by: I assume we can transpose matrices whenever we need to?) $$ x^T(A^T+A) = x^T(A^T+A^T) = x^T(2A^T) $$  (by: transpose property) $$ x^T(2A^T) = (x^T2A^T)^T = 2Ax $$  Is that how this happens?
4688
It's stated that the gradient of:  $$\frac{1}{2}x^TAx - b^Tx +c$$  is   $$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$  How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$?
57909
I'm trying to show that  $$\delta\big(f(x)\big) = \sum_{i}\frac{\delta(x-a_{i})}{\left|{\frac{df}{dx}(a_{i})}\right|}$$  Where $a_{i}$ are the roots of the function $f(x)$. I've tried to proceed by using a dummy function $g(x)$ and carrying out:  $$\int_{-\infty}^{\infty}dx\,\delta\big(f(x)\big)g(x)$$  Then making the coordinate substitution $u$ = $f(x)$ and integrating over $u$. This seems to be on the right track, but I'm unsure where the absolute value comes in in the denominator, and also why it becomes a sum.  $$\int_{-\infty}^{\infty}\frac{du}{\frac{df}{dx}}\delta(u)g\big(f^{-1}(u)\big) = \frac{g\big(f^{-1}(0)\big)}{\frac{df}{dx}\big(f^{-1}(0)\big)}$$  Can any one shed some light? Wikipedia just states the formula and doesn't actually show where it comes from.
78129
The options are:  A) $\dfrac{(n-1)!}{x}$;  B) $\dfrac{n}{x}$;  C) $(-1)^{n-1}\dfrac{(n-1)!}{x}$;  D) $\dfrac{1}{x}$  My attempt:  $$f'(x)= (n-1)x^{n-2}\log x+ x^{n-2}$$ $$f''(x)=(n-2)(n-1)x^{n-3}\log x+ (n-1)x^{n-3}+(n-2)x^{n-3}$$  But I fail to see any pattern...
4753
I am reading Elements of Statistical Learning, and they derive the equation for linear regression using differentiation on a function which takes a matrix as input. This is making my head spin a bit and I am looking for some resources to practice this type of differentiation, and become more comfortable with it.  This is the problem. Let $X \in \mathbb{R}^{N \times p}$, $b \in \mathbb{R}^{p \times k}$, and $y \in \mathbb{R}^{N \times k}$.  Define $RSS(X, y) = \operatorname{tr}[(Xb - y)^t(Xb -  y)]$.  I want to differentiate $RSS$ with respect to $b$ and set the derivative to zero since I want to minimize the function, $RSS$.  The book claims this is $X^t(Xb - y) = 0$, which gets us, $b = (X^tX)^{-1}Xy$ if $(X^tX)$ is singular. It does not show how it applied the derivatives (which is expected since thats not what this book is about).  I can sort of reason that we have some sort of product rule happening. It seems likely that $d/db(\operatorname{tr}(A)) = \operatorname{tr}(d/db(A))$. But right now I am just writing down symbols that somehow represent some ideas in my head.  In my Principles of Mathematical Analysis book (by Rudin) he covers when the input is a vector, and the output can be a vector. However, this seems to be input is a matrix, and the output in some of the subproblems is a matrix.  Any recommendations for books I can look at?
83265
Let's define  $$ e^x := \lim_{n\to\infty}\left(1+\frac{x} {n}\right)^n, \forall x\in\Bbb R $$  and  $$ \frac{d} {dx} f(x) := \lim_{\Delta x\to0} \frac{f(x+\Delta x) - f(x)} {\Delta x} $$  Prove that  $$ \frac{d} {dx} e^x = e^x $$  using the definition of $e^x$ and derivation above, without using L'Hôpital's rule or the "" and/or the "inverse function derivative trick".  $$ \left( \frac{d} {dx} f^{-1}(x)= \frac{1} {\left(\frac{d}{d(f^{-1}(x))}f(x)\right)(f^{-1}(x))}\right) $$  Or equivalently prove that the following two definiton of $e$ are identical $$ 1)\space\space\space\space e =\lim_{n\to\infty}(1+\frac{1} {n})^n $$  $$ 2) \space\space\space\space e\in\Bbb R,\space\space(\frac{d} {dx} e^x)(x=0) = 1   $$  What I've got is $$ \frac{d}{dx}e^x=e^x \lim_{\Delta x\to0}\frac{e^{\Delta x} - 1} {\Delta x} = e^x \lim_{\Delta x\to0}\lim_{n\to\infty}\frac{\left(1+\frac{\Delta x}{n}\right)^{n}-1}{\Delta x} =  e^x \lim_{\Delta x\to0}\frac{e^{0+\Delta x}-e^0}{\Delta x}  $$  If i assume that $n\in\Bbb N$ I could use binomial theorem but I didn't got much out of it. Wolframalpha just uses L'Hospital rule to solve it, but I am looking for an elementary solution. What I'm interested in is basically is the equivalence of the two definition of $e$ mentioned above. And I'd like to get a direct proof rather than an indirect(I mean which involves logarithms or the derivatives of invers functions).  I look forward getting your aswers.
123550
Thanks everyone for viewing my question! I'm getting stucked on this problem: Suppose $A,B,C,D,E$ are real $n\times n$ matrices such that for any real number $t\neq 1,2$: $$ I_n=(tI_n-A)(\frac{B}{(t-1)^3}+\frac{C}{(t-1)^2}+\frac{D}{t-1}+\frac{E}{t-2}) $$ The problem asks me to prove $C=(A-I)D=D(A-I)$. My progress: Multiply $t-1$ to both sides we obtain: $$ (t-1)I_n=(tI_n-A)(\frac{B}{(t-1)^2}+\frac{C}{t-1}+D+\frac{(t-1)E}{t-2}) $$ Then take limits when $t\rightarrow 1$: $$0=\lim_{t\rightarrow1}(I_n-A)(\frac{B}{(t-1)^2}+\frac{C}{t-1}+D) \\(A-I_n)D=\lim_{t\rightarrow1}(I_n-A)(\frac{B}{(t-1)^2}+\frac{C}{t-1}) $$ This is really close to the required form but I do not know how to continue, I saw some answer concluding that: $$ \lim_{t\rightarrow1}(I_n-A)(\frac{B}{(t-1)^2}+\frac{C}{t-1})=C $$ I don't know if this is correct and it makes no sense to me. Thanks so much for your answers in advance!
1034
(Just some recalls)  We have an action on which we want to apply Least action principle.  $$ S=\int_{t_i}^{t_f} L(q,\dot{q},t)dt$$  We assume that $t \mapsto q(t)$ is the function that will extremise the action, thus if we replace :   $$q(t) \rightarrow q(t)+\delta q(t)$$  with $ \delta q(t_i)=\delta q(t_f)=0$  We will have at first order in $\delta q$ the variation of the action that will be zero.  $$ \delta S = \int_{t_i}^{t_f} \frac{\partial L}{\partial q} \delta q + \frac{\partial L}{\partial \dot{q}} \delta \dot{q} = 0  $$  And we say that because : $ \delta \frac{dx}{dt} = \frac{d \delta x}{dt} $, we can integer by part (the boundary term vanish because of (*)): $$ \delta S = \int_{t_i}^{t_f} (\frac{\partial L}{\partial q} - \frac{d}{dt} \frac{\partial L}{\partial \dot{q}} )\delta q  = 0  $$    My question :  It makes sense to me to say that, as :  $$q(t) \rightarrow q(t)+\delta q(t)$$  then by derivation I will have :  $$\dot{q}(t) \rightarrow \dot{q}(t)+\frac{d}{dt} \delta q(t)$$  We immediately find that $ \delta \frac{dx}{dt} = \frac{d \delta x}{dt} $, so it seems to be very general for any transformation.  But my teacher said it could'nt be the case for any transformation. I don't see why ??  What's more, just to understand :  In a Lagrangian, we consider $q(t)$ and $\dot{q}(t)$ as independant variables, because at a given point, if I have a value of a function, its derivative can take any values.  But the functions $q$ and $\dot{q}$ are not independant functions (they are linked by derivative operation).  So, we always have $\delta q(t)$ and $\delta \dot{q}(t)$ that are independant, and to proove that  $ \delta \frac{dx}{dt} = \frac{d \delta x}{dt} $, we use the "trick" that, the functions associated are not independant.  In a sense, we "go" in the function space where functions are not independent and then we go back in "variable" spaces where they are independent.  Am I right ? And why did my teacher said that $ \delta \frac{dx}{dt} = \frac{d \delta x}{dt} $ is not always true, was him wrong?
97835
$x, a$ in $\mathbb R^n$, $A$ in $\mathbb  R^{n\times n}$. Compute $d(x^T a)/dx$ and $d(x^T A x)/dx$.  I'm not sure about how to think about these and how to do these. Can someone explain how to derive the expressions for the two?  Finally, what happens when we have $A$ and $X$, BOTH in $\mathbb R^{n\times n}$, and we want to find $dTrace(XA)/dX$?
53337
I was trying to derive equation (65) in the following review:    This slightly unusual then usual classical mechanics because it includes a variation of time also, $\delta t$.  Usually one would define, $$\delta S~=~ \int \left[ L \left(\tilde{q}(t),\dot{\tilde{q}}(t),t\right) -  L \left(q(t),\dot{q}(t),t\right)\right] dt , $$ where,  $$\tilde{q}~=~q+\delta q. $$  We have then (before applying int by parts),  $$ \delta S ~=~ \int \left(\frac{\partial L}{\partial q}\delta q +\frac{\partial L}{\partial \dot q}\delta \dot q\right)dt.$$  How does one proceed if both $q$ and $t$ vary and further that $q$ depends on a varying $t$?  Is the definition now  $$\delta S~=~ \int \left[ L \left(\tilde{q}(t+\delta t),\dot{\tilde{q}}( t+\delta t), t+\delta t\right) -  L \left(q(t),\dot{q}(t),t\right)\right] dt~?  $$  If so how does one proceed?
88515
Let A be some $2x2$ matrix with real entries. Prove that $A^T$$A$ = $I$ if and only if $A$ is the rotation matrix or the reflection matrix.  My Progress: It can be shown that if $A$ is either the rotation or reflection matrix, then $A^T$$A$ = $I$ holds by matrix multiplication. Where I get suck is showing that if $A$ is a $2x2$ orthogonal matrix, then $A$ must either be the Rotation or Reflection Matrix. I suppose that since $A$ is orthogonal, it is distance preserving - and the only $2x2$ matrices that preserve distance are the Rotation and Reflection Matrices, but this isn't really a proof.  Any advice and help would be much appreciated.
45377
I have been forewarned about it, I have read the  here, but I haven't seen a counter example where it doesn't work. I know that it isnt really a fraction, but does it effectively get the same result in all cases, or are there counterexamples I must be warned about.   I do not mean obvious counter examples, like $$\frac{dy}{dx}+\frac{du}{dv} = \frac{dydv+dxdu}{dx dv}$$ which as far as I know doesn't really mean anything.
4998
I'm asked to show that  $$\frac{d(\hat{A}\hat{B})}{d\lambda} ~=~ \frac{d\hat{A}}{d\lambda}\hat{B} + \hat{A}\frac{d\hat{b}}{d\lambda}$$ With $\lambda$ a continuous parameter. Should I use the definition $$\frac{d\hat{A}}{d\lambda} ~=~ \lim_{\epsilon \to 0} \frac{\hat{A}(\lambda + \epsilon) - \hat{A}(\lambda)}{\epsilon}$$ applied to $\hat{A}\hat{B}$ like $$\frac{d(\hat{A}\hat{B})}{d\lambda} ~=~ \lim_{\epsilon \to 0} \frac{\hat{A}(\lambda + \epsilon)\hat{B}(\lambda + \epsilon) - \hat{A}(\lambda)\hat{B}(\lambda)}{\epsilon}$$ and do some algebra to get the RHS of the first equation, or I'm missing something? Another interesting derivative to pay attention to is: $$\frac{d}{d\lambda}\exp(\hat{A}(\lambda) )~?$$
99450
If a given first derivative is: $\ {dy \over dx} = {-48x \over (x^2+12)^2} $  What are the steps using the quotient rule to derive the second derivative: $\ {d^2y \over dx^2} = {-144(4-x^2) \over (x^2+12)^3} $  My Steps:  \begin{align*} {d^2y \over dx^2} &amp;= {-48(x^2 +12)^2 - 2(x^2+12)(2x)(-48x)\over (x^2+12)^4} \\ &amp;=-48{(x^2 +12)^2 - 4x^2(x^2+12)\over (x^2+12)^4} \\ &amp;= -48{(x^2 +12)( - 4x^2 +(x^2+12))\over (x^2+12)^4} \\ &amp;=  -48{(x^2 +12)( - 4x^2)\over (x^2+12)^3} \\ &amp;= {???} \end{align*}
43823
In general, $\frac{d}{dx}(f(x) \cdot g(x)) \neq  \frac{d}{dx}f(x) \cdot \frac{d}{dx}g(x)$   When does this result hold true? My first try is to use product rule on left side and compare the two sides, but this hasn't helped at all. Any suggestions?
21417
I've tried to find any solution or proof for $$\langle x| x' \rangle=\delta(x-x'),$$ but I only came to this post:   So I got the information, that the vector $|x\rangle$ form a dirac-normalized basis for the Hilbert Space.   I know that the dirac-delta distribution is defined like this: $$\delta(x-x') = \begin{cases} 0 &amp;\mbox{if } x\neq x' \\ \infty &amp; \mbox{if } x=x' \end{cases},$$ this means that my x' is a point on my x axis where I have my infinite high peak. And also $$\int_{-\infty}^{\infty}dx\cdot \delta(x-x')=1.$$  But how actually correlate this with the scalar product of vectors x, x' in the Hilbert Space that form a so-called 'diracl-normailzed" basis of it?  Can you give me some tips on this please? Or maybe you actually know a link, where this is explained.
36032
I do not understand a remark in Adams' Calculus (page 628 $7^{th}$ edition). This remark is about the derivative of a determinant whose entries are functions as quoted below.      $``$Since every term in the expansion of a determinant of any order is a product involving one element from each row, the general product rule implies that the derivative of an $n\times n$ determinant whose elements are functions will be the sum of $n$ such $n\times n$ determinants, each with the elements of one of the rows differentiated. For the $3\times 3$ case we have      $$\frac{d}{dt}\begin{vmatrix}   a_{11}(t) &amp; a_{12}(t) &amp; a_{13}(t) \\   a_{21}(t) &amp; a_{22}(t) &amp; a_{23}(t) \\   a_{31}(t) &amp; a_{32}(t) &amp; a_{33}(t) \end{vmatrix}=\begin{vmatrix}   a'_{11}(t) &amp; a'_{12}(t) &amp; a'_{13}(t) \\   a_{21}(t) &amp; a_{22}(t) &amp; a_{23}(t) \\   a_{31}(t) &amp; a_{32}(t) &amp; a_{33}(t) \end{vmatrix}+\begin{vmatrix}   a_{11}(t) &amp; a_{12}(t) &amp; a_{13}(t) \\   a'_{21}(t) &amp; a'_{22}(t) &amp; a'_{23}(t) \\   a_{31}(t) &amp; a_{32}(t) &amp; a_{33}(t) \end{vmatrix}+\begin{vmatrix}   a_{11}(t) &amp; a_{12}(t) &amp; a_{13}(t) \\   a_{21}(t) &amp; a_{22}(t) &amp; a_{23}(t) \\   a'_{31}(t) &amp; a'_{32}(t) &amp; a'_{33}(t) \end{vmatrix}."$$   It is not difficult to check this equality by simply expanding both sides. However, the remark sounds like using some clever trick to get this result. Can anyone explain it to me, please? Thank you!
27333
In a scientific paper, I've seen the following $$\frac{\delta K^{-1}}{\delta p} = -K^{-1}\frac{\delta K}{\delta p}K^{-1}$$ where $K$ is a $n \times n$ matrix that depends on $p$. In my calculations I would have done the following $$\frac{\delta K^{-1}}{\delta p} = -K^{-2}\frac{\delta K}{\delta p}=-K^{-T}K^{-1}\frac{\delta K}{\delta p}$$ Is my calculation wrong? Note: I think $K$ is symmetric.