I am referring to Tom Minka's . I don't have the book by Magnus &amp; Neudecker so I can't refer to the details of the theory. Regarding rules (6): $d(XY) = (dX)Y + X(dY)$ and (12): $dX^*=(dX)^*$, I am not clear how to apply them. My notation used is numerator layout, i.e. $\dfrac{dx}{dx} = I$ Question 1. $f(x)=x^Tx$ , $\dfrac{df}{dx}=2x^T$ However, if I use $\dfrac{df}{dx}= x^T\dfrac{dx}{dx} + \dfrac{dx^T}{dx}x$, firstly, $\dfrac{dx^T}{dx}$ is $1^T$? Second, according to rule (12), $\dfrac{dx^T}{dx} = (\dfrac{dx}{dx})^T = I^T = I $? Question 2. $f(x) = x^TAx$ $\dfrac{df}{dx}=x^T\dfrac{dAx}{dx}+ \dfrac{dx^T}{dx}(Ax) = x^TA + ???$ $???$ is supposed to be $x^TA^T$, however, it seems to me no matter $\dfrac{dx^T}{dx}$ equals $1^T$ or $I$ it does not give the expected result.
4688
It's stated that the gradient of:  $$\frac{1}{2}x^TAx - b^Tx +c$$  is   $$\frac{1}{2}A^Tx + \frac{1}{2}Ax - b$$  How do you grind out this equation? Or specifically, how do you get from $x^TAx$ to $A^Tx + Ax$?
43823
In general, $\frac{d}{dx}(f(x) \cdot g(x)) \neq  \frac{d}{dx}f(x) \cdot \frac{d}{dx}g(x)$   When does this result hold true? My first try is to use product rule on left side and compare the two sides, but this hasn't helped at all. Any suggestions?
68973
MWE:  \documentclass{article}  \usepackage{bm}  \begin{document} $$x(t), \bf{x}(t), \bf{x(t)}$$ \end{document}   Result:    Question:  Why are the second and third x(t) typeset the same? In the second  expression, it was asked to apply \bf to only x.  EDIT:  I had not payed attention to it when asking the question, but I would like to have a bold and italic x.
104622
, "Note that the operators are in time order...we can insert $T$ without changing anything". This I agree with. But then on the next paragraph he states "The time order operator $T$, moves all...where they annihilate...". How correct is this paragraph? Is equation (5.14) always valid? Is equation (5.14) the reason why the time-ordering symbol $T$ appears all over the place in QFT?  The equations I'm referring to are (braket package available?):   $$\langle f|i\rangle = \langle0|~a_{1'}(+\infty)a_{2'}(+\infty)a_{1}^\dagger(-\infty)a_{2}^\dagger(-\infty)~|0\rangle \tag{5.13} $$  and since time goes from right to left inside the braket we put the $T$-symbol  $$\langle f|i\rangle = \langle 0|T ~a_{1'}(+\infty)a_{2'}(+\infty)a_{1}^\dagger(-\infty)a_{2}^\dagger(-\infty)~|0\rangle \tag{5.14} $$   without changing anything.   After this, the $T$-symbol appears everywhere!
115373
This seems a bit obvious, but I'm not clear on one part. My teacher had told us $d(x^2) =2xdx$. However on asking why, we were told $d(x^2)= (x+dx)^2-x^2= 2xdx+(dx)^2$. We were told that we can ignore the $(dx)^2$ because its small. For obvious reason I feel this step is wrong, why can we ignore $(dx)^2$, is their any other way of getting the result?
22832
I'm struggling with a proof of the normal equation, so I posted a  which hopefully will get resolved soon. However, I mentioned there that I'm uncomfortable with the proofs dealing with , particularly when it comes to derivatives of product of vectors or product of matrices and vectors.  Furthermore, there are a couple of "layout conventions" which are not even used consistently. Because of this, I thought I could get rid of all that insanity by using tensor notation instead.   In relation to the normal equation question, I would be very interested in looking at a proof of this equation (here you can see the ):  $$\beta = (X^{T}X)^{-1}X^{T}{\bf{y}}$$  using tensor notation.  And as a more general request, I'd like to get an advice on the issue of dealing with matrix calculus and derivatives. Particularly, I want to hear about your approach when you have to calculate something of this kind given the inconsistencies in notations and non-intuitive nature of some identities. Do you consult Wikipedia or some handbook every time you want to calculate a derivative of some product of vectors or matrices? Do you memorize some recurrent identities? Do you know an alternative procedure?
42960
One problem I have found when learning calculus is that there are many different ways to denote the derivative. If $y=f(x)=x^2$, then we could write \begin{align} f'(x)&amp;=2x \\ y'&amp;=2x \\ \frac{df}{dx}(x)&amp;=2x \\ \frac{df(x)}{dx}&amp;=2x \\ \frac{d}{dx}f(x)&amp;=2x \\ \frac{dy}{dx}&amp;=2x \end{align} And this is just Lagrange and Leibniz's notations alone. What I find troubling is that they all seem to be suggesting subtly different things about what the derivative actually is. Is it a function, a limit of a quotient, or both? In the interests of keeping my post brief, I'll focus my attention on $f'(x)=2x$ and $\frac{dy}{dx}=2x$, as these seem to be the most common notations.  $$ f'(x)=2x $$  It does make sense to think of the derivative as the gradient function: $$ f'\colon x\mapsto\lim_{\Delta x \to 0}\frac{f(x+\Delta x)-f(x)}{\Delta x} $$ In this case the limit expression is equal to $2x$, and so we can write $$ f' \colon x \mapsto 2x $$ However, this notation seems a little counter-intuitive when we consider what it means to differentiate a function with respect to a variable other than $x$. If I ask what is the derivative of $f(x)$ with respect to $\frac{x}{2}$, does this question make sense? Is it simply $f'(\frac{x}{2})$? Or do we have to express $x^2$ in terms of $\frac{x}{2}$? And how can we can express this derivative using Lagrange's notation?  $$ \frac{dy}{dx}=2x $$  There are many things which are nice about Leibniz's notation, including the fact that it is explicit which variable you are differentiating with respect to. However, in this case, it is unclear whether we are talking about a function, or something else entirely. There are other issues. Some people say they dislike the Leibniz formulation of the chain rule $$ \frac{dy}{dx}=\frac{dy}{du}\frac{du}{dx} $$ saying that they find it to be inaccurate. I don't really understand why this is the case. Could someone please elaborate?
70274
I know that $\frac{dy'(x)}{dy}=0$ (where $y'=\frac{dy(x)}{dx}$). The reason explained is that $y'$  does not depend explicitly on $y$. But intuitively, $y'$ depends on $y$, since if you vary $y$ you will modify $y'$. Why is my reasoning wrong (my reasoning sounds like it's related to functional calculus, instead of standard calculus)?  I tried to write  $\frac{dy'(x)}{dy}=\frac{d}{dy}\frac{dy}{dx}=\frac{d}{dx}\frac{dy}{dy}=\frac{d}{dx}1=0$, but this proof doesn't convince me.  I think that other way to see this is saying that if a function is of the form $f(y)$, it will not dependend on the variable $y'$.  But the same way you write $f(y)=y^2$, you could write $f(y)=\frac{d}{dx}y$, which clearly depends on $y'$. So I don't know if there are some types of operations which are restricted (for example, taking limits):  Note: The problem raises in the context of Classical Mechanics, where: $\frac{\partial L(\dot{x},t)}{\partial x}=0$.
47708
I am looking to find out how the second derivative formula works.  I can blindly apply it but I don't have a grasp of what is going on or why. The first derivitive has the formula $\frac{dy}{dx} = \frac{\frac{dy}{dt}}{\frac{dx}{dt}}$. I told this was because if you divide them you cancel out the dts and get dy/dx. The second derivative has the formula $\frac{d^2y}{dx^2} = \frac{d}{dx} \left(\frac{dy}{dx}\right) = \frac{ \frac{d}{dt}\left(\frac{dy}{dx}\right) }{\frac{dx}{dt}}$. Is this something that can be explained with only a knowledge of calculus or is this something I would have to take on faith until I learn more. Any help would be greatly appreciated.
57678
I took ${-x^2-x{\tau}} = k $ on differentiating it $dk=(-2x-\tau)dx$ but this substitution doesn't work how can i proceed further .
3973
I do not know much about fractional calculus, except what I have read in a few short posts at MSE and .  I know that order of a derivative can be extended from rational values to real values, but all I know is what is written .  So my question is, what is the simplest way to understand and/or define $\dfrac{d^n}{dx^n}f(x),$ for $n\in\mathbb{R}$?  For example what is $\dfrac{d^\pi}{dx^\pi}f(x)$?  Also, what about complex values?  What might we say about $\dfrac{d^{s}}{dz^s}f(z)$ for $s\in\mathbb{C}$?  And what are some uses for such derivatives?
13721
$$\dfrac{1}{2}\ln(x+7)-(2 \ln x+3 \ln y)$$  Our professor let's us solve this but i do not understand how $\ln$ works. He says it has same properties with $\log$ but i still don't get it. What's the difference of the two?
46489
I'm studying for actuarial exams, but I always pick up mathematics books because I like to challenge myself and try to learn new branches.  Recently I've bought Topology by D. Kahn and am finding it difficult.  Here is a problem that I think I'm am answering sufficiently but any help would be great if I am off.  If $d$ is a metric on a set $S$, show that  $$d_1(x,y)=\frac{d(x,y)}{1+d(x,y)}$$ is a metric on $S$.  The conditions for being a metric are $d(X,Y)\ge{0}, d(X,Y)=0$ iff $X=Y$, $d(X,Y)=d(Y,X)$, and $d(X,Y)\le{d(X,Z)+d(Z,Y)}$.  Thus, we simply go axiom by axiom.  1)  Since both $d(x,y)\ge{0}$ and $1+d(x,y)\ge{0},$ it is clear that $d_1(x,y)\ge{0}$.  (Is this a sufficient analysis?) 2) $d_1(x,x)=\frac{d(x,x)}{1+d(x,x)}=\frac{0}{1+0}=0$. 3) $d_1(x,y)=\frac{d(x,y)}{1+d(x,y)}=\frac{d(y,x)}{1+d(y,x)}=d_1(y,x).$ 4) $d_1(x,y)=\frac{d(x,y)}{1+d(x,y)}\le{\frac{d(x,z)+d(z,y)}{1+d(x,z)+d(z,y)}}=\frac{d(x,z)}{1+d(x,z)+d(z,y)}+\frac{d(z,y)}{1+d(x,z)+d(z,y)}\lt\frac{d(x,z)}{1+d(x,z)}+\frac{d(z,y)}{1+d(z,y)}=d_1(x,z)+d_1(z,y).$  However, #4 is strictly less, not less than or equal to, according to my analysis, so where did I go wrong?
112982
Reasoning for $\frac{\ln|x|}{2}+ C$ :  $$\int  \frac{1}{(2x)}  \, dx = \frac{\ln|2x|}{2}+ C$$   Check: By taking derivative we get $\frac{1}{x}$  Reasoning for $\frac{\ln|x|}{2}+ C$ :  $$\int  \frac{1}{2x}  \, dx = \frac{1}{2}\int  \frac{1}{x}  \, dx =\frac{\ln|x|}{2}+ C$$  Check: By taking derivative we get $\frac{1}{x}$  Question: Is  $\int  \frac{1}{2x}  \, dx $ equal to $\frac{\ln|2x|}{2}+ C$ or $\frac{\ln|x|}{2}+ C$?
79166
I've been interested in non-standard analysis recently. I was reading up on it and noticed the following interesting comment on the , right after giving an example of a nonstandard differentiation:     The use of the standard part in the definition of the derivative is a rigorous alternative to the traditional practice of neglecting the square of an infinitesimal quantity... the typical method from Newton through the 19th century would have been simply to discard the $dx^2$ term.   I've never heard anything like this before, and really find it fascinating that Newton's method was to define the relation $dx^2 = 0$. If we actually formalize the above structure by taking $\mathbb{R}$ and adjoining an element $dx^2 = 0$ to it, we get the "dual numbers," isomorphic to the quotient ring $\mathbb{R}[x]/x^2$. I'd seen some things about how this algebra plays into automated differentiation algorithms for some computer software systems, but I've never heard anything about Newton directly working in this algebra. So I have a few questions:   Does anyone have more historical information on the way that Newton performed differentiation, and its relation to the dual numbers? Does anyone know how effectively real analysis can be formalized with the dual numbers? Does the resulting system play nice enough to develop all of the important modern results? If we start with $\mathbb{C}[x]/x^2$ instead, can we likewise develop complex analysis?   Since this idea is so simple, I'm very curious how powerful it is. I'm also curious if it has any major drawbacks too, since I'm not sure why anyone would mess with the foundational baggage involved in defining the hyperreals if this simple 2-dimensional real algebra could really do the trick.
99618
While I do know that $\frac{dy}{dx}$ isn't a fraction and shouldn't be treated as such, in many situations, doing things like multiplying both sides by $dx$ and integrating, cancelling terms, doing things like $\frac{dy}{dx} = \frac{1}{\frac{dx}{dy}}$ works out just fine.  So I wanted to know: Are there any particular cases (in single-variable calculus) we have to look out for, where treating $\frac{dy}{dx}$ as a fraction gives incorrect answers, in particular, at an introductory level?  Note: Please provide specific instances and examples where treating $\frac{dy}{dx}$ as a fraction fails
34579
I'm reading Sakurai's Modern QM right now and in the first chapter he states a number of conditions required for a translation operator: unitarity, $$T(\mathrm{d}\mathbf{x})T(\mathrm{d}\mathbf{x^\prime}) = T(\mathrm{d}\mathbf{x}+\mathrm{d}\mathbf{x^\prime}),$$ $$T(-\mathrm{d}\mathbf{x}) = T^{-1}(\mathrm{d}\mathbf{x}),$$ and going to 1 for small translations.   He then makes the ansatz (1.6.20 in the new version): $$ T(\mathrm{d}\mathbf{x}) = 1-i \mathbf{K} \cdot \mathrm{d}\mathbf{x} $$ and demonstrates that this form of $T$ meets the requirements and is generally the correct choice. Here, $\mathbf K =(\mathrm K_x,\mathrm K_y,\mathrm K_z)$ with each component hermitian.  I was hoping somebody could provide some motivation for this particular choice. Is there any reason why it is to be expected or natural? How would you come up with this form of the operator by yourself?