How can we interpret the transpose of a matrix, that represents the cordinates of a vector with respect to a basis, geometrically?
53379
I can follow the definition of the transpose algebraically, i.e. as a reflection of a matrix across its diagonal, or in terms of dual spaces, but I lack any sort of geometric understanding of the transpose, or even symmetric matrices.  For example, if I have a linear transformation, say on the plane, my intuition is to visualize it as some linear distortion of the plane via scaling and rotation. I do not know how this distortion compares to the distortion that results from applying the transpose, or what one can say if the linear transformation is symmetric. Geometrically, why might we expect orthogonal matrices to be combinations of rotations and reflections?
20240
Mariano mentioned somewhere that everyone should prove once in their life that every matrix is conjugate to its transpose.  I spent quite a bit of time on it now, and still could not prove it. At the risk of devaluing myself, might I ask someone else to show me a proof?
104599
Here $X,Y$ are vector of length $n$ and $A$ is  a $n\times n$ matrix. Suppose the covariance matrix $D(X)$ is known?
46693
I would like to ask a question about affine matrix transformations, specifically a mathematical explanation about why these two ways to interpret them are equivalent.  Suppose I have a translation matrix T (for simplicity 2D) and a rotation matrix R as follows:  $$  T= \begin{bmatrix}         1 &amp; 0 &amp; tx \\         0 &amp; 1 &amp; ty \\         0 &amp; 0 &amp; 1 \\         \end{bmatrix}  R= \begin{bmatrix}         \cos \theta &amp; -\sin \theta &amp; 0 \\         \sin \theta &amp; \cos \theta &amp; 0 \\         0 &amp; 0 &amp; 1 \\         \end{bmatrix} $$  Suppose: $$ tx = 0 , ty = 2, \theta = 45^{ \circ } $$  Now if I transform a quad applying translation and rotation to every vertex in this order: R * T * Vertex (Column vector), I obtain this:      Since both translation and rotation transforms are done around the origin, considering T first and R after. However this is equivalent to considering R first and T after modifying the quad relative to itself and not the origin:       The result is the same. Why is this equivalent?
95101
I hope it is alright to ask something like this here, I am having trouble keeping up with all the special cases and my book is being kind of vague.  I know how to do the standard method of finding diagonal matrices, but I know that triangular matrices are special and my book makes a connection, but it is hard to tell what it is.  Thanks.
50986
This question regards the problem of . Vectors and matrices will be denoted in bold.  Premises. Let $N,K$ be given integers, with $K \gg N &gt; 1$. The transpose of matrix $\mathbf{A}$ will be denoted with $\mathbf{A}^T$. Suppose the following statistical model holds $$ (*) \quad \mathbf{y} = \mathbf{Hx + n}, \quad \mathbf{n} \sim \mathcal{N}_{K}(\mathbf{0}, \mathbf{C})  $$ where $\mathbf{y} \in \mathbb{R}^{K \times 1}$ are the observables, $\mathbf{H} \in \mathbb{R}^{K \times N}$ is a known full-rank matrix, $\mathbf{x} \in \mathbb{R}^{N \times 1}$ is a deterministic vector of unknown parameters (which we want to estimate) and finally $\mathbf{n} \in \mathbb{R}^{K \times 1}$ is a disturbance vector (noise) with a known (positive definite) covariance matrix $\mathbf{C} \in \mathbb{R}^{K \times K}$. The  (ML) estimate of $\mathbf{x}$, denoted with $\hat{\mathbf{x}}_{ML}$, is given by $$ (1) \quad \hat{\mathbf{x}}_{ML} = (\mathbf{H}^T \mathbf{C^{-1}} \mathbf{H})^{-1} \mathbf{H}^T \mathbf{C}^{-1} \mathbf{y} $$ and this is also the standard formula of Generalized Linear Least Squares (GLLS). Consider the standard formula of Ordinary Least Squares (OLS) for a linear model, i.e. $$ (2) \quad \hat{\mathbf{x}}_{OLS} = (\mathbf{H}^T \mathbf{H})^{-1} \mathbf{H}^T \mathbf{y} $$ As a final note on notation, $\mathbf{I}_K$ is the $K \times K$ identity matrix and $\mathbf{O}$ is a matrix of all zeros (with appropriate dimensions). Now, for the problem at hand, assume that $\mathbf{C}^{-1} = \mathbf{I}_K + \mathbf{X}$, where $\mathbf{X} \in \mathbb{R}^{K \times K}$ is a symmetric, invertible matrix (and, for the formalism to make sense, $\mathbf{X}$ is such that $\mathbf{I}_K + \mathbf{X}$ is invertible and the inverse is positive definite). In this case, it can be proven (using matrix inversion lemma) that $$ (3) \quad (\mathbf{H}^T \mathbf{C^{-1}} \mathbf{H})^{-1} \mathbf{H}^T \mathbf{C}^{-1} = (\mathbf{H}^T \mathbf{H})^{-1} \mathbf{H}^T + \mathbf{Q} $$ where the expression for the matrix $\mathbf{Q} \in \mathbb{R}^{N \times K}$ can be found (I will omit it here). Now, finally,  Proposition 1. If $\mathbf{H}^T\mathbf{X} = \mathbf{O}_{N,K}$, then equation $(1)$ degenerates in equation $(2)$, i.e., there exists no difference between GLLS and OLS.  The proof is straigthforward and is valid even if $\mathbf{X}$ is singular. Now, my question is  Question: Can an equation similar to eq. $(3)$ (which "separates" an OLS-term from a second term) be written when $\mathbf{X}$ is a singular matrix?  I am not interested in a closed-form of $\mathbf{Q}$ when $\mathbf{X}$ is singular. But I do am interested in understanding the concept beyond that expression: what is the actual role of $\mathbf{Q}$? In which space does it operate?  I found this problem during a numerical implementation where both OLS and GLLS performed roughly the same (the actual model is $(*)$), and I cannot understand why OLS is not strictly sub-optimal. I found this slightly counter-intuitive, since you know a lot more in GLLS (you know $\mathbf{C}$ and make full use of it, why OLS does not), but this is somehow "useless" if some conditions are met. What are these conditions?  As a final note, I am rather new to the world of Least Squares, since I generally work within a ML-framework (or MMSE in other cases) and never studied the deep aspects of GLLS vs OLS, since, in my case, they are just intermediate steps during the derivation of MLE for a given problem.  If the question is, in your opinion, a bit too broad, or if there is something I am missing, could you please point me in the right direction by giving me references? Preferably well-known books written in standard notation.
71315
I need your expertise in understanding and hopefully solving the following problem:  Let $C \subset \mathbb{R}^d$, and lets assume that $C$ is not centrally symmetric, thus when computing the $MVEE(C)$ (i.e the minimum volume enclosing ellipsoid), they use the following "lifting step":  $$ C^\prime = \left\lbrace \pm \begin{bmatrix} x\\ 1 \end{bmatrix} \middle| x \in C \right\rbrace,$$ where $C^\prime$ is centrally symmetric.  It is proven that for any $\varepsilon &gt; 0$, $MVEE(C^\prime) \cap \Pi$ is a $(1 + \varepsilon)$-approximation to $MVEE(C) \times \left\lbrace 1 \right\rbrace$, where $\Pi = \left\lbrace x \in \mathbb{R^{d+1}} \middle| x_{d+1} = 1 \right\rbrace$.  Let $G \in \mathbb{d \times d}$ be an invertible matrix, let $ v \in \mathbb{R}^d$ such that the pair $(G,v)$ represents the matrix and the center of $MVEE(C)$ respectively. Let $\hat{G} \in \mathbb{R}^{(d + 1) \times (d + 1)}$ be an invertible matrix such that the pair $(\hat{G},\overset{\to}{0})$ represents the matrix and center of $MVEE(C^\prime)$ respectively.  Then, $\forall x \in MVEE(C)$ $$ \left| \left| \left[ x , 1 \right] \hat{G} \begin{bmatrix} x\\1 \end{bmatrix} \right| \right|_2 = (1 + \varepsilon) \cdot \left| \left| (x - v) G (x - v)^T \right| \right|_2 .$$  Regarding $x \not\in MVEE(C) $, then it is easy to see that the above term is not satisfied and moreover the right term is bigger or equal to the left term.  What I need to know is whether it possible the bound the right term by the left term (maybe because $MVEE(C) \times \left\lbrace 1 \right\rbrace \subseteq MVEE(C^\prime)$), i.e. $$ poly(d) \cdot \left| \left| \left[ x , 1 \right] \hat{G} \begin{bmatrix} x\\1 \end{bmatrix} \right| \right|_2 \geq (1 + \varepsilon) \cdot \left| \left| (x - v) G (x - v)^T \right| \right|_2 \ ?$$  Please Do correct if I am wrong and thanks in advance.
71019
Consider the vectorspace of all real $m \times n$ vectors and define an    inner product $\langle A,B\rangle = \operatorname{tr}(B^T   A)$.    "tr" stands for "trace"        which is the sum of the diagonal entries of a matrix.   How do you prove that $\operatorname{tr}(B^T   A)$ is indeed a inner product?  Kind regards
97848
Why is it, geometrically, that the row space is ALSO orthonormal? What exactly does the transpose LOOK like? Normally the row and column space are two separate things, but in the case of an orthonormal matrix, you have that the row space is actually the inverse of the column space, so also, what does an inverse space look like? I feel there should be a very good geometric interpretation, but I don't see it.
117422
I'm solving a problem from the book, Mathematics for 3D Game Programming and Computer Graphics, Third Edition, by Eric Lengley. The problem goes:  Let $N$ be the normal vector to a surface at a point $P$, and let $S$ and $T$ be tangent vectors at the point $P$ such that $S \times T = N$. Given an invertible 3 $\times$ 3 matrix $M$, show that $(MS) \times (MT) = (\text{det}M(M^{-1})^{T}(S \times T)$, supporting the fact that normals are correctly transformed by the inverse transpose of the matrix $M$.  The author provided a hint stating we can represent $(MS) \times (MT)$ as $$ (MS) \times (MT) = \begin{bmatrix} 0 &amp; -(MS)_{z} &amp; (MS)_{y} \\  (MS)_{z} &amp; 0 &amp; -(MS)_{x} \\  -(MS)_{y} &amp; (MS)_{x} &amp; 0  \end{bmatrix} MT $$ We then find a matrix $G$ such that $$ GU = \begin{bmatrix} 0 &amp; -(MS)_{z} &amp; (MS)_{y} \\  (MS)_{z} &amp; 0 &amp; -(MS)_{x} \\  -(MS)_{y} &amp; (MS)_{x} &amp; 0  \end{bmatrix} M $$ where $$ U = \begin{bmatrix} 0 &amp; -S_{z} &amp; S_{y} \\  S_{z} &amp; 0 &amp; -S_{x} \\  -S_{y} &amp; S_{x} &amp; 0  \end{bmatrix} $$ and show that $G = (\text{det}M)(M^{-1})^{T}$ to solve the problem. I am aware that there is an  to this problem, but I would like to solve it through the hints provided. Unfortunately, I am only able to go as far doing: $$ G = \begin{bmatrix} 0 &amp; -(MS)_{z} &amp; (MS)_{y} \\  (MS)_{z} &amp; 0 &amp; -(MS)_{x} \\  -(MS)_{y} &amp; (MS)_{x} &amp; 0  \end{bmatrix} M U^{-1} $$ At this point, I do not know how to proceed with showing that $G = (\text{det}M)(M^{-1})^{T}$. How would you proceed? I'd like to ask for hints on solving the problem.
28322
We transpose a matrix $A$ by replacing $A_{ij}$ with $A_{ji}$, for all $i$ and $j$.  However, in case $A$ has more than two dimensions (that is, it is a tensor), I don't know how to apply the transpose operation.   If A has dimensions $3\times 3 \times 8$, then what will replace $A_{ijk}$?  If $A$ has shape $3\times 3\times 8\times 8$, then what will replace $A_{ijkl}$?
120290
I'm trying to find a basis for the kernel for the following mapping:  Considering the linear transformation T: $M_{33} \rightarrow M_{33}  $ defined by $T(A) = .5(A + A^T)$. I know that this is basically asking for the basis under the condition that $T(A)=0$ which means that $A+A^T=0$ so $A^T = -A$.  I found that matrices that fit this condition are Skew Symmetric Matrices.  However, I'm not sure how to find the basis for the kernel of these matrices.
68464
I want to explicitly specify dimension of matrices in some expressions, something like   $$\boldsymbol{A}_{m \times n} \boldsymbol{B}_{n \times m} = \boldsymbol{C}_{m \times m} \, .$$  Is there any more or less standard notation for this?    While this notation is generally unambiguous, I think it become ambiguous, for example, in the following case. Suppose that I have some column or row vector which is conjugate transpose of correspondingly row or column vector. If I write it in the following way  $$\boldsymbol{A}_{n \times 1}^{\dagger} \, ,$$ we can have 2 different interpretations:   matrix $\boldsymbol{A}_{n \times 1}$ is $n \times 1$ column vector and by $\boldsymbol{A}_{n \times 1}^{\dagger}$ I'm referring to its conjugate transpose which is $1 \times n$ row vector; matrix $\boldsymbol{A}_{n \times 1}^{\dagger}$ itself is $n \times 1$ column vector which is conjugate transpose of $1 \times n$ row vector $\boldsymbol{A}$.
25250
In the following question we regard vectors in $\Bbb R^n$ as columns and define the dot product in the usual way which means that $x · y$ = $x^{t}y$  a) If A is an n × n matrix show that $A_{ij}$ = $e_i \cdot A e_j$ where $e_i$, $i =1, \dots , n$ are the standard basis vectors in $\Bbb R^n$  b) Show that a matrix $P$ is orthogonal if and only if ($Px$) · ($Py$) =$x · y$ for all $x, y$ ∈ $\Bbb R^n$  Working -   I got part (a) alright but have no idea of where to start with part (b) except to use the information that an n x n matrix $P$ is orthogonal if $P^{-1}$= $P^{t}$ and that $P^{t}$$P$=$I$, $P$$P^{t}$=$I$, where $I$ is the identity matrix. Any suggests are gracefully appreciated. Feel free to edit my question for clarity. Thank you
105770
Let us look the the matrix  $\left(\begin{array}{ccccc} a &amp; b &amp; b &amp; b &amp; b\\ b &amp; a &amp; b &amp; b &amp; b\\ b &amp; b &amp; a &amp; b &amp; b\\ b &amp; b &amp; b &amp; a &amp; b\\ b &amp; b &amp; b &amp; b &amp; a \end{array}\right)$  It has one value, $a$, on the main diagonal, and another value, $b$ everywhere else. Let us assume that we are over a ring and that $a$ is invertible.  I wish to find the determinant of every $n\times n$ matrix of this form ($a$ on the diagonal, $b$ everywhere else). Using row and column operations I have managed to transform the matrix to upper-triangular form and found formula for specific cases. Generalizing it I got to the following formula:  $\det(A) = a\left(a-b\right)^{n-2}\left(a+\left(n-2\right)b-\frac{\left(n-1\right)b^{2}}{a}\right)$  I think I can prove it with row-operations in the general case with a little patience.  However, I'm wondering if there is a "smart" way of getting to this formula that I'm missing and if there is a nicer representation of it. Also, what can be said when $a$ is not invertible? (esp. the case where we are over a field and $a=0$).
90841
How do you prove the following fact about the transpose of a product of matrices? Also can you give some intuition as to why it is so.  $(AB)^T = B^TA^T$
122053
Find a basis for the vector space of symmetric matrices with an order of $n \times n$  This is my thought: by definition of symmetry, $a_{i,j}=a_{j,i}$. Therefore, the basis should consist ${n^2-n} \over 2$ matrices to determine each symmetric pair. In addition, it should also consist $n$ matrices to determine each term in the diagonal.    Therefore, the dimension of the vector space is ${n^2+n} \over 2$.   It's not hard to write down the above mathematically (in case it's true).    Two questions:     Am I right? Is that the desired basis? Is there a more efficent alternative to reprsent the basis?     Thanks!