Let's say we have a white noise process $x(t)$ such that:  $E(X(t)X(t+\tau))=N\delta(\tau)$  $E(X(t))=0$  In particular, with $\tau=0$, $E(X(t)X(t))=E(X^2(t))$ is infinite.  Now, I want $X(t)$ at each time $t$ to have a normal distribution of 0 mean and $\sigma^2$ variance. That is:  $E(X^2(t))=\sigma^2$  This is not consistent. I guess the expectations mean something different in both cases, but I don't find an explanation. This prevents me from moving ahead in a study of the mean, variance and autocorrelation of a process $y(t)$ defined as $y(t)=1$ if $a&lt;x(t)&lt;b$ and 0 otherwise.
123644
What is meant by a continuous-time white noise process?  In a discussion following a  a few months ago, I stated that as an engineer, I am used to thinking  of a continuous-time wide-sense-stationary white noise process  $\{X(t) \colon -\infty &lt; t &lt; \infty\}$ as a zero-mean process having autocorrelation function $R_X(\tau) = E[X(t)X(t+\tau)] =  \sigma^2\delta(\tau)$ where $\delta(\tau)$ is the Dirac delta or impulse, and power spectral density $S_X(f) = \sigma^2, -\infty &lt; f &lt; \infty$. At that time, several people with very high reputation on Math.SE assured me that this was an unduly restrictive notion, and that no difficulties arise if one takes the autocorrelation function to be $$E[X(t)X(t+\tau)] = \begin{cases}\sigma^2, &amp; \tau = 0,\\ 0, &amp; \tau \neq 0. \end{cases}$$  What engineers like to call a white noise process is a hypothetical beast that is never observed directly in any physical system, but  which can be used to account for the fact that the output of a linear time-invariant system whose input is thermal noise is well-modeled by a wide-sense-stationary Gaussian process whose power spectral density is proportional to $|H(f)|^2$ where $H(f)$ is the transfer function of the linear system. Standard second-order random process theory says that the input and output power spectral densities $S_X(f)$ nd $S_Y(f)$ are related as $$S_Y(f) = S_X(f)|H(f)|^2.$$ Thus, pretending that thermal noise is a white Gaussian noise process in the engineering sense and pretending that the second-order theory  extends to white noise processes (even though their variance is  not finite) allows us to get to the result that the output power spectral density is proportional to $|H(f)|^2$.  My query about the definition of a white noise process is occasioned by a  regarding the variance of a random variable $Y$ defined as $$Y = \int_0^T h(t)X(t)\ \mathrm dt$$ where $\{X(t)\}$ is a white Gaussian noise process. The  leads to $$\operatorname{var}(Y) = \sigma^2 \int_0^T |h(t)|^2\ \mathrm dt$$ (as I pointed out in a comment on the answer) if the autocorrelation function is taken to be $R_X(\tau) = \sigma^2\delta(\tau)$ (the engineering definition).  However, the OP on that question specified $R_X(0) = \sigma^2$, not $\sigma^2\delta(\tau)$, that is, the definition accepted by mathematicians.  For this autocorrelation function, the variance is $$\int_0^T \int_0^T E[X(t)X(s)]h(t)h(s)\mathrm dt\mathrm ds = 0$$ since the integrand is nonzero only on a set of measure $0$.  So, what is the variance of the random variable $Y$? and what do readers of Math.SE understand by the phrase white noise process?  Perhaps this question should be converted to a Community wiki?
74953
Given that a series {$x_t$} where $x_t=\sin(t+U)$, $U$ has uniform distrubution on $(0,2\pi)$. Is {$x_t$} second-order stationary? I know here we need to show $E(x_t)$ is constant and $cov(x_{t+s},x_{t})$ only depends on $s$. Now $E(U)=\pi$, $\mathrm{Var}(U)=\frac{\pi^2}{3}$. My problem is how to find the expection value and covariance of $\sin$ function? Indeed $\mathrm{Cov}(x_{t+s},x_{t})$ = $E(x_{t+s}x_{t})-E(x_{t+s})E(x_{t})$, so the probelm is expected value.. Here I think it is hard to find the p.d.f of this $\sin$ function. Can anyone help me with that? Thank you in advance!
105744
Let $X, Y$ be independent standard normal random variables.  We already know that $X+Y$ is normal with mean 0 and variance 2. However, I am trying to prove this result in a slightly different way than usual, i.e., using convolutions.  We can address the problem in the following way: the sum of the two variables has a density $h(x)$ given by the convolution product of the densities of the two variables, say $f$ and $g$, so that   $$ h(x) = \int_{-\infty}^{\infty} f(x-y)g(y)dy. $$  However,  $$ f(y) = g(y) = \frac{1}{\sqrt {2\pi}}\exp(-y^2/2). $$  Hence now the integral becomes:  $$ h(x) =\frac{1}{2\pi}\int_{-\infty}^{\infty}\exp(-(x-y)^2/2)\exp(-y^2/2)dy. $$  However, I am puzzled here. How should we go on from here? I cannot see how to transform this integral into the integral of a normal r.v of type $N(0,2)$. Thanks in advance for your help.
90516
I'm trying to understand the basic concept of random processes.I already understood that a continuous-time random process is defined by X(¥,t) where ¥ is each element in the Sample Space and t is the time index.     By this definition, we can see the random process as an ensemble of sample realizations ( time-functions defined for each outcome in the Sample Space) or as a set of random variables ( each one defined for a specific time t ). In my mind, the value of the time-function ( sample realizations ) X(¥1,t) reflects the changing value of the probability of outcome ¥1 being choosen in the sample space, is that correct ?    For example, we have a boy B in the sample space representing outcome ¥1.Having this outcome ¥1 at time t=0 equals 30%.As the time passes, the sample spaces gets more crowded and at time t=5, the probabily of choosing boy B equals 20%.   Can i think that the time-function X(¥1,t) will equal 0,3 for t=0 and 0,2 for t=5 ? Does X(¥1,0)=0,3      and     X(¥1,5)=0,2         ?  Thanks in advance
95283
Let $Y_0$ and $Y_1$ be both identically (not necessarily independent)    normally distributed with mean $\mu$ and $\sigma^2$, i.e., $Y_i \sim  N(\mu, \sigma^2)$ for $i = 1, 2$. Let $\rho$ denote the correlation    between $Y_0$ and $Y_1$. Show that the conditional expectation    $E[Y_1|Y_0 = y_0] = (1-\rho)\mu + \rho y_0$.   I know that in general if $(X, Y)$ are bivariate normally distributed, then $E[X|Y] = E[X] + \rho \frac{\sigma_X}{\sigma_Y}(Y-E[Y])$. But in this question, there is no indication that $Y_0$ and $Y_1$ are jointly normally distributed; they are simply marginally normally distributed. If they were joint normal, the result falls out simply from the result I just mentioned. Any ideas on how I can prove the result?
57133
In a statistics book i'm reading, it is postulated that asymptotic normality of an estimator implies consistency. That is $$ \hat{\theta}_n \stackrel{as}{\sim} \mathcal{N}(\theta_0, \frac{1}{n}\sigma(\theta_0)) \Rightarrow P_{\theta_0}(|\hat{\theta}_n - \theta_0|&gt;\epsilon ) \to 0 $$ when $n\to\infty$, for all $\theta_0 \in \Theta$ and $\epsilon&gt;0$.  I am trying to prove this, but i can't seem to get a breakthrough.   If anyone could shed some light on how this is proven, i would be very grateful.
32239
If two continuous random variables have exactly the same expected value and variance, do they always have the same distribution?
96005
Let $X_1,\ldots,X_n$ be i.i.d. $B(1,\theta)$ random variables, $0&lt;\theta&lt;1$. Then, as an estimator $\theta$, check if $T(X_1,\ldots,X_n)= \dfrac{\sum_{i=1}^n X_i +\sqrt{n}/2}{n+\sqrt{n}}$ is consistent and/or unbiased.  $$T=\frac{\frac{1}{n}\sum_{i=1}^n X_i +\frac{1}{2\sqrt{n}}}{1+\frac{1}{\sqrt{n}}}$$  $$T=\frac{\bar{X} +\frac{1}{2\sqrt{n}}}{1+\frac{1}{\sqrt{n}}}$$  $$E(T)=\frac{E(\bar{X} +\frac{1}{2\sqrt{n}})}{1+\frac{1}{\sqrt{n}}}=\dfrac{\theta +\frac{1}{2\sqrt{n}}}{1+\frac{1}{\sqrt{n}}}.$$  So, $T$ is biased.  Consistency: $\lim\limits_{n \to\infty}E(T)=\lim\limits_{n \to\infty}\dfrac{E(\theta +\frac{1}{2\sqrt{n}})}{1+\frac{1}{\sqrt{n}}}$  So $T$ is consistent.   But The given answer is "neither unbiased nor consistent"  Where did I go wrong ? Please advise.
72308
I've been reading over this , trying to make sense of how the mean and variance of a gaussian conditional was derived. I've come to accept that unless I allocate a dozen or so hours to refreshing my linear algebra knowledge, it's out of my reach for the time being. that being said, I'm looking for a conceptual explanation for that these equations represent: $$\mu_{1|2} = \mu_1 + \Sigma_{1,2} * \Sigma^{-1}_{2,2}(x_2 - \mu_2)$$ I read the first as &quot;Take $\mu1$ and augment it by some factor, which is the covariance scaled by the precision (measure of how closely $X_2$ is clustered about $\mu_2$, maybe?) and projected onto the distance of the specific $x_2$ from $mu_2$.&quot; $$\Sigma_{1|2} = \Sigma_{1,1} - \Sigma_{1,2} * \Sigma^{-1}_{2,2} * \Sigma_{1,2}$$ I read the second as, &quot;take the variance about $\mu_1$ and subtract some factor, which is covariance squared scaled by the precision about $x_2$.&quot; In either case, the precision $\Sigma^{-1}_{2,2}$ seems to be playing a really important role. A few questions:  Am I right to treat precision as a measure of how closely observations are clustered about the expectation? Why is the covariance squared in the latter equation? (Is there a geometric interpretation?) So far, I've been treating $\Sigma_{1,2} * \Sigma^{-1}_{2,2}$ as a ratio, (a/b), and so this ratio acts to scale the (second) $\Sigma_{1,2}$, essentially accounting for/damping the effect of the covariance; I don't know if this is valid. Anything else you'd like to add/clarify?
42977
I'm sorry if this is a duplicate, but I can't seem to find the answer to this.  If $Z_t$ is a white noise process and $X_t$ satisfies  $$ \phi(B) X_t = \theta(B) Z_t $$  (where $B$ is the lag operator), then when is $X_t$ stationary?  If $\theta(z)$ and $\phi(z)$ have no common roots and $\phi(z)$ has a root with $|z| = 1 $ then $X_t$ is necessarily non-stationary [Brockwell and Davis, Remark 3 in Chapter 3].  I also know that a moving average of a stationary process with absolutely summable coefficients is also stationary [Brockwell an Davis, Proposition 3.1.2]. So if $\phi(z)$ has no roots in with $|z| \leq 1$ then the Maclaurin series for $1/\phi(z)$ has radius of convergence greater than 1. Therefore $X_t$ is a moving average of $\theta(B) Z_t$ with absolutely summable coefficients, and hence stationary.  By stationarity, I mean weak (=second order) stationarity.  The lecturer has a habit of putting this on the final, so I'd like to know if there's a general theorem governing it.  From a past paper:
105717
Let $X_1$ and $X_2$ be independent Normal random variables with mean $\mu_1$ and $\mu_2$, and variances $\sigma_1$ and $\sigma_2$. Let $Y = X_2-X_1 + c$, where $c$ is a constant. For notational simplicity, let $Y = X' + c$, where $X' = X_2-X_1$ The moment generating function of a Normal random variable $X_n\sim N(μ_n,σ_n^2)$ is $$M_{X_n } (t)=\text{exp}\Big(μ_n t+\frac{σ_n^2 t^2}{2}\Big)$$ Then, $$M_{X'} (t)=\text{exp}\Big[t(\mu_1+\mu_2)+\frac{t^2}{2}(\sigma^2_1+\sigma^2_2)\Big]$$ Thus, by the uniqueness property of moment generating function, $X'$ is said to follow a Normal distribution with mean $(\mu_1+\mu_2)$ and variance $(\sigma^2_1+\sigma^2_2)$. How do I proceed from here to derive the following:  First, I want to establish that $Y$ is also a Normal random variable using the moment generating function. The constant $c$ bugs me, but I cannot ignore it. As per my understanding, $Y\sim N\big(c+\mu_1+\mu_2,\sigma^2_1+\sigma^2_2\big)$ (and I could be completely wrong). But I don't understand how to apply the moment generating function to prove that.  Secondly, I want to derive $P(Y \leq a)$, where $a$ is a constant. If I am not mistaken, $P(Y \leq a)$ can be written as $$P(Y \leq a)=\Phi\Bigg(\frac{a-X_2-X_1 + c}{\sqrt{2(\sigma^2_1+\sigma^2_2)}}\Bigg).$$ But how do I arrive at that.   Any help is high solicited and appreciated.
99189
I am considering a stochastic variable X that is standard normal distributed i.e. $$ F_X(x) = \int_{-\infty}^x\frac{1}{\sqrt{2\pi}}e^{-\frac{t^2}{2}}dt $$ How do I find out the distribution of $X^2$?  /Gus
34234
I asked this question on mathoverflow yesterday and got a suggestion to try it here. I apologize if this is an easy question, but I can't seem to find an answer anywhere.  I'm trying to duplicate a macroeconomic paper that uses MCMC analysis to derive time series and parameter values. For the variance of a white noise term the authors of the original paper choose an inverse gamma prior of mean 0.5 and having a 90% confidence interval between 0.21 and 0.79.  I'm using Dynare to run my estimation and Dynare only accepts these kinds of parameters in terms of standard error. Therefore, my question basically amounts to "how do I find the square root of an inverse gamma distribution"?  In case it's not clear what I'm asking, here's how the model equation appears in the paper:  $a_t=\rho_a\times{a_{t-1}}+\epsilon_a;\hspace{20 pt}\epsilon_a\sim{N}(0,\sigma_a)$   $a_t$ follows a simple AR(1) process and $\epsilon_a$ is meant to be white noise with mean 0 and variance $\sigma_a$. The prior for $\sigma_a$ is supposed to follow an inverse gamma distribution with the mean and confidence interval above. My question then is how can I find the "square root of the distribution" so I can write an expression for the standard error of $\epsilon_a$, instead of its variance?  If it's not possible to come up with a simple expression for this, would you have any suggestions for roughly equivalent specifications of the prior?  Thanks in advance
72864
The  that is defined as $Y_{t} = Y_{t-1} + e_t$, where $e_t$ is white noise. Denotes that the current position is the sum of the previous position + an unpredicted term. You can prove that the mean function $\mu_t = 0 $, since $$E(Y_{t}) = E(e_1+ e_2+ ... +e_t) = E(e_1) + E(e_2) +... +E(e_t) = 0 + 0 + \cdots + 0$$ But why is it that the variance increases linearly with time? Does this have something to do with it's not “pure” random, since the new position is very correlated with the previous one? EDIT: Now I have a much better understanding by visualizing a big sample of random walks, and here we can easily observe that the overall variance does increase over time,  and the mean is as expected around zero. Maybe this was trivial after all, since in the very early stages of the time series (compare time = 10 with time = 100) the random walkers have not had yet the time to explore as much.
113041
Last minute studying please tell me how to:     Prove that if the expected conditional expected value of the random variable $X$ given the random variable $Y$ - denoted by $E(X\mid Y)$ - is constant then $X, Y$ are uncorrelated.
92502
As I understand, we can get correlation by normalizing covariance using the equation  $$\rho_{i,j}=\frac{cov(X_i, X_j)}{\sigma_i \sigma_j}$$  where $\sigma_i=\sqrt{E[(X_i-\mu_i)^2]}$ is the standard deviation of $X_i$.  My concern is what if the standard deviation equals zero? Is there any condition that guarantees it cannot be zero?  Thanks.
3642
Would someone be able to offer a layman's explanation of what is means when two stochastic processes are a Modification of each other and when they are Indistinguishable?  My Stochastic Analysis notes define the following:  a) $\textit{The stochastic processes X and Y are called a modification of each other if}$ \begin{align} P(X(t) = Y(t)) = 1 \quad \textit{for all t $\in$ I} \end{align}  b) $\textit{The stochastic processes X and Y are called indistinguishable}$ \begin{align} P(X(t) = Y(t) \,\,\,\textit{for all $t \in I$}) = 1 \end{align}  I interpret this to mean that:  (a) for each $t$ in $I$ the probability that the processes are equal is equal to 1.  (b) the probability that the entire path of the processes X is equal to the entire path of the process Y, is equal to 1.  I don't understand the difference between the two statements. The first seems to examine the processes at each point. The second examines the paths of the processes (which are made up of the points!).  Is anyone able to provide a motivational explanation as to why (a) does not mean that the paths are the same? I find it counter intuitive since if the processes are equal at each point then surely their paths are the same?  The classic example I've been given which (apparently) shows the two processes to be a modification and not indistinguishable is:  $\textbf{Example:}$ Let $\Omega = [0,\infty), \mathcal{A} = \mathcal{B}([0,\infty))$ and $P$ be a probability measure on $\mathcal{A}$ which has a density. Define two stochastic processes $(X(t): t \ge 0)$ and $(Y(t): t \ge 0)$ by \begin{align} X(t)(\omega) =  \begin{cases}  1, \text{ if $t = \omega$},\\ 0, \text{ otherwise} \end{cases} \quad Y(t)(\omega) = 0 \quad \text{for all $t \ge 0$ and all $\omega \in \Omega$.} \end{align} Then $X$ and $Y$ are modifications of each other but $X$ and $Y$ are not indistinguishable.   $\textbf{Question:}$ With regard to the example above if $t = 0$ and $\omega = 0$ then $X(0)(0)= 1$ but $Y(0)(0) = 0$, then how can their probability be equal to 1 at this point? Hence how can they be a modification of each other?  Many thanks,  John