Quite often in published research we see researchers apply log transformation to their data, and some claim that this makes the data closer to normal distribution. My questions are:  Mathematically, why this might be true? In particular, it would be great if you could illustrate how log transformation brings the characteristics of the sample or data (such as: dispersion, skewness, etc. ) closer to those of a normal.  Does it always bring data closer to normal (Or, are there situations in which it fails)?
120617
I have a highly right skewed data set with a large range of values (from 1 ~ 10^6) (can't share the actual data for work related reasons).   When I plot the log of the data instead, the distribution looks a lot more like a normal distribution.   Have I stumbled on a meaningful insight in the data set, or is just a general property of the log transform that it brings the distribution closer to normal?
98537
I get a fan-shaped scatter plot of the relation between two different quantitative variables:    I am trying to fit a linear model for this relation. I think I should apply some kind of transformation to the variables in order to unify the ascent variance in the relation before fitting a linear regression model, but I can't find the way to do it. Or maybe, there is a better model to use in these cases, I can't either find it.  I have tried rlm, but the residuals still have heteroscedasticity. I have also tried to apply a SD ratio calculated from all the y of each x and other similar erratic approaches.  My questions:   Is there any typical way of fitting a model for a fan-shaped relation or a typical model to use in these cases? Is there any typical transformation that could be applied to the variables in order to reduce its variance?
76263
neatly explains how to interpret a log transformed predictor in OLS. Does the interpretation change if there are 0s in the data and the transformation becomes log(1 + x) instead?   Some authors (e.g. Fox and Weisberg 2011) recommend adding a start (i.e. a positive constant) if a log transformation is necessary to correct skewness and improve symmetry, but the data contains zeros.   Consider a variation of the Ornstein example in CAR (p. 303):   require(car) data(Ornstein) boxplot(Ornstein$interlocks, horizontal = T)      The data is clearly right skewed, and contains 0s.   summary(powerTransform(1 + Ornstein$interlocks)) ## bcPower Transformation to Normality  ##  ##                         Est.Power Std.Err. Wald Lower Bound Wald Upper Bound ## 1 + Ornstein$interlocks    0.1248    0.053           0.0209           0.2287 ##  ## Likelihood ratio tests about transformation parameters ##                              LRT df      pval ## LR test, lambda = (0)   5.502335  1 0.0189911 ## LR test, lambda = (1) 262.431991  1 0.0000000   The powerTransform() function suggests that a log(1 + x) transformation here could be useful.   boxplot(log(1 + Ornstein$interlocks), horizontal = T)     As you can see, symmetry is indeed improved.   Question: If this transformed variable were to be included in an OLS regression as an IV, would the coefficient estimates still have the usual interpretation of log transformed variables?
50007
I've read that "If the error distribution is significantly non-normal, confidence intervals may be too wide or too narrow" (). So, can anyone elaborate on this? When are the confidence intervals narrow and when are they wide? Does it have anything to do with skewness?
107510
I have two dependent variables y1 and y2 with highly skewed distributions. In order to do ANOVA, I was trying to transform the data to normality.      y1 is a proportion expressed as percentage.  For y1 I tried log of reflected data. log10(k - y1), where k = max(y1) + 1.   y1 being a proportion, I also tried a logit transformation. log(y1/(100 - y1)). However, there were many Inf values.  For y2 I tried log10 of the data.   The residuals of the ANOVA model in both the cases are heavy tailed distributions.      Should I transform the data for ANOVA? If so, what is the most appropriate strategy?
72982
I am trying to calculate a mean and standard deviation from 2 percentiles for a lognormal distribution.   I was successful in performing the calculation for a normal distribution using X = mean + sd * Z and solving for mean and sd.  I think I am missing an equation when I try to do the same thing for a lognormal distribution. I looked at  and trying to use ln(X) = mean + sd * Z but I'm getting confused whether the mean and sd in this case are for the normal distribution or the lognormal.  Which equations should I be using? and will I need more than 2 percentile to solve the calculations?
74312
uses generalised linear models (both binomial and negative binomial error distributions) to analyse data. But then in the statistical analysis section of the methods, there is this statement:     ...and second by modelling the presence data using Logistic Regression   Models, and the foraging time data using a Generalized Linear Model   (GLM). A negative binomial distribution with a log link function was   used to model the foraging time data (Welsh et al. 1996) and model   adequacy was verified by examination of resi- duals (McCullagh &amp;   Nelder 1989). Shapiro–Wilk or Kolmogorov–Smirnov tests were used to   test for normality depending on sample size; data were log-transformed   before analyses to adhere to normality.   If they assume binomial and negative binomial error distributions, then surely they shouldn't be checking for normality of residuals?
83880
I have the mean and median values for a sample drawn from a lognormal distribution. Note that this is not the mean and median of the logs of the variable, though I can of course calculate the logs of the mean and median. Is there a closed form solution for μ and σ from this information? If there is only a numeric solution, could you tell me how to find it, ideally with R?  I note that this question has been answered for deriving μ and σ from the sample mean and the sample variance, here:   However, I do not have the sample variance, only the mean and median.  If there is no closed-form or straightforward numeric solution, I'd like to know if using the logs of the sample mean and median, or some transform of them, will provide a reasonable answer for a large sample (in the hundreds of millions).
16111
I am always struggling with normality testing for quantitative predictors (no factors) and transforming them to normality.    If I am running a GLMM and my predictors are really non-normal, should I  transform them as well to try to make them normally distributed?  I know that this is important for the response variable but what should be done with predictors?   P.S.: I really could not find a similar question.
3668
I have a question regarding interpretation of regression results based on data where I have some negative values. Since the residuals were positively skewed I needed to log-transform my data, and because of negative values my log transformation looked like $\ln(Y-\min(Y)+1)$. Normally, if we have $\ln$ in both sides of the equation, then our $b$ coefficient means a percentage change of DV for 1 % change of IV. But I have big doubts whether the same is valid for such log-transformed data, where I added a constant to my DV in order to make the minimum value equal to 1. I can't also find any literature on this topic. Does anyone know for sure what would be the correct interpretation in such a situation?
105550
Recently I came across the following result in a paper, which I can't seem to find right.  Anyway the authors had cited  by Johnson &amp; Kotz, page 20 as the source. Since I do not have access to this book I am unable to check on the derivation of this result. I was hoping someone could shed some light on how to prove this.  Let $X$ and $Y$ denote a pair of bivariate log normal random variables with their means and variances given by the ordered pairs $(\zeta_{X}, \eta_{X})$ and $(\zeta_{Y}, \eta_{Y})$ and with correlation coefficient $\rho_{(X, Y)}$, derived from the bivariate normal with marginal distributions having parameters $(\xi_{X}, \nu_{X}^2)$ and $(\xi_{Y}, \nu_{Y}^2)$ and correlation coefficient $\rho_{N}$.  The expression for $\rho_{(X, Y)}$ is given as \begin{equation} \rho_{(X, Y)} = \dfrac{\mathrm e^{(\rho_{N} \nu_{X} \nu_{Y})} - 1}{\sqrt {(\mathrm e^{\nu_{X}^2} - 1)(\mathrm e^{\nu_{Y}^2} - 1)}} \end{equation}  The below is a manipulation I came up with, any error in that would be completely mine.  Upon rearranging we get \begin{equation}\begin{aligned} \rho_N &amp;= \dfrac{\ln (1 + \rho_{(X, Y)} \sqrt {(\mathrm e^{\nu_{X}^2} - 1)(\mathrm e^{\nu_{Y}^2} - 1)})}{\nu_{X}\nu_{Y}}\\ &amp;= \dfrac{\ln (1 + \rho_{(X, Y)} \frac{\eta_{X}\eta_{Y}}{\zeta{X}\zeta{Y}})}{\sqrt {\ln ((\frac{\eta_{X}}{\zeta_{X}})^2 + 1)\ln ((\frac{\eta_{Y}}{\zeta_{Y}})^2 + 1)}}. \end{aligned} \end{equation}  I am very interested in learning if this result would hold true between $X$ and $Y$'s correlations if we had a quad-variate situation as in there were 2 more random variables which are also correlated with $X$ and $Y$ would this result still be true ? I suspect it should be, but I am newbie and due to lack of a credible source I was wondering if some one would be kind to confirm it.
49240
I'm working on a VAR model and am doing a log transformation of the raw data.   x = log(x)   After differencing, running various tests and running a VAR(2) model, I add the predicted differences cumulatively to the original time series (logged) like this:  prd.diffs &lt;- as.ts(prd$mean$d.x1) U &lt;- tail(undiff,2) cumu &lt;- cumsum(prd.diffs) + U[1]   However, while the output now looks as it should, it is still in logged values.  My question is as follows:  How does one return the logged output to original values? I'm under the impression that using exp() will not give the correct results, and having tried some variations it hasn't worked for me.   Does the fact that it is a VAR($p$) model (and possibly what lag it has) influence the method, and how should that be implemented to get the correct output?  P.S. I realise that the above code does not constitute a replicable example as I didn't deem that necessary for the question, but should that be desired I can add some more code.
59670
Because of larger values of variables, I did a log- transformation in my dataset. Now I want to give a descriptive table regarding my variables like mean, max, min, median, skewness, kurtosis. Can I give standard measures like arithmetic mean or should I give geometric mean? In terms of skewness or kurtosis, do I need a transformation?
75298
To understand what I mean, let's use two well-known distributions: the normal and lognormal ones.  From the dataset point of view, if you take normally-distributed data and take their exponential, you get lognormally-distributed data (norm -exp-> lognorm).  So the "idea" was: why not using the PDF directy and transform the normal PDF to the lognormal one by taking the exponential of its x-axis. The reasoning was that this x-axis represents my data: taking the exponential of these data or the exponential of the x-axis of the corresponding PDF should do the same.  But I did not even bother to do it: mathematically, this is not how one can transform the normal PDF to the lognormal one!  What I don't understand is why just taking the exponential of the x-axis of the PDF does not work. I mean, not mathematically (I get that there is a $1/x$ in the lognormal PDF!), but what is conceptually wrong with "just stretching" the axis?
90723
I have two distributions which are derived from 2 separate sets of data. These distributions are not normal, and it is not clear at this point if they belong to any family of known pdfs (they are not symmetric either). Given a data point, I need to decide which distribution it is most likely to belong to. If these were normal distributions, I could do usual parametric tests and go from there, but in this case I am not sure how to proceed. I searched around a bit, but I couldn't find anything, probably because I am not using the right keywords. Any help is much appreciated.  Edit to clarify: I should have mentioned that it was univariate. I might as well explain the actual problem here as well. We have time-spent data for users on websites. We also have information on whether a user liked or disliked some of those sites (about 4% of all the sites). So we know the distribution of time spent for like and dislike. An obvious question, then, is for a random user who spent x amount of time, are they more likely to have liked the page or dislike the page. Our time spent information is based on seconds, so the distributions are really discreet, but in real life, they are continuous.
3493
I am working with a small dataset (21 observations) and have the following normal QQ plot in R:     Seeing that the plot does not support normality, what could I infer about the underlying distribution? It seems to me that a distribution more skewed to the right would be a better fit, is that right? Also, what other conclusions can we draw from the data?
40997
I have come across 3 questions on the title subject.   Why is it necessary to do a normality test? To check if data is imbalanced or not? Are these 4 methods of checking if the data follows normal distribution criteria both applicable to numerical and categorical variable? I am trying to check if the data follows normal distribution by following 4 methods.   Checking Distribution Drawing Box Plot Drawing QQ Plot Use skewness, kurtosis criteria  Skewness for Normal Dist is 0, Kurtosis for Normal Dist is 3. Is there a certain bound that I can use to guarantee that the data is normally distributed? (such as, 0 +/- 1 OR 3 +/- 1)