I'm currently writing a program that checks for the multiplicity of a number by a factor of 10. However, this program is violated by what I assume is a double-precision error in python. I have attached an example below. # correct representation in line with expectation &gt;&gt; 71885 / 71.885 = 1000.0  # this fails, when we should be reporting 10^6 (1000000) &gt;&gt; 310888679 / 310.888679 = 999999.99999999
27
Consider the following code:   0.1 + 0.2 == 0.3  -&gt;  false   0.1 + 0.2         -&gt;  0.30000000000000004   Why do these inaccuracies happen?
60608
I have the value 25.00 in a float, but when I print it on screen it is 25.0000000. How can I display the value with only two decimal places?
50690
I am writing an algorithm where, given a model, I compute likelihoods for a list of datasets and then need to normalize (to probability) each one of the likelihood. So something like [0.00043, 0.00004, 0.00321] might be converted to may be like [0.2, 0.03, 0.77].   My problem is that the log likelihoods, I am working with, are quite small (for instance, in log space, the values are like -269647.432, -231444.981 etc). In my C++ code, when I try to add two of them (by taking their exponent) I get an answer of "Inf". I tried to add them in log-space , but again stumbled upon the same problem.  Can anybody share his/her expert opinion on this?
53630
If I run this code:  select 99999 * 9999   result = 999890001  If I run this code:  select 9999923 * 99999999   result:     Msg 8115, Level 16, State 2, Line 1   Arithmetic overflow error converting expression to data type int.   And If I run the following code:  select 922337299987987689745893 * 999999994564   result = 922337294974162127011191918541325652  Why am I getting this arithmetic overflow error within a specific range?
41282
Here's my code:  x = 1.0 y = 100000.0     print x/y   My quotient displays as 1.00000e-05.  Is there any way to suppress scientific notation and make it display as 0.00001? I'm going to use the result as a string.
67402
Official Javadoc  that Math.floor() returns a double that is "equal to a mathematical integer", but then why shouldn't it return an int?
80740
Using 5000-digit precision in PARI/GP, I discovered that the fractional part of $(\sqrt{2}+\sqrt{3})^{2008}$ is extremely small, less than $10^{-999}$.  Is there a simple explanation for this fact ?  This looks like a Pisot number issue (similar questions have already been studied on MSE, see for example ), but it’s a more complicated situation.   Related :
32263
Can you please tell me how much is (-2) % 5? According to my Python interpreter is 3, but do you have a wise explanation for this?  I've read that in some languages the result can be machine-dependent, but I'm not sure though.
40771
I know that I should use the definition of an odd integer ($2k+1$), but that's about it.  Thanks in advance!
9476
Imagine you have a large array of floating point numbers, of all kinds of sizes. What is the most correct way to calculate the sum, with the least error? For example, when the array looks like this:  [1.0, 1e-10, 1e-10, ... 1e-10.0]   and you add up from left to right with a simple loop, like  sum = 0 numbers.each do |val|     sum += val end   whenever you add up the smaller numbers might fall below the precision threshold so the error gets bigger and bigger. As far as I know the best way is to sort the array and start adding up numbers from lowest to highest, but I am wondering if there is an even better way (faster, more precise)?  EDIT: Thanks for the answer, I now have a working code that perfectly sums up double values in Java. It is a straight port from the Python post of the winning answer. The solution passes all of my unit tests. (A longer but optimized version of this is available here )  /**  * Adds up numbers in an array with perfect precision, and in O(n).  *   * @see http://code.activestate.com/recipes/393090/  */ public class Summarizer {      /**      * Perfectly sums up numbers, without rounding errors (if at all possible).      *       * @param values      *            The values to sum up.      * @return The sum.      */     public static double msum(double... values) {         List&lt;Double&gt; partials = new ArrayList&lt;Double&gt;();         for (double x : values) {             int i = 0;             for (double y : partials) {                 if (Math.abs(x) &lt; Math.abs(y)) {                     double tmp = x;                     x = y;                     y = tmp;                 }                 double hi = x + y;                 double lo = y - (hi - x);                 if (lo != 0.0) {                     partials.set(i, lo);                     ++i;                 }                 x = hi;             }             if (i &lt; partials.size()) {                 partials.set(i, x);                 partials.subList(i + 1, partials.size()).clear();             } else {                 partials.add(x);             }         }         return sum(partials);     }      /**      * Sums up the rest of the partial numbers which cannot be summed up without      * loss of precision.      */     public static double sum(Collection&lt;Double&gt; values) {         double s = 0.0;         for (Double d : values) {             s += d;         }         return s;     } }
95140
I have the number 0.101 in binary. I want to round it to 2 places after the radix point using the algorithm rounding to the nearest. Here is how I do it.  If I want to have two places, then I will have one of the two possible resulting numbers: 0.10 or 0.11. Let's see how by how much each is different from the original:   1) 0.101 - 0.10 = 0.11  2) 0.101 - 0.11 = 0.10  The second one gives us less difference then the first, so using the algorithm rounding to the nearest, 0.101 should be rounded to 0.11. Is this how the algorithm works?
27648
double r = 11.631; double theta = 21.4;   In the debugger, these are shown as 11.631000000000000 and 21.399999618530273.  How can I avoid this?
42817
Possible Duplicate:          What's a way to see if a number is a ?   bool IsPerfectSquare(long input) {    // TODO }   I'm using C# but this is language agnostic.  Bonus points for clarity and simplicity (this isn't meant to be code-golf).    Edit: This got much more complex than I expected! It turns out the problems with double precision manifest themselves a couple ways. First, Math.Sqrt takes a double which can't precisely hold a long (thanks Jon).   Second, a double's precision will lose small values ( .000...00001) when you have a huge, near perfect square. e.g., my implementation failed this test for Math.Pow(10,18)+1 (mine reported true).
2201
I want a to be rounded to 13.95.  &gt;&gt;&gt; a 13.949999999999999 &gt;&gt;&gt; round(a, 2) 13.949999999999999   The  function does not work the way I expected.
97059
How is it that Java's BigDecimal can be this painful?  Double d = 13.3D;  BigDecimal bd1 = new BigDecimal(d); BigDecimal bd2 = new BigDecimal(String.valueOf(d));   System.out.println("RESULT 1: "+bd1.toString()); System.out.println("RESULT 2: "+bd2.toString());  RESULT 1: 13.300000000000000710542735760100185871124267578125 RESULT 2: 13.3   Is there any situation where Result 1 would be desired? I know that Java 1.5 changed the toString() method but was this the intended consequence?   Also I realise that BigDecimal has doubleValue() etc, but the library that I am working with helpfully uses a toString() and I can't change that :-(  Cheers.
73179
On , $N_A = 6.022\;141\;29 \times 10^{23} \text{ mol}^{-1}$ has 9 significant figures and a standard uncertainty of $0.000\;000\;27 \times 10^{23} \text{ mol}^{-1}$.  First, can it be written as the following?  $$ N_A = 6.022\;141\;29 \times 10^{23} \pm 0.000\;000\;27 \times 10^{23} \text{ mol}^{-1} $$ If yes, consider the following quote from  (emphasis added):     Whenever you make a measurement, the number of meaningful digits that you write down implies the error in the measurement. For example if you say that the length of an object is 0.428 m, you imply an uncertainty of about 0.001 m. To record this measurement as either 0.4 or 0.42819667 would imply that you only know it to 0.1 m in the first case or to 0.00000001 m in the second. You should only report as many significant figures as are consistent with the estimated error. The quantity 0.428 m is said to have three significant figures, that is, three digits that make sense in terms of the measurement. Notice that this has nothing to do with the "number of decimal places". The same measurement in centimeters would be 42.8 cm and still be a three significant figure number. The accepted convention is that only one uncertain digit is to be reported for a measurement. In the example if the estimated error is 0.02 m you would report a result of 0.43 ± 0.02 m, not 0.428 ± 0.02 m.   and the following quote from  (emphasis added):     The absolute error places bounds telling you that the actual value is expected to fall between x - D x and x + D x. The absolute error is an easy way to report the error and makes it easy to check that you are not reporting too many significant figures. For example, suppose you had a measurement that was reported to be 10.02 ± 0.3 m. Because the reported value is uncertain in the tenths place, it is wrong to report it to the hundredths place. It should be reported to the same decimal place as the absolute error: 10.0 ± 0.3 m. Another point is that the absolute error and the reported value must have the same units.   If there is such a convention (cf. Rice's lab guide) and it is wrong to report more significant figures than is warranted by the absolute error (cf. the joint project's guidelines), and NIST should have adhered to such convention and striven to be correct, is it true that NIST adheres and is correct because the standard uncertainty has two least significant digits (i.e., $27$)?  Had the standard uncertainty been $0.000\;000\;2\text{ mol}^{-1}$, would NIST have reported Avogadro's Number as $N_A = 6.022\;141\;3\times 10^{23}$ with 8 significant figures?
44298
From the original data, we know that they are nearly 0.087 rather that 0.087 x 10^-2. Why is there scientific notation (10^-2)? It looks ridiculous. Please help me figure it out. Thanks very much!   MWE:  \documentclass[border=5mm]{standalone} \usepackage{pgfplots} \pgfplotsset{compat=newest} \begin{document} \definecolor{mycolor1}{rgb}{1,0,1}  \pgfplotstableread{ X           Y1 1    0.0873514639102964  2    0.0873656441069913  3    0.0873509320240032  4    0.0873490368814644  5    0.0873430960617957  6    0.0873334626756388  7    0.0873430596012905  8    0.0873485872405474  9    0.0873519301953318  10  0.0873474824394306 11  0.08735183859007     12  0.0873408783047386 13  0.0873439016123011 14  0.0873424452271129 15  0.0873452848079798 16  0.0873467228778811 17  0.087348210873026    18  0.0873464310394191 19  0.0873469131127228 20  0.0873449430894595 }\datatable  \pgfplotsset{     width=9cm,     height=6cm,     scale only axis,     xmin=0.5, xmax=20.5,     tick align = outside,     yticklabel style={/pgf/number format/fixed},   } \begin{tikzpicture}  \begin{axis}[     xtick={1,...,20},     ymin=0.08715,     ymax=0.08740,     ytick={0.08715,0.08720,0.08725,0.08730,0.08735,0.08740},     yticklabels={0.08715,0.08720,0.08725,0.08730,0.08735,0.08740},     xlabel={$x$},     ylabel={$y$},     y axis line style={mycolor1},     ytick style={mycolor1},     yticklabel style=mycolor1,     yticklabel style={/pgf/number format/fixed},     axis x line*=bottom,     axis y line*=left] \addplot [     color=mycolor1,     solid,     line width=1.5pt,     mark size=2.7pt,     mark=square*,     mark options={solid,fill=mycolor1}]     table [x=X, y=Y1]{\datatable}; \label{plot1} % So we can recall the plot style later on \end{axis} \end{tikzpicture}% \end{document}