I am trying to match values between arrays.   I realize some operations are calculating these as False when if should be true. One of the values it seems contains more decimal values.           Debug.Print x / 100         Debug.Print y         Debug.Print (x / 100) - y   This results in:  0.9248  0.9248  1.11022302462516E-16   I've tried using round &amp; formatcurrency. While it will show the values  as rounded, it still retains the scientific notation when calculating.  Does anyone have any suggestion on how i can get x = y to flag as 'true'?  Thanks.
27
Consider the following code:   0.1 + 0.2 == 0.3  -&gt;  false   0.1 + 0.2         -&gt;  0.30000000000000004   Why do these inaccuracies happen?
61612
I'm looking for a  way to convert numbers to string format, dropping any redundant '.0'   The input data is a mix of floats and strings. Desired output:  0   --> '0'  0.0 --> '0'  0.1 --> '0.1'  1.0 --> '1'  I've come up with the following generator expression, but I wonder if there's a faster way:  (str(i).rstrip('.0') if i else '0' for i in lst)   The truth check is there to prevent 0 from becoming an empty string.  EDIT: The more or less acceptable solution I have for now is this:  ('%d'%i if i == int(i) else '%s'%i for i in lst)   It just seems strange that there is no elegant way to handle this (fairly straightforward) case in python.
80556
What's going on?! Subtraction works fine until I get to 0.1 - 0.1. I'm in visual c# 2008 using the nonoba.com API.  Console.WriteLine("hit! " + Users[targetNum].character.health + " : " + player.character.profile.attackPower);  Users[targetNum].character.health -= player.character.profile.attackPower;  Console.WriteLine("health! " + Users[targetNum].character.health);   output:  hit! 0.1 : 0.1 health! 1.490116E-08   Thanks all - I might use the decimal type, as I'm normally adding/subtracting nice "round" numbers. For now I'll just go with:  if (Users[targetNum].character.health &lt;= 0.00001)   By the way I knew this wasn't really going to be a "bug" in c# - I thought it would either by a bug in my code or some lack of understanding, which it was.  Having read all the recommended reading, I'm going to conclude that my folly was due to normally using the ActionScript Number type, which maybe has a decimal rather than binary floating point - anyway, it would never give this output.
9683
Hey all I asked this over  =) and I got a good answer but I still have no idea whats going on. I want to know how the expression got to the answer and maybe the math behind how it was reasoned.. the steps if you would be so kind. The likelihood is I don't understand the equation.   I'm not a big maths person please be gentle :p     For all those playing along at home I'm reading Absolute Java 5th   Edition by Walter Savitch (Chapter 2 Page 66).      The Expression  double d = 12345.123456789; System.out.printf("START%12.5e END %n", d);       The Answer  START 1.23451e+04END       I understand the basic principles of this printf method's arguments   for example I know the '%' represents the start of the parameters.   '12' is the number of spacing, '.5' is the times the decimal point   will move.... I see the decimal point has moved 4 places towards the   left... can someone explain the principles of e-notation. Also how   this expression came to this answer =).      As far as e-notation go's its meant to be scientific notation;      So like 5.89e-4 would mean 0.000589 (move the decimal place if minus   left if not move the decimal place right).
8128
What is the best possible way to check if a string can be represented as a number in Python?   The function I currently have right now is:  def is_number(s):     try:         float(s)         return True     except ValueError:         return False   Which, not only is ugly and slow, seems clunky. However I haven't found a better method because calling float in the main function is even worse.
38898
I need to format a decimal value into a string where I always display at lease 2 decimals and at most 4.   So for example  "34.49596" would be "34.4959"  "49.3" would be "49.30"   Can this be done using the String.format command? Or is there an easier/better way to do this in Java.
34299
How do you convert between hexadecimal numbers and decimal numbers in C#?
44849
Does anyone know why, using SQLServer 2005  SELECT CONVERT(DECIMAL(30,15),146804871.212533)/CONVERT(DECIMAL (38,9),12499999.9999)   gives me 11.74438969709659,  but when I increase the decimal places on the denominator to 15, I get a less accurate answer:  SELECT CONVERT(DECIMAL(30,15),146804871.212533)/CONVERT(DECIMAL (38,15),12499999.9999)   give me 11.74438969
4177
Is there a way to step between 0 and 1 by 0.1?   I thought I could do it like the following, but it failed:  for i in range(0, 1, 0.1):     print i   Instead, it says that the step argument cannot be zero, which I did not expect.
31696
I need to bitwise shift a value 64 times in JavaScript. But JavaScript starts rounding after 32.  For example:  for(var j = 0; j &lt; 64; j++) {     mask = mask &lt;&lt; 1;     console.log(mask); }   This prints value from 0 to 1073741824 but then rounds of and starts printing 0.
63628
Here's an example:  Double d = (1/3); System.out.println(d);   This returns 0, not 0.33333... as it should.  Does anyone know?
42817
Possible Duplicate:          What's a way to see if a number is a ?   bool IsPerfectSquare(long input) {    // TODO }   I'm using C# but this is language agnostic.  Bonus points for clarity and simplicity (this isn't meant to be code-golf).    Edit: This got much more complex than I expected! It turns out the problems with double precision manifest themselves a couple ways. First, Math.Sqrt takes a double which can't precisely hold a long (thanks Jon).   Second, a double's precision will lose small values ( .000...00001) when you have a huge, near perfect square. e.g., my implementation failed this test for Math.Pow(10,18)+1 (mine reported true).
80880
My code:  a = '2.3'   I wanted to display a as a floating point value.  Since a is a string, I tried:  float(a)   The result I got was :  2.2999999999999998   I want a solution for this problem. Please, kindly help me.  I was following .
38011
I am developing a PHP application in which I am in need to store values bigger than 1 million, but with the flexibility of having floating points. I know the traditional float and double. I am looking to store only up to two decimal points, and will only handle operations with numbers that have 2 decimals exclusively.  I have chosen my column type of double and unsigned, but whenever I store a row with a value bigger than one million, it gets truncated to 999999.99  Why is this? I am looking for the right datatype/solution. If you could point me in the right direction, I will appreciate it a lot!  Thank you for your time in advance!  Cheers.
99240
I write financial applications where I constantly battle the decision to use a double vs using a decimal.  All of my math works on numbers with no more than 5 decimal places and are not larger than ~100,000. I have a feeling that all of these can be represented as doubles anyways without rounding error, but have never been sure.  I would go ahead and make the switch from decimals to doubles for the obvious speed advantage, except that at the end of the day, I still use the ToString method to transmit prices to exchanges, and need to make sure it always outputs the number I expect. (89.99 instead of 89.99000000001)  Questions:   Is the speed advantage really as large as naive tests suggest? (~100 times) Is there a way to guarantee the output from ToString to be what I want? Is this assured by the fact that my number is always representable?   UPDATE: I have to process ~ 10 billion price updates before my app can run, and I have implemented with decimal right now for the obvious protective reasons, but it takes ~3 hours just to turn on, doubles would dramatically reduce my turn on time. Is there a safe way to do it with doubles?
23208
I have been asked to test a library provided by a 3rd party. The library is known to be accurate to n significant figures. Any less-significant errors can safely be ignored. I want to write a function to help me compare the results:  def nearlyequal( a, b, sigfig=5 ):   The purpose of this function is to determine if two floating-point numbers (a and b) are approximately equal. The function will return True if a==b (exact match) or if a and b have the same value when rounded to sigfig significant-figures when written in decimal.   Can anybody suggest a good implementation? I've written a mini unit-test. Unless you can see a bug in my tests then a good implementation should pass the following:  assert nearlyequal(1, 1, 5)  assert nearlyequal(1.0, 1.0, 5)  assert nearlyequal(1.0, 1.0, 5)  assert nearlyequal(-1e-9, 1e-9, 5)  assert nearlyequal(1e9, 1e9 + 1 , 5)  assert not nearlyequal( 1e4, 1e4 + 1, 5)  assert nearlyequal( 0.0, 1e-15, 5 )  assert not nearlyequal( 0.0, 1e-4, 6 )    Additional notes:   Values a and b might be of type int, float or numpy.float64. Values a and b will always be of the same type. It's vital that conversion does not introduce additional error into the function. Lets keep this numerical, so functions that convert to strings or use non-mathematical tricks are not ideal. This program will be audited by somebody who is a mathematician who will want to be able to prove that the function does what it is supposed to do. Speed... I've got to compare a lot of numbers so the faster the better. I've got numpy, scipy and the standard-library. Anything else will be hard for me to get, especially for such a small part of the project.
26280
What is an easy way in Python to format integers into strings representing thousands with K, and millions with M, and leaving just couple digits after comma?  I'd like to show 7436313 as 7.44M, and 2345 as 2,34K.  Is there some % string formatting operator available for that? Or that could be done only by actually dividing by 1000 in a loop and constructing result string step by step?