When I run the following query Oracle returns 0.9999999999999999999999999999999999999999.  select 1/3 * 3 from dual;   However when I execute the following Oracle returns 1.0:  select 1/to_float(3) * 3 from dual;   Why the difference?
27
Consider the following code:   0.1 + 0.2 == 0.3  -&gt;  false   0.1 + 0.2         -&gt;  0.30000000000000004   Why do these inaccuracies happen?
9915
Query 1:  select count(UserID) from users where UserID IN (select UserID from otherTable)   Returns: 178  Query 2:: (simply changed IN to NOT IN)  select count(UserID) from users where UserID NOT IN (select UserID from otherTable)   Returns: 0  Query 3: (remove the IN-clause completely)  select count(UserID) from users   Returns: 1047123 (over a million)  What? I can't... Even...   The only explanation I have is my database in inconsistent/corrupted, I have to run DBCC CHECKDB or something.  PS. UserID is a primary key, so - no dupes.
63628
Here's an example:  Double d = (1/3); System.out.println(d);   This returns 0, not 0.33333... as it should.  Does anyone know?
33306
Hey! I was looking at this code at   I noticed that in some situations they used hex numbers, like in line 134:  for (j = 1; val &amp;&amp; j &lt;= 0x80; j &lt;&lt;= 1, q++)   Now why would they use the 0x80? I am not that good with hex but I found an online hex to decimal and it gave me 128 for 0x80.  Also before line 134, on line 114 they have this:  small_n = (n &amp; 0xffff0000) == 0;   The hex to decimal gave me 4294901760 for that hex number. So here in this line they are making a bit AND and comparing the result to 0??  Why not just use the number? Can anyone please explain and please do give examples of other situations.  Also I have seen large lines of code where it's just hex numbers and never really understood why :(
10027
What would be the most efficient way to compare two double or two float values?  Simply doing this is not correct:  bool CompareDoubles1 (double A, double B) {    return A == B; }   But something like:  bool CompareDoubles2 (double A, double B)  {    diff = A - B;    return (diff &lt; EPSILON) &amp;&amp; (-diff &lt; EPSILON); }   Seems to waste processing.  Does anyone know a smarter float comparer?
60259
What is the difference between decimal, float and double in .NET?  When would someone use one of these?
28516
How do I format a number in Java? What are the &quot;Best Practices&quot;? Will I need to round a number before I format it?  32.302342342342343 =&gt; 32.30 .7323 =&gt; 0.73  etc.
70051
How can I format Floats in Java so that the float component is displayed only if it's not zero? For example:   123.45 -> 123.45 99.0   -> 99 23.2   -> 23.2 45.0   -> 45   Edit: I forgot to mention - I'm still on Java 1.4 - sorry!
4177
Is there a way to step between 0 and 1 by 0.1?   I thought I could do it like the following, but it failed:  for i in range(0, 1, 0.1):     print i   Instead, it says that the step argument cannot be zero, which I did not expect.
56409
I have found a query to find the nth highest salary from Employee table , but i don't understand the logic of (N-1)?  EmpID         Salary 1             90000 2             80000 3             54000 4             37000 5             12000 6             69000 7             50000  SELECT * FROM Employee E1 WHERE (N-1) = (                 SELECT COUNT(DISTINCT(E2.Salary))                 FROM Employee E2                 WHERE E2.Salary &gt; E1.Salary               )   If N= 4, then how does the query work? I'm a complete beginner in SQL, please help!
41282
Here's my code:  x = 1.0 y = 100000.0     print x/y   My quotient displays as 1.00000e-05.  Is there any way to suppress scientific notation and make it display as 0.00001? I'm going to use the result as a string.
96922
Why when I save a value of say 40.54 in SQL Server to a column of type Real does it return to me a value that is more like 40.53999878999 instead of 40.54?  I've seen this a few times but have never figured out quite why it happens.  Has anyone else experienced this issue and if so causes it?
13936
Is there a way to make an Oracle query behave like it contains a MySQL limit clause? In MySQL, I can do this: select *  from sometable order by name limit 20,10  to get the 21st to the 30th rows (skip the first 20, give the next 10). The rows are selected after the order by, so it really starts on the 20th name alphabetically. In Oracle, the only thing people mention is the rownum pseudo-column, but it is evaluated before order by, which means this: select *  from sometable where rownum &lt;= 10 order by name  will return a random set of ten rows ordered by name, which is not usually what I want. It also doesn't allow for specifying an offset.
50537
I have PG server (9.0.4) installed on Debian x86. In one of my tables I use numeric(6,1):  CREATE TABLE mytable(name text, numfield numeric(6,1))   Also index created for this field:  CREATE INDEX mytable_numfield_idx ON mytable USING btree (numfield);   Everything is going fine if I use this field with real numeric value. As I see the query uses the index:  EXPLAIN ANALYZE SELECT * FROM kodiall WHERE kodgo=123.0      Index Scan using mytable_numfield_idx on mytable  (cost=0.00..8.27 rows=1   width=193) (actual time=0.085..0.087 rows=1 loops=1)      Index Cond: (numfield = 123.0)      Total runtime: 0.131 ms   But the index is ignored for some reason if I use 0.0 or NULL as condition value:  EXPLAIN ANALYZE SELECT * FROM kodiall WHERE numfield=0.0 --EXPLAIN ANALYZE SELECT * FROM kodiall WHERE numfield=NULL      Seq Scan on mytable  (cost=0.00..57.80 rows=1080 width=193) (actual   time=0.033..1.853 rows=1088 loops=1)      Filter: (numfield = 0.0)   What wrong here? Is 0.0 not a numeric value?  In real query I use LEFT JOIN with numfield and values in joined table can be NULL.
121518
Debugging some finance-related SQL code found a strange issue with numeric(24,8) mathematics precision.  Running the following query on your MSSQL you would get A + B * C expression result to be 0.123457  SELECT A,        B,        C,        A + B * C FROM ( SELECT CAST(0.12345678 AS NUMERIC(24,8)) AS A,        CAST(0 AS NUMERIC(24,8)) AS B,        CAST(500 AS NUMERIC(24,8)) AS C ) T  So we have lost 2 significant symbols. Trying to get this fixed in different ways i got that conversion of the intermediate multiplication result (which is Zero!) to numeric (24,8) would work fine.  And finally a have a solution. But still I hace a question - why MSSQL behaves in this way and which type conversions actually occured in my sample?
5995
When displaying the value of a decimal currently with .ToString(), it's accurate to like 15 decimal places, and since I'm using it to represent dollars and cents, I only want the output to be 2 decimal places.   Do I use a variation of .ToString() for this?
46383
I accept that two numbers can have the same supremum depending on how you generate a decimal representation. So $2.4999\ldots = 2.5$ etc. Can anyone point me to resources that would explain what the below argument that shows $999\ldots = -1$ is about? Here is the most usual proof I see that $0.999\ldots = 1$:  $x=0.999\ldots$ $10x=9.999\ldots$ $10x - x = 9$ $x=1$  Using this same argument template I can show $999\ldots=-1$:  $x= \ldots9999.0 $ $0.1x= \ldots9999.9$ $0.1x - x = 0.9$ $x=-1$  What might this mean? Edit from one of the comments: $$\sum_{k=0}^{\infty}{9 \cdot 10^k}=-1$$