Currently I have the following code:     Declare @FloatVal float set @FloatVal = 777777111111.555343 print @FloatVal print @FloatVal - FLOOR(@FloatVal) select @FloatVal select @FloatVal - FLOOR(@FloatVal)   And it outputs 7.77777e+011 for the first print, 0.555298 for the second print, 777777111111.556 for the first select and it outputs 0.5552978515625 for the second select.  Why is this the case?  Ultimately we want to receive the decimal places accurately, however, it seems to never give us this.  Unfortunately we cannot change the datatype from float to anything else because the system is setup in a way where this is not possible.  Thank you
121518
Debugging some finance-related SQL code found a strange issue with numeric(24,8) mathematics precision.  Running the following query on your MSSQL you would get A + B * C expression result to be 0.123457  SELECT A,        B,        C,        A + B * C FROM ( SELECT CAST(0.12345678 AS NUMERIC(24,8)) AS A,        CAST(0 AS NUMERIC(24,8)) AS B,        CAST(500 AS NUMERIC(24,8)) AS C ) T  So we have lost 2 significant symbols. Trying to get this fixed in different ways i got that conversion of the intermediate multiplication result (which is Zero!) to numeric (24,8) would work fine.  And finally a have a solution. But still I hace a question - why MSSQL behaves in this way and which type conversions actually occured in my sample?
44860
I am looking to format a number like 188518982.18 to £188,518,982.18 using Python.  How can I do this?
111324
I declare a variable for a 64 bit counter as :  long long call_count;   What is the format specifier that I should use in print statements?  I tried, %l, %ld, %ll. None seems to be correct.  I use Diab C compiler for compiling my application code to run on pSOS operating system.
17181
echo "scale=3;1/8" | bc   shows .125 on the screen. How to show 0.125 if the output result is less than one?
2846
Why does the following behave unexpectedly in Python?  &gt;&gt;&gt; a = 256 &gt;&gt;&gt; b = 256 &gt;&gt;&gt; a is b True           # This is an expected result &gt;&gt;&gt; a = 257 &gt;&gt;&gt; b = 257 &gt;&gt;&gt; a is b False          # What happened here? Why is this False? &gt;&gt;&gt; 257 is 257 True           # Yet the literal numbers compare properly   I am using Python 2.5.2. Trying some different versions of Python, it appears that Python 2.3.3 shows the above behaviour between 99 and 100.  Based on the above, I can hypothesize that Python is internally implemented such that "small" integers are stored in a different way than larger integers and the is operator can tell the difference. Why the leaky abstraction? What is a better way of comparing two arbitrary objects to see whether they are the same when I don't know in advance whether they are numbers or not?
66958
I want to do:  int a = 255;  cout &lt;&lt; a;   and have it show FF in the output, how would I do this?
21536
I have a raster with float number values for example: 2.544459104537964 and I want to round these up to integers. I have seen other posts on ArcGIS like  but when I use Raster Calculator in QGIS, Int(myraster + 0.5) only makes the first value (which is 0) to 0,5 and the last one (which is 100) to 100,5. The other pixels stay float.
70051
How can I format Floats in Java so that the float component is displayed only if it's not zero? For example:   123.45 -> 123.45 99.0   -> 99 23.2   -> 23.2 45.0   -> 45   Edit: I forgot to mention - I'm still on Java 1.4 - sorry!
47935
I need to convert lat/long that is expressed as degrees, minutes, and seconds in the data into decimal degrees. For example, in the data, they are listed as N335042.06 in the Latitude column, and W86031.04 in the Longitude column. I have done this problem before where I created a script which converted DMS to DD, and vice-versa, so I guess I could use bits from that. But the problem I'm having is how to ignore (for lack of a better word) the 'N' and 'W' in the data? Can I skip them? And DMS are listed all together without any symbols or spaces.  Can I use len(), range(), split() to specify what part to read from the value? For example, can do the following?  N335042.06 where, 33 = degrees 50 = minutes 42.06 = seconds ...?  I came across  ESRI article, but it's in VB. Will probably use it as a reference but some of the terminology/syntax is different from Python.  Final code that works!   # Pre-logic def latDD(x):   D = int(x[1:3])   M = int(x[3:5])   S = float(x[5:])   DD = D + float(M)/60 + float(S)/3600   return DD  # Expression latDD(!Latitude!)
118343
I'd like to display the binary (or hexadecimal) representation of a floating point number.  I know how to convert by hand (using the method ), but I'm interested in seeing code samples that do the same.  Although I'm particularly interested in the C++ and Java solutions, I wonder if any languages make it particularly easy so I'm making this language agnostic.  I'd love to see some solutions in other languages.  EDIT: I've gotten good coverage of C, C++, C#, and Java.  Are there any alternative-language gurus out there who want to add to the list?
63306
How can we extract the decimal part of a floating point number and store the decimal part and the integer part into two separate integer variables?
32795
Why is it that scanf() needs the l in "%lf" when reading a double, when printf() can use "%f" regardless of whether its argument is a double or a float?  Example code:  double d; scanf("%lf", &amp;d); printf("%f", d);
61612
I'm looking for a  way to convert numbers to string format, dropping any redundant '.0'   The input data is a mix of floats and strings. Desired output:  0   --> '0'  0.0 --> '0'  0.1 --> '0.1'  1.0 --> '1'  I've come up with the following generator expression, but I wonder if there's a faster way:  (str(i).rstrip('.0') if i else '0' for i in lst)   The truth check is there to prevent 0 from becoming an empty string.  EDIT: The more or less acceptable solution I have for now is this:  ('%d'%i if i == int(i) else '%s'%i for i in lst)   It just seems strange that there is no elegant way to handle this (fairly straightforward) case in python.
112920
How can I round a decimal number (floating point) to the nearest integer?   e.g.  1.2 = 1 1.7 = 2
8128
What is the best possible way to check if a string can be represented as a number in Python?   The function I currently have right now is:  def is_number(s):     try:         float(s)         return True     except ValueError:         return False   Which, not only is ugly and slow, seems clunky. However I haven't found a better method because calling float in the main function is even worse.
27648
double r = 11.631; double theta = 21.4;   In the debugger, these are shown as 11.631000000000000 and 21.399999618530273.  How can I avoid this?
111701
I'm messing around with some C code using floats, and I'm getting 1.#INF00, -1.#IND00 and -1.#IND when I try to print floats in the screen. What does those values mean?  I believe that 1.#INF00 means positive infinity, but what about -1.#IND00 and -1.#IND? I also saw sometimes this value: 1.$NaN which is Not a Number, but what causes those strange values and how can those help me with debugging?  I'm using  which I believe uses  representation for float point numbers.  Can someone list all those invalid values and what they mean?