in

ASP.NET Weblogs

Jason Nadal

Restless C#ding

Javascript Bad Math

I really wish someone could explain to me how, in javascript, the following calculation happens:

0.2320 * 100 = 23.20000000000003

Check out the following script, which I only managed to fix by setting the precision of the offending result to 4 significant digits. The glitch can be seen with the following script:

<html>
<body> 
 <script language="javascript">
  var x = Number('0.2320');
  alert(x);
  var y = x*100;
  alert(y);
 </script> 
</body>
</html>

Comments

 

Raymond Chen said:

April 14, 2004 11:03 PM
 

Jason said:

Yes, I knew it was a floating point error, but I didn't realize the process for fp arithmatic was inherently inexact.
April 15, 2004 8:29 AM
 

Doug Reilly said:

Yep. Something as simple as this (C#) will show it:

int loop;
double foo=1.0;
for ( loop=0 ; loop<100 ; loop++ )
{
System.Console.WriteLine(foo.ToString());
foo+=0.1;
}
System.Console.ReadLine();

All languages suffer from this, I fear (Decimal in .NET helps).
April 20, 2004 8:14 PM

Leave a Comment

(required)  
(optional)
(required)  
Add