An Architect's View

CFML, Clojure, Software Design, Frameworks and more...

An Architect's View

More on Floating Point

January 14, 2005 · 4 Comments

A little while ago I explained why decimalFormat() did not have the bug that certain folks were claiming. I wasn't entirely happy with my explanation. Here's what I should have said: Eric Lippert explains why floating point numbers aren't accurate on binary digital computers. Eric's blog is one of only a few Microsoft blogs that I read regularly. He writes some great stuff (although I wish he wouldn't use purple text).
View count: 279

Tags: programming

4 responses so far ↓

  • 1 Qasim Rasheed // Jan 14, 2005 at 9:18 AM

    I just found out the other day that numberformat also perform a rounding. Is this an expected behaviour or a bug

    e.g #NumberFormat( "1.234567", "999999.9999" )# will give 1.2346 instead of 1.2345

  • 2 Sean Corfield // Jan 14, 2005 at 10:15 AM

    That looks like exactly the right result to me. Rounding 1.234567 to 4 d.p. should be 1.2346

    But go read my previous blog entry about decimalFormat() anyway (December 12th).

  • 3 Qasim Rasheed // Jan 14, 2005 at 10:18 AM

    So if I understood you correctly, numberformat will implicitly round numbers in addition to formatitng them?

  • 4 Eric Lippert // Jan 17, 2005 at 2:28 PM

    I'll be discussing rounding issues in floating point in the next few blog entries.

    I'm glad that you enjoy the blog. Do let me know if you have suggestions for future articles.

    And one of these days I'll get the style sheets set up correctly so that people who inexplicably do not like purple Lucida Sans can more easily turn it off. One of these days == don't hold your breath waiting.

Leave a Comment

Leave this field empty: