opensubscriber
   Find in this group all groups
 
Unknown more information…

a : advanced-dotnet@discuss.develop.com 29 August 2008 • 5:45AM -0400

Re: [ADVANCED-DOTNET] Converting doubles into integers without rounding errors
by David Nicholson

REPLY TO AUTHOR
 
REPLY TO GROUP




Eddie

No, you should be truncating to an int, not using Math.Round. Here's an
example:

using System;
namespace DoubleRound
{
    class Program
    {
        static void Main(string[] args)
        {
            // initial number may be slightly low or high
            double d1 = 137.889999;
            double d2 = 137.890000;
            double d3 = 137.890001;

            Console.WriteLine("Rounded d1 = " + Round(d1).ToString());
            Console.WriteLine("Rounded d2 = " + Round(d2).ToString());
            Console.WriteLine("Rounded d3 = " + Round(d3).ToString());
        }

        static decimal Round(Double d)
        {
            double rd1 = d + 0.0001;    // add 1/100th of a cent
            double rd2 = rd1 * 100.0;   // integral part of double is cents
            int i = (int) rd2;          // integer is cents

            decimal dec1 = (decimal) i;  // decimal is cents
            decimal dec2 = dec1 / 100m;  // decimal is dollars.cents

            // no precision errors with decimal divide

            return dec2;
        }
    }
}

I laid out the steps in Round() for clarity, you could easily write it in
fewer lines. The result is:

Rounded d1 = 137.89
Rounded d2 = 137.89
Rounded d3 = 137.89

Double has a precision of 64 bits - there's no way your number is going to
be half a cent out because of rounding.

David.


On Thu, 28 Aug 2008 11:39:42 -0400, Eddie Lascu <elascu@IBIG...>
wrote:

>David,
>
>It all depends on what method are you using: Math.Round or Math.Truncate.
>Round rounds the double to the nearest integer. So
>
>Math.Round( 13788.999 ) will return 13789 as opposed to Math.Truncate who
>will return 13788.
>
>So far so good. The problem will appear when the number is 13788.5. This
>will be rounded to the nearest even number so Math.Round will return 13788
>(incorrect). Hopefully this will never happen, because as some of you
>suggested, that means that the initial amount of $137.89 is stored in a
>double as 137.885000, off by 0.005, a very big precision error.
>
>My point is that adding anything will not help me because I can introduce
>the error. Just for the sake of argument, let's assume that a precision
>error of 0.005 can happen. If that, then an error of 0.0045 can happen
too.
>If my amount of $137.89 is actually stored as $137.8945 and I add 0.001 I
>get 137.8955 which will round up to 137.90 - wrong number. We have already
>established that Math.Truncate is not good so I am only left with
Math.Round
>and with the hope that a precision error of 0.005 is too big to actually
>happen.
>
>Thanks to everyone who chimed in and shared experience.
>Best regards,
>Eddie
>
>
>
>
>-----Original Message-----
>From: Discussion of advanced .NET topics.
>[mailto:ADVANCED-DOTNET@DISC...]On Behalf Of David Nicholson
>Sent: Wednesday, August 27, 2008 8:04 PM
>To: ADVANCED-DOTNET@DISC...
>Subject: Re: [ADVANCED-DOTNET] Converting doubles into integers without
>rounding errors
>
>
>Since there seems to be no clear solution so far, I've gone back to the
>beginning. I suggest you get more information by writing some code to
>display your numbers to higher precision. My expectation is that the
>137.89 is actually 137.889999 (probably more nine's). So your code below
>converts this to 13788.9999, then truncates to 137.88.
>
>So add a small value (e.g. .0001) first. Then the outcomes for numbers
>above and below the intended number look something like:
>
>137.889999 -> 137.890099 -> 13789.0099 -> 13789
>137.890000 -> 137.890100 -> 13789.0100 -> 13789
>137.890001 -> 137.890101 -> 13789.0101 -> 13789
>
>David.
>
>
>On Mon, 25 Aug 2008 12:27:52 -0400, Eddie Lascu <elascu@IBIG...>
>wrote:
>
>>Hello everyone,
>>
>>I have some objects that contain an amount field that is declared as
>double.
>>Since it contains amounts, it always has only two decimal digits that are
>>significant. During the process I need to convert that double into an
>>integer by removing the decimal point. For example, $78.59 should be
>>converted to integer 7859 and $101.53 to 10153. in my code I have
>>uint nIntAmount = (uint)(objMyObject.Amount * 100);
>>
>>The problem I am facing is that sometimes, very rarely, there is a
>rounding
>>error that is introduced and the integer obtained is off by a cent (plus
>or
>>minus). For example, this is a line that was traced in my log file:
>>
>>"Updating the batch with $137.89 as the amount in the transaction. This
>>amount was converted to 13788."
>>
>>Can either of you suggest a different way to convert the amounts in
>integers
>>without this nagging rounding error?
>>
>>Any help will be appreciated,
>>
>>Eddie
>>
>>===================================
>>This list is hosted by DevelopMentor®  http://www.develop.com
>>
>>View archives and manage your subscription(s) at
>http://discuss.develop.com
>
>===================================
>This list is hosted by DevelopMentor®  http://www.develop.com
>
>View archives and manage your subscription(s) at
http://discuss.develop.com
>
>===================================
>This list is hosted by DevelopMentor®  http://www.develop.com
>
>View archives and manage your subscription(s) at
http://discuss.develop.com

===================================
This list is hosted by DevelopMentor®  http://www.develop.com

View archives and manage your subscription(s) at http://discuss.develop.com

Bookmark with:

Delicious   Digg   reddit   Facebook   StumbleUpon

Related Messages

opensubscriber is not affiliated with the authors of this message nor responsible for its content.