opensubscriber
   Find in this group all groups
 
Unknown more information…

a : advanced-dotnet@discuss.develop.com 26 August 2008 • 1:35AM -0400

Re: [ADVANCED-DOTNET] Converting doubles into integers without rounding errors
by Helen Warn

REPLY TO AUTHOR
 
REPLY TO GROUP




Will you ever have to deal with negative numbers? If so, you need to test
that case as well.

helen

-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:ADVANCED-DOTNET@DISC...] On Behalf Of Eddie Lascu
Sent: Monday, August 25, 2008 10:24 AM
To: ADVANCED-DOTNET@DISC...
Subject: Re: [ADVANCED-DOTNET] Converting doubles into integers without
rounding errors

Hi John,

I just tested this and I am happy with the return as I got what I expected
in each case:

Console.WriteLine(Math.Round((123.454 * 100.0) - 0.5));  // expect 12345
Console.WriteLine(Math.Round((123.455 * 100.0) - 0.5));  // expect 12345
Console.WriteLine(Math.Round((123.456 * 100.0) - 0.5));  // expect 12345

Console.WriteLine(Math.Round((123.464 * 100.0) - 0.5));  // expect 12346
Console.WriteLine(Math.Round((123.465 * 100.0) - 0.5));  // expect 12346
Console.WriteLine(Math.Round((123.466 * 100.0) - 0.5));  // expect 12346

The problem though is that I can never be sure that amount 123.45 will not
be stored as 123.4499999. According to the above approach, this will be
converted to 12344 which is not what I want.


-----Original Message-----
From: Discussion of advanced .NET topics.
[mailto:ADVANCED-DOTNET@DISC...]On Behalf Of John Warner
Sent: Monday, August 25, 2008 1:11 PM
To: ADVANCED-DOTNET@DISC...
Subject: Re: [ADVANCED-DOTNET] Converting doubles into integers without
rounding errors
Importance: Low


What result do you want from the examples you borrow from MSDN? Again have
you considered 1000.0 instead of 100.0 and do your own round?

John Warner




> -----Original Message-----
> From: Discussion of advanced .NET topics.
> [mailto:ADVANCED-DOTNET@DISC...] On Behalf Of Eddie Lascu
> Sent: Monday, August 25, 2008 1:04 PM
> To: ADVANCED-DOTNET@DISC...
> Subject: Re: [ADVANCED-DOTNET] Converting doubles into
> integers without rounding errors
>
>
> According to MSDN, Math.Round will return the following:
>
> Math.Round(4.4); //Returns 4.0.
> Math.Round(4.5); //Returns 4.0.
> Math.Round(4.6); //Returns 5.0.
>
> So, in my case doing Math.Round(objMyObject.Amount * 100.0)
> will not do the trick always. For example:
>
> Math.Round(123.454 * 100.0); // returns 12345
> Math.Round(123.455 * 100.0); // returns 12345
> Math.Round(123.456 * 100.0); // returns 12346
>
> So I suppose I need to execute this:
>
> Math.Round((objMyObject.Amount * 100.0) - 0.5) to be sure
> that I get what I want.
>
> Am I missing something?
>
>
>
> -----Original Message-----
> From: Discussion of advanced .NET topics.
> [mailto:ADVANCED-DOTNET@DISC...]On Behalf Of Peter Ritchie
> Sent: Monday, August 25, 2008 12:48 PM
> To: ADVANCED-DOTNET@DISC...
> Subject: Re: [ADVANCED-DOTNET] Converting doubles into
> integers without rounding errors
>
>
> As John has alluded to, what you're seeing is a difference in
> rounding between formatting a double value as text and
> multiplying by 100 and dropping the decimals.
>
> If you have specific rounding logic, perform it before you
> convert to int.  If you want the same value that you'd see as
> formatted text, convert to text first.
>
> I would recommend something like this:
>
> double tempAmount = objMyObject.Amount * 100.0;
> uint nIntAmount = Math.Round(tempAmount);
>
> ===================================
> This list is hosted by DevelopMentorR  http://www.develop.com
>
> View archives and manage your subscription(s) at
> http://discuss.develop.com
>
> ===================================
> This list is hosted by DevelopMentorR  http://www.develop.com
>
> View archives and manage your subscription(s) at
> http://discuss.develop.com
>

===================================
This list is hosted by DevelopMentor.  http://www.develop.com

View archives and manage your subscription(s) at http://discuss.develop.com

===================================
This list is hosted by DevelopMentor.  http://www.develop.com

View archives and manage your subscription(s) at http://discuss.develop.com

===================================
This list is hosted by DevelopMentorĀ®  http://www.develop.com

View archives and manage your subscription(s) at http://discuss.develop.com

Bookmark with:

Delicious   Digg   reddit   Facebook   StumbleUpon

Related Messages

opensubscriber is not affiliated with the authors of this message nor responsible for its content.