opensubscriber
   Find in this group all groups
 
Unknown more information…

a : advanced-dotnet@discuss.develop.com 26 August 2008 • 12:49AM -0400

Re: [ADVANCED-DOTNET] Converting doubles into integers without rounding errors
by Adam Tuliper

REPLY TO AUTHOR
 
REPLY TO GROUP




any chance of fixing the underlying issue and moving to the Decimal data
type? double is not fun to work with because of this exact issue.
double uses basically binary fractions to store data, decimal data type does
not and as such should be able to represent your number perfectly.



On Mon, Aug 25, 2008 at 9:38 AM, Gregory Miley <GregoryM@leet...> wrote:

> You could convert to string (bleh) and strip the decimal out, then to
> int. Not the best approach, but it should work without any errors if you
> don't mind it using up more resources.
>
> -----Original Message-----
> From: Discussion of advanced .NET topics.
> [mailto:ADVANCED-DOTNET@DISC...] On Behalf Of Eddie Lascu
> Sent: Monday, August 25, 2008 12:28
> To: ADVANCED-DOTNET@DISC...
> Subject: Converting doubles into integers without rounding errors
>
> Hello everyone,
>
> I have some objects that contain an amount field that is declared as
> double.
> Since it contains amounts, it always has only two decimal digits that
> are
> significant. During the process I need to convert that double into an
> integer by removing the decimal point. For example, $78.59 should be
> converted to integer 7859 and $101.53 to 10153. in my code I have
> uint nIntAmount = (uint)(objMyObject.Amount * 100);
>
> The problem I am facing is that sometimes, very rarely, there is a
> rounding
> error that is introduced and the integer obtained is off by a cent (plus
> or
> minus). For example, this is a line that was traced in my log file:
>
> "Updating the batch with $137.89 as the amount in the transaction. This
> amount was converted to 13788."
>
> Can either of you suggest a different way to convert the amounts in
> integers
> without this nagging rounding error?
>
> Any help will be appreciated,
>
> Eddie
>
> ===================================
> This list is hosted by DevelopMentor(r)  http://www.develop.com
>
> View archives and manage your subscription(s) at
> http://discuss.develop.com
>
> ===================================
> This list is hosted by DevelopMentor(R)  http://www.develop.com
>
> View archives and manage your subscription(s) at
> http://discuss.develop.com
>

===================================
This list is hosted by DevelopMentorĀ®  http://www.develop.com

View archives and manage your subscription(s) at http://discuss.develop.com

Bookmark with:

Delicious   Digg   reddit   Facebook   StumbleUpon

Related Messages

opensubscriber is not affiliated with the authors of this message nor responsible for its content.