#105 ✓resolved
Peter Johnson

Incorrect float declaration

Reported by Peter Johnson | June 25th, 2011 @ 07:50 PM

Originally posted on Trac by BraD
Original Trac Ticket

  Description

Im using yasm to build binary files executables on a Virtual Machine oriented to build a 2d game engine, all variables and operands for math ops, are type double or float, when in the asm codes:

dd 3.141592 ; In the binary it's 3.141592
dd 1.000000 ; In the binary it's 0.500000
dd 1.00000 ; In the binary it's 1.000000

only when it's has 6 decimal digits & they are 0

Trac Attachments

h3. Trac Comments
comment:1

              Changed 4 years ago by peter


(In [1836]) Fix #107: Float input "1.000000" hit an edge case in the code that caused
the rounding increment at the end of float conversion to wrap the mantissa
from all 1's to 0, resulting in an incorrect result.

      comment:1

                  Changed 4 years ago by peter

      Status
        changed from new to closed
    
      Resolution
        set to fixed

(In [1836]) Fix #107: Float input "1.000000" hit an edge case in the code that caused
the rounding increment at the end of float conversion to wrap the mantissa
from all 1's to 0, resulting in an incorrect result.

      Status
        changed from new to closed
    
      Resolution
        set to fixed

Comments and changes to this ticket

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

The Yasm Modular Assembler Project

Shared Ticket Bins

People watching this ticket

Pages