#99 ✓resolved
Peter Johnson

Incorrect bytecode generated when using bin format with BITS 64 assembled for Long Mode

Reported by Peter Johnson | June 25th, 2011 @ 07:50 PM

Originally posted on Trac by dave.poirier@ieee.org
Original Trac Ticket

  Description

I'm attempting to use -fbin to generate a 32 bits binary code that receives control from Grub, then jumps into another part of that same bin file into Long Mode with 64bits registers. The following code doesn't get assembled properly:

   151 00000088 48C7C1D0070000          mov rcx, (80*2*50)/4
   152 0000008F 48B820072007200720-     mov rax, 0x0720072007200720
   153 0000008F 07                      
   154 00000099 48C7C700800B00          mov rdi, 0xB8000
   155 000000A0 F348AB                  rep stosq
   156 000000A3 F4                      hlt 
   157 000000A4 EBFC                    jmp $

It should be:

   141 00000068 B9                      db 0xB9 ; mov rcx, (80*2*50)/4
   142 00000069 D007000000000000        dq (80*2*50)/4
   143 00000071 B8                      db 0xB8 ; mov rax, 0x072007200720
   144 00000072 2007200720072007        dq 0x0720072007200720
   145 0000007A BF                      db 0xBF ; mov rdi, 0xB8000
   146 0000007B 00800B0000000000        dq 0xB8000
   147 00000083 F3AB                    db 0xF3, 0xAB ; rep stosq
   148 00000085 F4                      db 0xF4 ; hlt
   149 00000086 EBFC                    db 0xEB, 0xFC ; jmp $

I tried using BITS 64 and USE64 with the same result.

Trac Attachments

h3. Trac Comments
comment:1

                      follow-up:
  ↓ 2

              Changed 4 years ago by peter@tortall.net


Your "should be" code is incorrect, and is not valid x86-64 code (or rather does not do what you think it does).  The 0x48 prefix is required to set 64-bit operation size; see the AMD64 or Intel64 manuals for details.

Some of the immediate values yasm generates are smaller (e.g. 32-bit instead of 64-bit) as it optimizes for the smallest possible size; note the code generated by yasm is correct, it just doesn't use 64-bit constants.  If you really want the 64-bit constants, use "strict qword" on the immediate; e.g. "mov rcx, strict qword (80*2*50)/4" generates "48 b9 d0 07 00 00 00 00 00 00".

      comment:1
    
                          follow-up:
      ↓ 2
                  
                  Changed 4 years ago by peter@tortall.net

      Status
        changed from new to closed
    
      Resolution
        set to worksforme

Your "should be" code is incorrect, and is not valid x86-64 code (or rather does not do what you think it does).  The 0x48 prefix is required to set 64-bit operation size; see the AMD64 or Intel64 manuals for details.

Some of the immediate values yasm generates are smaller (e.g. 32-bit instead of 64-bit) as it optimizes for the smallest possible size; note the code generated by yasm is correct, it just doesn't use 64-bit constants.  If you really want the 64-bit constants, use "strict qword" on the immediate; e.g. "mov rcx, strict qword (80*2*50)/4" generates "48 b9 d0 07 00 00 00 00 00 00".

      Status
        changed from new to closed
    
      Resolution
        set to worksforme

comment:2

                    in reply to:
  ↑ 1

              Changed 4 years ago by anonymous


Apparently there are two long-modes, long-mode with compatibility for 32-bits, which your code apparently assembles to, and the 64-bit long-mode, which I'm attempting to use.  Is it just me again not understanding the encoding?

comment:3

              Changed 4 years ago by peter@tortall.net


No, there's only one 64-bit mode, called "long mode" by AMD.  From page 6 of the AMD64 architecture programmer's manual, volume 3: "In 64-bit mode, most instructions default to a 32-bit operand size. For these instructions, a REX prefix (page 16) can specify a 64-bit operand size, and a 66h prefix specifies a 16-bit operand size. The REX prefix takes precedence over the 66h prefix.  However, if an instruction defaults to a 64-bit operand size, it does not need a REX prefix and it can only be overridden to a 16-bit operand size. It cannot be overridden to a 32-bit operand size, because there is no 32-bit operand-size override prefix in 64-bit mode."

What AMD calls "compatibility mode" is really a 32-bit mode at the opcode level, e.g. the registers are all 32-bit, none of the extended 64-bit registers are available, etc (section 2.4.1 in volume 2: "In compatibility and legacy modes, the GPRs consist only of the eight legacy 32-bit registers. All legacy rules apply for determining operand size.").  The only thing that's 64-bit-like in compatibility mode is things like the size of stack pushes and pops (all are 8 byte), but that's not visible at the opcode encoding level.  See page 50 in volume 2 (Table 2-6, Difference between long mode and legacy mode) for details.  If you're writing for compatibility mode, just write 32-bit code (BITS 32)--note you can't use 64-bit registers like rax in this mode--and keep in mind the size differences.

comment:4

              Changed 4 years ago by peter@tortall.net


I guess I didn't directly answer part of your question, which is that Yasm in BITS 64 mode assembles for the 64-bit long mode as described in the first paragraph of my previous response.  The key part of that paragraph is that "most instructions default to a 32-bit operand size."  That's why there's a lot of 0x48 (REX) prefixes generated by Yasm in BITS 64 mode: mov, for example, defaults to 32-bit operand size and thus needs a REX prefix to access a 64-bit register (like rax).  If the REX prefix wasn't there, the mov would access eax, not rax (as it would be using a 32-bit operand size).

Comments and changes to this ticket

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

The Yasm Modular Assembler Project

Shared Ticket Bins

People watching this ticket

Pages