GDB seems to have been clearly made with source-level debugging in mind, the kind where the source code of the program is available. Sometimes this is not the case, or you just don’t want to bother looking for the source code. Here I’ll put a bunch of useful commands.
layout asm: enable TUI mode and show assembly instructions
layout regs: enable TUI mode and show CPU registers
set disassembly-flavor intel: use Intel-like disassembly, which is more familiar to those writing or debugging assembly code on Windows
Ctrl+N: serve as alternatives to “up” and “down” when the active TUI window is not the command window
Ctrl+X, O: switch the active TUI window
Ctrl+X, (A or Shift+A or Ctrl+A): toggle TUI mode
x/5i $pc: check out 5 instructions, starting at the program counter
stepi): next/step, but on an instruction level
b *0x<addr>: breakpoint on a specific address
info registers: look at CPU registers
info frame: look at current stack frame, potentially useful for finding function arguments and local vars
x/8g $sp: look at (64-bit words) in the stack
x64 Linux C/C++ conventions
This is a set of additional notes written during the process of disassembling a library.
For function calls, the arguments are stored in the following order:
- RDI register
- RSI register
- RDX register
- RCX register
- R8 register
- R9 register
- rest are pushed onto the stack in reverse order
When a function is called, it will typically perform the following operations:
mov rbp, rsp
sub rsp, (amount of space used for local variables)
add rsp, (same amount)
leave ; or mov rsp,rbp and pop rbp
This means that any reference to memory at [rbp-0x…] is using the function’s local variables.
If a function returns a value, it will be stored in the EAX register.
If it is a C++ function, its name will be “mangled”: its argument types will be
included in the name. The utility
c++filt can convert these mangled names to
their standard forms. If the function is part of an object, it will require the
first parameter to be the object’s address.
Seems like the same is true for constructors and destructors: their first parameter is an address that will point to the object in question.