Advanced GDB Debugging
In this post, I would like to point out some tips/tricks to make debugging easier with GDB. I will be focusing on Linux x86_64 binaries, but most of the things should work on other architectures as well. I will try to cover both
I am gonna assume basic familiarity with GDB, such as setting breakpoints/stepping through code, etc. If you are not familiar with GDB, I would recommend reading/watching some basic tutorial for GDB.
Using the .gdbinit file
Gdb uses a file called the .gdbinit
file which is used to store all the settings needed when you start gdb. This can be configured on both a systemwide basis and also on a projectwide basis. The root .gdbinit
is usually found in the /home
directory, and the project specific .gdbinit
is usually found in the root of the project directory.
However to load a project specific .gdbinit file when you start gdb, you need to pass the following command to gdb:
gdb -x <path to .gdbinit file>
GDB Advanced Commands
GDB has a lot of commands, I plan to try and cover the commands that helped me the most while debugging applications. A good starting tip - is that whenever you are stuck or can’t remember the command that you were looking for, try using the apropos
command. This is basically a regex search through the commands list and you can usually find the command you were looking for. To get more information about a command, use the help
command.
Watchpoints
Watchpoints are used to stop exectuing the program when a certain memory location is read/written. This is useful when you want to know when a certain variable is accessed. But you don’t want to set a breakpoint on every single line where the variable may be accessed.
(gdb) watch __afl_prev_loc
Note: watchpoints can be called on any memory address, but you need to cast the address to help determine how many bytes need to be watched. For example, if you want to watch a 4 byte integer, you need to cast the address to
(int *)
(gdb) watch *(int *)0x7fffffffe3e0
You need to use the
*
to dereference the address else gdb will try to watch the constant address itself.
watch
- watchpoint for write accessrwatch
- watchpoint for read accessawatch
- watchpoint for read/write access
Conditional Breakpoints and Watchpoints
Conditional breakpoints are a amazing feature of GDB, which allows you to set a breakpoint only if a certain condition is met. This is very useful when you are debugging a basic block that’s called multiple times, and you want to stop only when a certain condition is met.
For ex, if I want to stop at the call to __afl_maybe_log only if the the value of rcx is 0xdaef, I can use the following command:
(gdb) break __afl_maybe_log if $rcx == 0xdaef
If symbols are available, you can also have checks on condition on the values of variables. For ex, if I want to stop at the call to __afl_maybe_log only if the the value of __afl_prev_loc is 0xdaef, I can use the following command:
(gdb) break __afl_maybe_log if __afl_prev_loc == 0xdaef
Note: Sometimes if the type of the variable is not known you might have to cast it to the correct type, also useful if you are checking the value of a memory address for ex:
(gdb) break __afl_maybe_log if (unsigned long)__afl_prev_loc == 0xdaef (gdb) break __afl_maybe_log if *(int *)($rbp - 0x10) == 0xdaef (gdb) break __afl_maybe_log if *(int *)0x7fffffffe0a0 == 0xdaef
if the breakpoint is already set, you can add a condition to it using the condition
command:
(gdb) condition 1 $rcx == 0xdaef
and you can remove the condition using the command :
(gdb) condition 1
which resets the condition of the breakpoint.
You can also choose to ignore the breakpoint a certain number of times by using the ignore
command:
(gdb) ignore 1 5
which will not stop at the breakpoint for the next 5 times, and then start stopping at the breakpoint again.
Catchpoints
Another useful feature, especially when debugging multithreaded C++ applications is the catch
command. It allows you to break the execution when a certain event occurs.
For ex, if I want to break when a thread is created, I can use the following command:
(gdb) catch thread create
Some common uses are :
catch syscall
: Break when a syscall is madecatch syscall <number/name>
: Break when a specific syscall is madecatch throw <regex>
: Break when a specific exception is thrown (C++)catch catch <regex>
: Break when a specific exception is caught (C++)catch signal <number/name>
: Break when a specific signal is sentcatch load/unload <regex>
: Break when a library is loaded/unloaded
Custom commands
TODO: Briefly explain how to write custom commands
GDB Python Scripting
GDB has a python scripting interface, which I have found to be useful exclusively for CTFs. I will try to quickly go over it, so that people can try it out for themselves.
Running a python script with gdb is as simple as running the following command:
gdb <target/executable> -x <path/to/script.py>
if you want to run a python script on a running gdb instance, you can use the source
command:
(gdb) source <path/to/script.py>
Defining custom commands in GDB
GDB allows you to define custom commands, in python. This is very useful to run a bunch of commands together and add some custom logic to it.
import gdb
class TestCommand(gdb.Command):
def __init__(self):
super(TestCommand, self).__init__("test", gdb.COMMAND_USER)
def invoke(self, arg, from_tty):
print("Test command invoked")
counter = 0
try:
total_count = int(arg.split(" ")[0])
skip_count = int(arg.split(" ")[1])
except:
print("Invalid arguments passed")
return
print(f"Total count: {total_count}")
print(f"Skip count: {skip_count}")
while counter < total_count:
if counter % skip_count == 0:
# Skip every skip count iteration
gdb.execute("continue")
continue
gdb.execute("info registers")
counter += 1
gdb.execute("continue")
TestCommand()
The above script defines a custom command called test
which takes 2 arguments - total_count
and skip_count
. The command will print the register values for every iteration, except for every skip_count
iteration.
Defining custom pretty printers
Say you have a struct defined as follows:
struct Node {
int value;
struct Node *next;
};
While debugging, the program you wish to print the structure of the linked list. You can define a custom pretty printer to do that.
TODO
CTF challenge solving
I have used GDB python scripting to solve a few CTF challenges. It’s a quick way to get some information from the binary, if you need to extract some runtime information.
For ex, here I am extracting characters which are generated during runtime, so that I can use them to reverse the encryption algorithm.
TODO
GDB Plugins
There are a lot of extensions available for GDB, which can make your life easier.
GEF
GEF is a GDB extension which adds a lot of useful features to GDB. It’s a must have for CTFs. I will try to do another blog post on features specific to GEF that can be really helpful for CTFs.
Decomp2dbg
This is a plugin which allows you to connect GDB to a decompiler. It’s shows you the decompiled code in GDB, and allows you to set breakpoints in the decompiled code. It’s extremely useful when you are debugging a binary without source code, and allows you to keep in sync with the decompiled code for faster debugging or reverse engineering.