1 .\" Copyright (c) 2003 Greg Lehey
2 .\" All rights reserved.
4 .\" Redistribution and use in source and binary forms, with or without
5 .\" modification, are permitted provided that the following conditions
7 .\" 1. Redistributions of source code must retain the above copyright
8 .\" notice, this list of conditions and the following disclaimer.
9 .\" 2. Redistributions in binary form must reproduce the above copyright
10 .\" notice, this list of conditions and the following disclaimer in the
11 .\" documentation and/or other materials provided with the distribution.
13 .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
14 .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
15 .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
16 .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
17 .\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
18 .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
19 .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
20 .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
21 .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
22 .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
25 .\" $FreeBSD: src/share/man/man4/gdb.4,v 1.9 2004/01/28 00:02:10 grog Exp $
26 .\" $DragonFly: src/share/man/man4/gdb.4,v 1.2 2004/12/20 14:21:19 swildner Exp $
33 .Nd external kernel debugger
35 .Cd "makeoptions DEBUG=-g"
37 .Cd "options GDB_REMOTE_CHAT"
41 kernel debugger is a variation of
43 which understands some aspects of the
46 It can be used in a number of ways:
49 It can be used to examine the memory of the processor on which it runs.
51 It can be used to analyse a processor dump after a panic.
53 It can be used to debug another system interactively via a serial or firewire
55 In this mode, the processor can be stopped and single stepped.
57 With a firewire link, it can be used to examine the memory of a remote system
58 without the participation of that system.
59 In this mode, the processor cannot be stopped and single stepped, but it can be
60 of use when the remote system has crashed and is no longer responding.
63 When used for remote debugging,
65 requires the presence of the
68 Commands exist to switch between
72 .Sh PREPARING FOR DEBUGGING
73 When debugging kernels, it is practically essential to have built a kernel with
75 .Pq Cd "makeoptions DEBUG=-g" .
76 It is easiest to perform operations from the kernel build directory, by default
77 .Pa /usr/obj/usr/src/sys/GENERIC .
79 First, ensure you have a copy of the debug macros in the directory:
83 This command performs some transformations on the macros installed in
84 .Pa /usr/src/tools/debugscripts
85 to adapt them to the local environment.
86 .Ss "Inspecting the environment of the local machine"
87 To look at and change the contents of the memory of the system you are running
90 .Dl "gdb -k -wcore kernel.debug /dev/mem"
92 In this mode, you need the
99 is a kernel data file.
100 You can look at live data, and if you include the
102 option, you can change it at your peril.
103 The system does not stop (obviously), so a number of things will not work.
104 You can set breakpoints, but you cannot
106 execution, so they will not work.
107 .Ss "Debugging a crash dump"
108 By default, crash dumps are stored in the directory
110 Investigate them from the kernel build directory with:
112 .Dl "gdb -k kernel.debug /var/crash/vmcore.29"
114 In this mode, the system is obviously stopped, so you can only look at it.
115 .Ss "Debugging a live system with a remote link"
116 In the following discussion, the term
118 refers to the system running the debugger, and
120 refers to the live system being debugged.
122 To debug a live system with a remote link, the kernel must be compiled with the
126 .Cd "options BREAK_TO_DEBUGGER"
127 enables the debugging machine stop the debugged machine once a connection has
128 been established by pressing
130 .Ss "Debugging a live system with a remote serial link"
131 When using a serial port for the remote link on the i386 platform, the serial
132 port must be identified by setting the flag bit
134 for the specified interface.
135 Generally, this port will also be used as a serial console (flag bit
138 .Pa /boot/device.hints
141 .Dl hint.sio.0.flags="0x90"
143 To share a console and debug connection on a serial line, use the
144 .Cd "options GDB_REMOTE_CHAT"
146 .Ss "Debugging a live system with a remote firewire link"
147 As with serial debugging, to debug a live system with a firewire link, the
148 kernel must be compiled with the option
151 .Cd "options GDB_REMOTE_CHAT"
152 is not necessary, since the firewire implementation uses separate ports for the
153 console and debug connection.
155 A number of steps must be performed to set up a firewire link:
158 Ensure that both systems have
160 support, and that the kernel of the remote system includes the
165 If they are not compiled into the kernel, load the KLDs:
167 .Dl "kldload firewire"
169 On the remote system only:
170 .Bd -literal -offset indent
175 You should see something like this in the
177 output of the remote system:
178 .Bd -literal -offset indent
180 fwohci0: node_id=0x8800ffc0, gen=2, non CYCLEMASTER mode
181 firewire0: 2 nodes, maxhop <= 1, cable IRM = 1
182 firewire0: bus manager 1
183 firewire0: New S400 device ID:00c04f3226e88061
184 dcons_crom0: <dcons configuration ROM> on firewire0
185 dcons_crom0: bus_addr 0x22a000
188 It is a good idea to load these modules at boot time with the following entry in
189 .Pa /boot/loader.conf :
191 .Dl dcons_crom_enable="YES"
193 This ensures that all three modules are loaded.
194 There is no harm in loading
198 on the local system, but if you only want to load the
200 module, include the following in
201 .Pa /boot/loader.conf :
203 .Dl firewire_enable="YES"
207 to find the firewire node corresponding to the remote machine.
208 On the local machine you might see:
209 .Bd -literal -offset indent
211 2 devices (info_len=2)
213 1 0x00c04f3226e88061 0
214 0 0x000199000003622b 1
217 The first node is always the local system, so in this case, node 0 is the remote
219 If there are more than two systems, check from the other end to find which node
220 corresponds to the remote system.
221 On the remote machine, it looks like this:
222 .Bd -literal -offset indent
224 2 devices (info_len=2)
226 0 0x000199000003622b 0
227 1 0x00c04f3226e88061 1
230 Next, establish a firewire connection with
233 .Dl "dconschat -br -G 5556 -t 0x000199000003622b"
235 .Li 0x000199000003622b
236 is the EUI64 address of the remote node, as determined from the output of
239 When started in this manner,
241 establishes a local tunnel connection from port
243 to the remote debugger.
244 You can also establish a console port connection with the
246 option to the same invocation
250 manpage for further details.
255 does not return control to the user.
256 It displays error messages and console output for the remote system, so it is a
257 good idea to start it in its own window.
259 Finally, establish connection:
260 .Bd -literal -offset indent
262 GNU gdb 5.2.1 (FreeBSD)
263 .Em "(political statements omitted)"
264 Ready to go. Enter 'tr' to connect to the remote target
265 with /dev/cuaa0, 'tr /dev/cuaa1' to connect to a different port
266 or 'trf portno' to connect to the remote target with the firewire
267 interface. portno defaults to 5556.
269 Type 'getsyms' after connection to load kld symbols.
271 If you're debugging a local system, you can use 'kldsyms' instead
272 to load the kld symbols. That's a less obnoxious interface.
279 macro assumes a connection on port 5556.
280 If you want to use a different port (by changing the invocation of
285 For example, if you want to use port 4711, run
289 .Dl "dconschat -br -G 4711 -t 0x000199000003622b"
291 Then establish connection with:
292 .Bd -literal -offset indent
293 (gdb) tr localhost:4711
297 .Ss "Non-cooperative debugging a live system with a remote firewire link"
298 In addition to the conventional debugging via firewire described in the previous
299 section, it is possible to debug a remote system without its cooperation, once
300 an initial connection has been established.
301 This corresponds to debugging a local machine using
303 It can be very useful if a system crashes and the debugger no longer responds.
304 To use this method, set the
307 .Va hw.firewire.fwmem.eui64_hi
309 .Va hw.firewire.fwmem.eui64_lo
310 to the upper and lower halves of the EUI64 ID of the remote system,
312 From the previous example, the remote machine shows:
313 .Bd -literal -offset indent
315 2 devices (info_len=2)
317 0 0x000199000003622b 0
318 1 0x00c04f3226e88061 1
322 .Bd -literal -offset indent
323 # sysctl -w hw.firewire.fwmem.eui64_hi=0x00019900
324 hw.firewire.fwmem.eui64_hi: 0 -> 104704
325 # sysctl -w hw.firewire.fwmem.eui64_lo=0x0003622b
326 hw.firewire.fwmem.eui64_lo: 0 -> 221739
329 Note that the variables must be explicitly stated in hexadecimal.
330 After this, you can examine the remote machine's state with the following input:
331 .Bd -literal -offset indent
332 # gdb -k kernel.debug /dev/fwmem0.0
333 GNU gdb 5.2.1 (FreeBSD)
334 .Em "(messages omitted)"
335 Reading symbols from /modules/dcons.ko...done.
336 Loaded symbols for /modules/dcons.ko
337 Reading symbols from /modules/dcons_crom.ko...done.
338 Loaded symbols for /modules/dcons_crom.ko
339 #0 sched_switch (td=0xc0922fe0) at /usr/src/sys/kern/sched_4bsd.c:621
343 In this case, it is not necessary to load the symbols explicitly.
344 The remote system continues to run.
346 The user interface to
353 This section discusses only the extensions for kernel debugging that get
354 installed in the kernel build directory.
355 .Ss "Debugging environment"
356 The following macros manipulate the debugging environment:
357 .Bl -tag -width indent
361 This command is only meaningful when performing remote debugging.
365 information for the target machine and invite user to paste it back in.
366 This is required because
368 does not allow data to be passed to shell scripts.
369 It is necessary for remote debugging and crash dumps; for local memory debugging
374 Read in the symbol tables for the debugging machine.
375 This does not work for
376 remote debugging and crash dumps; use
379 .It Ic tr Ar interface
380 Debug a remote system via the specified serial or firewire interface.
382 Debug a remote system via serial interface
385 Debug a remote system via serial interface
388 Debug a remote system via firewire interface at default port 5556.
395 are convenience commands which invoke
397 .Ss "The current process environment"
398 The following macros are convenience functions intended to make things easier
402 .Bl -tag -width indent
404 Select stack frame 0 and show assembler-level details.
406 Select stack frame 1 and show assembler-level details.
408 Select stack frame 2 and show assembler-level details.
410 Select stack frame 3 and show assembler-level details.
412 Select stack frame 4 and show assembler-level details.
414 Select stack frame 5 and show assembler-level details.
416 Show 12 words in hex, starting at current
420 List the next 10 instructions from the current
424 Show the register contents and the first four parameters of the current stack
427 Show the first parameter of current stack frame in various formats.
429 Show the second parameter of current stack frame in various formats.
431 Show the third parameter of current stack frame in various formats.
433 Show the fourth parameter of current stack frame in various formats.
435 Show the fifth parameter of current stack frame in various formats.
437 Show the last 12 words on stack in hexadecimal.
439 Show the register contents and the first ten parameters.
441 Single step 1 instruction (over calls) and show next instruction.
443 Single step 1 instruction (through calls) and show next instruction.
445 .Ss "Examining other processes"
446 The following macros access other processes.
450 does not understand the concept of multiple processes, so they effectively
454 .Bl -tag -width indent
456 Show a backtrace for the process
459 Show backtraces for all processes in the system.
461 Show a backtrace for the process previously selected with
464 Show a backtrace from the
467 .It Ic defproc Ar pid
468 Specify the PID of the process for some other commands in this section.
472 of the stack of the process previously selected with
475 Show some PCB contents of the process
478 .Ss "Examining data structures"
481 commands to look at most data structures.
482 The macros in this section are
483 convenience functions which typically display the data in a more readable
484 format, or which omit less interesting parts of the structure.
485 .Bl -tag -width indent
487 Show information about the buffer header pointed to by the variable
489 in the current frame.
495 in the current frame.
497 Show detailed information about the buffer header
499 pointed at by the local variable
502 Show summary information about the buffer header
504 pointed at by the parameter
507 Print a number of fields from the buffer header pointed at in by the pointer
509 in the current environment.
511 Show some information of the
513 pointed to by the local variable
516 .Ss "Miscellaneous macros"
517 .Bl -tag -width indent
519 Check unallocated memory for modifications.
520 This assumes that the kernel has been compiled with
521 .Cd "options DIAGNOSTIC"
522 This causes the contents of free memory to be set to
525 Print the system message buffer.
526 This corresponds to the
529 This macro used to be called
531 It can take a very long time over a serial line,
532 and it is even slower via firewire
533 or local memory due to inefficiencies in
535 When debugging a crash dump or over firewire, it is not necessary to start
537 to access the message buffer: instead, use an appropriate variation of
538 .Bd -literal -offset indent
539 dmesg -M /var/crash/vmcore.0 -N kernel.debug
540 dmesg -M /dev/fwmem0.0 -N kernel.debug
545 utility without options.
547 Print the command name of the current process.
550 This corresponds in concept, but not in appearance, to the
553 When debugging a crash dump or over firewire, it is not necessary to start
557 output: instead, use an appropriate variation of
558 .Bd -literal -offset indent
559 ps -M /var/crash/vmcore.0 -N kernel.debug
560 ps -M /dev/fwmem0.0 -N kernel.debug
563 Kludge for writing macros.
564 When writing macros, it is convenient to paste them
568 Unfortunately, if the macro is already defined,
574 It will not give up until you answer
576 This command is that answer.
577 It does nothing else except to print a warning
578 message to remind you to remove it again.
581 This man page was written by
582 .An "Greg Lehey" Aq grog@FreeBSD.org .
597 was never designed to debug kernels, and it is not a very good match.
602 implementation is very inefficient, and many operations are slow.
604 Serial debugging is even slower, and race conditions can make it difficult to
605 run the link at more than 9600 bps.
606 Firewire connections do not have this problem.
610 In general, the person who wrote them did so while looking for a specific
611 problem, so they may not be general enough, and they may behave badly when used
612 in ways for which they were not intended, even if those ways make sense.
614 Many of these commands only work on the ia32 architecture.