-
Notifications
You must be signed in to change notification settings - Fork 0
/
debugging_with_gdb.html
507 lines (422 loc) · 15 KB
/
debugging_with_gdb.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
<!--{
"Title": "Debugging Go Code with GDB",
"Path": "/doc/gdb"
}-->
<p><i>
This applies to the <code>gc</code> toolchain. Gccgo has native gdb support.
Besides this overview you might want to consult the
<a href="http://sourceware.org/gdb/current/onlinedocs/gdb/">GDB manual</a>.
</i></p>
<p>
GDB does not understand Go programs well.
The stack management, threading, and runtime contain aspects that differ
enough from the execution model GDB expects that they can confuse
the debugger, even when the program is compiled with gccgo.
As a consequence, although GDB can be useful in some situations, it is
not a reliable debugger for Go programs, particularly heavily concurrent ones.
Moreover, it is not a priority for the Go project to address these issues, which
are difficult.
In short, the instructions below should be taken only as a guide to how
to use GDB when it works, not as a guarantee of success.
</p>
<p>
In time, a more Go-centric debugging architecture may be required.
</p>
<h2 id="Introduction">Introduction</h2>
<p>
When you compile and link your Go programs with the <code>gc</code> toolchain
on Linux, Mac OS X, FreeBSD or NetBSD, the resulting binaries contain DWARFv3
debugging information that recent versions (>7.1) of the GDB debugger can
use to inspect a live process or a core dump.
</p>
<p>
Pass the <code>'-w'</code> flag to the linker to omit the debug information
(for example, <code>go build -ldflags "-w" prog.go</code>).
</p>
<p>
The code generated by the <code>gc</code> compiler includes inlining of
function invocations and registerization of variables. These optimizations
can sometimes make debugging with <code>gdb</code> harder. To disable them
when debugging, pass the flags <code>-gcflags "-N -l"</code> to the
<a href="/cmd/go"><code>go</code></a> command used to build the code being
debugged.
</p>
<h3 id="Common_Operations">Common Operations</h3>
<ul>
<li>
Show file and line number for code, set breakpoints and disassemble:
<pre>(gdb) <b>list</b>
(gdb) <b>list <i>line</i></b>
(gdb) <b>list <i>file.go</i>:<i>line</i></b>
(gdb) <b>break <i>line</i></b>
(gdb) <b>break <i>file.go</i>:<i>line</i></b>
(gdb) <b>disas</b></pre>
</li>
<li>
Show backtraces and unwind stack frames:
<pre>(gdb) <b>bt</b>
(gdb) <b>frame <i>n</i></b></pre>
</li>
<li>
Show the name, type and location on the stack frame of local variables,
arguments and return values:
<pre>(gdb) <b>info locals</b>
(gdb) <b>info args</b>
(gdb) <b>p variable</b>
(gdb) <b>whatis variable</b></pre>
</li>
<li>
Show the name, type and location of global variables:
<pre>(gdb) <b>info variables <i>regexp</i></b></pre>
</li>
</ul>
<h3 id="Go_Extensions">Go Extensions</h3>
<p>
A recent extension mechanism to GDB allows it to load extension scripts for a
given binary. The tool chain uses this to extend GDB with a handful of
commands to inspect internals of the runtime code (such as goroutines) and to
pretty print the built-in map, slice and channel types.
</p>
<ul>
<li>
Pretty printing a string, slice, map, channel or interface:
<pre>(gdb) <b>p <i>var</i></b></pre>
</li>
<li>
A $len() and $cap() function for strings, slices and maps:
<pre>(gdb) <b>p $len(<i>var</i>)</b></pre>
</li>
<li>
A function to cast interfaces to their dynamic types:
<pre>(gdb) <b>p $dtype(<i>var</i>)</b>
(gdb) <b>iface <i>var</i></b></pre>
<p class="detail"><b>Known issue:</b> GDB can’t automatically find the dynamic
type of an interface value if its long name differs from its short name
(annoying when printing stacktraces, the pretty printer falls back to printing
the short type name and a pointer).</p>
</li>
<li>
Inspecting goroutines:
<pre>(gdb) <b>info goroutines</b>
(gdb) <b>goroutine <i>n</i> <i>cmd</i></b>
(gdb) <b>help goroutine</b></pre>
For example:
<pre>(gdb) <b>goroutine 12 bt</b></pre>
</li>
</ul>
<p>
If you'd like to see how this works, or want to extend it, take a look at <a
href="/src/runtime/runtime-gdb.py">src/runtime/runtime-gdb.py</a> in
the Go source distribution. It depends on some special magic types
(<code>hash<T,U></code>) and variables (<code>runtime.m</code> and
<code>runtime.g</code>) that the linker
(<a href="/src/cmd/ld/dwarf.c">src/cmd/ld/dwarf.c</a>) ensures are described in
the DWARF code.
</p>
<p>
If you're interested in what the debugging information looks like, run
'<code>objdump -W 6.out</code>' and browse through the <code>.debug_*</code>
sections.
</p>
<h3 id="Known_Issues">Known Issues</h3>
<ol>
<li>String pretty printing only triggers for type string, not for types derived
from it.</li>
<li>Type information is missing for the C parts of the runtime library.</li>
<li>GDB does not understand Go’s name qualifications and treats
<code>"fmt.Print"</code> as an unstructured literal with a <code>"."</code>
that needs to be quoted. It objects even more strongly to method names of
the form <code>pkg.(*MyType).Meth</code>.
<li>All global variables are lumped into package <code>"main"</code>.</li>
</ol>
<h2 id="Tutorial">Tutorial</h2>
<p>
In this tutorial we will inspect the binary of the
<a href="/pkg/regexp/">regexp</a> package's unit tests. To build the binary,
change to <code>$GOROOT/src/regexp</code> and run <code>go test -c</code>.
This should produce an executable file named <code>regexp.test</code>.
</p>
<h3 id="Getting_Started">Getting Started</h3>
<p>
Launch GDB, debugging <code>regexp.test</code>:
</p>
<pre>
$ <b>gdb regexp.test</b>
GNU gdb (GDB) 7.2-gg8
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv 3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
Type "show copying" and "show warranty" for licensing/warranty details.
This GDB was configured as "x86_64-linux".
Reading symbols from /home/user/go/src/regexp/regexp.test...
done.
Loading Go Runtime support.
(gdb)
</pre>
<p>
The message <code>"Loading Go Runtime support"</code> means that GDB loaded the
extension from <code>$GOROOT/src/runtime/runtime-gdb.py</code>.
</p>
<p>
To help GDB find the Go runtime sources and the accompanying support script,
pass your <code>$GOROOT</code> with the <code>'-d'</code> flag:
</p>
<pre>
$ <b>gdb regexp.test -d $GOROOT</b>
</pre>
<p>
If for some reason GDB still can't find that directory or that script, you can load
it by hand by telling gdb (assuming you have the go sources in
<code>~/go/</code>):
</p>
<pre>
(gdb) <b>source ~/go/src/runtime/runtime-gdb.py</b>
Loading Go Runtime support.
</pre>
<h3 id="Inspecting_the_source">Inspecting the source</h3>
<p>
Use the <code>"l"</code> or <code>"list"</code> command to inspect source code.
</p>
<pre>
(gdb) <b>l</b>
</pre>
<p>
List a specific part of the source parametrizing <code>"list"</code> with a
function name (it must be qualified with its package name).
</p>
<pre>
(gdb) <b>l main.main</b>
</pre>
<p>
List a specific file and line number:
</p>
<pre>
(gdb) <b>l regexp.go:1</b>
(gdb) <i># Hit enter to repeat last command. Here, this lists next 10 lines.</i>
</pre>
<h3 id="Naming">Naming</h3>
<p>
Variable and function names must be qualified with the name of the packages
they belong to. The <code>Compile</code> function from the <code>regexp</code>
package is known to GDB as <code>'regexp.Compile'</code>.
</p>
<p>
Methods must be qualified with the name of their receiver types. For example,
the <code>*Regexp</code> type’s <code>String</code> method is known as
<code>'regexp.(*Regexp).String'</code>.
</p>
<p>
Variables that shadow other variables are magically suffixed with a number in the debug info.
Variables referenced by closures will appear as pointers magically prefixed with '&'.
</p>
<h3 id="Setting_breakpoints">Setting breakpoints</h3>
<p>
Set a breakpoint at the <code>TestFind</code> function:
</p>
<pre>
(gdb) <b>b 'regexp.TestFind'</b>
Breakpoint 1 at 0x424908: file /home/user/go/src/regexp/find_test.go, line 148.
</pre>
<p>
Run the program:
</p>
<pre>
(gdb) <b>run</b>
Starting program: /home/user/go/src/regexp/regexp.test
Breakpoint 1, regexp.TestFind (t=0xf8404a89c0) at /home/user/go/src/regexp/find_test.go:148
148 func TestFind(t *testing.T) {
</pre>
<p>
Execution has paused at the breakpoint.
See which goroutines are running, and what they're doing:
</p>
<pre>
(gdb) <b>info goroutines</b>
1 waiting runtime.gosched
* 13 running runtime.goexit
</pre>
<p>
the one marked with the <code>*</code> is the current goroutine.
</p>
<h3 id="Inspecting_the_stack">Inspecting the stack</h3>
<p>
Look at the stack trace for where we’ve paused the program:
</p>
<pre>
(gdb) <b>bt</b> <i># backtrace</i>
#0 regexp.TestFind (t=0xf8404a89c0) at /home/user/go/src/regexp/find_test.go:148
#1 0x000000000042f60b in testing.tRunner (t=0xf8404a89c0, test=0x573720) at /home/user/go/src/testing/testing.go:156
#2 0x000000000040df64 in runtime.initdone () at /home/user/go/src/runtime/proc.c:242
#3 0x000000f8404a89c0 in ?? ()
#4 0x0000000000573720 in ?? ()
#5 0x0000000000000000 in ?? ()
</pre>
<p>
The other goroutine, number 1, is stuck in <code>runtime.gosched</code>, blocked on a channel receive:
</p>
<pre>
(gdb) <b>goroutine 1 bt</b>
#0 0x000000000040facb in runtime.gosched () at /home/user/go/src/runtime/proc.c:873
#1 0x00000000004031c9 in runtime.chanrecv (c=void, ep=void, selected=void, received=void)
at /home/user/go/src/runtime/chan.c:342
#2 0x0000000000403299 in runtime.chanrecv1 (t=void, c=void) at/home/user/go/src/runtime/chan.c:423
#3 0x000000000043075b in testing.RunTests (matchString={void (struct string, struct string, bool *, error *)}
0x7ffff7f9ef60, tests= []testing.InternalTest = {...}) at /home/user/go/src/testing/testing.go:201
#4 0x00000000004302b1 in testing.Main (matchString={void (struct string, struct string, bool *, error *)}
0x7ffff7f9ef80, tests= []testing.InternalTest = {...}, benchmarks= []testing.InternalBenchmark = {...})
at /home/user/go/src/testing/testing.go:168
#5 0x0000000000400dc1 in main.main () at /home/user/go/src/regexp/_testmain.go:98
#6 0x00000000004022e7 in runtime.mainstart () at /home/user/go/src/runtime/amd64/asm.s:78
#7 0x000000000040ea6f in runtime.initdone () at /home/user/go/src/runtime/proc.c:243
#8 0x0000000000000000 in ?? ()
</pre>
<p>
The stack frame shows we’re currently executing the <code>regexp.TestFind</code> function, as expected.
</p>
<pre>
(gdb) <b>info frame</b>
Stack level 0, frame at 0x7ffff7f9ff88:
rip = 0x425530 in regexp.TestFind (/home/user/go/src/regexp/find_test.go:148);
saved rip 0x430233
called by frame at 0x7ffff7f9ffa8
source language minimal.
Arglist at 0x7ffff7f9ff78, args: t=0xf840688b60
Locals at 0x7ffff7f9ff78, Previous frame's sp is 0x7ffff7f9ff88
Saved registers:
rip at 0x7ffff7f9ff80
</pre>
<p>
The command <code>info locals</code> lists all variables local to the function and their values, but is a bit
dangerous to use, since it will also try to print uninitialized variables. Uninitialized slices may cause gdb to try
to print arbitrary large arrays.
</p>
<p>
The function’s arguments:
</p>
<pre>
(gdb) <b>info args</b>
t = 0xf840688b60
</pre>
<p>
When printing the argument, notice that it’s a pointer to a
<code>Regexp</code> value. Note that GDB has incorrectly put the <code>*</code>
on the right-hand side of the type name and made up a 'struct' keyword, in traditional C style.
</p>
<pre>
(gdb) <b>p re</b>
(gdb) p t
$1 = (struct testing.T *) 0xf840688b60
(gdb) p t
$1 = (struct testing.T *) 0xf840688b60
(gdb) p *t
$2 = {errors = "", failed = false, ch = 0xf8406f5690}
(gdb) p *t->ch
$3 = struct hchan<*testing.T>
</pre>
<p>
That <code>struct hchan<*testing.T></code> is the runtime-internal representation of a channel. It is currently empty, or gdb would have pretty-printed it's contents.
</p>
<p>
Stepping forward:
</p>
<pre>
(gdb) <b>n</b> <i># execute next line</i>
149 for _, test := range findTests {
(gdb) <i># enter is repeat</i>
150 re := MustCompile(test.pat)
(gdb) <b>p test.pat</b>
$4 = ""
(gdb) <b>p re</b>
$5 = (struct regexp.Regexp *) 0xf84068d070
(gdb) <b>p *re</b>
$6 = {expr = "", prog = 0xf840688b80, prefix = "", prefixBytes = []uint8, prefixComplete = true,
prefixRune = 0, cond = 0 '\000', numSubexp = 0, longest = false, mu = {state = 0, sema = 0},
machine = []*regexp.machine}
(gdb) <b>p *re->prog</b>
$7 = {Inst = []regexp/syntax.Inst = {{Op = 5 '\005', Out = 0, Arg = 0, Rune = []int}, {Op =
6 '\006', Out = 2, Arg = 0, Rune = []int}, {Op = 4 '\004', Out = 0, Arg = 0, Rune = []int}},
Start = 1, NumCap = 2}
</pre>
<p>
We can step into the <code>String</code>function call with <code>"s"</code>:
</p>
<pre>
(gdb) <b>s</b>
regexp.(*Regexp).String (re=0xf84068d070, noname=void) at /home/user/go/src/regexp/regexp.go:97
97 func (re *Regexp) String() string {
</pre>
<p>
Get a stack trace to see where we are:
</p>
<pre>
(gdb) <b>bt</b>
#0 regexp.(*Regexp).String (re=0xf84068d070, noname=void)
at /home/user/go/src/regexp/regexp.go:97
#1 0x0000000000425615 in regexp.TestFind (t=0xf840688b60)
at /home/user/go/src/regexp/find_test.go:151
#2 0x0000000000430233 in testing.tRunner (t=0xf840688b60, test=0x5747b8)
at /home/user/go/src/testing/testing.go:156
#3 0x000000000040ea6f in runtime.initdone () at /home/user/go/src/runtime/proc.c:243
....
</pre>
<p>
Look at the source code:
</p>
<pre>
(gdb) <b>l</b>
92 mu sync.Mutex
93 machine []*machine
94 }
95
96 // String returns the source text used to compile the regular expression.
97 func (re *Regexp) String() string {
98 return re.expr
99 }
100
101 // Compile parses a regular expression and returns, if successful,
</pre>
<h3 id="Pretty_Printing">Pretty Printing</h3>
<p>
GDB's pretty printing mechanism is triggered by regexp matches on type names. An example for slices:
</p>
<pre>
(gdb) <b>p utf</b>
$22 = []uint8 = {0 '\000', 0 '\000', 0 '\000', 0 '\000'}
</pre>
<p>
Since slices, arrays and strings are not C pointers, GDB can't interpret the subscripting operation for you, but
you can look inside the runtime representation to do that (tab completion helps here):
</p>
<pre>
(gdb) <b>p slc</b>
$11 = []int = {0, 0}
(gdb) <b>p slc-></b><i><TAB></i>
array slc len
(gdb) <b>p slc->array</b>
$12 = (int *) 0xf84057af00
(gdb) <b>p slc->array[1]</b>
$13 = 0</pre>
<p>
The extension functions $len and $cap work on strings, arrays and slices:
</p>
<pre>
(gdb) <b>p $len(utf)</b>
$23 = 4
(gdb) <b>p $cap(utf)</b>
$24 = 4
</pre>
<p>
Channels and maps are 'reference' types, which gdb shows as pointers to C++-like types <code>hash<int,string>*</code>. Dereferencing will trigger prettyprinting
</p>
<p>
Interfaces are represented in the runtime as a pointer to a type descriptor and a pointer to a value. The Go GDB runtime extension decodes this and automatically triggers pretty printing for the runtime type. The extension function <code>$dtype</code> decodes the dynamic type for you (examples are taken from a breakpoint at <code>regexp.go</code> line 293.)
</p>
<pre>
(gdb) <b>p i</b>
$4 = {str = "cbb"}
(gdb) <b>whatis i</b>
type = regexp.input
(gdb) <b>p $dtype(i)</b>
$26 = (struct regexp.inputBytes *) 0xf8400b4930
(gdb) <b>iface i</b>
regexp.input: struct regexp.inputBytes *
</pre>