Programming Languages on Power

 View Only

Basic debugging techniques for .NET applications

By Amit Sirohi posted Fri April 14, 2023 10:38 AM

  

Enabling verbosity options allows you to get more information while building C# applications. In this blog, you will learn how to enable verbosity options using the ‘dotnet CLI command’ using .NET environment variables to make debugging easier. In the latter sections, you will learn how to debug using GNU Debugger (referred to as GDB, hereafter) and capture coredumps for a crashing C# application.

Verbosity

Verbosity is used to display extended (additional information) while building or running the C# application. You can enable the verbosity level at build time using following command:

dotnet build -v|--verbosity

For example:

dotnet build --verbosity diagnostic

This enables the diagnostic output.

For verbosity, there are different options available. Permissible values are q[uiet], m[inimal], n[ormal], d[etailed], and diag[nostic]. The default value is minimal. By default, MSBuild displays warnings and errors at all verbosity levels. To exclude warnings, use /property:WarningLevel=0. Different values for verbosity are shown in the following table.

S.NO

Fields

Value

Explanation

1

Detailed

3

Relatively verbose, but not exhaustive

2

Diagnostic

4

The most verbose and informative verbosity

3

Minimal

1

Relatively little output

4

Normal

2

Standard output. This is the default value if verbosity level is not set

5

Quiet

0

The most minimal output

Examples

This section includes several examples of verbose messages for different options of HelloWorld program.

Create a HelloWorld program using following command:

dotnet new console -o HelloWorld  -f net7.0
cd Helloworld

Enable log messages using the verbosity option and run the HelloWorld program. Check the following examples that are set with different values of verbosity.

Example 1: Verbosity set to minimal

dotnet run --verbosity minimal
MSBuild version 17.4.0+18d5aef85 for .NET
  Determining projects to restore...
  All projects are up-to-date for restore.
  HelloWorld -> /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/bin/Debug/net7.0/HelloWorld.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:05.83

Example 2: Verbosity set to normal

dotnet run –verbosity normal
MSBuild version 17.4.0+18d5aef85 for .NET
Build started 2/28/2023 7:53:42 AM.
     1>Project "/home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj" on node 1 (Restore target(s)).
     1>_GetAllRestoreProjectPathItems:
         Determining projects to restore...
       Restore:
         X.509 certificate chain validation will use the fallback certificate bundle at '/home/ubuntu/amit/new-dotnet-7.0.1/output/.dotnet/sdk/7.0.100/trustedroots/codesignctl.pem'.
         X.509 certificate chain validation will use the fallback certificate bundle at '/home/ubuntu/amit/new-dotnet-7.0.1/output/.dotnet/sdk/7.0.100/trustedroots/timestampctl.pem'.
         Assets file has not changed. Skipping assets file writing. Path: /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/obj/project.assets.json
         Restored /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj (in 212 ms).
         
         NuGet Config files used:
             /root/.nuget/NuGet/NuGet.Config
         
         Feeds used:
             https://api.nuget.org/v3/index.json
         All projects are up-to-date for restore.
     1>Done Building Project "/home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj" (Restore target(s)).
   1:7>Project "/home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj" on node 1 (default targets).
     1>GenerateTargetFrameworkMonikerAttribute:
       Skipping target "GenerateTargetFrameworkMonikerAttribute" because all output files are up-to-date with respect to the input files.
       CoreGenerateAssemblyInfo:
       Skipping target "CoreGenerateAssemblyInfo" because all output files are up-to-date with respect to the input files.
       CoreCompile:
       Skipping target "CoreCompile" because all output files are up-to-date with respect to the input files.
       _CreateAppHost:
       Skipping target "_CreateAppHost" because all output files are up-to-date with respect to the input files.
       _CopyOutOfDateSourceItemsToOutputDirectory:
       Skipping target "_CopyOutOfDateSourceItemsToOutputDirectory" because all output files are up-to-date with respect to the input files.
       GenerateBuildDependencyFile:
       Skipping target "GenerateBuildDependencyFile" because all output files are up-to-date with respect to the input files.
       GenerateBuildRuntimeConfigurationFiles:
       Skipping target "GenerateBuildRuntimeConfigurationFiles" because all output files are up-to-date with respect to the input files.
       CopyFilesToOutputDirectory:
         HelloWorld -> /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/bin/Debug/net7.0/HelloWorld.dll
     1>Done Building Project "/home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:05.84

Example 3: Verbosity set to diagnostic

When verbosity is set to diagnostic, the size of the log messages is very large. To simplify the information, this examples uses a build hierarchy.

dotnet run --verbosity diagnostic
Detailed Build Summary
======================
     (TaskId:0)
                     
                     ============================== Build Hierarchy (IDs represent configurations) =====================================================
                     Id                  : Exclusive Time   Total Time   Path (Targets)
                     ----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
                     0                   : 3.554s           4.274s       /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj (Restore)  (TaskId:0)
                     | 1                 : 0.133s           0.133s       /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj (_IsProjectRestoreSupported)  (TaskId:0)
                     | 2                 : 0.023s           0.023s       /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj (_GenerateRestoreProjectPathWalk)  (TaskId:0)
                     | 3                 : 0.529s           0.529s       /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj (_GenerateRestoreGraphProjectEntry)  (TaskId:0)
                     . 4                 : 0.031s           0.031s       /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj (_GenerateProjectRestoreGraph)  (TaskId:0)
                     
                     ============================== Node Utilization (IDs represent configurations) ====================================================
                     Timestamp:            1        Duration   Cumulative
                     ----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
                     638131675728116122:   0        2.526s     2.526s .................................................. (TaskId:0)
                     638131675753373668:   1        0.136s     2.662s .. (TaskId:0)
                     638131675754731518:   0        0.087s     2.748s . (TaskId:0)
                     638131675755598601:   2        0.024s     2.772s  (TaskId:0)
                     638131675755834258:   0        0.016s     2.788s  (TaskId:0)
                     638131675755996086:   3        0.529s     3.317s .......... (TaskId:0)
                     638131675761287680:   0        0.004s     3.321s  (TaskId:0)
                     638131675761326755:   4        0.031s     3.352s  (TaskId:0)
                     638131675761634841:   0        0.947s     4.299s .................. (TaskId:0)
                     -----------------------------------------------------------------------------------------------------------------------------------
                     Utilization:          100.0    Average Utilization: 100.0 (TaskId:0)
                     
                     Detailed Build Summary
                     ======================
                          (TaskId:0)
                     
                     ============================== Build Hierarchy (IDs represent configurations) =====================================================
                     Id                  : Exclusive Time   Total Time   Path (Targets)
                     ----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
                     5                   : 1.413s           1.413s       /home/ubuntu/amit/new-dotnet-7.0.1/HelloWorld/HelloWorld.csproj ()  (TaskId:0)
                     
                     ============================== Node Utilization (IDs represent configurations) ====================================================
                     Timestamp:            1        Duration   Cumulative
                     ----------------------------------------------------------------------------------------------------------------------------------- (TaskId:0)
                     638131675728116122:   0        2.526s     2.526s .................................................. (TaskId:0)
                     638131675753373668:   1        0.136s     2.662s .. (TaskId:0)
                     638131675754731518:   0        0.087s     2.748s . (TaskId:0)
                     638131675755598601:   2        0.024s     2.772s  (TaskId:0)
                     638131675755834258:   0        0.016s     2.788s  (TaskId:0)
                     638131675755996086:   3        0.529s     3.317s .......... (TaskId:0)
                     638131675761287680:   0        0.004s     3.321s  (TaskId:0)
                     638131675761326755:   4        0.031s     3.352s  (TaskId:0)
                     638131675761634841:   0        0.947s     4.299s .................. (TaskId:0)
                     638131675771471043:   5        1.413s     5.748s ............................ (TaskId:0)
                     -----------------------------------------------------------------------------------------------------------------------------------
                     Utilization:          99.4     Average Utilization: 99.4 (TaskId:0)

The data is split into two sections: The Build Hierarchy and the Node Utilization. Let us see each of these one by one.

Build Hierarchy

The Build Hierarchy section shows all of the projects that were built. Each of the columns contains the following information:

  • ID – The request ID which was built. A request is any request to build a target on a project. This can be from the command-line or through an MSBuild task. Note that if the same project file is invoked with the same target, using the same global properties and tools version multiple times during the build, it may show up multiple times in the graph in different places. This is okay. However, sometimes it is invoked multiple times, but does not show up. This is due to the way MSBuild works internally, meaning, it can reduce the instances of such multiple entries. Thus, we do not see them in the graph.

  • Exclusive Time – The amount of time MSBuild spends actually executing the tasks and targets in that request. This does NOT include the time spent waiting for an MSBuild task to build the projects it depends on.

  • Total Time – This is the time spent executing tasks and targets plus the time spent waiting for other dependency requests to build.

  • Path – Displays the path to the project file which was invoked for that request.

  • (Targets) – The list of targets which were specified. It is always expressed in parenthesis. If this is empty, it means the default targets were executed instead. This list never shows the default or initial targets, but only those explicitly specified on the command-line or in the Targets parameter of the MSBuild task.

Node Utilization

The Node Utilization section shows how MSBuild has allocated requests to build using the nodes available. The columns are explained as follows:

  • Timestamp – This is the wall-clock time for the current event. We generate a new event any time something has changed about how work is distributed. The time between events may vary significantly because of this.

  • (numbered columns) – This is the ‘current work’ display for each node. The following symbols may appear:

    • (number) – This represents a request and corresponds to the requests in the Build Hierarchy. It specifically means that the specified request has either started or resumed on the node.
    • x – The node is idle and doing no work.
    • | – The node is still working on the current request. The request number is the one at the top of the | symbols.
  • Duration – This is the amount of time the system spent in this state.

  • Cumulative – This is the total amount of time which has elapsed from the beginning of the build until this state ends.

The detailed build summary diagnostic output can provide some useful information about how MSBuild sees and builds your projects. You can use it to determine the actual project dependencies and relative build times of all of your projects. This information can then be used to better organize your projects for the purposes of more efficient builds, and also get performance summary of each component.

COREHOST_TRACE

COREHOST_TRACE controls diagnostics tracing from the hosting components, such as dotnet.exe, hostfxr, and hostpolicy.

COREHOST_TRACE=[0/1] - default setting is 0 -it denotes tracing is disabled. If set to 1, diagnostics tracing is enabled.

COREHOST_TRACEFILE=

COREHOST_TRACE_VERBOSITY=[1/2/3/4] - default setting is 4. The setting is used only when tracing is enabled via COREHOST_TRACE=1.

4 - all tracing information is written.

3 - only informational, warning, and error messages are written.

2 - only warning and error messages are written.

1 - only error messages are written.

The typical way to get detailed trace information about application startup is to set COREHOST_TRACE=1 and COREHOST_TRACEFILE=host_trace.txt and then run the application. A new file host_trace.txt will be created in the current directory with the detailed information.

Trace levels

The Mono runtime will output different amounts of information based on the log level. The log level is set by the MONO_LOG_LEVEL environment variable. Modifying this variable after startup will not change the log level. It should always be set before running the application. Possible values for the logging level are “error”, “critical”, “warning”, “message”, “info”, and “debug”. All messages with a higher level than the current value will be displayed. So setting the log level to “warning” will display all “warning”, “critical”, and “error” messages. Setting the log level to “debug” will display all messages. The default log level is “error”.

Trace Filters

The runtime can display a number of different types of trace information. The MONO_LOG_MASK environment variable should be used to filter the data, so that it can be interpreted more easily. The mask should be set before the application is run and can not be altered while the application is running. Setting the mask will enable logging for only certain runtime functions. Possible values for the log mask are:

"all" - Display all messages regardless of type

"aot" - the Ahead Of Time precompiler

"asm" - The assembly loader

"cfg" - The configuration file loader

"dll" - The native library (pinvoke) loader

"gc" - Garbage collector information

"io-layer" - I/O layer. You can select I/O layers subsystems, processes, files, sockets, events, semaphores, mutexes and handleswith the following categories instead of io-layer: "io-layer-process", "io-layer-file", "io-layer-socket", "io-layer-event", "io-layer-semaphore", "io-layer-mutex", "io-layer-handle".

"io-selector" - async socket operations

"security" - e.g. Moonlight CoreCLR support

"threadpool" - thread pool generic

"type" - Type load information

The MONO_LOG_MASK can be set to one or more values. Values are separated by a comma. When you want to display only the messages for native library loading and config file loading, the log mask will look like this: MONO_LOG_MASK=”dll,cfg”. The default log mask is “all”. Thus, if you do not set the mask, all message types will be displayed.

Example 4: Debugging System.DllNotFoundException error

If you are facing a DLLImport issue and are getting the typical System.DllNotFoundException, here is how you can debug this problem.

If you get a DllNotFoundException when loading an unmanaged library:

  • Set the below mentioned environment variables

    export MONO_LOG_LEVEL=debug
    export MONO_LOG_MASK=dll
  • Run your application. You should get trace information on the library names that the runtime is trying to use, and the errors in loading those libraries.

Mono --trace[=expression]

Mono has a trace-like feature built into the runtime. This is useful to see which methods are being called by your application. After setting MONO_ENV_OPTIONS environment variable just execute dotnet run command to print the traces according to requirement. The sections below explain the options available to enable tracing.

Note: .NET for Linux on Power is built using the .NET Core Mono Runtime Engine

export MONO_ENV_OPTIONS="--trace=all”

Mono shows method names as they are invoked. By default all methods are traced. The trace can be customized to include or exclude methods, classes or assemblies. A trace expression is a comma separated list of targets. Each target can be prefixed with a minus sign to turn off a particular target. The words `program', `all' and `disabled' have special meaning. `Program' refers to the main program being executed, and `all' means all the method calls. The `disabled' option is used to start up with tracing disabled.

Assemblies are specified by their name, for example, to trace all calls in the System assembly, use:

# Trace all calls in the System assembly
export MONO_ENV_OPTIONS=”--trace=System”

Classes are specified with the T: prefix. For example, to trace all calls to the System.String class, use:

# trace all call to the System.String class
export MONO_ENV_OPTIONS=”--trace=T:System.String” 

Individual methods are referenced with the M: prefix, and the standard method notation:

# Traching the System.Console.WriteLine Method
export MONO_ENV_OPTIONS=”--trace=M:System.Console:WriteLine” 

Exceptions can also be traced. It will cause a stack trace to be printed every time an exception of the specified type is thrown. The exception type can be specified with or without the namespace. To trace all exceptions, specify 'all' as the type name use –trace=E:all which will report on all exceptions.

# Tracing the exception System.Exception
export MONO_ENV_OPTIONS=”--trace= E:System.Exception”

Trace managed to unmanaged transitions using the wrapper qualifier:

# Trace the managed to unmanaged transitions
export MONO_ENV_OPTIONS=”--trace=wrapper”

Namespaces can be specified using the N: prefix:

# Trace the System.Xml namespace
export MONO_ENV_OPTIONS=”--trace=N:System.Xml”

Use --trace=disabled to disable tracing in mono.

# Disable mono tracing
export set  MONO_ENV_OPTIONS="--trace=disabled"

Binary Log - binlog

The binary log is a detailed description of the build process that can later be used to reconstruct text logs and used by other analysis tools. A binary log is usually 10-20x smaller than the most detailed text diagnostic-level log, but it contains more information. The binary logger by default collects the source text of project files, including all imported projects and target files encountered during the build.

Note: You can also produce binary logs from the dotnet build command by passing the -bl argument.

# Generate binary logs  using the -bl option
# dotnet build -bl:<log-file-path> <path-of-solution-file>
dotnet build -bl:/home/ubuntu/amit/logs/msbuild.binlog /home/ubuntu/amit/sol/Solution.sln

Creating and debugging coredump

In this section we learn how to generate coredump and debug it using GDB to get info of current state when crash occurs.

Run the following commands to generate coredump on defined path.

echo "/home/ubuntu/amit/core.%p" > /proc/sys/kernel/core_pattern
ulimit -c unlimited

Run the following command to debug coredump.

gdb <executable> --core=<path-of-core>

GDB commands to analyze coredump

Use the gdb dotnet -c <fully qualified core file name> command 

$ gdb dotnet -c /home/ubuntu/amit/core.3774750
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "powerpc64le-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from dotnet...
(No debugging symbols found in dotnet)
[New LWP 3774750]
[New LWP 3774751]
[New LWP 3774752]
[New LWP 3774753]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
Core was generated by `/home/ubuntu/amit/preview7/output_old_2/.dotnet/dotnet /home/ubuntu/amit/previe'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007d3e45a3d168 in __libc_signal_restore_set (set=0x7fffceb8ac58) at ../sysdeps/unix/sysv/linux/internal-signals.h:86
86      ../sysdeps/unix/sysv/linux/internal-signals.h: No such file or directory.
[Current thread is 1 (Thread 0x7d3e46145260 (LWP 3774750))]
(gdb)

Use the backtrace or bt command to display the backtrace of functions.

(gdb) bt
#0  0x00007d3e45a3d168 in __libc_signal_restore_set (set=0x7fffceb8ac58) at ../sysdeps/unix/sysv/linux/internal-signals.h:86
#1  __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:48
#2  0x00007d3e45a14850 in __GI_abort () at abort.c:79
#3  0x00007d3e457925d0 in mono_post_native_crash_handler (signal=<optimized out>, mctx=<optimized out>, info=<optimized out>,
    crash_chaining=<optimized out>) at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-posix.c:882
#4  0x00007d3e4574242c in mono_handle_native_crash (signal=0x7d3e45874317 "SIGSEGV", mctx=0x7fffceb8b0a8, info=0x7d3e44d95c2c)
    at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-exceptions.c:3011
#5  0x00007d3e4568c3e8 in mono_sigsegv_signal_handler_debug (_dummy=11, _info=0x7fffceb8c0e8, context=0x7fffceb8b370, debug_fault_addr=0x6e)
    at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-runtime.c:3773
#6  <signal handler called>
#7  0x00007d3e457454dc in common_call_trampoline (regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", m=0x67670473660, vt=0x0, vtable_slot=0x0,
    error=0x7fffceb8c498) at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:539
#8  0x00007d3e457450f8 in mono_magic_trampoline (regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", arg=<optimized out>, tramp=<optimized out>)
    at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:759
#9  0x00007d3e450901c4 in ?? ()
#10 0x00007d3e44d95c2c in ?? ()
(gdb)

Use the frame <frame-number> 

(gdb) f 3
#3  0x00007d3e457925d0 in mono_post_native_crash_handler (signal=<optimized out>, mctx=<optimized out>, info=<optimized out>,
    crash_chaining=<optimized out>) at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-posix.c:882
882     /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-posix.c: No such file or directory.
(gdb) f 7
#7  0x00007d3e457454dc in common_call_trampoline (regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", m=0x67670473660, vt=0x0, vtable_slot=0x0,
    error=0x7fffceb8c498) at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:539
539     /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c: No such file or directory.
(gdb)

Use the info frame command to display the information about current frame.

(gdb) info frame
Stack level 7, frame at 0x7fffceb8c460:
 pc = 0x7d3e457454dc in common_call_trampoline (/root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:539);
    saved pc = 0x7d3e457450f8
 called by frame at 0x7fffceb8c510, caller of frame at 0x7fffceb8b2f0
 source language c.
 Arglist at 0x7fffceb8c370, args: regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", m=0x67670473660, vt=0x0, vtable_slot=0x0, error=0x7fffceb8c498
 Locals at 0x7fffceb8c370, Previous frame's sp is 0x7fffceb8c460
 Saved registers:
  r19 at 0x7fffceb8c3f8, r20 at 0x7fffceb8c400, r21 at 0x7fffceb8c408, r22 at 0x7fffceb8c410, r23 at 0x7fffceb8c418, r24 at 0x7fffceb8c420,
  r25 at 0x7fffceb8c428, r26 at 0x7fffceb8c430, r27 at 0x7fffceb8c438, r28 at 0x7fffceb8c440, r29 at 0x7fffceb8c448, r30 at 0x7fffceb8c450,
  pc at 0x7fffceb8c470, lr at 0x7fffceb8c470, fpscr at 0x7fffceb8c468
(gdb)

Use the down command to move to the immediate ‘called’ frame.

(gdb) down
#6  <signal handler called>
(gdb) down
#5  0x00007d3e4568c3e8 in mono_sigsegv_signal_handler_debug (_dummy=11, _info=0x7fffceb8c0e8, context=0x7fffceb8b370, debug_fault_addr=0x6e)
    at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-runtime.c:3773
3773    /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-runtime.c: No such file or directory.
(gdb)

Use the up command to move to the immediate ‘calling frame.

(gdb) up
#6  <signal handler called>
(gdb) up
#7  0x00007d3e457454dc in common_call_trampoline (regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", m=0x67670473660, vt=0x0, vtable_slot=0x0,
    error=0x7fffceb8c498) at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:539
539     /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c: No such file or directory.
(gdb) up
#8  0x00007d3e457450f8 in mono_magic_trampoline (regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", arg=<optimized out>, tramp=<optimized out>)
    at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:759
759     in /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c
(gdb)

Use the ptype <variable-name> command to get the type of variable.

(gdb) f 7
#7  0x00007d3e457454dc in common_call_trampoline (regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", m=0x67670473660, vt=0x0, vtable_slot=0x0,
    error=0x7fffceb8c498) at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:539
539     in /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c
(gdb) ptype regs
type = long *
(gdb) ptype m
type = struct _MonoMethod {
    guint16 flags;
    guint16 iflags;
    guint32 token;
    MonoClass *klass;
    MonoMethodSignature *signature;
    const char *name;
    unsigned int inline_info : 1;
    unsigned int inline_failure : 1;
    unsigned int wrapper_type : 5;
    unsigned int string_ctor : 1;
    unsigned int save_lmf : 1;
    unsigned int dynamic : 1;
    unsigned int sre_method : 1;
    unsigned int is_generic : 1;
    unsigned int is_inflated : 1;
    unsigned int skip_visibility : 1;
    unsigned int _unused : 2;
    int slot : 16;
} *
(gdb)

Use print <variable-name> command to print the value of variable.

(gdb) f 7
#7  0x00007d3e457454dc in common_call_trampoline (regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", m=0x67670473660, vt=0x0, vtable_slot=0x0,
    error=0x7fffceb8c498) at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:539
539     in /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c
(gdb) print m
$1 = (MonoMethod *) 0x67670473660
(gdb) print *m
$2 = {flags = 22, iflags = 0, token = 100672073, klass = 0x67670447c60, signature = 0x676703bcda0,
  name = 0x7d3e42255966 <error: Cannot access memory at address 0x7d3e42255966>, inline_info = 0, inline_failure = 0, wrapper_type = 0, string_ctor = 0,
  save_lmf = 0, dynamic = 0, sre_method = 0, is_generic = 0, is_inflated = 1, skip_visibility = 0, _unused = 0, slot = -1}
(gdb) print m->klass
$3 = (MonoClass *) 0x67670447c60
(gdb) print *(m->klass)
$4 = {element_class = 0x67670447c60, cast_class = 0x67670447c60, supertypes = 0x67670302bd0, idepth = 2, rank = 0 '\000', class_kind = 1 '\001',
  instance_size = 16, inited = 1, size_inited = 1, valuetype = 0, enumtype = 0, blittable = 1, unicode = 0, wastypebuilder = 0,
  is_array_special_interface = 0, is_byreflike = 0, min_align = 1 '\001', packing_size = 0, ghcimpl = 0, has_finalize = 0, delegate = 0,
  gc_descr_inited = 1, has_cctor = 0, has_references = 0, has_ref_fields = 0, has_static_refs = 0, no_special_static_fields = 0, is_com_object = 0,
  nested_classes_inited = 0, interfaces_inited = 1, simd_type = 0, has_finalize_inited = 1, fields_inited = 1, has_failure = 0, has_weak_fields = 0,
  has_dim_conflicts = 0, any_field_has_auto_layout = 0, parent = 0x6766ff7e228, nested_in = 0x6767042cc70, image = 0x676700d78f0,
  name = 0x7d3e4223c373 <error: Cannot access memory at address 0x7d3e4223c373>,
  name_space = 0x7d3e422323ec <error: Cannot access memory at address 0x7d3e422323ec>, type_token = 33556493, vtable_size = 4, interface_count = 0,
  interface_id = 0, max_interface_id = 0, interface_offsets_count = 0, interfaces_packed = 0x67670302c00, interface_offsets_packed = 0x67670302c00,
  interface_bitmap = 0x67670302c00 "", interfaces = 0x0, sizes = {class_size = 0, element_size = 0, generic_param_token = 0}, fields = 0x67670302bd0,
  methods = 0x0, this_arg = {data = {klass = 0x67670447c60, type = 0x67670447c60, array = 0x67670447c60, method = 0x67670447c60,
      generic_param = 0x67670447c60, generic_class = 0x67670447c60}, attrs = 0, type = MONO_TYPE_CLASS, has_cmods = 0, byref__ = 1, pinned = 0},
  _byval_arg = {data = {klass = 0x67670447c60, type = 0x67670447c60, array = 0x67670447c60, method = 0x67670447c60, generic_param = 0x67670447c60,
      generic_class = 0x67670447c60}, attrs = 0, type = MONO_TYPE_CLASS, has_cmods = 0, byref__ = 0, pinned = 0}, gc_descr = 0x13,
  runtime_vtable = 0x67670302c18, vtable = 0x6767000b560, infrequent_data = {head = 0x0}}
(gdb)

Use the  info registers command to display the list of registers and their values.

(gdb) info registers
r0             0x7d3e457453f8      137706406695928
r1             0x7fffceb8c370      140736661603184
r2             0x7d3e458d7a00      137706408344064
r3             0x6e                110
r4             0x7d3e44d95c2c      137706396539948
r5             0x676703bed00       7105758883072
r6             0x7d3e45896264      137706408075876
r7             0x0                 0
r8             0x7fffceb8c498      140736661603480
r9             0x1                 1
r10            0x7d3e461453e0      137706417181664
r11            0x200               512
r12            0x7d3e454e06b0      137706404185776
r13            0x7d3e4614c9d0      137706417211856
r14            0x8                 8
r15            0x67670307568       7105758131560
r16            0x165129ff          374417919
r17            0xc8                200
r18            0x1                 1
r19            0x6766ff35c50       7105754127440
r20            0x0                 0
r21            0x6766ffaf720       7105754625824
r22            0x67670473660       7105759622752
r23            0x0                 0
r24            0x0                 0
r25            0x7fffceb8c498      140736661603480
r26            0x0                 0
r27            0x0                 0
r28            0x7fffceb8c550      140736661603664
r29            0x67670473660       7105759622752
r30            0x7d3e44d95c2c      137706396539948
r31            0x7fffceb8c6a0      140736661604000
pc             0x7d3e457454dc      0x7d3e457454dc <common_call_trampoline+860>
msr            0x800000000000f033  9223372036854837299
cr             0x42020c40          1107430464
lr             0x7d3e457454d8      0x7d3e457454d8 <common_call_trampoline+856>
ctr            0x7d3e454e06b0      137706404185776
xer            0x0                 0
fpscr          0x82024000          2181185536
vscr           0x0                 0
vrsave         0xffffffff          -1
ppr            0xc000000000000     3377699720527872
dscr           0x0                 0
tar            0x0                 0
bescr          <unavailable>
ebbhr          <unavailable>
ebbrr          <unavailable>
mmcr0          0x0                 0
mmcr2          0x0                 0
siar           0x0                 0
sdar           0x0                 0
sier           0x0                 0
orig_r3        0x7d3e4578ce64      137706406989412
trap           0x300               768
(gdb)

Use the p/x command to print the value in hex. Use the x/x command to examine the value at address in hex.

(gdb) f 7
#7  0x00007d3e457454dc in common_call_trampoline (regs=0x7fffceb8c550, code=0x7d3e44d95c2c "\260", m=0x67670473660, vt=0x0, vtable_slot=0x0,
    error=0x7fffceb8c498) at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:539
539     in /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c
(gdb) p regs[0]
$10 = 7105759622752
(gdb) p/x regs[0]
$11 = 0x67670473660
(gdb) x/x regs[0]
0x67670473660:  0x00000016
(gdb)

Use the disassemble <function-name> command to display the assembly of a function.

(gdb) disassemble mono_magic_trampoline
Dump of assembler code for function mono_magic_trampoline:
   0x00007d3e45745060 <+0>:     addis   r2,r12,25
   0x00007d3e45745064 <+4>:     addi    r2,r2,10656
   0x00007d3e45745068 <+8>:     mflr    r0
   0x00007d3e4574506c <+12>:    std     r27,-40(r1)
   0x00007d3e45745070 <+16>:    std     r28,-32(r1)
   0x00007d3e45745074 <+20>:    std     r29,-24(r1)
   0x00007d3e45745078 <+24>:    std     r30,-16(r1)
   0x00007d3e4574507c <+28>:    std     r0,16(r1)
   0x00007d3e45745080 <+32>:    stdu    r1,-176(r1)
   0x00007d3e45745084 <+36>:    mr      r28,r4
   0x00007d3e45745088 <+40>:    mr      r27,r3
   0x00007d3e4574508c <+44>:    li      r3,0
   0x00007d3e45745090 <+48>:    addis   r4,r2,-4
   0x00007d3e45745094 <+52>:    mr      r29,r5
   0x00007d3e45745098 <+56>:    stw     r3,56(r1)
   0x00007d3e4574509c <+60>:    addi    r3,r1,40
   0x00007d3e457450a0 <+64>:    addi    r4,r4,-8162
   0x00007d3e457450a4 <+68>:    std     r3,40(r1)
   0x00007d3e457450a8 <+72>:    std     r4,48(r1)
   0x00007d3e457450ac <+76>:    bl      0x7d3e454d0ae0 <0000001a.plt_call.mono_threads_enter_gc_unsafe_region_internal>
   0x00007d3e457450b0 <+80>:    ld      r2,24(r1)
   0x00007d3e457450b4 <+84>:    mr      r30,r3
   0x00007d3e457450b8 <+88>:    bl      0x7d3e45612368 <mono_thread_is_gc_unsafe_mode+8>
   0x00007d3e457450bc <+92>:    nop
   0x00007d3e457450c0 <+96>:    cmplwi  r3,0
   0x00007d3e457450c4 <+100>:   beq-    0x7d3e4574514c <mono_magic_trampoline+236>
   0x00007d3e457450c8 <+104>:   addis   r3,r2,1
   0x00007d3e457450cc <+108>:   addi    r8,r1,56
   0x00007d3e457450d0 <+112>:   mr      r5,r29
   0x00007d3e457450d4 <+116>:   li      r6,0
   0x00007d3e457450d8 <+120>:   li      r7,0
   0x00007d3e457450dc <+124>:   li      r29,0
   0x00007d3e457450e0 <+128>:   lwz     r4,-14720(r3)
   0x00007d3e457450e4 <+132>:   addi    r4,r4,1
   0x00007d3e457450e8 <+136>:   stw     r4,-14720(r3)
   0x00007d3e457450ec <+140>:   mr      r3,r27
   0x00007d3e457450f0 <+144>:   mr      r4,r28
   0x00007d3e457450f4 <+148>:   bl      0x7d3e45745188 <common_call_trampoline+8>
   0x00007d3e457450f8 <+152>:   lhz     r4,56(r1)
   0x00007d3e457450fc <+156>:   cmplwi  r4,0
   0x00007d3e45745100 <+160>:   beq     0x7d3e45745114 <mono_magic_trampoline+180>
   0x00007d3e45745104 <+164>:   addi    r3,r1,56

Use disassemble addr1,addr2 command to show the assembly from addr1 to addr2.

(gdb) disassemble 0x00007d3e457454dc-20, 0x00007d3e457454dc+20
Dump of assembler code from 0x7d3e457454c8 to 0x7d3e457454f0:
   0x00007d3e457454c8 <common_call_trampoline+840>:     beq     0x7d3e45745410 <common_call_trampoline+656>
   0x00007d3e457454cc <common_call_trampoline+844>:     mr      r3,r28
   0x00007d3e457454d0 <common_call_trampoline+848>:     mr      r4,r30
   0x00007d3e457454d4 <common_call_trampoline+852>:     bl      0x7d3e4578ce60 <mono_arch_find_static_call_vtable>
   0x00007d3e457454d8 <common_call_trampoline+856>:     nop
=> 0x00007d3e457454dc <common_call_trampoline+860>:     ld      r4,0(r3)
   0x00007d3e457454e0 <common_call_trampoline+864>:     ld      r19,16(r3)
   0x00007d3e457454e4 <common_call_trampoline+868>:     ld      r21,0(r4)
   0x00007d3e457454e8 <common_call_trampoline+872>:     cmpldi  r21,0
   0x00007d3e457454ec <common_call_trampoline+876>:     beq-    0x7d3e45745b38 <common_call_trampoline+2488>
End of assembler dump.
(gdb)

Use the disassemble /m <function-name> command to show the disassembly with source line.

How to debug a specific DLL in dotnet using GDB

Use the command gdb --args dotnet exec

$ gdb --args dotnet exec /home/ubuntu/amit/preview7/output_old_2/.dotnet/sdk/7.0.100-preview.7.22377.5/FSharp/fsc.dll
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "powerpc64le-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from dotnet...
(No debugging symbols found in dotnet)
(gdb)

Use run or r command to run the program in GDB.

(gdb) break common_call_trampoline
Breakpoint 1 at 0x7ffff75f51cc: file /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c, line 440.
(gdb) break mono_magic_trampoline
Breakpoint 2 at 0x7ffff75f5090: file /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c, line 753.
(gdb)

Use break or b <function-name> to add breakpoint to function and line.

(gdb) break common_call_trampoline
Breakpoint 1 at 0x7ffff75f51cc: file /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c, line 440.
(gdb) break mono_magic_trampoline
Breakpoint 2 at 0x7ffff75f5090: file /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c, line 753.
(gdb)

Use info breakpoints or i b to list the breakpoint info.

(gdb) info breakpoints
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00007ffff75f51cc in common_call_trampoline
                                                   at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:440
2       breakpoint     keep y   0x00007ffff75f5090 in mono_magic_trampoline
                                                   at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:753
(gdb)

Use run or r command and program stops at breakpoint, use next or n to go to next statement and step or s to move inside the function definition.

(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /home/ubuntu/amit/preview7/output_old_2/.dotnet/dotnet exec /home/ubuntu/amit/preview7/output_old_2/.dotnet/sdk/7.0.100-preview.7.22377.5/FSharp/fsc.dll
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff67ff150 (LWP 3794627)]
[New Thread 0x7ffff491f150 (LWP 3794628)]
[New Thread 0x7ffff470f150 (LWP 3794629)]

Thread 1 "dotnet" hit Breakpoint 2, mono_magic_trampoline (regs=0x7fffffffd400, code=0x7ffff4b72408 "p", arg=0x10015ffc0, tramp=<optimized out>)
    at /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c:753
753     /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c: No such file or directory.
(gdb)n
751     in /root/amit-devel/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-trampolines.c

Conclusion

Hope this blog gave you some tools to debug your C# applications. Please drop us a note if you found this information useful.

References

Permalink