Programming Languages on Power

 View Only

Debugging .NET: Unraveling managed method names

By Vikas Gupta posted Thu July 13, 2023 08:08 AM

  

Let's embark on a journey of debugging .NET with a powerful tool called GDB (GNU Debugger). You might have heard of it before—it's a favourite among UNIX system debuggers, giving us a behind-the-scenes view of what's going on inside our programs as they run. This blog is a must-read for developers working with .NET code, including those involved in the runtime and contributors to .NET repositories.

Note: The purpose of this document is to debug .NET code itself, not the .NET applications (C#, F# etc.)

Cracking the code: Understanding managed methods

Have you ever encountered cryptic frames marked with "??()" while debugging a .NET application crash or coredump? Wonder no more! We'll dive into the depths of these managed methods and decipher their significance. But before we delve deeper, let's get familiar with some key definitions.

Key definitions

  1. Managed code:

    Managed code refers to code that is controlled by the Common Language Runtime (CLR). This code is written in high-level languages like C# or F# and is compiled into an intermediate language (IL). Subsequently, the JIT compiler translates this IL code into architecture-specific native code. If you want to learn more about managed code, refer to What is manged code?.

  2. Unmanaged code:

    Unlike its managed counterpart, unmanaged code is executed directly by the operating system.

However, the CLR enables seamless interaction between managed and unmanaged code. Even within the .NET class libraries, code occasionally needs to traverse these boundaries. This crossover is commonly referred to as interoperability or interop.

Debugging coredump

Let's refer to the following example that demonstrates how to debug a coredump and extract the name of the managed function.

To obtain the managed function name, use the following command:

$ gdb -e /home/ubuntu/sapana/preview7/output/.dotnet/dotnet -c core.dotnet.796807
(gdb) bt
#0  mono_dump_mem (d=0x7384efbc6980, len=-272864896) at /root/dotnet-ppc64le-preview7/runtime/src/mono/mono/utils/mono-logger.c:553
#1  0x000073842bc1edc0 in dump_memory_around_ip (mctx=0x7384efbc6990) at /root/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-posix.c:726
#2  mono_dump_native_crash_info (signal=<optimized out>, mctx=0x7384efbc6990, info=<optimized out>)
    at /root/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-posix.c:871
#3  0x000073842bbcdd6c in mono_handle_native_crash (signal=0x73842bd01e68 "SIGSEGV", mctx=0x7fffd2d5d148, info=0x7fffd2d5e188)
    at /root/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-exceptions.c:2995
#4  0x000073842bb159e8 in mono_sigsegv_signal_handler_debug (_dummy=11, _info=0x7fffd2d5e188, context=0x7fffd2d5d410, debug_fault_addr=0x7384efbc6990)
    at /root/dotnet-ppc64le-preview7/runtime/src/mono/mono/mini/mini-runtime.c:3770
#5  <signal handler called>
#6  0x00007384efbc6990 in ?? ()
#7  0x00007383efaac384 in ?? ()
#8  0x00007383efaaa710 in ?? ()
#9  0x00007383efaafd9c in ?? ()
#10 0x00007383efaaf8f8 in ?? ()
#11 0x00007383efaaf7b4 in ?? ()
#12 0x00007383efa95cb4 in ?? ()
#13 0x00007383efa951b8 in ?? ()
#14 0x00007383efa94fa4 in ?? ()
#15 0x0000738420974808 in ?? ()
#16 0x00007384209740c0 in ?? ()
#17 0x0000738420972c28 in ?? ()
#18 0x000073842097266c in ?? ()
#19 0x00007384209683a0 in ?? ()
#20 0x0000738420968218 in ?? ()
#21 0x0000738420968130 in ?? ()
#22 0x0000738420967ff0 in ?? ()
#23 0x0000738420967cf4 in ?? ()
#24 0x0000738420967300 in ?? ()
#25 0x0000738420966c88 in ?? ()

Retrieve the name of managed method

To obtain the name of managed method, follow these steps:

  1. Firstly, use the command ‘f 9’ in the GDB prompt to check the function at frame, #9(in this case). Take note of the address associated with frame #9. The output might look like this.
    (gdb) f 9
    #9  0x00007383efaafd9c in ?? ()
  2. Next, retrieve information about the chunks using the command ‘p *jit_info_table’. The output might look like this.
    (gdb) p *jit_info_table
    $1 = {domain = 0x7383e80007f0, num_chunks = 610, num_valid = 19823, chunks = 0x7383e807dc50}
    (gdb)
    

    From the output, we can see that there are 610 chunks in the ‘jit_info_table’.

    To locate the correct chunk containing the desired address (0x00007383efaafd9c), we need to examine the chunks one by one. Let's say we want to check the 50th chunk. Use the following command to investigate the chunk.

     

    (gdb) p *jit_info_table->chunks[50]
    
    $5 = {refcount = 1, num_elements = 32, last_code_end = 0x7383efa9be14 "8\a0\f\204s", next_tombstone = 0x0, data = {0x48536535aa0, 0x48536535ae0,
    
        0x48536535b20, 0x48536535b60, 0x48536535be8, 0x7383fc080128, 0x7383fc080168, 0x7383fc084b20, 0x7383fc084b60, 0x7383fc0a8730, 0x7383fc0a8880,
    
        0x7383fc0a8918, 0x7383fc0a8958, 0x7383fc0a8998, 0x7383fc0a89d8, 0x7383fc0a8a18, 0x7383fc0a8a58, 0x7383fc0a8a98, 0x7383fc0a8de8, 0x7383fc0a8e38,
    
        0x7383f80592b0, 0x7383f805a340, 0x7383f805a508, 0x7383f805a580, 0x7383f805a5c0, 0x7383f805a668, 0x7383f805a6d8, 0x7383f805a718, 0x7383f805a758,
    
        0x7383f805a798, 0x7383f805a7f0, 0x7383f805a878, 0x0 <repeats 32 times>}}
    
     
    
    (gdb) p *jit_info_table->chunks[53]
    
    $7 = {refcount = 1, num_elements = 32, last_code_end = 0x7383efaaf7f4 "@\370\252\357\203s", next_tombstone = 0x0, data = {0x7383fc0c4e78, 0x7383fc0c5038,
    
        0x7383fc0c50c8, 0x7383fc0c51a8, 0x7384000f0cc0, 0x7384000f0d58, 0x7384000f0d98, 0x7384000f0ee0, 0x7384000f0f30, 0x7384000f0f70, 0x7384000f10b0,
    
        0x7384000e4708, 0x7384000e4990, 0x7384000c7c48, 0x7383f805fc90, 0x7383f805fdd8, 0x7383f805fe18, 0x7383f805feb0, 0x7383f805fef0, 0x7383f00894c0,
    
        0x7383f0089500, 0x7383f0089540, 0x7383f00895a8, 0x7383f00895e0, 0x7383f0089620, 0x7383f00896a0, 0x7383f0089660, 0x7383f00896e0, 0x7383f0089720,
    
        0x7383f0089760, 0x7383f00897a0, 0x7383f00897e0, 0x0 <repeats 32 times>}}
    
     
    
    (gdb) p *jit_info_table->chunks[54]
    
    $8 = {refcount = 1, num_elements = 33, last_code_end = 0x7383efab55bc "", next_tombstone = 0x0, data = {0x7383f0089820, 0x7383f0089b80, 0x7383f0089bd0,
    
        0x7383f0089e60, 0x7383f0089eb0, 0x7383f008a1d0, 0x7383f80623a0, 0x7384000f6b78, 0x7384000f6bc0, 0x7384000f7468, 0x7384000f7590, 0x7384000f75d0,
    
        0x7383f008f380, 0x7383f0042958, 0x7383f0042998, 0x7383f00429d8, 0x7384000fb680, 0x7384000ff760, 0x738400100a80, 0x7384000f9018, 0x7384000f90c0,
    
        0x7384000f9318, 0x7384000f9358, 0x7384000f96b0, 0x7384000f96e8, 0x7384000f9728, 0x7384000f9b68, 0x7384040a34d0, 0x7384040a3668, 0x7384040a37b0,
    
        0x7384040a3830, 0x7384040a37f0, 0x7384040a38f0, 0x0 <repeats 31 times>}}
    
    (gdb)

     

    Great! Now that we have identified that the address 0x00007383efaafd9c lies between chunks 53 and 54, and we choose the higher address (chunk 54) as it is the end of code.

    Let's proceed to examine the data field to find the 'code_start' and compare it with the address 0x00007383efaafd9c.To do this, run the following command.

     

    (gdb) p *jit_info_table->chunks[54].data[1]
    $12 = {d = {method = 0x7383f00593d8, image = 0x7383f00593d8, aot_info = 0x7383f00593d8, tramp_info = 0x7383f00593d8}, n = {
        next_jit_code_hash = 0x485361eaff0, next_tombstone = 0x485361eaff0}, code_start = 0x7383efaafb08, unwind_info = 351, code_size = 864, num_clauses = 0,
      has_generic_jit_info = 0, has_try_block_holes = 0, has_arch_eh_info = 0, has_thunk_info = -1, has_unwind_info = 0, from_aot = 0, from_llvm = 0,
      dbg_attrs_inited = 0, dbg_hidden = 0, async = 0, dbg_step_through = 0, dbg_non_user_code = 0, is_trampoline = 0, is_interp = 0, gc_info = 0x0,
      seq_points = 0x7383f8057f50, clauses = 0x7383f0089bb8}
    
    (gdb) p *jit_info_table->chunks[54].data[2]
    $13 = {d = {method = 0x7383f416fd30, image = 0x7383f416fd30, aot_info = 0x7383f416fd30, tramp_info = 0x7383f416fd30}, n = {
        next_jit_code_hash = 0x4853623ce70, next_tombstone = 0x4853623ce70}, code_start = 0x7383efaaff70, unwind_info = 259, code_size = 500, num_clauses = 0,
      has_generic_jit_info = 0, has_try_block_holes = 0, has_arch_eh_info = 0, has_thunk_info = -1, has_unwind_info = 0, from_aot = 0, from_llvm = 0,
      dbg_attrs_inited = 0, dbg_hidden = 0, async = 0, dbg_step_through = 0, dbg_non_user_code = 0, is_trampoline = 0, is_interp = 0, gc_info = 0x0,
      seq_points = 0x7383f804cca0, clauses = 0x7383f0089c08}
    
    (gdb)
    

    Now that we have identified that the address 0x00007383efaafd9c lies between data[1] and data[2], and we choose data[1] as it denotes the start.

    (gdb) p *jit_info_table->chunks[54].data[1].d.method
    $15 = {flags = 129, iflags = 0, token = 100693793, klass = 0x48533f0eab8, signature = 0x7383f0059400,
      name = 0x738423ed563b <error: Cannot access memory at address 0x738423ed563b>, inline_info = 0, inline_failure = 0, wrapper_type = 0, string_ctor = 0,
      save_lmf = 0, dynamic = 0, sre_method = 0, is_generic = 0, is_inflated = 0, skip_visibility = 0, _unused = 0, slot = -1}
    
    (gdb) p *jit_info_table->chunks[54].data[1].d.method.klass
    $16 = {element_class = 0x48533f0eab8, cast_class = 0x48533f0eab8, supertypes = 0x48535bd28d8, idepth = 6, rank = 0 '\000', class_kind = 1 '\001',
      instance_size = 192, inited = 1, size_inited = 1, valuetype = 0, enumtype = 0, blittable = 0, unicode = 0, wastypebuilder = 0,
      is_array_special_interface = 0, is_byreflike = 0, min_align = 8 '\b', packing_size = 0, ghcimpl = 0, has_finalize = 0, delegate = 0, gc_descr_inited = 1,
      has_cctor = 1, has_references = 1, has_ref_fields = 0, has_static_refs = 1, no_special_static_fields = 0, is_com_object = 0, nested_classes_inited =  v0,
      interfaces_inited = 1, simd_type = 0, has_finalize_inited = 1, fields_inited = 1, has_failure = 0, has_weak_fields = 0, has_dim_conflicts = 0,
      any_field_has_auto_layout = 1, parent = 0x48533f0ebb8, nested_in = 0x0, image = 0x48533bb65b0,
      name = 0x738423ea2302 <error: Cannot access memory at address 0x738423ea2302>,
      name_space = 0x738423ed0c78 <error: Cannot access memory at address 0x738423ed0c78>, type_token = 33555823, vtable_size = 236, interface_count = 0,
      interface_id = 0, max_interface_id = 2884, interface_offsets_count = 20, interfaces_packed = 0x48535bd2908, interface_offsets_packed = 0x48535bd29a8,
      interface_bitmap = 0x48535bd29d0 "@", interfaces = 0x0, sizes = {class_size = 32, element_size = 32, generic_param_token = 32}, fields = 0x48535bdbd78,
      methods = 0x0, this_arg = {data = {klass = 0x48533f0eab8, type = 0x48533f0eab8, array = 0x48533f0eab8, method = 0x48533f0eab8,
          generic_param = 0x48533f0eab8, generic_class = 0x48533f0eab8}, attrs = 0, type = MONO_TYPE_CLASS, has_cmods = 0, byref__ = 1, pinned = 0},
      _byval_arg = {data = {klass = 0x48533f0eab8, type = 0x48533f0eab8, array = 0x48533f0eab8, method = 0x48533f0eab8, generic_param = 0x48533f0eab8,
          generic_class = 0x48533f0eab8}, attrs = 0, type = MONO_TYPE_CLASS, has_cmods = 0, byref__ = 0, pinned = 0}, gc_descr = 0x1ffefa,
      runtime_vtable = 0x48535bfb938, vtable = 0x48535c20200, infrequent_data = {head = 0x0}}
    (gdb)
    
  3. Note the address of the method name, Klass name, and namespace.

    Method :          0x738423ed563b

    Klass :              0x738423ea2302

    Namespace:   0x738423ed0c78

  4. Excellent progress! Now, let's find the range in which the addresses of the method, klass, and namespace fall to identify the binary file.

    Run the following command in the GDB prompt to identifying the range and the corresponding DLL file that contains the method, klass, and namespace we are interested in.

    (gdb) info proc mappings
    Mapped address spaces:
              Start Addr           End Addr       Size     Offset objfile
    …
     0x738423a20000     0x738424000000   0x5e0000        0x0 /root/.nuget/packages/microsoft.net.compilers.toolset/4.4.0-2.22412.11/tasks/net6.0/bincore/Microsoft.CodeAnalysis.CSharp.dll
    …
    

    Keep up the great work!

  5. Exit from the GDB prompt and subtract address of the DLL file obtained in the previous step from the addresses of method, klass, and namespace we found in the step 3.

    Method :          0x738423ed563b - 0x738423a20000     = 4b563b

    Klass :              0x738423ea2302 - 0x738423a20000     = 482302

    Namespace:   0x738423ed0c78 - 0x738423a20000     = 4b0c78

  6. Generate the hexdump of the DLL file b running the following command and save the hexdump into a file.
    $ hexdump -C /root/.nuget/packages/microsoft.net.compilers.toolset/4.4.0-2.22412.11/tasks/net6.0/bincore/Microsoft.CodeAnalysis.CSharp.dll > /home/ubuntu/vikas/CodeAnalysis.CSharp.dll_hexdump.txt
  7. Now, open the hexdump file in a text editor and lookup for the previously determined difference value found in step 5.
    $ vim /home/ubuntu/vikas/CodeAnalysis.CSharp.dll_hexdump.txt

    For example, we found ‘4b563b’ in the following line.

    The letter 'b' in the array is located at position 11, corresponding to the letter 'M'. This allows us to identify the method name as 'MakeAllMembers'.

    004b5630  41 6c 6c 4d 65 6d 62 65  72 73 00 4d 61 6b 65 41  |AllMembers.MakeA|

    004b5640  6c 6c 4d 65 6d 62 65 72  73 00 67 65 74 5f 45 52  |llMembers.get_ER|

    Method :   0x738423ed563b - 0x738423a20000     = 4b563b -> MakeAllMembers

     

    Repeat this process to find the Klass and namespace names.

    482302 can be found in the following line.

    00482300  6c 00 53 6f 75 72 63 65  4d 65 6d 62 65 72 43 6f  |l.SourceMemberCo|

    00482310  6e 74 61 69 6e 65 72 54  79 70 65 53 79 6d 62 6f  |ntainerTypeSymbo|

    00482320  6c 00 49 50 6f 69 6e 74  65 72 54 79 70 65 53 79  |l.IPointerTypeSy|

    The letter '2' in the array is located at position 2, corresponding to the letter 'S'. This allows us to identify the Klass name as ' SourceMemberContainerTypeSymbol '.

    Klass :    0x738423ea2302 - 0x738423a20000   = 482302 -> SourceMemberContainerTypeSymbol

     

    4b0c78 can be found in the following line. 

    004b0c70  61 6c 43 61 6c 6c 73 00  4d 69 63 72 6f 73 6f 66  |alCalls.Microsof|

    004b0c80  74 2e 43 6f 64 65 41 6e  61 6c 79 73 69 73 2e 43  |t.CodeAnalysis.C|

    004b0c90  53 68 61 72 70 2e 53 79  6d 62 6f 6c 73 00 4d 69  |Sharp.Symbols.Mi|

    The letter '8' in the array is located at position 8, corresponding to the letter 'M'. This allows us to identify the Klass name as 'Microsoft.CodeAnalysis.Csharp'.

    Namespace:   0x738423ed0c78 - 0x738423a20000     = 4b0c78 -> Microsoft.CodeAnalysis.CSharp

    Hurray! We did it! We have successfully identified the method, Klass, and namespace from the coredump.

     

    Method:   0x738423ed563b - 0x738423a20000   = 4b563b -> MakeAllMembers

    Klass:    0x738423ea2302 - 0x738423a20000     = 482302 -> SourceMemberContainerTypeSymbol

    Namespace: 0x738423ed0c78 - 0x738423a20000 = 4b0c78 -> Microsoft.CodeAnalysis.CSharp 

Conclusion:

In this blog post, we have demonstrated a method to extract the names of managed code functions from coredumps where the backtrace fails to display managed functions (marked as '?? ()'). By following the outlined steps, you can debug your own .NET coredumps and successfully identify the managed code responsible for failures.

Permalink