Skip to content

Instantly share code, notes, and snippets.

@MaskRay
Last active August 26, 2024 03:42
Show Gist options
  • Save MaskRay/6c3910c7ab1df208c36cfedf965f6b51 to your computer and use it in GitHub Desktop.
Save MaskRay/6c3910c7ab1df208c36cfedf965f6b51 to your computer and use it in GitHub Desktop.

If you link a program with a compiler driver (clang/gcc) in a standard way (not -nostdlib), the following components are usually on the linker command line.

  • crt1.o (glibc/musl): -no-pie/-pie/-static-pie
    • crt1.o: -no-pie
    • Scrt1.o: -pie, -shared
    • rcrt1.o: -static-pie
    • gcrt1.o:
  • crti.o (glibc/musl)
  • crtbegin.o
    • crtbegin.o: -no-pie
    • crtbeginS.o: -pie, -shared
    • crtbeginT.o: -static-pie
  • user input
  • -lstdc++
  • Some combination of -lc -lgcc_s -lgcc -lgcc_eh
  • crtn.o (glibc/musl)
  • crtend.o
    • crtend.o: -no-pie
    • crtendS.o: -pie, -shared
    • crtendT.o: -static-pie

crt1.o

This file is only used by executables.

In glibc, the file is -r linked from csu/start.c csu/abi-note.c csu/init.c csu/static-reloc.c. It used to call __libc_start_main with arguments main, __libc_csu_init, __libc_csu_fini (defined by libc_nonshared.a(elf-init.oS)). From BZ #23323 onwards, on most architectures, start.S:_start calls __libc_main_start with two zero arguments instead, and __libc_csu_init and __libc_csu_fini are moved into csu/libc-start.c.

In musl, this file calls __libc_start_main with main, _init, and _fini.

crti.o/crtn.o

crti.o defines _init in the .init section and _fini in the .fini section. The defined _init is a fragment which is expected to be concatenated with other files and finally crtn.o to get the full definition.

In glibc x86-64,

# crti.o
Disassembly of section .init:

0000000000000000 <_init>:
       0: 48 83 ec 08                   subq    $8, %rsp
       4: 48 8b 05 00 00 00 00          movq    (%rip), %rax  # b <_init+0xb>
                0000000000000007:  R_X86_64_REX_GOTPCRELX       __gmon_start__-0x4
       b: 48 85 c0                      testq   %rax, %rax
       e: 74 02                         je      0x12 <_init+0x12>
      10: ff d0                         callq   *%rax

Disassembly of section .fini:

0000000000000000 <_fini>:
       0: 48 83 ec 08                   subq    $8, %rsp

# crtn.o
Disassembly of section .init:

0000000000000000 <.init>:
       0:       addq    $8, %rsp
       4:       retq

Disassembly of section .fini:

0000000000000000 <.fini>:
       0:       addq    $8, %rsp
       4:       retq

The linker defines DT_INIT if _init (default value for -init) is defined, and DT_FINI if _fini is defined.

The section fragment idea is fragile. On RISC-V, DT_INIT is not used.

glibc defines the files in sysdeps/aarch64/crt[in].S.

crti.o calls __gmon_start__ (gmon profiling system) if defined. This is used by gcc -pg.

crtbegin.o/crtend.o

libgcc/crtstuff.c

If __LIBGCC_INIT_ARRAY_SECTION_ASM_OP__ is not defined and __LIBGCC_INIT_SECTION_ASM_OP__ is defined,

  • crtend.o defines a .init section which calls __do_global_ctors_aux. __do_global_ctors_aux calls the static constructors in the .ctors section.
  • crtbegin.o defines a .fini section which calls __do_global_dtors_aux. __do_global_dtors_aux calls the static constructors in the .dtors section.
  • crtbegin.o defines .ctors and .dtors with a single -1 value.
  • crtend.o defines .ctors and .dtors with a single 0 value.

On modern distributions, __LIBGCC_INIT_ARRAY_SECTION_ASM_OP__ is 0 and crtend.o contains no .text/.ctors/.dtors.

glibc startup sequence

Below the control flows are flattened.

Dynamically linked executable

In rtld:

  • sysdeps/x86_64/dl-machine.h:_user
  • elf/rtld.c:_dl_start
  • sysdeps/x86_64/dl-machine.h:_dl_start_user
  • elf/dl-init.c:_dl_init
  • Jump to the main executable e_entry

In the main executable:

  • sysdeps/x86_64/start.S:_start
  • csu/libc-start.c:__libc_start_main, the SHARED branch
  • (if ELF_INITFINI is defined) Run DT_INIT
  • Run DT_INITARRAY
  • Run main
  • Run exit

Statically linked executable

In the main executable:

  • sysdeps/x86_64/start.S:_start
  • csu/libc-start.c:__libc_start_main, the !SHARED branch
  • _dl_relocate_static_pie
  • ARCH_SETUP_IREL
  • ARCH_SETUP_TLS
  • csu/libc-start.c:call_init
    • Run [__preinit_array_start, __preinit_array_end)
    • (if ELF_INITFINI is defined) Run _init
    • Run [__init_array_start, __init_array_end)
  • Run main
  • Run exit

musl startup sequence

For a dynamically linked executable, the rtld process:

  • arch/x86_64/crt_arch.h:_dlstart
  • ldso/dlstart.c:_dlstart_c
  • ldso/dynlink.c:__dls2 relocate rtld
  • ldso/dynlink.c:__dls2b setup early thread pointer
  • ldso/dynlink.c:__dls3
  • Jump to the main executable e_entry

In the main executable:

  • arch/x86_64/crt_arch.h:_start
  • crt/crt1.c:_start_c
  • src/env/__libc_start_main.c:__libc_start_main
  • __init_libc initialize auxv/TLS/stack protector/etc
  • libc_start_main_stage2
  • __libc_start_init
  • exit(main(argc, argv, envp));

__libc_start_init has different behaviors for dynamically and statically linked executables. For a dynamically linked executable: it runs DT_INIT (unless NO_LEGACY_INITFINI) then DT_INIT_ARRAY. Note: libc.so has a dummy _init.

@somebodytoldme
Copy link

helpful, i'm confused in musl and libgcc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment