C++ links: Executables - Linking and Loading
See also:
- Building
- Compilers
- Executables - executable & object file formats (DLL, ELF, Mach-O, PE); debugging data formats (DWARF, PDB)
- Readings:
- Blogs
- Books
- Linker Scripts
- LTO
- OS: POSIX (macOS, Linux, Solaris)
- OS: Windows
- Relocations
- Security
- Shared Libraries
- Software:
- Talks: 2019, 2018, 2017, 2016, 2012
- Beginner's Guide to Linkers
- 2010; David Drysdale
- https://www.lurklurk.org/linkers/linkers.html
- Binary File Descriptor library (BFD)
- Concurrent Linking with the GNU Gold Linker
- 2013 Thesis; Sander Mathijs van Veen
- https://smvv.io/gold.pdf
- Linkers and Loaders
- ACM Computing Surveys, Volume 4, Number 3, September 1972
- Leon Presser, John R. White
- http://www-inst.eecs.berkeley.edu/~cs162/sp06/hand-outs/p149-presser-linker-loader.pdf
- Optimizing the LLVM ELF linker for a distributed compilation environment: Concurrent Linking with LLVM LLD
- 2020 Master’s thesis; Alexander Wilkens
- https://www.diva-portal.org/smash/record.jsf?pid=diva2:1484152
- Oracle Solaris 11.1 Linkers and Libraries Guide
- The Missing Link: Explaining ELF Static Linking, Semantically
- OOPSLA 2016
- Stephen Kell, Dominic P. Mulligan, Peter Sewell
- http://www.cl.cam.ac.uk/~pes20/rems/papers/oopsla-elf-linking-2016.pdf
- The Sad State of Symbol Aliases
- The Theory, History and Future of System Linkers
- HelloGCC Workshop 2013; Luba Tang
- https://github.com/hellogcc/HelloGCC2013/blob/master/linker_overall.pdf
- Eli Bendersky - Linkers and loaders
- https://eli.thegreenplace.net/tag/linkers-and-loaders
- Load-time relocation of shared libraries
- Position Independent Code (PIC) in shared libraries
- Position Independent Code (PIC) in shared libraries on x64
- How statically linked programs run on Linux
- Library order in static linking
- Fun with weak symbols
- 2012; Mike Hommey
- https://glandium.org/blog/?p=2388
- Ian Lance Taylor
- 20 part linker essay by Ian Lance Taylor
- Notes: Day 40: Linkers are amazing.
- 2013; Julia Evans
- http://jvns.ca/blog/2013/12/10/day-40-learning-about-linkers/
- TOC: https://checkoway.net/musings/linkers/
- TOC: https://lwn.net/Articles/276782/
- 1. Introduction, personal history, first half of what's-a-linker - http://www.airs.com/blog/archives/38
- 2. What's-a-linker: Dynamic linking, linker data types, linker operation - http://www.airs.com/blog/archives/39
- 3. Address spaces, Object file formats - http://www.airs.com/blog/archives/40
- 4. Shared Libraries - http://www.airs.com/blog/archives/41
- 5. More Shared Libraries -- specifically, linker implementation; ELF Symbols - http://www.airs.com/blog/archives/42
- 6. Relocations, Position Dependent Shared Libraries - http://www.airs.com/blog/archives/43
- 7. Thread Local Storage (TLS) optimization - http://www.airs.com/blog/archives/44
- 8. ELF Segments and Sections - http://www.airs.com/blog/archives/45
- 9. Symbol Versions, Relaxation optimization - http://www.airs.com/blog/archives/46
- 10. Parallel linking - http://www.airs.com/blog/archives/47
- 11. Archive format - http://www.airs.com/blog/archives/48
- 12. Symbol resolution - http://www.airs.com/blog/archives/49
- 13. Symbol resolution from the user's point of view; Static Linking vs. Dynamic Linking - http://www.airs.com/blog/archives/50
- 14. Link time optimization, aka Whole Program optimization; Initialization Code - http://www.airs.com/blog/archives/51
- 15. COMDAT sections - http://www.airs.com/blog/archives/52
- 16. C++ Template Instantiation, Exception Frames - http://www.airs.com/blog/archives/53
- 17. Warning Symbols - http://www.airs.com/blog/archives/54
- 18. Incremental Linking - http://www.airs.com/blog/archives/55
- 19.
__startand__stopSymbols, Byte Swapping - http://www.airs.com/blog/archives/56 - 20. Last post; Update on gold's status - http://www.airs.com/blog/archives/57
- Notes: Day 40: Linkers are amazing.
- Protected Symbols - https://www.airs.com/blog/archives/307
- gold
- Gold Workqueues - https://www.airs.com/blog/archives/78
- STT_GNU_IFUNC - https://www.airs.com/blog/archives/403
- A New ELF Linker
- GCC Summit 2008
- Proceedings of the GCC Developers' Summit (2008)
- https://research.google.com/pubs/pub34417.html
- https://gcc.gnu.org/wiki/HomePage?action=AttachFile&do=view&target=gcc-2008-proceedings.pdf
- http://airs.com/ian/gold-slides.pdf
- 20 part linker essay by Ian Lance Taylor
- Nick Desaulniers
- Part 1 – Object Files and Symbols - https://nickdesaulniers.github.io/blog/2016/08/13/object-files-and-symbols/
- Part 2 – Static and Dynamic Libraries - https://nickdesaulniers.github.io/blog/2016/11/20/static-and-dynamic-libraries/
- Stephen Kell
- Linking and loading: what's incidental?
- Dynamic linking and security
- C libraries and linking
- Link order
- Weak dynamic symbols
- Trevor Pounds
- Versioning Symbols for Shared Libraries (glibc)
- Linking to Older Versioned Symbols (glibc)
- Advanced C and C++ Compiling
- 2014; Milan Stevanovic
- "Engineering guide to C/C++ compiling, linking, and binary files structure"
- http://www.apress.com/9781430266679
- http://link.springer.com/book/10.1007%2F978-1-4302-6668-6
- https://github.com/apress/adv-c-cpp-compiling
- Linkers and Loaders
- 1999; John R. Levine
- https://www.iecc.com/linker/
- OSDev Wiki
- LD - Linker Scripts
- From Zero to main(): Demystifying Firmware Linker Scripts
- The Most Thoroughly Commented Linker Script in The World (Probably)
- "a linker script for the Atmel/Microchip SAM D21 with an absolutely obscene amount of documentation"
- https://blog.thea.codes/the-most-thoroughly-commented-linker-script/
- https://github.com/theacodes/Winterbloom_Castor_and_Pollux/blob/master/firmware/scripts/samd21g18a.ld
- (Ab)using LTO plugin API for system-wide shrinking of dynamic libraries
- GNU Tools Cauldron 2018
- Vladislav Ivanishin
- https://www.youtube.com/watch?v=BNYGB7dHgkc
- https://gcc.gnu.org/wiki/cauldron2018?action=AttachFile&do=view&target=lto-plugin-api-abuse.pdf
- GCC Wiki - LinkTimeOptimization
- Honza Hubička's Blog
- Linktime optimization in GCC, part 1 - brief history
- Linktime optimization in GCC, part 2 - Firefox
- Linktime optimization in GCC, part 3 - LibreOffice
- Link time and inter-procedural optimization improvements in GCC 5
- Building libreoffice with GCC 6 and LTO
- GCC 8: link time and interprocedural optimization
- LLVM Link Time Optimization: Design and Implementation
- Optimizing Large Applications
- 2014 Master's Thesis; Martin Liška
- https://arxiv.org/abs/1403.6997
- Optimizing real world applications with GCC Link Time Optimization
- 2010 GCC Developers' Summit; T. Glek, J. Hubicka
- https://arxiv.org/abs/1010.2196
- ThinLTO
- ThinLTO: Scalable and Incremental LTO
- ThinLTO: Scalable and Incremental Link-Time Optimization
- CppCon 2017; Teresa Johnson
- https://www.youtube.com/watch?v=p9nH2vZ2mNo
- ThinLTO: Scalable and incremental LTO
- CGO 2017
- Teresa Johnson, Mehdi Amini, and Xinliang David Li
- https://research.google/pubs/pub47584/
- Debugging Dyld
- dyld: Dynamic Linking On OS X
- Friday Q&A 2012-11-09
- https://www.mikeash.com/pyblog/friday-qa-2012-11-09-dyld-dynamic-linking-on-os-x.html
- Dynamic Library Programming Topics
- Awesome LD_PRELOAD
- List of resources related to LD_PRELOAD, a mechanism for changing application behavior at run-time
- https://github.com/gaul/awesome-ld-preload
- Linker limitations on 32-bit architectures
- 2019; Alexander E. Patrakov
- https://lwn.net/Articles/797303/
- Linux EDR Evasion With Meterpreter and LD_PRELOAD
- Linux Internals
- Dynamic Linking Wizardry
- The Art Of Symbol Resolution
- Understanding ld-linux.so.2
- Surfing With The Linker Aliens: Solaris Linking & ELF Blogs
- http://www.linker-aliens.org/
- The Link-editors - a source tour
- What Are "Tentative" Symbols?
- Raymond Chen - The Old New Thing
- Understanding the classical model for linking
- https://blogs.msdn.microsoft.com/oldnewthing/tag/linker
- Groundwork: The algorithm
- Taking symbols along for the ride
- You can override an LIB with another LIB, and a LIB with an OBJ, but you can’t override an OBJ
- Sometimes you don’t want a symbol to come along for a ride
- Understanding errors in classical linking: The delay-load catch-22
- Using linker segments and
__declspec(allocate(…))to arrange data in a specific order - Gotchas when using linker sections to arrange data, part 1
- Gotchas when using linker sections to arrange data, part 2
- Why would the incremental linker insert padding between section fragments?
- Understanding the classical model for linking
- Binary Dodo - Arun
- Investigating linking with COMMON symbols in ELF
- Symbol binding types in ELF and their effect on linking of relocatable files
- Symbol resolution during link-editing
- Copy relocations, canonical PLT entries and protected visibility
- ELF Binary Relocations and Thread Local Storage - Stafford Horne
- TLS Examples
- ELF Binaries and Relocation Entries
- Thread Local Storage
- How Relocations and Thread Local Storage are Implemented
- Executable and Linkable Format 101 - Ignacio Sanmillan
- Part 3: Relocations - https://www.intezer.com/executable-and-linkable-format-101-part-3-relocations/
- GOT and PLT
- Ian Lance Taylor
- Linkers 2. What's-a-linker: Dynamic linking, linker data types, linker operation
- Linkers 5. More Shared Libraries -- specifically, linker implementation; ELF Symbols
- Linkers 6. Relocations, Position Dependent Shared Libraries
- Linkers 9. Symbol Versions, Relaxation optimization
- Linker combreloc
- Linker relro
- Introduction to the ELF Format - Keith Makan
- Part VI(1): The Symbol Table and Relocations - https://blog.k3170makan.com/2018/10/introduction-to-elf-format-part-vi.html
- Part VI(2): The Symbol Table and Relocations - https://blog.k3170makan.com/2018/10/introduction-to-elf-format-part-vi_18.html
- Part VI(3): More Relocation tricks - r_addend execution - https://blog.k3170makan.com/2018/10/introduction-to-elf-format-part-vi-more.html
- Linkers and loaders - Eli Bendersky - http://eli.thegreenplace.net/tag/linkers-and-loaders
- Load-time relocation of shared libraries
- Making our own executable packer - Amos Wenger
- ELF relocations - https://fasterthanli.me/blog/2020/elf-relocations/
- More ELF relocations - https://fasterthanli.me/blog/2020/more-elf-relocations/
- Moving code around - Thiago Macieira
- Oracle Solaris 11.1 Linker and Libraries Guide
- Relocation Processing
- Relocation Sections
- Relocations in ELF Toolchains - Palmer Dabbelt
- Relocations: fantastic symbols, but where to find them? - Siddhesh Poyarekar
- Resolving ELF Relocation Name / Symbols - Chris Rohlf
- An Evil Copy: How the Loader Betrays You
- Network and Distributed System Security Symposium (NDSS) 2017
- Xinyang Ge, Mathias Payer, Trent Jaeger
- https://nebelwelt.net/publications/files/17NDSS.pdf
- https://www.microsoft.com/en-us/research/publication/evil-copy-loader-betrays/
- Breaking the links: Exploiting the linker
- Dynamic Loader Oriented Programming on Linux
- Reversing and Offensive-oriented Trends Symposium (ROOTS) 2017
- Julian Kirsch, Bruno Bierbaumer, Thomas Kittel, and Claudia Eckert
- https://kirschju.re/projects/static/kirsch-roots-2017-paper.pdf
- https://github.com/kirschju/wiedergaenger
- Types for the Chain of Trust: No (loader) write left behind
- 2018 PhD dissertation; Rebecca .bx Shapiro
- http://typedregions.com/
- Dissertation: http://typedregions.com/main.pdf
- Defense slides: http://typedregions.com/defense-slides.pdf
- Chain loading, not preloading: the dynamic linker as a virtualization vector
- https://www.cs.kent.ac.uk/people/staff/srk21/blog/2021/01/04/#elf-chain-loading
- donald: the Mickey Mouse of dynamic linkers
- Fun with weak dynamic linking
- Guided Linking: Dynamic Linking without the Costs
- OOPSLA 2020
- Sean Bartell, Will Dietz, Vikram S. Adve
- https://dl.acm.org/doi/abs/10.1145/3428213
- http://design.cs.iastate.edu/splash20/oopsla20/oopsla20main-p80-p.pdf
- https://www.youtube.com/watch?v=apzR151q1wE
- BCDB: The Bitcode Database with Guided Linking
- How To Write Shared Libraries - Ulrich Drepper
- Inlining — shared libraries are special
- PLT and GOT - the key to code sharing and dynamic libraries
- Shared Libraries: Understanding Dynamic Loading
- Software Multiplexing: Share Your Libraries and Statically Link Them Too
- GNU Linker Output Viewer & Editor
- ld-limiter: Limit number of parallel link jobs
- libdlbind: Dynamic creation and update of ELF files, or: an allocator for JIT compilers
- an extension to System V-style dynamic loaders to allow dynamic allocation of objects and dynamic binding of symbols
- https://github.com/stephenrkell/libdlbind
- libwhich: Like
which, for dynamic libraries - linkermapviz
- Interactive visualization of GNU ld’s linker map with a tree map.
- https://github.com/PromyLOPh/linkermapviz
- linksem: Semantic model for aspects of ELF static linking and DWARF debug information
- ShlibVisibilityChecker: Tool for locating internal symbols unnecessarily exported from shared libraries
- ld - the GNU linker
- LLD - The LLVM Linker
- mold: A Modern Linker
- cctools-port: Apple cctools and ld64 port for Linux, *BSD and macOS
- ld64: Instructions on how to build the ld64 linker on macOS
- [lld] Initial commit for new Mach-O backend
- macdylibbundler: Utility to ease bundling libraries into executables for OSX
- osxcross: Mac OS X cross toolchain for Linux, FreeBSD, OpenBSD and Android (Termux)
- zld: A faster version of Apple's linker
- dlinject.py: Inject a shared library (i.e. arbitrary code) into a live linux process, without ptrace
- Faulty.lib: Dynamic linker for compressed libraries, with on-demand decompression (ELF Linux systems)
- Fun with LD_PRELOAD
- linux.conf.au 2009; Kevin Pulo
- https://www.youtube.com/watch?v=WocKUD5a4O0
- http://kev.pulo.com.au/publications/lca2009/lca2009-kevin-pulo-fun-with-ld_preload.pdf
- LD_PRELOAD libraries
- libsysconfcpus: override number of CPUs reported by sysconf()
- xlibtrace: trace libX11 calls
- xmultiwin: transparently "clone" X11 windows
- tunerlimit: control access to setrlimit(2)
- GOTCHA: a library for wrapping function calls in shared libraries
- Gotcha is a library that wraps functions. Tools can use gotcha to install hooks into other libraries, for example putting a wrapper function around libc's malloc.
- It is similar to LD_PRELOAD, but operates via a programmable API. This enables easy methods of accomplishing tasks like code instrumentation or wholesale replacement of mechanisms in programs without disrupting their source code.
- https://github.com/LLNL/GOTCHA
- Implib.so: POSIX equivalent of Windows DLL import libraries
- libosuction: A tool for stripping dynamic libraries of unneeded symbols
- https://github.com/ispras/libosuction
- System-Wide Elimination of Unreferenced Code and Data in Dynamically Linked Programs
- Ivannikov ISPRAS Open Conference (ISPRAS) 2017
- V. Ivanishin, E. Kudryashov, A. Monakov, D. Melnik, J. Lee
- https://doi.org/10.1109/ISPRAS.2017.00007
- libpreloadvaccine: Whitelisting LD_PRELOAD libraries using LD_AUDIT
- https://github.com/ForensicITGuy/libpreloadvaccine
- Whitelisting LD_PRELOAD for Fun and No Profit
- libprocesshider: Hide a process under Linux using the ld preloader
- Dynamic Linking On GNU/Linux
- emBO++ 2019; Florian Sowade
- https://www.youtube.com/watch?v=G2r5JhVDddw
- https://www.slideshare.net/FlorianSowade/dynamic-linking-on-gnulinux/
- Pardon the Interposition—Modifying and Improving Software Behavior with Interposers
- USENIX LISA 2019; Danny Chen
- https://www.usenix.org/conference/lisa19/presentation/chen
- The ACM-NVIDIA Compiler summer school lectures (2019)
- https://nptel.ac.in/courses/128/106/128106009/
- https://www.youtube.com/playlist?list=PLPeEbErKGwN15rF-BnlngobapgIRZbAK4
- Program Execution Environment
- part 1: https://www.youtube.com/watch?v=Sf4JnUcJxSQ
- part 2: https://www.youtube.com/watch?v=e3DnPTUunZk
- part 3: https://www.youtube.com/watch?v=h__sj8IfONg
- part 4: https://www.youtube.com/watch?v=yzYXrwxu6YM
- part 5: https://www.youtube.com/watch?v=4_LK7nOSlbM
- part 6: https://www.youtube.com/watch?v=CzAwzTkbMDw
- part 7: https://www.youtube.com/watch?v=IWU5Av6YpEw
- part 8: https://www.youtube.com/watch?v=Wd4NaS0_7Tc
- What makes LLD so fast?
- FOSDEM 2019; Peter Smith
- https://fosdem.org/2019/schedule/event/llvm_lld/
- https://www.youtube.com/watch?v=CeHhveHHzII
- Behind the Scenes of the Xcode Build Process
- WWDC 2018; Jake Petroules, Jürgen Ributzka, Devin Coughlin, Louis Gerbarg
- https://developer.apple.com/videos/play/wwdc2018/415/
- The Bits Between the Bits: How We Get to main()
- CppCon 2018; Matt Godbolt
- https://www.youtube.com/watch?v=dOfucXtyEsU
- Slides: https://mattgodbolt.github.io/cppcon-bits-between-bits/
- App Startup Time: Past, Present, and Future
- WWDC 2017; Louis Gerbarg
- https://developer.apple.com/videos/play/wwdc2017/413/
- "Learn about the dyld dynamic linker used on Apple platforms, how it's changed over the years, and where it's headed next. Find out how improved tooling makes it easier to optimize your app's launch time, and see how new changes coming in dyld will bring even further launch time improvements."
- LLD from a user's perspective
- lld: A Fast, Simple, and Portable Linker
- 2017 LLVM Developers’ Meeting; Rui Ueyama
- https://www.youtube.com/watch?v=yTtWohFzS6s
- https://llvm.org/devmtg/2017-10/slides/Ueyama-lld.pdf
- My Little Object File: How Linkers Implement C++
- CppCon 2017; Michael Spencer
- https://youtu.be/a5L66zguFe4
- The Missing Link: The Curious symbiosis between C++ and the Linker
- C++ Meetup Sydney hosted by IMC; December 6, 2017
- Dave Gittins
- https://www.youtube.com/watch?v=twNoIhGKIoo
- New LLD linker for ELF: A high performance linker from the LLVM project
- 2016 EuroLLVM Developers' Meeting; Rui Ueyama
- https://www.youtube.com/watch?v=CYCRqjVa6l4
- https://llvm.org/devmtg/2016-03/Presentations/EuroLLVM%202016-%20New%20LLD%20linker%20for%20ELF.pdf
- Optimizing App Startup Time: Linkers, loaders, and you
- WWDC 2016; Nick Kledzik, Louis Gerbarg
- https://developer.apple.com/videos/play/wwdc2016/406/
- "Launching an App is a complicated and subtle process and the ramifications on launch times of different App design patterns are often non-obvious. Come learn what happens in the time between when an App begins launching and when the main() function gets control and how that time relates to the code and structure of your App. Learn about the inner details of the dynamic loader, dyld, and best practices for structuring your code to perform at its best from the very start."
- lld - the LLVM Linker
- 2012 EuroLLVM Developers’ Meeting; Michael Spencer
- https://www.youtube.com/watch?v=zCaFF3aOabg
- http://llvm.org/devmtg/2012-04-12/Slides/Michael_Spencer.pdf