IBM Open XL 17.1.2 introduces hotpatching functionality on AIX, allowing patches to be applied to a running program without interruption. The concept of hotpatching also exists in the Linux kernel.
The clang option -fpatchable-function-entry=N,8 is used to enable hotpatching in Open XL. When this option is specified, the compiler instruments the generated code by padding around the function entry such that at runtime, the padded section can be modified to jump to the patched version of the function. Open XL documentation on -fpatchable-function-entry.
For example:
$ cat test.c
int foo() {
return 5;
}
Disassembly without hotpatching:
$ ibm-clang test.c -c
$ llvm-objdump -d test.o
Disassembly of section .text:
00000000 <.foo>:
0: 38 60 00 05 li 3, 5
4: 4e 80 00 20 blr
Disassembly with hotpatching which represents the function label padding:
$ ibm-clang -fpatchable-function-entry=9,8 test.c -c
$ llvm-objdump -d test.o
Disassembly of section .text:
00000000 <.text>:
0: 68 6f 74 70 xori 15, 3, 29808 --> (This is the encoding for the characters of "hotp" string.)
4: 61 74 63 68 ori 20, 11, 25448 --> (This is the encoding for the characters of "atch" string.)
8: 68 6f 74 70 xori 15, 3, 29808
c: 61 74 63 68 ori 20, 11, 25448
10: 68 6f 74 70 xori 15, 3, 29808
14: 61 74 63 68 ori 20, 11, 25448
18: 68 6f 74 70 xori 15, 3, 29808
1c: 7f e0 00 08 trap
00000020 <.foo>:
20: 60 00 00 00 nop
24: 38 60 00 05 li 3, 5
28: 4e 80 00 20 blr
In this code snippet, there is an 8-word block before the function entry label that contains the 3.5 repetition of the "hotpatch" string followed by an unconditional trap, and one NOP after the function entry label. This is similar to what legacy XL generates with -qxflag=hotpatch except that the last 4 byte is modified to be an unconditional trap to prevent the accidental branching to this area.
Furthermore, Open XL 17.1.2.2 extends hotpatching support to work with the -ffunction-sections option, which was not available in the initial 17.1.2 release.
Open XL does not currently support hotpatching at the function level. Instead, hotpatching is applied to all functions within a file when using the -fpatchable-function-entry option during compilation.
For hotpatching to work properly, we need to control how the compiler optimizes code to make sure hotpatching maintains program correctness.
When LLVM optimizes one function based on the information from another function, this optimized function is termed an 'impacted function' of the other. If we patch a function, we must also patch its impacted functions. For example, inlining is one such inter-procedural optimization (IPO), and if the callee 'foo' is inlined into caller 'bar' and 'foo' had to be hotpatched, then for correctness, 'bar' needs to be hotpatched as well, as it is impacted by 'foo'.
The more inter-procedural optimizations we apply, the more functions each function impacts. To minimize the number of impacted functions we can choose to enable only specific IPO optimizations.
To do so, Open XL 17.1.2.5 implements the -flive-patching=inline-only-static option similar to GCC. When specified, this option disables any inter-procedural optimizations that rely on details of one function to modify other functions, and it only allows inlining of static functions, C inline functions and C++ explicit instantiation declaration of inline template functions. This means to correctly use the hotpatching option, it is not needed to disable incompatible passes using compiler flags like -fno-inline. Open XL documentation on -flive-patching.
The following are some -mllvm options to support debugging for -flive-patching. (note that mllvm options in general are not officially supported and might change)
- live-patching-allow-inlining to enable unrestricted inlining for debugging.
- live-patching-pass-index-map to print an index:pass mapping for the passes that are disabled with the -flive-patching option.
- live-patching-allow-pass=<index>[,<index>...] that takes a comma separated list of indexes and enables them with the -flive-patching option. The index of the disabled passes can be printed using the live-patching-pass-index-map option.
Example:
$ ibm-clang -mllvm --live-patching-pass-index-map test.c
<0:ArgumentPromotionPass>
<1:CalledValuePropagationPass>
<2:DeadArgumentEliminationPass>
...
<28:CoroConditionalWrapper>
<29:CoroEarlyPass>
<30:CoroSplitPass>
To enable a set of passes from the above list:
$ ibm-clang -O3 -flive-patching=inline-only-static -mllvm --live-patching-allow-pass=0,1 test.c
It allows the ArgumentPromotionPass and CalledValuePropagationPass passes to run when -flive-patching is defined.