WebSphere Application Server & Liberty

 View Only

Lessons from the Field #29: Linux core_pattern best practices

By Kevin Grigorenko posted Tue May 16, 2023 09:00 AM


In general, it has become an industry best practice to set the Linux core_pattern to a piped core dump processing program. The most common modern such programs are |/usr/lib/systemd/systemd-coredump and |/usr/share/apport/apport.

Conversely, in general, it has become an industry malpractice to set the Linux core_pattern to core (which produces a core dump in the process current working directory).

There are two main reasons for these updated practices:

  1. A piped core dump processing program can limit the disk usage of core dumps thus avoiding disk space exhaustion which may cause production outages.
  2. In container environments (e.g. Kubernetes, OpenShift, etc.), a core_pattern of core produces core dumps inside the container, most often on an ephemeral filesystem that gets deleted when the container stops. In contrast, a piped core dump processing program runs on the worker node and thus saves off the container's core dump even if it stops.
    1. Moreover, the common practice of overriding core_pattern to core is much less palatable in such environments because it would apply to all containers, some of which are not well designed to handle core dumps. The Linux kernel currently does not support masking core_pattern on a per-container/pod basis.

Another common core_pattern is |/opt/dynatrace/oneagent/agent/rdp. Consult the value of cat /opt/dynatrace/oneagent/agent/conf/original_core_pattern to determine the underlying core_pattern. If /opt/dynatrace/oneagent/agent/conf/original_core_pattern is core, than that is still, in general, a malpractice.

Previous preferences for producing core dumps for IBM Java and IBM Semeru Runtimes with core_pattern=core are no longer applicable. The JVMPORT030W and JVMDUMP012E messages have been clarified and a new message JVMPORT049I has been added to emphasize this. Additional documentation describes these messages and how to configure piped core dump processing programs.

As covered in the aforementioned documentation, the largest remaining issue is that versions of systemd-coredump before v251 truncate 64-bit dumps at 2 GB by default. Later versions truncate at 32GB by default. Ensure the piped core dump processing program is configured not to truncate dumps. In environments such as OpenShift, use MachineConfigs. In contrast, apport does not truncate by default.

#automation-portfolio-specialists-app-platform #WebSphereLiberty #java #RedHatOpenShift #WebSphereApplicationServer