The Core Pattern (core_pattern), or how to specify filename and path for core dumps

1. Introduction to Core Dumps

In most GNU/Linux systems (all of those I personally have used, at least), core dump files generated after an uncaught signal in a process (as a SIGSEGV or SIGQUIT), are generated in the base directory where the program was executed, and named as “core” or “core.PID”.

For example:

$> cd /home/user
$> ulimit -c unlimited
$> kill -s SIGSEGV $$

This will trigger a segmentation fault in your current shell (you probably guessed it after seeing that the shell session where you executed it was closed), and generate a core file in:
/home/user/core

Now… is it possible to change where that file is generated by default instead of the current directory? And is it possible to change the name of that generated file? The answer is YES! to both. Let’s see how we can get this.

2. The Core Pattern in Kernel

Since some years ago, the kernel configuration includes a file named “core_pattern”:
/proc/sys/kernel/core_pattern

In my system, that file contains just this single word:
core

As expected, this pattern shows how the core file will be generated. Two things can be understood from the previous line: The filename of the core dump file generated will be “core”; and second, the current directory will be used to store it (as the path specified is completely relative to the current directory).

Now, if we change the contents of that file… (as root, of course)

$> mkdir -p /tmp/cores
$> chmod a+rwx /tmp/cores
$> echo "/tmp/cores/core.%e.%p.%h.%t" > /proc/sys/kernel/core_pattern

And we run the same as before:

$> cd /home/user
$> ulimit -c unlimited
$> $> kill -s SIGSEGV $$

We get… voilá!
/tmp/cores/core.bash.8539.drehbahn-mbp.1236975953

Not only the program name (“bash“) or the PID (“8539“), but also the hostname (“drehbahn-mbp“) and the unix time (“1236975953“) are appended in the name of the core file!! And of course, it is stored in the absolute path we specified (“/tmp/cores/“).

You can use the following pattern elements in the core_pattern file:

%p: pid
%: '%' is dropped
%%: output one '%'
%u: uid
%g: gid
%s: signal number
%t: UNIX time of dump
%h: hostname
%e: executable filename
%: both are dropped

Isn’t is great?! Imagine that you have a cluster of machines and you want to use a NFS directory to store all core files from all the nodes. You will be able to detect which node generated the core file (with the hostname), which program generated it (with the program name), and also when did it happen (with the unix time).

3. Configure it forever

The changes done before are only applicable until the next reboot. In order to make the change in all future reboots, you will need to add the following in “/etc/sysctl.conf“:

# Own core file pattern...
kernel.core_pattern=/tmp/cores/core.%e.%p.%h.%t

sysctl.conf is the file controlling every configuration under /proc/sys

Posted on March 13, 2009, in Development and tagged , , , , . Bookmark the permalink. 20 Comments.

  1. Really good explanation 🙂 Thanks Alex for it 🙂

  2. o/

    just wanted to say that there is no need to edit the file manually. simply run the sysctl command, which does the stuff for ya.

    greetZ

  3. I have the following in a simulator environment

    echo “|/etc/cmpressdump.sh /misc/scratch/core/%h_%e_%p.core” > /proc/sys/kernel/core_pattern

    where cmpressdump.sh has
    /pkg/bin/gzip – > $1.gz
    /pkg/bin/(my_program) > /tmp/core

    When I kill a process, I see that the core file is zipped and placed correctly at /misc/scratch/core. But /pkg/bin/(my_program) which has lot of prints does not show up in /tmp/core. ie /tmp/core is an empty file

    If I simply run etc/cmpressdump.sh, then all the prints show up in the /tmp/core

    What could be wrong?

  4. Very good explanation.
    Thanks! I didn’t know it was so flexible.

  5. James Cuzella

    savvy Ubuntu users may have noticed that their /proc/sys/kernel/core_pattern file contains the following:

    |/usr/share/apport/apport %p %s %c

    This is how Ubuntu is able to detect when a program has crashed/dumped core, and apport is able to send crash reports to launchpad. This also led me to find out that ‘%c’ (another unmentioned ‘pattern element’) is supported in the core_pattern file. Running ‘apport’ with –help reveals what it is:

    /usr/share/apport/apport –help
    Usage: /usr/share/apport/apport
    The core dump is read from stdin.

    So:

    %c = core file ulimit

  6. Thanks Aleksander for the very good clear explanation.

  7. is there anything same in AIX.
    I tried ‘chore’ but it changes it to default naming convention . Can it be tailored according to need like here in linux.

  8. No idea in AIX, sorry.

  9. Thanks! It really helped me on tracking down a process that was crashing randomly. So I dont need to get gdb running to track down the problem.

  10. I really like your writing style, good information, regards for putting up acadkddcekab

  11. I should just let it be but, given the year it was posted. But well.

    just wanted to say that there is no need to edit the file manually. simply run the sysctl command, which does the stuff for ya.

    Yes, true. Until you reboot, that is. Then it magically ‘disappears’ because you set it temporarily (variables that are not saved and loaded to/from disk only remain – because this is all that is possible – while the process that owns it, exists). sysctl changes it at runtime – the configuration file itself makes it persistent. While it is less important for this setting (though why not make it persist if you have a use for it beyond one time, is beyond me), other settings it is rather important that you do indeed make it persistent, if you actually have a need for the setting (networking settings come to mind).

  12. slightly tangential to topic at hand but, dumping core on nfs mounted file-systems, sigh ! what could possibly go wrong 🙂 ?

  13. Just like James Cuzella says, ubuntu uses apport

    Actually, it seems to override /etc/sysctl.conf (ubuntu 16.04)
    because /etc/init.d/apport modifies /proc/sys/kernel/core_pattern

  14. Writing a coredump to /tmp is just a terrible idea.

    There are many problems there and one happens to be there are potential security problems. One might argue that it’s a single user system but that’s not always so simple. There are in any case other problems. Including clean up scripts. And there are others too. Fine if you want to write it to some place other than the idiotic (and it is idiotic and when this was first introduced it really irked me as a programmer even for experiments directly causing segfaults) of writing to a system directory. But if you’re going to do that might as well keep it the same even if only changing the file name format.

    Of course it might be of use to some to have the seconds since the epoch but having just the PID is extremely valuable and this includes for reporting segfaults automatically though of course one could argue that there are ways to do that with other formats. Fair enough.

    Whatever the case writing to /tmp is a bad idea. But if you prefer it so be it. Still not something I would recommend.

  15. > Writing a coredump to /tmp is just a terrible idea.

    That path is obviously just an example, never said that everyone should write coredumps to /tmp, because not every setup is the same. I’m sorry if that was inferred from the post. But hey, if you ask me, if I’m debugging my own stuff I wouldn’t mind writing them in /tmp. Not that I’ve done that in the past 10 years since I wrote the post, but well… 🙂

  1. Pingback: Kernel Core Pattern « wirelined

  2. Pingback: Learning the core_pattern in linux kernel | Life in Linux Kernel

  3. Pingback: GDB 调试dumped core文件 – kow's website

  4. Pingback: Finding coredump file – Robin on Linux

  5. Pingback: Server Bug Fix: How to disable core dumps on Debian 8 - TECHPRPR

Leave a comment