Originally posted by: SystemAdmin

Hi,

I hope this is the correct forum and some of you can help us on this core dump issue. We are basically stuck with this issue where our webserver program (which utilise pthreads library) keep getting core dump at certain point.

Running dbx on the program, we got this:
Type 'help' for help.
http://using memory image in ../cfg/core
reading symbolic information ...warning: no source compiled with -g

warning: Unable to access address 0x6e65743f from core
pthreaded.c, 5785: PTHDB_CALLBACK (callback failed)

Segmentation fault in pth_locks_ppc_mp.global_lock_ppc_mp [/usr/lib/libpthreads.a] at 0xd006c44c
0xd006c44c (global_lock_ppc_mp+0x1c) 810b0010 lwz r8,0x10(r11)
(dbx)

(dbx) where
pth_locks_ppc_mp.global_lock_ppc_mp() at 0xd006c44c
rec_mutex._rec_mutex_lock(??) at 0xd01f4cbc
printf.fprintf(0xf0249fb0, 0x100ea7c0, 0x6f, 0x4, 0xa, 0xeb, 0x12, 0x6d) at 0xd0207d28
server_loop__FP4soap(0x2fefd520) at 0x1005e768
main(0xb, 0x2ff229c4) at 0x1005dac0
(dbx) quit

The core dump happened both in AIX 5.2 and 5.3 machine. We have increased and set the AIXTHREADSTK set at 1024K and AIXTHREADSCOPE = S.
On AIX 5.2 machine, we get core dump almost after 120 requests being processed (not necessarily concurrently, only 10 concurrent at 1 time) and on AIX 5.3 we got core dump after 360 requests.
We compiled the program on AIX 4.3 though (if that matters).

Does anybody experience this kind of situation before? Or anybody has a clue on how to troubleshoot this?
Any response will be appreciated.

Thanks.
#AIX-Forum

Originally posted by: SystemAdmin

FWIW,
I don't have experience with this particular problem, however I was looking at a pthread issue where pthreads seems to hang and the APAR listed below seems to be the fix to our problem.

I wonder if you could be encountering similar issues.

http://www-1.ibm.com/support/docview.wss?uid=isg1IY80846

If you are using round robin scheduling (SCED_RR) in pthreads, then you might try changing it to something else like SCHED_FIFO. If that fixes the problem, then perhaps you are encountering the issue discussed in the APAR linked to above.
#AIX-Forum

Originally posted by: SystemAdmin

Thank you! :-) I will give it a try and let you know the result.
#AIX-Forum

Originally posted by: dickdunbar

Did that AIX fix help?

One thing you should always do when you get a call stack is to dump the registers. It often provides a clue for the problem.

For example, the address in the callback doesn't look quite right;
0x6e65743f

My dbx sessions almost always start out with variations of:

x; where
($link-4)/2i
($iar-64)/16i
($ctr)/4i
($r3)/8X
#AIX-Forum