Originally posted by: SystemAdmin

Ok, I've got an AIX 5.3 partition built on a p550q and having trouble with NTP. I've got it running on the two other partitions on this piece of hardware, but whenever I try to start ntp on this particular partition, it runs for about 30 seconds and then errors out. I get a generic "Software Program Error" from errpt. I've got my ntp.conf set up the exact same as the other two partitions, so I feel pretty confident it's ok.

I've ran the ntpdate -d {IP of NTP server } and it communicates with and adjusts the time via that utility.

I'm baffled!!

Originally posted by: HajoEhlers

1) Use IP address in the /etc/ntp.conf file for remote time servers for trouble shooting
2) use " lssrc -ls xntpd " and compare with an working system

hth
Hajo

Originally posted by: SystemAdmin

Thanks for the response.

I'm using the IP address in the ntp.conf already. When I start the xntpd daemon using startsrc -s xntpd, it only "runs" for about 30 seconds before I get the error from errpt.
In that time, I tried running the lssrc -ls xntpd and it says the request can't be sent to the xntpd subsystem. It says to restart the subsystem and try again.
It's like the subsystem never gets fully started and then dies.

The following is output from errpt -a

LABEL: SRC_SVKO
IDENTIFIER: BC3BE5A3

Date/Time: Tue Nov 21 10:18:01 EST 2006
Sequence Number: 59
Machine Id: 000B1212D600
Node Id: sceptre_2
Class: S
Type: PERM
Resource Name: SRC

Description
SOFTWARE PROGRAM ERROR

Probable Causes
APPLICATION PROGRAM

Failure Causes
SOFTWARE PROGRAM

Recommended Actions
MANUALLY RESTART SUBSYSTEM IF NEEDED

Detail Data
SYMPTOM CODE
768
SOFTWARE ERROR CODE
-9017
ERROR CODE
0
DETECTING MODULE
'srchevn.c'@line:'350'
FAILING MODULE
xntpd

Originally posted by: HajoEhlers

0) Check that not a xntpd is already running
$ ps -ef | grep ntp

1) Verify that the ntpd.conf is correct - no ^M ( CR/LF ) from a dos editor
Example from our site:

driftfile /etc/ntp.drift
tracefile /etc/ntp.trace
server ntp03
server ntp04

1.1 ) Check the permission on the file
ls -l /etc/ntp.conf
-rw-rw-r-- 1 root system 1042 Nov 17 14:18 /etc/ntp.conf

2) Start the xntpd from the command line in debug mode or trace the startup

$ /usr/sbin/xntpd -d -d -d -d -d -d -d -d -d -d
or
$ truss -f /usr/sbin/xntpd

and try to figure out whats going on

hth
Hajo

Originally posted by: SystemAdmin

Thanks again for your reply. I started xntpd with the debugging mode as you suggested. I sent it to a file, and this is what it said:

28 Nov 12:11:46 xntpd213120: logging to file /tmp/ntp.log
28 Nov 12:11:46 xntpd213120: 3.4y
adding new filegen
adding new filegen
adding new filegen
adding new filegen
28 Nov 12:11:46 xntpd213120: Clock init couldn't find kernel as either /vmunix or /unix

Now, I'm running the 64-bit kernel, so /unix_64 is present, but it appears this is looking for just /unix. The weird thing is that it runs fine on another partition that was loaded with the same set of OS cds and is also running 64-bit mode.
The following is the last screen of output from the trace:

217232: kfcntl(3, F_SETFD, 0x00000001) = 0
217232: connext(3, 0xF029AFCC, 16) = 0
217232: _getpid() = 217232
217232: open("/dev/null", O_WRONLY|O_CREAT|O_TRUNC) = 4
217232: kioctl(4, 22528, 0x00000000, 0x00000000) Err#19 ENODEV
217232: kioctl(4, 22528, 0x00000000, 0x00000000) Err#19 ENODEV
217232: kwrite(4, " 3 . 4 y\n", 5) = 5
217232: close(4) = 0
217232: send(3, 0x2FF21E04, 40, 0) = 40
217232: _setpriority(0, 0, -12) = 0
217232: _sigaction(2, 0x2FF22C20, 0x00000000) = 0
217232: _sigaction(3, 0x2FF22C20, 0x00000000) = 0
217232: _sigaction(15, 0x2FF22C20, 0x00000000) = 0
217232: _sigaction(30, 0x2FF22C20, 0x00000000) = 0
217232: _sigaction(31, 0x2FF22C20, 0x00000000) = 0
217232: _sigaction(13, 0x2FF22C20, 0x00000000) = 0
217232: open("/dev/kmem", O_RDONLY) = 4
217232: statx("/kernel/unix", 0x2FF22B18, 76, 0) Err#2 ENOENT
217232: statx("/unix", 0x2FF22B18, 76, 0) = 0
217232: open("/unix", O_RDONLY|O_LARGEFILE) = 5
217232: kioctl(5, 22528, 0x00000000, 0x00000000) Err#25 ENOTTY
217232: kioctl(5, 22528, 0x00000000, 0x00000000) Err#25 ENOTTY
217232: kread(5, "\0\0\0\0\0\0\0\0\0\0\0\0".., 4096) = 0
217232: close(5) = 0
217232: open("/usr/lib/nls/msg/en_US/xntpd.cat", O_RDONLY) = 5
217232: kioctl(5, 22528, 0x00000000, 0x00000000) Err#25 ENOTTY
217232: kfcntl(5, F_SETFD, 0x00000001) = 0
217232: kioctl(5, 22528, 0x00000000, 0x00000000) Err#25 ENOTTY
217232: kread(5, "\0\001 ù\01B\01B I S O 8".., 4096) = 4096
217232: lseek(5, 0, 1) = 4096
217232: lseek(5, 376, 1) = 4472
217232: kread(5, "\01A\002\001\01A\0\0 ^ È".., 4096) = 4096
217232: lseek(5, 0, 1) = 8568
217232: lseek(5, 0, 1) = 8568
217232: lseek(5, 0, 0) = 0
217232: kread(5, "\0\001 ù\01B\01B I S O 8".., 4096) = 4096
217232: kread(5, "\017\0 1\0\0 T ü\018\0 I".., 4096) = 4096
217232: _getpid() = 217232
217232: lseek(5, 0, 1) = 8192
217232: lseek(5, 16143, 0) = 16143
217232: kread(5, " C l o c k i n i t c".., 4096) = 4096
217232: _getpid() = 217232
217232: open("/dev/null", O_WRONLY|O_CREAT|O_TRUNC) = 6
217232: kioctl(6, 22528, 0x00000000, 0x00000000) Err#19 ENODEV
217232: kioctl(6, 22528, 0x00000000, 0x00000000) Err#19 ENODEV
217232: kwrite(6, " C l o c k i n i t c".., 59) = 59
217232: close(6) = 0
217232: send(3, 0x2FF21C94, 94, 0) = 94
217232: __loadx(0x04000000, 0x2FF22200, 0x00000800, 0x0000D0B2, 0x00000000) = 0x00000000
217232: kfcntl(1, F_GETFL, 0x00000001) = 0
217232: _exit(3)

I really can't figure out why it's having a problem with the kernel.

Originally posted by: SystemAdmin

Sounds like you could have a bad /unix symlink in the root directory, check that it resolves correctly to your 64 bit kernel.

Should be.....
TESTBED:/ #ls -la /unix
lrwxrwxrwx 1 root system 21 Apr 28 2006 /unix -> /usr/lib/boot/unix_64

TESTBED:/ #ls -al /usr/lib/boot/unix_64
-r-xr-xr-x 1 root system 14149648 Jul 7 04:22 /usr/lib/boot/unix_64

You may have to re-link it as it could be still pointing to the 32bit kernel.

Here's a pretty good xntpd article http://unix.ittoolbox.com/groups/technical-functional/ibm-aix-l/ntp-network-time-protocol-set-up-on-aix-52-944293

Cheers...Steve

Originally posted by: SystemAdmin

That certainly was the issue. Something had "stomped" all over the unix_64 file in /usr/lib/boot had a zero-byte size. I've corrected the problem and ntp started up just fine.

Thanks to everyone for their help.

Originally posted by: HajoEhlers

Sound not to good. You are the second person an a AIX 5.3 system where the unix_64 has got nulled. Since you mention on a previou post /unix_64 i hope you are aware that you are talking about /usr/lib/boot/unix_64 where /unix is a link to it

$ ls -l /unix
/unix -> /usr/lib/boot/unix_64

hth
Hajo