Eleven Reasons that causes INIT: Cannot create /var/adm/utmpx – Solaris

sol-troubleshootingThe utmpx and wtmpx files are extended database files that have superseded the obsolete utmp and wtmp database files.  The utmpx database contains user access and accounting information for commands such as who, write, and login. The wtmpx database contains the history of user access and accounting information for the utmpx database. We occasionally see following error in solaris during system startup, 

INIT: Cannot create /var/adm/utmp or /var/adm/utmpx

INIT failed write of utmpx entry:” “

INIT: SINGLE USER MODE

This console message indicates that init cannot write in the /var directory, which is usually part of the / (root) file system. Some other messages follow, and the system usually comes up single-user. The problem is often that / or /var is mounted read-only. Sometimes a brief power outage leaves the system believing that many file systems are still mounted. But most of the times it is just not the issue with the read-only filesystem but there could be several other reasons could cause this errors to appear.   I will be listing them here for the reference purpose:

 

 

 

Cause 1. Root Filesystem is in  Read-Only Mount State due to filesystem inconsistency.

Solution :  For troubleshooting, boot from CD-ROM (the same Solaris[TM] version that the system is running if possible) into single user mode, and mount the core Solaris filesystems under /a

ok boot cdrom -sw

# /usr/sbin/fsck /dev/rdsk/c0t0d0s0

# mount /dev/dsk/c0t0d0s0 /a

If they are on separate disk slices, repeat the /usr/sbin/fsck and mount steps for the /usr, /var, and /opt filesystems. Mount them to /a/usr, /a/var, and /a/opt, respectively.

Example:

# /usr/sbin/fsck /dev/rdsk/c0t0d0s4

# mount /dev/dsk/c0t0d0s4 /a/var

 

 

 

Cause 2:  Invaid Entries In /etc/vfstab where the Solaris Core Filesystems are referring to wrong or invalid devices, when configured with Solarice Volume Manager or Veritas Volume Manager.

 

 

Solution : For troubleshooting, boot from CD-ROM (the same Solaris[TM] version that the system is running if possible) into single user mode, and mount the core Solaris filesystems under /a.

If the core Solaris OE filesystems are mounted from physical disk slices, not Veritas Volume Manager or Solstice DiskSuite[TM] volumes, then make sure the devices in /a/etc/vfstab point to valid entries in /a/dev and that the entries in /a/dev are symbolic links to valid devices in the /a/device directory, especially /var if it is a separate slice

Check the each and every device path for the filesystems mentioned in the  vfstab for both block and raw devices using ls commands. And confirm the path and device exists.

 

 

Cause 3 :  / and /var filesystem are full

 

Solutions :   For troubleshooting, boot from CD-ROM (the same Solaris[TM] version that the system is running if possible) into single user mode, and mount the core Solaris filesystems under /a

# /usr/sbin/df -k

 Filesystem           kilobytes    used      avail   capacity    Mounted on

/dev/dsk/c0t0d0s0     3277234    2837450   407012     88%           /a

If /a or /a/var has 100% in the capacity column, clean out files on the full filesystem

 

 

Cause 4 :  Run Level Scripts either from  /etc/rc [S,2,3]  or /sbin/rc[S,2,3] are corrupted

 

Solution :  Run the scripts in shell debug mode by adding “set -x” in these scripts. This will enable verbose output from the script as it runs, which may help pinpoint the problem or its source.

Example:

 

#!/sbin/sh set -x

 

 

Cause 5 :  Invalid hostname files exists in /etc directory  /etc/hostname.hme0.bakcup.

 

Solution :  The startup script /etc/init.d/network will fail to parse these file names correctly , if they are similar to below

Example of problem filename:  /etc/hostname.hme0.9mar2005

 

Cause 6  :  Incorrect permissions for /var/adm/wtmpx and /var/adm/utmpx

 

Solution :   For troubleshooting, boot from CD-ROM (the same Solaris[TM] version that the system is running if possible) into single user mode, and mount the core Solaris filesystems under /a.

Since the files may be corrupted, clear their contents and again check permissions and ownership. Do the following:

# /usr/bin/cp /dev/null /a/var/adm/utmpx

# /usr/bin/cp /dev/null /a/var/adm/wtmpx

# /usr/bin/chown root:bin /a/var/adm/utmpx

# /usr/bin/chmod 644 /a/var/adm/utmpx

# /usr/bin/chown adm:adm /a/var/adm/wtmpx

# /usr/bin/chmod 644 /a/var/adm/wtmpx

There should also be a /a/etc/utmpx and /a/etc/wtmpx that are symbolic links to the files in ../var/adm/

Example:

# /usr/bin/ls -al /a/etc/tmpx

lrwxrwxrwx 1 root root 16 Aug 15 2001 /etc/utmpx -> ../var/adm/utmpx

lrwxrwxrwx 1 root root 16 Aug 15 2001 /etc/wtmpx -> ../var/adm/wtmpx

# /usr/bin/ls -al /a/var/adm/ tmpx

-rw-r–r– 1 root bin 11160 Apr 27 00:00 /var/adm/utmpx

-rw-r–r– 1 adm adm 334056 Apr 25 17:53 /var/adm/wtmpx

 

 

Cause 7 :  Corrputed /etc/inittab

 

Solution :  Check the /a/etc/inittab file for corruption by comparing to another system running the same Solaris OE. The /etc/inittab file should not be used for comparison while booted from CD-ROM, as its configuration is geared toward the installation process not system recovery. If /etc/inittab is corrupt, copy one the one from /cdrom//Product/SUNWcsr/reloc/etc to /a/etc. 

 

 

Cause 8 :  Missing  Console Device files  i.e.  /dev/console linked to devices/pseudo/cn@0:console 

 

Solution :   Make sure /dev/console exists. It should be a symbolic link to ../devices/pseudo/cn@0:console The major and minor number of the device file in /devices should be zero. The /etc/name_to_major file should also have a cn device listed with major number zero.

Example:

# /usr/bin/ls -al /a/dev/console

lrwxrwxrwx 1 root other 30 Aug 15 2001 /a/dev/console -> ../devices/pseudo/cn@0:console

 

 

Cause 9 :  Corrupted /etc/mnttab

 

Solution : For troubleshooting, boot from CD-ROM (the same Solaris[TM] version that the system is running if possible) into single user mode, and mount the core Solaris filesystems under /a.  Make a backup of /a/etc/mnttab and cat /dev/null to the original. 

 

 

Cause 10 :  Corrupted Kernel due to kernel patch upgrade

 

Solution : For troubleshooting, boot from CD-ROM (the same Solaris[TM] version that the system is running if possible) into single user mode, and mount the core Solaris filesystems under /a.

This is a common one if the error started after a patch installation (possibly in multi-user mode). Often, a restore from backup is required, but the kernel patch can possibly be backed out:

Example:

# /usr/sbin/ patchrm -R /a kernel_patch-revision

 

Cause 11 :  /Var filesystem unavailable. The /var partition did not mount on booting

 

Solution :   If the /var directory is a symbolic link to a filesystem that is not either / or /usr, then the above errors will appear on boot up. Note: Making /var a separate partition (in /etc/vfstab) is okay. After making a backup copy, edit /etc/init.d/standardmounts, modify the file so that the file system which contains /var is mounted. The following is an example where /var resides in /export/home/var:

exec < ${vfstab}; readvfstab “/export/home”

if [ “${mountp}” -a -d /export/home ]

then /sbin/mount -m /export/home > /dev/null 2>&1

mntlist=”${mntlist}/export/home /export/home”

fi

 

 

 

Ramdev

Ramdev

I have started unixadminschool.com ( aka gurkulindia.com) in 2009 as my own personal reference blog, and later sometime i have realized that my leanings might be helpful for other unixadmins if I manage my knowledge-base in more user friendly format. And the result is today's' unixadminschool.com. You can connect me at - https://www.linkedin.com/in/unixadminschool/

What is in your mind, about this post ? Leave a Reply

Close
  Our next learning article is ready, subscribe it in your email

What is your Learning Goal for Next Six Months ? Talk to us